EP3924795A1

EP3924795A1 - Trajectory prediction for driving strategy

Info

Publication number: EP3924795A1
Application number: EP19914873.5A
Authority: EP
Inventors: Yao Ge; Maximilian DOEMLING; Dominik Notz; Gaowei Xu
Original assignee: Bayerische Motoren Werke AG
Current assignee: Bayerische Motoren Werke AG
Priority date: 2019-02-15
Filing date: 2019-02-15
Publication date: 2021-12-22
Also published as: EP3924795A4; CN113454555A; WO2020164090A1

Abstract

Examples of the present disclosure describe a method, an apparatus, and a vehicle for trajectory prediction. The method comprises obtaining sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle (502); feeding the obtained sensor data into a hybrid predictor consisting of a plurality of predictors each with a weight assigned by the hybrid predictor, wherein the plurality of predictors comprise at least a free space predictor and a roadmodel-based predictor (504); for a current time instant, each of the plurality of predictors giving its own predictions for a predetermined period of time from the current time instant based on historical sensor data from the at least one sensor (506); the hybrid predictor determining a reliability of each one of the plurality of predictors (508); the hybrid predictor outputting weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors and the reliability of each one of the plurality of predictors, with assigning a greater weight to the predictor that gave a better prediction in the previous one or more time instants (510); and providing a predicted trajectory based on the weighted hybrid predictions (512).

Description

Trajectory Prediction for Driving Strategy

FIELD OF THE INVENTION

The present disclosure relates in general to automated driving vehicles, and in more particular, to trajectory predictions for driving strategies.

BACKGROUND OF THE INVENTION

An automated driving vehicle (also known as a driverless car, self-driving car, or robotic car) is a kind of vehicle that is capable of sensing its environment and navigating without human input. Automated driving vehicles (hereinafter, called as ADV) use a variety of techniques to detect their surroundings, such as radar, laser light, GPS, odometry and computer vision. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage.
To achieve a high level (Level 3-5) autonomous driving, the future trajectory of objects should be precisely modelled for driving strategy decision and collision avoidance. Currently, the prior art in trajectory prediction includes the following methodologies:
(1) Applying a Kalman filter or its variants to estimate agents' kinematic states such as velocities, positions and accelerations, etc.;
(2) Using model predictive control (MPC) approaches to minimize incidences for road collisions, which consider the obstacles in road environment;
(3) Predefining various maneuvers and utilize a hidden Markov model (HMM) to select the most reasonable one for each object based on its current kinematic features; and
(4) Exploiting dynamic Bayesian network (DBN) to model objects' feature states.
The above prior art methodologies may have at least the following disadvantages:
(1) Current predictions are mainly based on lanes, i.e., not in-between lanes;
(2) Quite a lot of assumptions are made to apply these algorithms and approaches such as straight lane, only a limited number of agents considered in lanes, a constant velocity model is assumed, etc. These assumptions usually don't hold in practice;
(3) Predefined trajectory is not suitable for various complex scenarios in urban driving environment;
(4) Sometimes, there doesn't exist lanes at all, so these prior arts cannot be adapted easily under such circumstances because these algorithms depends on lanes significantly;
(5) These prior arts are built based on constant velocity models or constant acceleration models, which are useful when the time horizon is within a second, but it becomes less efficient for time horizons of a few seconds in that agents' kinematic features usually change over time;
(6) Some static agents are detected by one or more Lidars, but due to that they are static, the cloud points are oscillating sometimes, and the fused object is also not stable. It will induce bad prediction performance when the detection noise is not negligible;
(7) Autonomous driving vehicles will need more accurate predictions of the future trajectories of other vehicles for safe and efficient driving; and
(8) Lane-based driving assumptions limit the power of predictions and leads to inaccurate results.
Therefore, an improved solution of trajectory prediction is desired for automated driving vehicles.
SUMMARY OF THE INVENTION
The present disclosure aims to provide a method, an apparatus and a vehicle for trajectory prediction.
In accordance with a first exemplary embodiment of the present disclosure, a computer-implemented method for trajectory prediction is provided. The method comprises obtaining sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle; feeding the obtained sensor data into a hybrid predictor consisting of a plurality of predictors each with a weight assigned by the hybrid predictor, wherein the plurality of predictors comprise at least a free space predictor and a roadmodel-based predictor; for a current time instant, each of the plurality of predictors giving its own predictions for a predetermined period of time from the current time instant based on historical sensor data from the at least one sensor; the hybrid predictor determining a reliability of each one of the plurality of predictors; the hybrid predictor outputting weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors and the reliability of each one of the plurality of predictors, with assigning a greater weight to the predictor that gave a better prediction in the previous one or more time instants; and providing a predicted trajectory based on the weighted hybrid predictions.
In accordance with a second exemplary embodiment of the present disclosure, a trajectory prediction apparatus is provided. The apparatus comprises a sensor data obtaining module configured to obtain sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle; a hybrid predictor consisting of a plurality of predictors each with a weight assigned by the hybrid predictor, wherein the plurality of predictors comprises at least a free space predictor and a roadmodel-based predictor, and the hybrid predictor is configured to: receive the sensor data from the sensor data obtaining module; for a current time instant, use the free space predictor and the roadmodel-based predictor to each give its own predictions for a predetermined period of time from the current time instant based on historical sensor data from the at least one sensor, determine which one of the plurality of predictors gave a better prediction that is closer to the actual movement of the object in previous one or more time instants; and output weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors for the current time instant with assigning a greater weight to the one predictor that gave the better prediction in the previous one or more time instants; and a trajectory predicting module configured to provide a predicted trajectory based on the weighted hybrid predictions.
In accordance with a third exemplary embodiment of the present disclosure, a vehicle is provided. The vehicle comprises at least one sensor configured to capture sensing data for objects surrounding the vehicle; the trajectory prediction apparatus according to the second embodiment; and a decision module configured to make vehicle control decisions based on the trajectories for objects surrounding the vehicle predicted by the trajectory prediction apparatus.
With the above method, apparatus and vehicle for trajectory prediction, at least the following advantages may be achieved with respect to the prior art methodologies:
(1) The invention can handle the scenarios where there doesn't exist lanes such as road junctions.
(2) The invention does not require any assumptions about the roads.
(3) The invention can predict the future trajectories and kinematic states of agents which may not move in lanes, such as pedestrians, scooters, bicycles, etc., because the free space predictor will play a significant role in this scenarios.
(4) The invention can alleviate the phenomenon that the predicted results are oscillatory when agents are moving with very low velocities or even static in that the hybrid predictor has a static agent classifier.
(5) The invention improves the prediction accuracy of future vehicle trajectories and it is more flexible and independent of road structures.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the present disclosure. Note that the drawings are not necessarily drawn to scale.
Fig. 1 illustrates an exemplary road environment.
Fig. 2 shows a schematic block diagram of a trajectory prediction apparatus in accordance with one embodiment of the present invention.
Fig. 3 is a partial enlarged view of the road environment of Fig. 1.
Fig. 4 is another partial enlarged view of the road environment of Fig. 1.
Fig. 5 is a flow diagram of a method for trajectory prediction, in accordance with one embodiment of the present invention.
Fig. 6 illustrates an exemplary vehicle according to an embodiment of the present invention.
Fig. 7 illustrates a general hardware environment wherein the present disclosure is applicable in accordance with an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the described exemplary embodiments. It will be apparent, however, to one skilled in the art that the described embodiments can be practiced without some or all of these specific details. In other exemplary embodiments, well known structures or process steps have not been described in detail in order to avoid unnecessarily obscuring the concept of the present disclosure.
The term “vehicle” used through the specification refers to cars of any type, including but not limited to sedans, vans, trucks, buses, or the like. For simplicity, the invention is described with respect to “car” . The term “A or B” used through the specification refers to “A and B” and “A or B” rather than meaning that A and B are exclusive, unless otherwise specified.
As mentioned before, there are several methods of trajectory prediction in the prior art. Among these methods, more typical trajectory predictions are the roadmodel-based trajectory prediction and the movement-history-based trajectory prediction.
The roadmodel-based trajectory prediction is based on existing road models, and thus, apparently, such trajectory prediction is only suitable for those areas where road models exist. Also, it is only applicable to automobiles. In addition, such kind of trajectory prediction also assumes that cars comply with traffic rules, for example, cars will travel along the centerlines of lanes, cars will change direction according to the lane direction indicators, cars will not change lanes arbitrarily or illegally, and the like. The movement-history-based trajectory prediction is based on the movement of an object over a period of time (also referred to as "time horizon" ) to predict future trajectories. How these two trajectory predictions will work in a road environment will be explained with respect to Fig. 1.
Fig. 1 illustrates an exemplary road environment. Shown in Fig. 1 is an exemplary three-way intersection comprising a main road in a vertical direction and a branch in a horizontal direction. The main road has 2 lanes, one being a straight-only lane and the other being a right-turn-only lane for cars to turn right into the branch. Also shown in Fig. 1 is a car 102 traveling on the right-turn lane and is about to reach the intersection. At this moment, a roadmodel-based trajectory prediction is able to recognize that the car 102 is traveling on the right-turn lane due to the knowledge of the road model of this intersection. Meanwhile, according to the assumption of this model, i.e., the car 102 will follow the traffic rules, the roadmodel-based trajectory prediction thus gives a prediction that the car 102 will travel along the trajectory shown as the dashed line 104, i.e., turning right at the intersection and driving into the branch. On the other hand, the movement-history-based trajectory prediction is unaware of the road model of the intersection. It gives predictions only based on the movement history of an object over a period of time. The previous two observation positions 102' and 102" of the car 102 are schematically illustrated in Fig. 1. It will be appreciated that only two previous positions are shown for simplicity, and in practice, more historical locations (e.g., 5 or 10 locations) may be necessary to predict the trajectory. Since the previous positions 102' and 102" of the car and the current position 102 are almost on a straight line, the movement-history-based trajectory prediction thus predicts that the car 102 will travel along the trajectory shown by the dashed line 106, i.e., continually travelling straightforward.
It can be understood that, in the above example, for a vehicle traveling in compliance with traffic rules, the roadmodel-based trajectory prediction, if applicable, can generally predict the future trajectory of the vehicle more accurately. However, as mentioned before, such roadmodel-based trajectory prediction can only be applied in areas with road models, and it must be assumed that objects obey traffic rules, and such prediction is only for cars. In contrast, the movement-history-based trajectory prediction is much more flexible. It works without various assumptions and it is not limited in cars. Its disadvantage is that, since its prediction is based only on historical trajectories, such prediction is actually more similar to summarizing a past trajectory (such as finding a common smooth curve for location points) and extending this trajectory to the future. Therefore, for a motion trajectory with a sharp change (such as a right-angle turn) , the trajectory prediction may deviate greatly from the actual trajectory, and the prediction may be less accurate as the predicted future time is further away from the current time.
Fig. 2 shows a schematic block diagram of a trajectory prediction apparatus 200 in accordance with one embodiment of the present invention. As shown in Fig. 2, the trajectory prediction apparatus 200 may include a sensor data obtaining module 202, a hybrid predictor 204, and a trajectory predicting module 206. The sensor data obtaining module 202 is configured to obtain sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle. The hybrid predictor 204 may include a plurality of existing predictors. As a non-limiting example, the hybrid predictor 204 may include at least one free space predictor 208 and one road model based predictor 210. In addition, the hybrid predictor 204 may optionally include a static object classifier 212. The free space predictor 208 may be the previously mentioned track predictor based on the movement history of the object. As a non-limiting example, the free space predictor 208 may be a trained RNN-based predictor. It will be appreciated that such free space predictor 208 is applicable for a variety of objects, and thus objects for which the trajectory prediction of the present invention is applicable include, but are not limited to, cars, pedestrians, bicycles, scooters, or any other moving objects. As mentioned before, the free space predictor 208 may calculate the movement trajectory curve of an object based on sensor data provided by the sensor data obtaining module 202, such as historical locations of the object. This can be done by means of a Kalman filter, for example. Thereafter, the free space predictor 208 may give the positions of points on the movement trajectory curve corresponding to a plurality of time points in the future as the prediction result. The roadmodel-based predictor 210, as previously mentioned, may give a prediction of future locations of the object based on its knowledge of the road model where the current location of the object is.
At each time instant (i.e., a "current" time instant, t _c) , the free space predictor 208 (labeled as Predictor 1) and the roadmodel-based predictor 210 (labeled as Predictor 2) may respectively give a prediction of positions at a plurality of time instances (t _cˇ1, t _cˇ2, t _cˇ3, t _cˇ4, …) in a future period of time (for example, 5 seconds, 10 seconds) , and, The number of time instances may depend on the size of the time horizon and the sampling accuracy. Therefore, a data structure can be contemplated as shown in Table 1 below:
Table 1
Thus, it can be understood that and corresponding to the current time instance tc in the above table are the predictions for the position of the object at the current time instance tc respectively given by the Predictor 1 and the Predictor 2 at previous time instances. On the other hand, the actual current position of the object can already be determined by the sensor data obtained from the sensors. Therefore, assuming that the actual position is Gc, then by comparing Gc with and it is possible to determine which one of the predictors is currently more reliable or more accurate.
Fig. 3 is a partial enlarged view of the road environment of Fig. 1. In Fig. 3, it is illustrated that the car 102 has reached the intersection and has just begun to turn to the right. It is assumed that the current time point corresponding to the current position of the car 102 is tc, and its position is marked as Gc. In addition, it can be seen that the previous positions of the car 102 are Gc _ˉ1, Gc _ˉ2, Gc _ˉ3, Gc _ˉ4, Gc _ˉ5, …, respectively. As mentioned before, at time instance t _c-1, i.e., when the car 102 was still at G _c-1, predicted values and respectively given by the free space predictor 208 and the roadmodel-based predictor 210 are located at corresponding positions as shown in the figure. It can be seen that the free space predictor 208 considers that the car 102 is traveling in a straight line based on the current and previous positions Gc _ˉ1, Gc _ˉ2, Gc _ˉ3, Gc _ˉ4, and Gc _ˉ5 of the car 102, and thus the prediction values reflect that the car 102 will continue traveling forward along the straight line, which is the trajectory 106. In contrast, since the roadmodel-based predictor 210 is aware of that the car 102 is traveling on the right-turn lane, the predicted values given by it reflect that the car 102 will travel along the trajectory 104. Thus it can be seen that, at the past time instance t _c-1, the predicted value given by the roadmodel-based predictor 210 for the current time tc is closer to the actual value Gc, and therefore the roadmodel-based predictor 210 is currently a more accurate and reliable predictor.
The hybrid predictor 204 is configured to fuse the predictions from a plurality of predictors and output fused prediction results. Typically, the fusion can be done by assigning a weight to each predictor and using the weighted value as the fused prediction. For example, the Predictor 1 may be assigned a weight w ₁ and the Predictor 2 may be assigned a weight w ₂. The fused prediction results can thus be calculated as:
wherein the sum of the weights w ₁ and w ₂ is 1, and i is a positive integer.
Returning to Fig. 3, since the roadmodel-based predictor 210 is currently (at time tc) the more accurate predictor, the hybrid predictor 204 assigns the roadmodel-based predictor 210 a higher weight, i.e., w ₂ ＞ w ₁ when predicting the trajectory for future time instances, t _c+1, t _c+1, t _c+1, .... . The specific assigning of the values of w ₁ and w ₂ can be determined based on the differences between and Gc and and Gc, respectively.
As another embodiment, the assigning of the values of w ₁ and w ₂ may further take into account the deviation of historical predicted values from the actual value. Fig. 4 is also a partial enlarged view of the road environment of Fig. 1. Different from Fig. 3, now assuming that the current time instance is t _c+1, and the car 102 has traveled a certain distance forward along the trajectory 104. At the previous time instance tc, the free space predictor 208 considered that the car 102 was only slightly off the line based on the current and previous positions Gc, Gc _ˉ1, Gc _ˉ2, Gc _ˉ3, Gc _ˉ4, and Gc _ˉ5 of the car 102, and accordingly, the predicted positions given, reflect that the car 102 would travel along the trajectory 106', which only slightly deviates from the original trajectory 106. The roadmodel-based predictor 210 is aware of that the car 102 is traveling on the right-turn lane, so the predicted value given by it reflect that the car 102 will continue to travel along the trajectory 104. Similar to the case of Fig. 3, since the position of G _c+1 is closer to the prediction given at time tc by the roadmodel-based predictor 210, the hybrid predictor 204 will assign the roadmodel-based predictor 210 a higher weight. While, if further considering the position of G _c+1 and the predictions and respectively given by the two predictors at time t _c-1, the prediction given by the free space predictor 208 is still on the dashed line 106, and the distance between the prediction and the actual position G _c+1 is farther than the distance between the prediction given at time tc and the actual position G _c+1. Meanwhile, the predictions and given at time t _c-1 and time tc by the roadmodel-based prediction model 210 approximately coincide with G _c+1 . Therefore, the hybrid predictor 204 is further confident in that the reliability of the roadmodel-based predictor 210 is higher at this moment, and thus the roadmodel-based predictor 210 will be given a greater weight.
Returning to Fig. 2, after the hybrid predictor 204 gives fused prediction values, such as the trajectory prediction module 206 can provide predicted trajectories based on the fused predicted values.
As an alternative embodiment, the hybrid predictor 204 may also optionally include a static object classifier 212. It is well known that current sensors are more or less unstable or inaccurate. For example, for an object that is stationary, the observation data of sensors for that object may jitter over time. Therefore, performing a movement trajectory prediction for observation data of a static object is a waste of resources and may also lead to erroneous trajectory prediction. To avoid that, the static object classifier 212 may be configured to classify an object as static or non-static by analysis of sensor data. As a non-limiting example, the static object classifier 212 may distinguish between static and non-static by for example setting thresholds for different types of sensor data, such as classifying an object as a static object when detecting that the speed of the object is below a certain predetermined threshold. In addition, the static object classifier 212 may also analyze the distribution of sensor data over time, and classify an object as static when the data exhibits a small fluctuation around a certain value (e.g., a velocity value of zero and an acceleration value of zero) . In the case that the static object classifier 212 classifies an object as a static object, the free space predictor 208 and the roadmodel-based predictor 210 will not perform any trajectory prediction on such static object. Instead, the hybrid predictor 204 outputs the current location of the static object as its prediction result, i.e., predicting that the static object will remain stationary. In the case that the static object classifier 212 classifies an object as non-static, it may optionally further smooth the observation data using a Kalman filter, and then feed it to the free space predictor 208 and the roadmodel- based predictor 210.
Fig. 5 is a flow diagram of a method 500 for trajectory prediction in accordance with one embodiment of the present invention. The method 500 begins at block 502, where sensor data for an object surrounding a vehicle may be obtained from at least one sensor installed on the vehicle. The obtained sensor data may be fed into a hybrid predictor, at block 504. The hybrid predictor of the present invention may consist of a plurality of predictors, each with a weight assigned by the hybrid predictor. As aforementioned, the plurality of predictors comprise at least a free space predictor, such as the free space predictor 208 in Fig. 2, and a roadmodel-based predictor, such as the roadmodel-based predictor 210.
Then, the method 500 proceeds to block 506, where each of the plurality of predictors may give its own predictions for a predetermined period of time from the current time instant. For example, the free space predictor 208 may give its predictions based on historical sensor data from the at least one sensor as previously described with regard to Fig. 3. Meanwhile, the roadmodel-based predictor 210 may also give its predictions mainly based on its knowledge of the road model, while it also uses the historical sensor data to determine the position of the object in the road model.
At block 508, the hybrid predictor 204 determines the reliability of each of the plurality of predictors. As an example, this can be determined by evaluating which of the predictors gave a better prediction that is closer to the actual movement of the object in previous one or more time instants, as previously described with regard to Figs 3 and 4.
Thereafter, at block 510, the hybrid predictor may output weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors, with greater weights being assigned to the more reliable predictors. In a preferred embodiment, outliers may be dropped from the weighted hybrid predictions. For example, an iterative multivariate Gaussian model is used to fuse all the predictions of different sources with various time horizons after dropping outliers iteratively. Then, at block 512, a predicted trajectory may be provided based on the weighted hybrid predictions. The method 500 ends.
In the optional case that the hybrid predictor 204 has a static object classifier 212, before each of the plurality of predictors giving its own predictions, the obtained sensor data may be first fed into the static object classifier 212 to classify the object into static or non-static. In response to that the static object classifier 212 classifies the object into static, the predicted trajectory can be directly provided based on the static state of the object, which means the predictions of the plurality of predictors are skipped. Otherwise, if the object is classified into non-static, the static object classifier 212 may optionally smooth the sensor data, such as by using a Kalman filter, before feeding it into the plurality of predictors.
Fig. 6 illustrates an exemplary vehicle 600 according to an embodiment of the present invention. The vehicle 600 may comprise at least one sensor 602 configured to capture sensing data for objects surrounding the vehicle, a trajectory prediction apparatus 604 for providing trajectory predictions, such as the trajectory prediction apparatus 200 in Fig. 2, and a decision module 606 configured to make vehicle control decisions based on the trajectories for objects surrounding the vehicle predicted by the trajectory prediction apparatus 604.
Fig. 7 illustrates a general hardware environment 700 wherein the present disclosure is applicable in accordance with an exemplary embodiment of the present disclosure.
With reference to Fig. 7, a computing device 700, which is an example of the hardware device that may be applied to the aspects of the present disclosure, will now be described. The computing device 700 may be any machine configured to perform processing and/or calculations, may be but is not limited to a work station, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant, a smart phone, an on-vehicle computer or any combination thereof. The aforementioned system may be wholly or at least partially implemented by the computing device 700 or a similar device or system.
The computing device 700 may comprise elements that are connected with or in communication with a bus 702, possibly via one or more interfaces. For example, the computing device 700 may comprise the bus 702, and one or more processors 704, one or more input devices 706 and one or more output devices 708. The one or more processors 704 may be any kinds of processors, and may comprise but are not limited to one or more general-purpose processors and/or one or more special-purpose processors (such as special processing chips) . The input devices 706 may be any kinds of devices that can input information to the computing device, and may comprise but are not limited to a mouse, a keyboard, a touch screen, a microphone and/or a remote control. The output devices 708 may be any kinds of devices that can present information, and may comprise but are not limited to display, a speaker, a video/audio output terminal, a vibrator and/or a printer. The computing device 700 may also comprise or be connected with non-transitory storage devices 710 which may be any storage devices that are non-transitory and can implement data stores, and may comprise but are not limited to a disk drive, an optical storage device, a solid-state storage, a floppy disk, a flexible disk, hard disk, a magnetic tape or any other magnetic medium, a compact disc or any other optical medium, a ROM (Read Only Memory) , a RAM (Random Access Memory) , a cache memory and/or any other memory chip or cartridge, and/or any other medium from which a computer may read data, instructions and/or code. The non-transitory storage devices 710 may be detachable from an interface. The non-transitory storage devices 710 may have data/instructions/code for implementing the methods and steps which are described above. The computing device 700 may also comprise a communication device 712. The communication device 712 may be any kinds of device or system that can enable communication with external apparatuses and/or with a network, and may comprise but are not limited to a modem, a network card, an infrared communication device, a wireless communication device and/or a chipset such as a Bluetooth ^TM device, 802.11 device, WiFi device, WiMax device, cellular communication facilities and/or the like.
When the computing device 700 is used as an on-vehicle device, it may also be connected to external device, for example, a GPS receiver, sensors for sensing different environmental data such as an acceleration sensor, a wheel speed sensor, a gyroscope and so on. In this way, the computing device 700 may, for example, receive location data and sensor data indicating the travelling situation of the vehicle. When the computing device 700 is used as an on-vehicle device, it may also be connected to other facilities (such as an engine system, a wiper, an anti-lock Braking System or the like) for controlling the traveling and operation of the vehicle.
In addition, the non-transitory storage device 710 may have map information and software elements so that the processor 704 may perform route guidance processing. In addition, the output device 706 may comprise a display for displaying the map, the location mark of the vehicle and also images indicating the travelling situation of the vehicle. The output device 706 may also comprise a speaker or interface with an ear phone for audio guidance.
The bus 702 may include but is not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus. Particularly, for an on-vehicle device, the bus 702 may also include a Controller Area Network (CAN) bus or other architectures designed for application on an automobile.
The computing device 700 may also comprise a working memory 714, which may be any kind of working memory that may store instructions and/or data useful for the working of the processor 704, and may comprise but is not limited to a random access memory and/or a read-only memory device.
Software elements may be located in the working memory 714, including but are not limited to an operating system 716, one or more application programs 718, drivers and/or other data and codes. Instructions for performing the methods and steps described in the above may be comprised in the one or more application programs 718, and the units of the aforementioned apparatus 800 may be implemented by the processor 704 reading and executing the instructions of the one or more application programs 718. The executable codes or source codes of the instructions of the software elements may be stored in a non-transitory computer-readable storage medium, such as the storage device (s) 710 described above, and may be read into the working memory 714 possibly with compilation and/or installation. The executable codes or source codes of the instructions of the software elements may also be downloaded from a remote location.
Those skilled in the art may clearly know from the above embodiments that the present disclosure may be implemented by software with necessary hardware, or by hardware, firmware and the like. Based on such understanding, the embodiments of the present disclosure may be embodied in part in a software form. The computer software may be stored in a readable storage medium such as a floppy disk, a hard disk, an optical disk or a flash memory of the computer. The computer software comprises a series of instructions to make the computer (e.g., a personal computer, a service station or a network terminal) execute the method or a part thereof according to respective embodiment of the present disclosure.
Reference has been made throughout this specification to “one example” or “an example” , meaning that a particular described feature, structure, or characteristic is included in at least one example. Thus, usage of such phrases may refer to more than just one example. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more examples.
One skilled in the relevant art may recognize, however, that the examples may be practiced without one or more of the specific details, or with other methods, resources, materials, etc. In other instances, well known structures, resources, or operations have not been shown or described in detail merely to observe obscuring aspects of the examples.
While sample examples and applications have been illustrated and described, it is to be understood that the examples are not limited to the precise configuration and resources described above. Various modifications, changes, and variations apparent to those skilled in the art may be made in the arrangement, operation, and details of the methods and systems disclosed herein without departing from the scope of the claimed examples.

Claims

A computer-implemented method for trajectory prediction, characterized in comprising:

obtaining sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle;

feeding the obtained sensor data into a hybrid predictor consisting of a plurality of predictors each with a weight assigned by the hybrid predictor, wherein the plurality of predictors comprise at least a free space predictor and a roadmodel-based predictor;

for a current time instant, each of the plurality of predictors giving its own predictions for a predetermined period of time from the current time instant based on historical sensor data from the at least one sensor;

the hybrid predictor determining a reliability of each one of the plurality of predictors;

the hybrid predictor outputting weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors and the reliability of each one of the plurality of predictors, with assigning a greater weight to the predictor that gave a better prediction in the previous one or more time instants; and

providing a predicted trajectory based on the weighted hybrid predictions.
The method according to claim 1, wherein the hybrid further comprises a static object classifier, and the method further comprises:

before each of the plurality of predictors giving its own predictions, feeding the obtained sensor data into the static object classifier to classify the object into static or non-static.
The method according to claim 2, further comprising:

in response to the static object classifier classifying the object into static, directly providing the predicted trajectory based on the static state of the object.
The method according to claim 2 or 3, further comprising:

in response to the static object classifier classifying the object into non-static, smoothing the sensor data before feeding it into the into the hybrid predictor.
The method according to claim 4, wherein smoothing the sensor data includes smoothing the sensor data using a Kalman filter.
The method according to any one of the preceding claims, wherein the free space predictor is an RNN-based predictor.
The method according to any one of the preceding claims, further comprising:

dropping outliers from the weighted hybrid predictions before providing the predicted trajectory.
The method according to any one of the preceding claims, wherein the object comprising any one of the following:

pedestrians,

scooters,

bicycles,

cars,

trucks, or

buses.
A trajectory prediction apparatus, characterized in comprising:

a sensor data obtaining module configured to obtain sensor data for an object surrounding a vehicle from at least one sensor installed on the vehicle;

a hybrid predictor consisting of a plurality of predictors each with a weight assigned by the hybrid predictor, wherein the plurality of predictors comprises at least a free space predictor and a roadmodel-based predictor, and the hybrid predictor is configured to:

receive the sensor data from the sensor data obtaining module;

for a current time instant, use thefree space predictor and the roadmodel-based predictor to each give its own predictions for a predetermined period of time from the current time instant based on historical sensor data from the at least one sensor,

determine which one of the plurality of predictors gave a better prediction that is closer to the actual movement of the object in previous one or more time instants; and

output weighted hybrid predictions for the predetermined period of time from the current time instant based on the respective predictions given by each of the plurality of predictors for the current time instant with assigning a greater weight to the one predictor that gave the better prediction in the previous one or more time instants; and

a trajectory predicting module configured to provide a predicted trajectory based on the weighted hybrid predictions.
The trajectory prediction apparatus according to claim 9, the hybrid predictor further comprising a static object classifier configured to classify the object into static or non-static, and wherein the trajectory predicting module configured to, in response to the static object classifier classifying the object into static, directly provide a predicted trajectory based on the static state of the object.
The trajectory prediction apparatus according to claim 10, the static object classifier is further configured to, in response to the static object classifier classifying the object into non-static, smooth the sensor data and feed it into the into the hybrid predictor.
The trajectory prediction apparatus according to any one of claims 9-11, wherein the free space predictor is an RNN-based predictor.
The trajectory prediction apparatus according to any one of claims 9-12, wherein the trajectory predicting module configured to drop outliers from the weighted hybrid predictions before providing the predicted trajectory.
The trajectory prediction apparatus according to any one of claims 9-13, wherein the object comprising any one of the following:

cars,

pedestrians,

bicycles, or

scooters.
A vehicle, characterized in comprising

at least one sensor configured to capture sensing data for objects surrounding the vehicle;

the trajectory prediction apparatus according to any one of claims 9-14; and

a decision module configured to make vehicle control decisions based on the trajectories for objects surrounding the vehicle predicted by the trajectory prediction apparatus.