CN115071704B

CN115071704B - Trajectory prediction method, apparatus, medium, device, chip and vehicle

Info

Publication number: CN115071704B
Application number: CN202210848278.9A
Authority: CN
Inventors: 赵燕顺
Original assignee: Xiaomi Automobile Technology Co Ltd
Current assignee: Xiaomi Automobile Technology Co Ltd
Priority date: 2022-07-19
Filing date: 2022-07-19
Publication date: 2022-11-11
Anticipated expiration: 2042-07-19
Also published as: CN115071704A

Abstract

The disclosure relates to a trajectory prediction method, a trajectory prediction device, a trajectory prediction medium, a trajectory prediction apparatus, a trajectory prediction chip, and a vehicle. The method comprises the following steps: the method comprises the steps of obtaining a target historical movement track of a target object, inputting the target historical movement track into a target track prediction model, and obtaining a target prediction track of the target object output by the target track prediction model within a first preset time period, wherein the target track prediction model is a track prediction model obtained by performing mutual supervision training on the undetermined track prediction model and an undetermined track backtracking model according to a first training sample set, and the first training sample set is expanded in the training process, so that the prediction accuracy of the target track prediction model obtained after training is improved, and the safety of an automatic driving vehicle is also improved.

Description

Trajectory prediction method, apparatus, medium, device, chip and vehicle

Technical Field

The present disclosure relates to the field of automatic driving technologies, and in particular, to a trajectory prediction method, apparatus, medium, device, chip, and vehicle.

Background

With the development of computer technology, the application of automatic driving is more and more. In order to improve the safety of an autonomous vehicle, it is necessary to detect obstacles around the vehicle and perform relevant vehicle control according to the positions of the obstacles. In an actual environment, both static obstacles (such as traffic signs, trees, roadblocks, and the like) and dynamic obstacles (such as motor vehicles, non-motor vehicles, pedestrians, and the like) exist, and in order to avoid collision with the dynamic obstacles, the moving track of the dynamic obstacles needs to be predicted. However, in the related art, there is a case where the trajectory prediction is inaccurate, affecting the safety of the autonomous vehicle.

Disclosure of Invention

To overcome the above problems in the related art, the present disclosure provides a trajectory prediction method, apparatus, medium, device, chip, and vehicle.

According to a first aspect of embodiments of the present disclosure, there is provided a trajectory prediction method, the method including:

acquiring a target historical movement track of a target object;

inputting the historical movement track of the target into a target track prediction model to obtain a target prediction track of the target object output by the target track prediction model within a first preset time period;

the target trajectory prediction model is to obtain a first historical trajectory output by a backtracking model of an undetermined trajectory by taking a sample predicted trajectory in a first training sample set as an input of the backtracking model of the undetermined trajectory, wherein the first training sample set comprises a sample historical trajectory and the sample predicted trajectory; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training a to-be-determined track prediction model according to the second training sample set to obtain a model;

the undetermined track backtracking model is a model obtained by taking the historical sample track in the first training sample set as the input of the undetermined track prediction model to obtain a first predicted track output by the undetermined track prediction model, taking the first predicted track as a new sample predicted track corresponding to the historical sample track, adding the new sample predicted track to the first training sample set to obtain a third training sample set, and training according to the third training sample set.

In some embodiments, the target trajectory prediction model is trained by:

obtaining a plurality of the first training sample sets;

determining the undetermined track prediction model and the undetermined track backtracking model;

circularly executing the model training step until the trained undetermined trajectory prediction model meets the target iteration stopping condition, and acquiring the target trajectory prediction model according to the trained undetermined trajectory prediction model;

wherein the model training step comprises:

inputting the sample predicted track into the backtracking model of the to-be-determined track to obtain the first historical track;

taking the first historical track as a new sample historical track corresponding to the sample prediction track, and adding the new sample historical track to the first training sample set to obtain a second training sample set;

and training the undetermined trajectory prediction model according to the second training sample set, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

In some embodiments, before the step of inputting the sample predicted trajectory into the undetermined trajectory backtracking model to obtain the first historical trajectory, the model training step further includes:

inputting the historical sample track into the prediction model of the undetermined track to obtain a first predicted track;

taking the first predicted track as a new sample predicted track corresponding to the sample historical track, and adding the new sample predicted track to the first training sample set to obtain a third training sample set;

and training the backtracking model of the undetermined track according to the third training sample set, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

In some embodiments, the training the backtracking model of the undetermined trajectory according to the third training sample set, and taking the trained backtracking model of the undetermined trajectory as a new backtracking model of the undetermined trajectory includes:

inputting the first predicted track into the backtracking model of the to-be-determined track to obtain one or more second historical tracks corresponding to the first predicted track;

calculating to obtain a first loss value of the second historical track and the sample historical track through a first loss function; wherein the first loss value is used for representing the difference degree between the second historical track and the sample historical track;

and updating parameters of the backtracking model of the undetermined track according to the first loss value to obtain a trained backtracking model of the undetermined track, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

In some embodiments, the training the undetermined trajectory prediction model according to the second training sample set, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model includes:

inputting the first historical track into the undetermined track prediction model to obtain one or more second predicted tracks corresponding to the first historical track;

calculating a second loss value of the second predicted track and the sample predicted track through a second loss function; wherein the second loss value is used for representing the difference degree between the second predicted track and the sample predicted track;

and under the condition that the undetermined trajectory prediction model does not meet the target iteration stopping condition according to the second loss value, updating parameters of the undetermined trajectory prediction model according to the second loss value to obtain a trained undetermined trajectory prediction model, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

In some embodiments, the determining the pending trajectory prediction model and the pending trajectory backtracking model comprises:

according to the first training sample set, pre-training a preset trajectory prediction model to obtain a to-be-determined trajectory prediction model;

and pre-training a preset track backtracking model according to the first training sample set to obtain a to-be-determined track backtracking model.

According to a second aspect of the embodiments of the present disclosure, there is provided a trajectory prediction apparatus, the apparatus including:

the track acquisition module is configured to acquire a target historical movement track of a target object;

a track prediction module configured to input the target historical movement track into a target track prediction model to obtain a target prediction track of the target object output by the target track prediction model within a first preset time period;

the target trajectory prediction model is to obtain a first historical trajectory output by a backtracking model of an undetermined trajectory by taking a sample predicted trajectory in a first training sample set as an input of the backtracking model of the undetermined trajectory, wherein the first training sample set comprises a sample historical trajectory and the sample predicted trajectory; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training a to-be-determined track prediction model according to the second training sample set to obtain a model; the undetermined track backtracking model is a model obtained by taking the historical sample track in the first training sample set as the input of the undetermined track prediction model to obtain a first predicted track output by the undetermined track prediction model, taking the first predicted track as a new sample predicted track corresponding to the historical sample track, adding the new sample predicted track to the first training sample set to obtain a third training sample set, and training according to the third training sample set.

In some embodiments, the apparatus further comprises a model training module configured to be trained by:

obtaining a plurality of the first training sample sets;

wherein the model training step comprises:

In some embodiments, the model training module is further configured to input the sample historical trajectory into the undetermined trajectory prediction model, resulting in a first predicted trajectory; taking the first predicted track as a new sample predicted track corresponding to the sample historical track, and adding the new sample predicted track to the first training sample set to obtain a third training sample set; and training the backtracking model of the undetermined track according to the third training sample set, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

In some embodiments, the model training module is configured to input the first predicted trajectory into the backtracking model to obtain one or more second historical trajectories corresponding to the first predicted trajectory; calculating to obtain a first loss value of the second historical track and the sample historical track through a first loss function; the first loss value is used for representing the difference degree of the second historical track and the sample historical track; and updating parameters of the to-be-determined track backtracking model according to the first loss value to obtain a trained to-be-determined track backtracking model, and taking the trained to-be-determined track backtracking model as a new to-be-determined track backtracking model.

In some embodiments, the model training module is configured to input the first historical trajectory into the undetermined trajectory prediction model, so as to obtain one or more second predicted trajectories corresponding to the first historical trajectory; calculating a second loss value of the second predicted track and the sample predicted track through a second loss function; wherein the second loss value is used to characterize a degree of difference between the second predicted trajectory and the sample predicted trajectory; and under the condition that the undetermined trajectory prediction model does not meet the target iteration stopping condition according to the second loss value, updating parameters of the undetermined trajectory prediction model according to the second loss value to obtain a trained undetermined trajectory prediction model, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

In some embodiments, the model training module is configured to pre-train a preset trajectory prediction model according to the first training sample set, so as to obtain a to-be-determined trajectory prediction model; and pre-training a preset track backtracking model according to the first training sample set to obtain a backtracking model of the track to be determined.

According to a third aspect of an embodiment of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of the trajectory prediction method provided by the first aspect of the present disclosure.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the steps of the trajectory prediction method provided by the first aspect of the present disclosure.

According to a fifth aspect of embodiments of the present disclosure, there is provided a chip comprising a processor and an interface; the processor is configured to read instructions to perform the steps of the trajectory prediction method provided by the first aspect of the present disclosure.

According to a sixth aspect of the embodiments of the present disclosure, there is provided a vehicle including the electronic apparatus provided in the third aspect of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects: the method comprises the steps of obtaining a target historical moving track of a target object, inputting the target historical moving track into a target track prediction model, and obtaining a target prediction track of the target object output by the target track prediction model within a first preset time period, wherein the target track prediction model is a track prediction model obtained by performing mutual supervision training on the undetermined track prediction model and the undetermined track backtracking model according to a first training sample set, and the first training sample set is expanded in the training process, so that the prediction accuracy of the target track prediction model obtained after training is improved, and the safety of an automatic driving vehicle is also improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart illustrating a method of trajectory prediction, according to an exemplary embodiment.

FIG. 2 is a flow diagram illustrating a method of training a target trajectory prediction model in accordance with an exemplary embodiment.

FIG. 3 is a block diagram illustrating a trajectory prediction device according to an exemplary embodiment.

FIG. 4 is a block diagram illustrating another trajectory prediction device, according to an example embodiment.

FIG. 5 is a block diagram illustrating an electronic device in accordance with an example embodiment.

FIG. 6 is a block diagram of a vehicle shown in accordance with an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the disclosure, as detailed in the appended claims.

It should be noted that all the actions of acquiring signals, information or data in the present disclosure are performed under the premise of complying with the corresponding data protection regulation policy of the country of the location and obtaining the authorization given by the owner of the corresponding device.

In the description of the present disclosure, terms such as "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. In addition, in the description with reference to the drawings, the same reference numerals in different drawings denote the same elements, but not otherwise described.

In the description of the present disclosure, unless otherwise indicated, "plurality" means two or more, and other terms are similar; "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c can be single or multiple; "and/or" is an association describing an associated object, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural.

Although operations may be depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain environments, multitasking and parallel processing may be advantageous.

First, an application scenario of the present disclosure will be explained. The present disclosure may be applied to an autonomous driving scenario. In order to improve the safety of an autonomous vehicle, it is necessary to detect obstacles around the vehicle and predict the movement locus of a dynamic obstacle so as to control the vehicle to travel according to the predicted locus. In the related art, a neural network model may be trained according to sample trajectory data, and trajectory prediction may be performed by the trained model. However, the acquisition and labeling of sample trajectory data are complex, and it is often difficult to acquire enough samples to train the model, which may cause the trained model to have the situation of inaccurate trajectory prediction, and affect the safety of the automatic driving vehicle.

In order to solve the problems, the disclosure provides a trajectory prediction method, a trajectory prediction device, a medium, equipment, a chip and a vehicle, mutual supervision training is performed according to a first training sample set based on an undetermined trajectory prediction model and an undetermined trajectory backtracking model, and the first training sample set is expanded in the training process, so that the prediction accuracy of a target trajectory prediction model obtained after training is improved. The accuracy of the trajectory prediction is improved, and the safety of the autonomous vehicle is also improved.

The disclosure is described below with reference to specific examples.

Fig. 1 illustrates a trajectory prediction method according to an exemplary embodiment, which may be applied to an electronic device, which may include a terminal device, such as a smart phone, a smart wearable device, a smart speaker, a smart tablet, a PDA (Personal Digital Assistant), a CPE (Customer Premise Equipment), a Personal computer, or the like; the electronic device may also include a server, such as a local server or a cloud server; the electronic device may also include a vehicle-mounted terminal device. As shown in fig. 1, the method may include:

s101, obtaining a target historical movement track of a target object.

The target object may be any obstacle around the target vehicle, and may be, for example, a moving obstacle within a certain distance range from the target vehicle, which is detected by a camera provided in the target vehicle; the obstacle may be a moving obstacle detected by a camera outside the target vehicle within a certain distance range from the target vehicle. The moving barrier may include a motor vehicle, a non-motor vehicle, a pedestrian, an animal, etc., which the present disclosure does not limit.

In some embodiments, the camera external to the target vehicle may comprise a road camera. The road camera can transmit the shot image to the electronic equipment, and the electronic equipment detects the target object according to the image.

In other embodiments, the camera outside the target vehicle may also include a camera of the target drone corresponding to the target vehicle. For example, the target vehicle may be configured with an unmanned aerial vehicle, the unmanned aerial vehicle travels along with the target vehicle, an environmental image with a larger distance is captured by a camera of the unmanned aerial vehicle, the unmanned aerial vehicle may transmit the environmental image to the electronic device through a wireless communication network, and the electronic device may detect the target object according to the environmental image.

In this step, a movement trajectory of the target object within a preset time range may be taken as the target historical movement trajectory, the preset time range may be any time range before the current time, and for example, a time within 10 seconds or 30 seconds before the current time may be taken as the preset time range.

S102, inputting the historical movement track of the target into a target track prediction model to obtain the target prediction track of the target object output by the target track prediction model in a first preset time period.

The target trajectory prediction model is to take a sample prediction trajectory in a first training sample set as the input of the backtracking model of the undetermined trajectory to obtain a first historical trajectory output by the backtracking model of the undetermined trajectory, wherein the first training sample set comprises a sample historical trajectory and a sample prediction trajectory; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training the to-be-determined track prediction model according to the second training sample set to obtain a model;

the undetermined trajectory backtracking model is a model obtained by taking a sample historical trajectory in a first training sample set as an input of an undetermined trajectory prediction model, obtaining a first predicted trajectory output by the undetermined trajectory prediction model, taking the first predicted trajectory as a new sample predicted trajectory corresponding to the sample historical trajectory, adding the new sample predicted trajectory to the first training sample set, obtaining a third training sample set, and training according to the third training sample set.

It should be noted that the undetermined trajectory prediction model may be configured to perform prediction according to an input historical trajectory, so as to obtain a predicted trajectory within a first preset time period corresponding to the historical trajectory, where the first preset time period may include a time period (e.g., 5 seconds or 10 seconds) after an end time corresponding to the historical trajectory; the backtracking model of the undetermined trajectory may be configured to perform backtracking according to an input predicted trajectory to obtain a historical trajectory within a second preset time period corresponding to the predicted trajectory, where the second preset time period may include a time period (e.g., 5 seconds or 10 seconds) before a start time corresponding to the predicted trajectory.

By adopting the method, the target historical movement track of the target object is obtained, the target historical movement track is input into the target track prediction model, and the target prediction track of the target object output by the target track prediction model in a first preset time period is obtained, wherein the target track prediction model is a track prediction model obtained by performing mutual supervision training on the undetermined track prediction model and the undetermined track backtracking model according to the first training sample set, and the first training sample set is expanded in the training process, so that the prediction accuracy of the target track prediction model obtained after training is improved, and the safety of the automatic driving vehicle is also improved.

FIG. 2 is a flowchart illustrating a method of training a target trajectory prediction model according to an exemplary embodiment, which may include, as shown in FIG. 2:

s201, obtaining a plurality of first training sample sets.

Wherein the first set of training samples may include sample historical trajectories and sample predicted trajectories. For example, the first training sample set may include a plurality of first training samples, each of which includes a sample history trajectory and a sample prediction trajectory corresponding to each other. For example, the actual movement trajectories of a plurality of moving objects in a period of time may be acquired, the actual movement trajectories may be used as a training sample, the actual movement trajectories may be divided from a specific time, the trajectories before the specific time may be used as a sample history trajectory, the trajectories after the specific time may be used as a sample predicted trajectory, and a combination of the sample history trajectory and the sample predicted trajectory may be used as a first training sample.

S202, determining a prediction model and a backtracking model of the undetermined track.

The undetermined trajectory prediction model and the undetermined trajectory backtracking model can be preset neural network models.

In some embodiments, the undetermined trajectory prediction model may be a pre-trained model, and for example, the pre-trained model may be pre-trained according to the first training sample set to obtain the undetermined trajectory prediction model.

Wherein, the preset track prediction model may be a neural network model. Illustratively, the preset trajectory prediction model may include an acrenet, a MapNet, and a PredNet.

Wherein, the ActorNet, mapNet and PredNet can be convolutional neural networks. The acternet may be a trajectory feature extraction network, and may be configured to obtain trajectory features of the historical trajectory, for example, the acternet may output the trajectory features according to the input historical trajectory; the MapNet may be a map feature extraction network, and may be configured to obtain map features of a high-precision map corresponding to the historical track, for example, the acrenet may output the map features according to the input high-precision map corresponding to the historical track; the map feature may include one or more of a road feature, a lane feature, and an environmental feature; the track characteristic and the map characteristic can be input into PredNet, the PredNet can be a prediction network, and a prediction track corresponding to the historical track can be obtained through prediction according to the track characteristic and the map characteristic.

In other embodiments, the to-be-determined trajectory backtracking model may be a pre-trained model, and for example, the pre-trained model may be pre-trained according to the first training sample set to obtain the to-be-determined trajectory backtracking model.

The preset trajectory backtracking model may be a neural network model. Illustratively, the preset trajectory backtracking model may include an ActorNet, mapNet, and PredNet.

It should be noted that the preset trajectory prediction model and the preset trajectory backtracking model may have the same or different model structures.

And S203, circularly executing the model training step until the trained undetermined trajectory prediction model meets the target iteration stopping condition, and acquiring the target trajectory prediction model according to the trained undetermined trajectory prediction model.

In some embodiments, the model training step may include the steps of:

s11, inputting the sample predicted track into a backtracking model of the to-be-determined track to obtain a first historical track.

Wherein, the first history track can be one or more. For example, the tracks may be predicted according to one sample, to obtain N first historical tracks, where N is any positive integer greater than or equal to 1, and for example, N may be 3 or 5.

In some embodiments, after the sample predicted trajectory is input into the undetermined trajectory backtracking model, the undetermined trajectory backtracking model may output W undetermined historical trajectories and a first confidence of each undetermined historical trajectory, the undetermined historical trajectories may be sorted from large to small according to the first confidence, and the N undetermined historical trajectories before being sorted are used as the first historical trajectories. Wherein N is any positive integer greater than or equal to 1, and W is any positive integer greater than or equal to N.

And S12, adding the first historical track serving as a new sample historical track corresponding to the sample prediction track to the first training sample set to obtain a second training sample set.

And S13, training the undetermined trajectory prediction model according to the second training sample set, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

Therefore, the sample historical track corresponding to each sample prediction track can be expanded through the backtracking model of the undetermined track, so that the cost for manually obtaining and marking sample data can be saved, the model prediction accuracy after training is improved, and the sample data obtaining efficiency is improved.

In other embodiments, the model training step may include the steps of:

s21, inputting the historical sample track into a prediction model of the undetermined track to obtain a first predicted track.

Wherein the first predicted trajectory may be one or more. Illustratively, M first predicted tracks may be obtained according to one sample historical track, where M is any positive integer greater than or equal to 1, and for example, M may be 3 or 5. It should be noted that M and N may be equal or unequal.

In some embodiments, after the sample historical trajectory is input into the undetermined trajectory prediction model, the undetermined trajectory prediction model may output V undetermined predicted trajectories and a second confidence coefficient of each undetermined predicted trajectory, the undetermined predicted trajectories may be sorted from large to small according to the second confidence coefficients, and M undetermined predicted trajectories that are sorted in the front are used as the first predicted trajectory. Wherein M is any positive integer greater than or equal to 1, and V is any positive integer greater than or equal to M.

And S22, adding the first predicted track serving as a new sample predicted track corresponding to the sample historical track to the first training sample set to obtain a third training sample set.

And S23, training the backtracking model of the undetermined track according to the third training sample set, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

And S24, inputting the sample predicted track into a backtracking model of the to-be-determined track to obtain a first historical track.

Likewise, the first history track may be one or more. Illustratively, the tracks may be predicted according to one sample, and N first history tracks are obtained, where N is any positive integer greater than or equal to 1, and for example, N may be 3 or 5.

And S25, adding the first historical track serving as a new sample historical track corresponding to the sample prediction track to the first training sample set to obtain a second training sample set.

And S26, training the undetermined trajectory prediction model according to the second training sample set, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

In some embodiments, the step S23 may train the backtracking model of the to-be-determined trajectory by:

firstly, inputting a first predicted track into a backtracking model of the track to be predicted to obtain one or more second historical tracks corresponding to the first predicted track.

And secondly, calculating to obtain a first loss value of the second historical track and the sample historical track through a first loss function.

The sample historical track may be a sample historical track for generating the first predicted track, and the first loss value is used to characterize a degree of difference between the second historical track and the sample historical track.

In some embodiments, the first loss function may comprise a categorical loss function, which may, for example, comprise the following equation (1):

（1）

wherein the content of the first and second substances,

representing a first loss value, N representing the number of second history tracks,

representing the historical trace of the sample to which the first predicted trace corresponds,

representing an ith second historical track generated from the first predicted track.

In other embodiments, the first Loss function may comprise an Exponential L2 Loss function, which may illustratively comprise the following equation (2):

（2）

wherein the content of the first and second substances,

representing a first loss value, N representing a number of second historical tracks,

representing the sample history track corresponding to the first predicted track,

representing an ith second historical track generated from the first predicted track, e representing a mathematical constant,

representing a preset hyper-parameter.

It should be noted that, the first loss function may also use a loss function commonly used in the related art, and the disclosure is not limited thereto.

And finally, updating parameters of the backtracking model of the undetermined track according to the first loss value to obtain a trained backtracking model of the undetermined track, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

It should be noted that, if the first loss value already satisfies the first iteration stopping condition, the above steps S21 to S23 may not be executed in the subsequent loop execution.

In the process of executing the model training step in a loop, if the first loss value satisfies the first iteration stop condition when the model training step is executed a +1 th time, the steps S21 to S23 are not executed any more. Therefore, when the first loss value meets the first iteration stopping condition, the to-be-determined trajectory backtracking model is proved to be trained completely, and the to-be-determined trajectory backtracking model cannot obtain better performance any more after continuous training, so that the to-be-determined trajectory backtracking model can be trained only in a to-be-determined trajectory prediction model instead of being trained in a subsequent cycle process.

Wherein the first stop iteration condition may include one or more of:

the first loss value is less than or equal to a first preset loss target value;

the first loss value does not converge for X consecutive rounds, where X is any positive integer greater than or equal to 2. For example, the first loss values obtained by the successive X rounds are all increasing, and the difference from the first preset loss target value is increasing, where X is any positive integer greater than or equal to 2. For another example, the first loss values obtained from successive X rounds are not reduced. For another example, the difference between the maximum value and the minimum value of the first loss value obtained in the successive X iterations is less than or equal to the first preset difference target.

In some embodiments, the step S26 may train the orbit prediction model to be determined by:

firstly, inputting a first historical track into a prediction model of the undetermined track to obtain one or more second predicted tracks corresponding to the first historical track.

And secondly, calculating a second loss value of the second predicted track and the sample predicted track through a second loss function.

Wherein the sample predicted trajectory may be a sample predicted trajectory that generated the first historical trajectory; the second loss value is used to characterize a degree of difference between the second predicted trajectory and the sample predicted trajectory.

It should be noted that the second loss function may also use a loss function commonly used in the related art, and the second loss function may be the same as or different from the first loss function, which is not limited in this disclosure.

And finally, under the condition that the undetermined trajectory prediction model does not meet the target iteration stop condition according to the second loss value, updating parameters of the undetermined trajectory prediction model according to the second loss value to obtain a trained undetermined trajectory prediction model, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

Wherein the target stop iteration condition may include one or more of:

the second loss value is less than or equal to a second preset loss target value;

the second loss value does not converge for successive Y rounds, where Y is any positive integer greater than or equal to 2. For example, the first loss values obtained by successive Y rounds are all increasing, and the difference between the first loss values and the first preset loss target value is increasing, where Y is any positive integer greater than or equal to 2. For another example, none of the first loss values obtained for successive Y rounds was reduced. For another example, the difference between the maximum value and the minimum value of the first loss value obtained in successive Y iterations is less than or equal to the first preset difference target.

Therefore, through the mode, mutual supervision training is carried out based on the undetermined track prediction model and the undetermined track backtracking model, the first training sample set is expanded in the training process, the diversity of sample distribution is improved, and therefore the prediction accuracy of the target track prediction model obtained after training is improved.

FIG. 3 is a block diagram illustrating a trajectory prediction apparatus 300 according to an exemplary embodiment, and as shown in FIG. 3, the apparatus 300 may include:

a track obtaining module 301 configured to obtain a target historical movement track of a target object;

a track prediction module 302, configured to input the target historical movement track into a target track prediction model, so as to obtain a target prediction track of the target object output by the target track prediction model within a first preset time period;

the target track prediction model is used for taking a sample prediction track in a first training sample set as an input of a backtracking model of the undetermined track to obtain a first historical track output by the backtracking model of the undetermined track, and the first training sample set comprises a sample historical track and the sample prediction track; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training a to-be-determined track prediction model according to the second training sample set to obtain a model; the undetermined track backtracking model is a model obtained by taking the historical sample track in the first training sample set as the input of the undetermined track prediction model to obtain a first predicted track output by the undetermined track prediction model, taking the first predicted track as a new sample predicted track corresponding to the historical sample track, adding the new sample predicted track to the first training sample set to obtain a third training sample set, and training according to the third training sample set.

Fig. 4 is a block diagram illustrating another trajectory prediction apparatus 300 according to an exemplary embodiment, and as shown in fig. 4, the apparatus 300 may further include a model training module 303, where the model training module 303 is configured to be trained by:

obtaining a plurality of the first training sample sets;

wherein the model training step comprises:

In some embodiments, the model training module 303 is further configured to input the sample historical trajectory into the undetermined trajectory prediction model to obtain a first predicted trajectory; taking the first predicted track as a new sample predicted track corresponding to the sample historical track, and adding the new sample predicted track to the first training sample set to obtain a third training sample set; and training the backtracking model of the undetermined track according to the third training sample set, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

In some embodiments, the model training module 303 is configured to input the first predicted trajectory into the backtracking model of the pending trajectory, so as to obtain one or more second historical trajectories corresponding to the first predicted trajectory; calculating to obtain a first loss value of the second historical track and the sample historical track through a first loss function; wherein the first loss value is used for representing the difference degree between the second historical track and the sample historical track; and updating parameters of the backtracking model of the undetermined track according to the first loss value to obtain a trained backtracking model of the undetermined track, and taking the trained backtracking model of the undetermined track as a new backtracking model of the undetermined track.

In some embodiments, the model training module 303 is configured to input the first historical trajectory into the undetermined trajectory prediction model, so as to obtain one or more second predicted trajectories corresponding to the first historical trajectory; calculating a second loss value of the second predicted track and the sample predicted track through a second loss function; wherein the second loss value is used to characterize a degree of difference between the second predicted trajectory and the sample predicted trajectory; and under the condition that the undetermined trajectory prediction model does not meet the target iteration stopping condition according to the second loss value, updating parameters of the undetermined trajectory prediction model according to the second loss value to obtain a trained undetermined trajectory prediction model, and taking the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model.

In some embodiments, the model training module 303 is configured to pre-train a preset trajectory prediction model according to the first training sample set, so as to obtain a to-be-determined trajectory prediction model; and pre-training a preset track backtracking model according to the first training sample set to obtain a to-be-determined track backtracking model.

With regard to the apparatus in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be described in detail here.

Fig. 5 is a block diagram of an electronic device 2000 shown in accordance with an example embodiment. The electronic device 2000 may be a terminal device, such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, a router, or the like; the electronic device 2000 may also be a server, such as a local server or a cloud server; the electronic device 2000 may also be a vehicle-mounted terminal device.

Referring to fig. 5, the electronic device 2000 may include one or more of the following components: a processing component 2002, a memory 2004, a power component 2006, a multimedia component 2008, an audio component 2010, an input/output (I/O) interface 2012, a sensor component 2014, and a communications component 2016.

The processing component 2002 may be used to control overall operation of the electronic device 2000, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 2002 may include one or more processors 2020 that execute instructions to perform all or a portion of the steps of the trajectory prediction method described above. Further, the processing component 2002 can include one or more modules that facilitate interaction between the processing component 2002 and other components. For example, the processing component 2002 may include a multimedia module to facilitate interaction between the multimedia component 2008 and the processing component 2002.

The memory 2004 is configured to store various types of data to support operations at the electronic device 2000. Examples of such data include instructions for any application or method operating on the electronic device 2000, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 2004 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power component 2006 provides power to the various components of the electronic device 2000. The power components 2006 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 2000.

The multimedia assembly 2008 includes a screen providing an output interface between the electronic device 2000 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 2008 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 2000 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

Audio component 2010 is configured to output and/or input audio signals. For example, audio component 2010 includes a Microphone (MIC) configured to receive external audio signals when electronic device 2000 is in an operational mode, such as a call mode, a record mode, and a voice recognition mode. The received audio signals may further be stored in the memory 2004 or transmitted via the communication component 2016. In some embodiments, audio assembly 2010 also includes a speaker for outputting audio signals.

The input/output interface 2012 provides an interface between the processing component 2002 and peripheral interface modules, which can be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

Sensor assembly 2014 includes one or more sensors for providing various aspects of status assessment for electronic device 2000. For example, sensor assembly 2014 may detect an open/closed state of electronic device 2000, a relative positioning of components, such as a display and a keypad of electronic device 2000, a change in position of electronic device 2000 or a component of electronic device 2000, a presence or absence of user contact with electronic device 2000, an orientation or acceleration/deceleration of electronic device 2000, and a change in temperature of electronic device 2000. The sensor assembly 2014 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 2014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 2014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 2016 is configured to facilitate wired or wireless communication between the electronic device 2000 and other devices. The electronic device 2000 may access a wireless network based on a communication standard, such as Wi-Fi,2G, 3G, 4G, 5G, 6G, NB-IOT, eMTC, etc., or a combination thereof. In an exemplary embodiment, the communication component 2016 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 2016 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 2000 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the trajectory prediction method described above.

The electronic device 2000 may be a stand-alone electronic device or a part of a stand-alone electronic device, for example, in an embodiment, the electronic device may be an Integrated Circuit (IC) or a chip, where the IC may be one IC or a set of multiple ICs; the chip may include, but is not limited to, the following categories: a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an SOC (System on Chip, SOC, system on Chip, or System on Chip), and the like. The integrated circuit or chip may be configured to execute executable instructions (or code) to implement the trajectory prediction method. Where the executable instructions may be stored in the integrated circuit or chip or may be retrieved from another device or apparatus, for example, where the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions may be stored in the processor, and when executed by the processor, implement the trajectory prediction method described above; alternatively, the integrated circuit or chip may receive executable instructions through the interface and transmit the instructions to the processor for execution, so as to implement the trajectory prediction method.

In an exemplary embodiment, the present disclosure also provides a computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the steps of the trajectory prediction method provided by the present disclosure. Illustratively, the computer-readable storage medium may be a non-transitory computer-readable storage medium comprising instructions, e.g., the memory 2004 comprising instructions executable by the processor 2020 of the electronic device 2000 to perform the trajectory prediction method described above. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-mentioned trajectory prediction method when executed by the programmable apparatus.

Fig. 6 is a block diagram illustrating a vehicle that may include the electronic device 2000 described above, as shown in fig. 6, according to an example embodiment. It should be noted that the vehicle may be any type of vehicle, such as a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, an amusement car, a train, etc., and the embodiment of the present disclosure is not particularly limited.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of trajectory prediction, the method comprising:

acquiring a target historical movement track of a target object;

inputting the target historical movement track into a target track prediction model to obtain a target prediction track of the target object output by the target track prediction model within a first preset time period;

the target track prediction model is used for taking a sample prediction track in a first training sample set as an input of a backtracking model of the undetermined track to obtain a first historical track output by the backtracking model of the undetermined track, and the first training sample set comprises a sample historical track and the sample prediction track; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training a to-be-determined track prediction model according to the second training sample set to obtain a model;

the undetermined trajectory backtracking model is a model obtained by taking the sample historical trajectory in the first training sample set as the input of the undetermined trajectory prediction model to obtain a first predicted trajectory output by the undetermined trajectory prediction model, taking the first predicted trajectory as a new sample predicted trajectory corresponding to the sample historical trajectory, adding the new sample predicted trajectory to the first training sample set to obtain a third training sample set, and training according to the third training sample set;

the target track prediction model is obtained by training in the following way:

obtaining a plurality of the first training sample sets;

determining the undetermined trajectory prediction model and the undetermined trajectory backtracking model;

wherein the model training step comprises:

2. The method of claim 1, wherein before inputting the sample predicted trajectory into the backtracking model of the pending trajectory to obtain a first historical trajectory, the model training step further comprises:

3. The method according to claim 2, wherein the training the backtracking model of the undetermined trajectory according to the third training sample set, and the taking the trained backtracking model of the undetermined trajectory as a new backtracking model of the undetermined trajectory comprises:

4. The method of claim 1, wherein the training the undetermined trajectory prediction model according to the second training sample set, and wherein the using the trained undetermined trajectory prediction model as a new undetermined trajectory prediction model comprises:

calculating a second loss value of the second predicted track and the sample predicted track through a second loss function; wherein the second loss value is used to characterize a degree of difference between the second predicted trajectory and the sample predicted trajectory;

5. The method of claim 1, wherein the determining the model for predicting the pending trajectory and the model for backtracking the pending trajectory comprises:

6. A trajectory prediction device, characterized in that the device comprises:

the target trajectory prediction model is to obtain a first historical trajectory output by a backtracking model of an undetermined trajectory by taking a sample predicted trajectory in a first training sample set as an input of the backtracking model of the undetermined trajectory, wherein the first training sample set comprises a sample historical trajectory and the sample predicted trajectory; taking the first historical track as a new sample historical track corresponding to the sample prediction track, adding the new sample historical track to the first training sample set to obtain a second training sample set, and training a to-be-determined track prediction model according to the second training sample set to obtain a model; the undetermined trajectory backtracking model is a model obtained by taking the sample historical trajectory in the first training sample set as the input of the undetermined trajectory prediction model to obtain a first predicted trajectory output by the undetermined trajectory prediction model, taking the first predicted trajectory as a new sample predicted trajectory corresponding to the sample historical trajectory, adding the new sample predicted trajectory to the first training sample set to obtain a third training sample set, and training according to the third training sample set;

the device further comprises a model training module, wherein the model training module is configured to train a target track prediction model by the following steps:

obtaining a plurality of the first training sample sets;

wherein the model training step comprises:

7. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: the steps of performing the method of any one of claims 1 to 5.

8. A computer readable storage medium having computer program instructions stored thereon which, when executed by a processor, implement the steps of the method of any one of claims 1 to 5.

9. A chip comprising a processor and an interface; the processor is configured to read instructions to perform the steps of the method of any one of claims 1 to 5.

10. A vehicle characterized in that the vehicle comprises the electronic device of claim 7.