CN113537258A

CN113537258A - Action track prediction method and device, computer readable medium and electronic equipment

Info

Publication number: CN113537258A
Application number: CN202010301174.7A
Authority: CN
Inventors: 董博
Original assignee: Beijing Jingdong Qianshi Technology Co Ltd
Current assignee: Beijing Jingdong Qianshi Technology Co Ltd
Priority date: 2020-04-16
Filing date: 2020-04-16
Publication date: 2021-10-22
Anticipated expiration: 2040-04-16
Also published as: CN113537258B

Abstract

The application provides a motion trail prediction method, a motion trail prediction device, a computer readable medium and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: calculating a first characteristic vector corresponding to an area occupied by an obstacle object in a current scene, a second characteristic vector corresponding to a historical track of the obstacle object and a third characteristic vector corresponding to a current state of the obstacle object; determining a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector and the third feature vector; constructing a relation model for representing the relation between an object to be predicted and an obstacle object according to the current scene and calculating a relation characteristic vector corresponding to the relation model; and predicting the motion trail of the object to be predicted according to the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene. The method can improve the accuracy of predicting the action track of the object to be predicted.

Description

Action track prediction method and device, computer readable medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method for predicting a trajectory of an action, an apparatus for predicting a trajectory of an action, a computer-readable medium, and an electronic device.

Background

In the field of intelligent travel, generally, the action trajectory of an object (such as a pedestrian, a vehicle, etc.) is predicted according to acquired road condition information, and a driving route which can avoid a certain degree of danger is generated according to the prediction result, and thus, the more accurate the prediction of the object, the higher the safety of the generated driving route. Generally, when a moving object is detected, the next action trajectory of the moving object may be predicted according to the movement trajectory of the object, but there may be a problem that the prediction accuracy is not high if the actual road condition generally has a large number of moving objects and the action trajectory prediction is performed on each object by the above method.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present application and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.

Disclosure of Invention

An object of the present application is to provide a method, an apparatus, a computer-readable medium, and an electronic device for predicting a trajectory of an object to be predicted, which can improve the accuracy of predicting the trajectory of the object to be predicted by combining information of a plurality of obstacle objects around the object to be predicted.

Other features and advantages of the present application will be apparent from the following detailed description, or may be learned by practice of the application.

A first aspect of the present application provides a method for predicting an action trajectory, which may include the steps of:

calculating a first characteristic vector corresponding to an area occupied by an obstacle object in a current scene, a second characteristic vector corresponding to a historical track of the obstacle object and a third characteristic vector corresponding to a current state of the obstacle object;

determining a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector and the third feature vector;

constructing a relation model for representing the relation between an object to be predicted and an obstacle object according to the current scene and calculating a relation characteristic vector corresponding to the relation model;

and predicting the motion trail of the object to be predicted according to the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene.

In an exemplary embodiment of the present application, before calculating a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a historical track of the obstacle object, and a third feature vector corresponding to a current state of the obstacle object, the method may further include:

generating a simplified diagram according to a live-action diagram corresponding to the current scene;

the area occupied by the obstacle object is determined from the simplified diagram.

In an exemplary embodiment of the present application, calculating a first feature vector corresponding to an area occupied by an obstacle object in a current scene includes:

extracting a graph feature vector corresponding to the simplified graph through a deep learning network;

and calculating a first eigenvector corresponding to an area occupied by the obstacle object according to the position information and the direction information of the obstacle object in the simplified diagram.

In an exemplary embodiment of the present application, determining a current feature vector corresponding to an object to be predicted in a current scene according to a first feature vector, a second feature vector, and a third feature vector includes:

and splicing the first feature vector, the second feature vector and the third feature vector, and determining a splicing result as a current feature vector corresponding to an object to be predicted in a current scene.

In an exemplary embodiment of the present application, a current scene includes at least one obstacle object, and a relationship model for characterizing a relationship between an object to be predicted and the obstacle object is constructed according to the current scene, including:

determining a first influence range of an obstacle object in a current scene and a second influence range of an object to be predicted;

screening a target object from the barrier objects according to the intersection of the first influence range and the second influence range;

and constructing a relation model for representing the relation between the target object and the object to be predicted.

In an exemplary embodiment of the present application, calculating a relationship feature vector corresponding to a relationship model includes:

determining a current feature vector corresponding to a target object;

and performing multi-layer feature extraction on the current feature vector corresponding to the target object to obtain a relation feature vector corresponding to the relation model.

In an exemplary embodiment of the present application, predicting a motion trajectory of an object to be predicted according to a relationship feature vector and a current feature vector corresponding to the object to be predicted in a current scene includes:

collecting a plurality of sample tracks, carrying out normalization processing on the sample tracks and clustering normalization results to obtain a plurality of track sets;

respectively determining target tracks from the plurality of track sets;

training a track prediction model according to the target track;

and inputting the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene into the trained track prediction model so that the track prediction model predicts the motion track of the object to be predicted.

According to a second aspect of the present application, there is provided a motion trajectory prediction device including a first feature vector calculation unit, a second feature vector calculation unit, a third feature vector calculation unit, and a motion trajectory prediction unit, wherein:

the first feature vector calculation unit is used for calculating a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a historical track of the obstacle object and a third feature vector corresponding to a current state of the obstacle object;

the second feature vector calculation unit is used for determining a current feature vector corresponding to an object to be predicted in the current scene according to the first feature vector, the second feature vector and the third feature vector;

the third feature vector calculation unit is used for constructing a relation model for representing the relation between the object to be predicted and the obstacle object according to the current scene and calculating a relation feature vector corresponding to the relation model;

and the motion trail prediction unit is used for predicting the motion trail of the object to be predicted according to the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene.

In an exemplary embodiment of the present application, the apparatus may further include a thumbnail image generation unit and an occupied area determination unit:

the simple diagram generating unit is used for generating a simple diagram according to a real diagram corresponding to the current scene before the first characteristic vector calculating unit calculates a first characteristic vector corresponding to an area occupied by the obstacle object, a second characteristic vector corresponding to a historical track of the obstacle object and a third characteristic vector corresponding to the current state of the obstacle object in the current scene;

and an occupied area determining unit for determining the occupied area of the obstacle object from the simplified diagram.

In an exemplary embodiment of the present application, the calculating a first eigenvector corresponding to an area occupied by an obstacle object in a current scene by a first eigenvector calculating unit includes:

the first feature vector calculation unit extracts a graph feature vector corresponding to the simplified graph through a deep learning network;

the first eigenvector calculating unit calculates a first eigenvector corresponding to an area occupied by the obstacle object according to the position information and the direction information of the obstacle object in the simplified diagram.

In an exemplary embodiment of the present application, the determining, by the second feature vector calculating unit, a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector, and the third feature vector includes:

and the second feature vector calculation unit splices the first feature vector, the second feature vector and the third feature vector, and determines a splicing result as a current feature vector corresponding to an object to be predicted in the current scene.

In an exemplary embodiment of the present application, the current scene includes at least one obstacle object, and the third feature vector calculation unit constructs a relationship model for characterizing a relationship between the object to be predicted and the obstacle object according to the current scene, including:

the third feature vector calculation unit determines a first influence range of an obstacle object in the current scene and a second influence range of an object to be predicted;

the third feature vector calculation unit screens the obstacle objects according to the intersection of the first influence range and the second influence range to obtain target objects;

and the third feature vector calculation unit constructs a relation model for representing the relation between the target object and the object to be predicted.

In an exemplary embodiment of the present application, the third feature vector calculation unit calculates a relationship feature vector corresponding to the relationship model, including:

the third feature vector calculation unit determines a current feature vector corresponding to the target object;

and the third feature vector calculation unit performs multi-layer feature extraction on the current feature vector corresponding to the target object to obtain a relation feature vector corresponding to the relation model.

In an exemplary embodiment of the present application, the predicting a motion trajectory of an object to be predicted by a motion trajectory predicting unit according to a relationship feature vector and a current feature vector corresponding to the object to be predicted in a current scene includes:

the motion track prediction unit acquires a plurality of sample tracks, normalizes the sample tracks and clusters normalization results to obtain a plurality of track sets;

a motion trajectory prediction unit determines target trajectories from a plurality of trajectory sets respectively;

the motion track prediction unit trains a track prediction model according to the target track;

and the motion track prediction unit inputs the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene into the trained track prediction model so that the track prediction model predicts the motion track of the object to be predicted.

According to a third aspect of the present application, there is provided a computer-readable medium, on which a computer program is stored, which program, when executed by a processor, implements the method of action trajectory prediction as described in the first aspect of the embodiments above.

According to a fourth aspect of the present application, there is provided an electronic device comprising: one or more processors; a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement the method of action trajectory prediction as described in the first aspect of the embodiments above.

The technical scheme provided by the application can comprise the following beneficial effects:

in the technical scheme provided by the embodiment of the application, a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a historical track of the obstacle object, and a third feature vector corresponding to a current state of the obstacle object can be calculated. Furthermore, the current feature vector corresponding to the object to be predicted in the current scene can be determined according to the first feature vector, the second feature vector and the third feature vector. Furthermore, a relation model for representing the relation between the object to be predicted and the obstacle object can be constructed according to the current scene, and a relation characteristic vector corresponding to the relation model is calculated. Furthermore, the motion trail of the object to be predicted can be predicted according to the relation feature vector and the current feature vector corresponding to the object to be predicted in the current scene. According to the technical description, on one hand, the accuracy rate of the action track prediction of the object to be predicted can be improved by combining the information of a plurality of obstacle objects around the object to be predicted; on the other hand, by constructing the relational model, the barrier objects interacting with the objects to be predicted can be integrated, so that the analysis efficiency of the barrier objects is improved, and the waste of computing resources is reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:

FIG. 1 shows a flow diagram of a method of action trajectory prediction according to an exemplary embodiment of the present application;

FIG. 2 shows a schematic diagram of a schematic diagram according to an exemplary embodiment of the present application;

FIG. 3 shows a schematic view of the impact range of an obstacle object according to an exemplary embodiment of the present application;

FIG. 4 illustrates a schematic view of the impact ranges of objects in a current scene according to an exemplary embodiment of the present application;

FIG. 5 illustrates a relational model diagram according to an exemplary embodiment of the present application;

FIG. 6 is a diagram illustrating normalization results from normalizing a plurality of sample traces according to an exemplary embodiment of the present application;

FIG. 7 illustrates a schematic diagram of target trajectories respectively determined from a plurality of trajectory sets according to an exemplary embodiment of the present application;

FIG. 8 is a block diagram illustrating an architecture of a method for predicting an action trajectory according to an exemplary embodiment of the present application;

FIG. 9 is a block diagram illustrating an action trajectory prediction apparatus according to an exemplary embodiment of the present application;

FIG. 10 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an exemplary embodiment of the present application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

Referring to fig. 1, fig. 1 is a flowchart illustrating an action track prediction method according to an exemplary embodiment of the present application, where the action track prediction method may be implemented by a server or a terminal device.

As shown in fig. 1, a method for predicting an action trajectory according to an embodiment of the present application includes steps S110 to S140, where:

step S110: and calculating a first characteristic vector corresponding to an area occupied by the obstacle object in the current scene, a second characteristic vector corresponding to the historical track of the obstacle object and a third characteristic vector corresponding to the current state of the obstacle object.

Step S120: and determining a current feature vector corresponding to the object to be predicted in the current scene according to the first feature vector, the second feature vector and the third feature vector.

Step S130: and constructing a relation model for representing the relation between the object to be predicted and the obstacle object according to the current scene and calculating a relation characteristic vector corresponding to the relation model.

Step S140: and predicting the motion trail of the object to be predicted according to the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene.

By implementing the action trajectory prediction method shown in fig. 1, the accuracy of action trajectory prediction for the object to be predicted can be improved by combining the information of a plurality of obstacle objects around the object to be predicted. In addition, by constructing the relational model, the barrier objects interacting with the objects to be predicted can be integrated, so that the analysis efficiency of the barrier objects is improved, and the waste of computing resources is reduced.

The following describes the steps in detail:

in step S110, a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a history track of the obstacle object, and a third feature vector corresponding to a current state of the obstacle object are calculated.

The current scene may be a road condition scene, the current scene may include one or more objects, and if the current scene includes a plurality of objects, the plurality of objects include an object to be predicted and one or more obstacle objects. In addition, the area occupied by the obstacle object may be an area covered by the obstacle object in the current scene, the historical track of the obstacle object may be drawn from the position of the obstacle object at each historical time, the current state of the obstacle object is used to describe the motion of the obstacle object in the current scene, and the current state may include the type, length, width, height, action direction, speed, acceleration, angular speed, and the like of the obstacle object; the obstacle object can be a pedestrian, a motor vehicle, a bicycle, an electric vehicle or the like. In addition, the current scene also includes an object to be predicted, and the object to be predicted can also be a pedestrian, a motor vehicle, a bicycle, an electric vehicle or the like, and the embodiment of the application is not limited.

In this embodiment of the application, optionally, before calculating a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a history track of the obstacle object, and a third feature vector corresponding to a current state of the obstacle object, the method may further include the following steps:

The live-action picture can be a shot road condition poetry live-action, and the simplified picture can be an image which simplifies the live-action picture through lines, marks and the like. The schematic diagram may include an object to be predicted, an obstacle object, a road path, a bicycle lane, a pedestrian crossing, a road edge, a road middle line, and the like, which is not limited in the embodiment of the present application. Referring to fig. 2, fig. 2 shows a schematic diagram of a schematic diagram according to an exemplary embodiment of the present application. As shown in fig. 2, the obstacle object is represented by a rectangle, the object to be predicted is represented by a square, the road path is represented by a curve, the bicycle lane and the pedestrian crossing are represented by a long rectangular area perpendicular to the road on which they are located, the road edge is represented by an 1/4 circular curve passing through the vertices in fig. 2, and the road middle line is represented by a broken line. For example, in fig. 2, the object to be predicted (i.e., a square in fig. 2) may be a pedestrian, and the obstacle object (i.e., a rectangle in fig. 2) may be a motor vehicle.

In addition, optionally, the manner of generating the thumbnail according to the live-action view corresponding to the current scene may specifically be: performing edge tracing on the live-action image according to the adjacent pixel difference of the live-action image corresponding to the current scene, and further generating a simplified image according to an edge tracing result; the sketch map can represent the object to be predicted and the obstacle object and reduce live-action details which are not needed for prediction, so that the obstacle object can be analyzed, the action track of the object to be predicted can be predicted according to the analysis result, the influence of the obstacle object or the image detail content (such as the illumination effect) of the object to be predicted in the live-action map on the prediction result can be reduced, and the prediction efficiency is improved.

In addition, optionally, the manner of determining the area occupied by the obstacle object from the simplified diagram may specifically be: and detecting the coverage area of the obstacle object in the sketch map on the road surface, and determining the area corresponding to the coverage area as the area occupied by the obstacle object.

Therefore, by implementing the optional embodiment, the live-action diagram can be simplified, and a simplified diagram is obtained, so that the occupation of computing resources can be reduced, and the track prediction efficiency can be improved.

Further, calculating a first feature vector corresponding to an area occupied by an obstacle object in the current scene, including:

The deep learning network may be VGG, Resnet, MobilenetV2, or the like, and the embodiment of the present application is not limited thereto, and the deep learning network may be obtained by training images in a visual database. Specifically, the way of extracting the graph feature vector corresponding to the sketch map through the deep learning network may specifically be: performing multilayer convolution processing (for example, 3-layer convolution processing) on the simplified diagram through convolution checking in the deep learning network to obtain a first reference characteristic vector, and inputting the first reference characteristic vector into the full-connection layer so that the full-connection layer classifies the first reference characteristic vector to obtain and output a graph characteristic vector; the size of the convolution kernel may be 3 × 3, the convolution result output from the previous layer may be used as the input of the next layer, and the number of fully connected layers may be one or more layers (e.g., 3 layers), which is not limited in the embodiments of the present application. In the deep learning network, a pooling layer is further included between the convolutional layers, and the pooling layer is used for maximally pooling the output of the convolutional layer in the previous layer and taking the maximum pooling result as the input of the convolutional layer in the next layer.

Further, the way of calculating the first eigenvector corresponding to the area occupied by the obstacle object according to the position information and the direction information of the obstacle object in the simplified diagram may specifically be: acquiring position information of an obstacle object in the simplified diagram and acquiring direction information of the obstacle object in the simplified diagram according to a track of the obstacle object at the previous moment; extracting a second reference characteristic vector corresponding to an area occupied by the obstacle object from the image characteristic vector through the position information and the direction information; inputting the second reference feature vector into a Multilayer Perceptron (MLP) so that the MLP performs feature calculation through a plurality of hidden layers, and further outputting a first feature vector corresponding to an area occupied by the obstacle object; the position information is used for describing the position of the obstacle object in the sketch map, and the position information may include coordinates, longitude and latitude, and the like, which is not limited in the embodiment of the application; the direction information is used to describe the action direction of the obstacle object in the current scene. Among them, MLP may also be called Artificial Neural Network (ANN).

Therefore, by implementing the optional embodiment, the feature of the region corresponding to the obstacle object can be determined from the image feature vector, so that the feature vector corresponding to the obstacle object in the simple diagram can be obtained through calculation, the calculation efficiency of the feature vector is improved, and the prediction efficiency of the action track of the object to be predicted is improved.

In this embodiment of the application, optionally, the manner of calculating the second feature vector corresponding to the historical trajectory of the obstacle object in the current scene may specifically be: intercepting historical tracks corresponding to the intercepting time periods (such as 12: 00-13: 00 in 1 month and 1 day of 2020); calculating a second feature vector of the historical track corresponding to the interception time period through a Long Short-Term Memory network (LSTM); LSTM typically includes, among other things, a forgetting gate, an input gate, and an output gate. Specifically, the way of calculating the second feature vector of the history track corresponding to the truncation period by the LSTM may specifically be:

the method comprises the following steps: calculating candidate state information, input weight of the candidate state information, forgetting weight of target state information at the previous moment and output weight of the target state information at the current moment according to the historical track at the current moment and the implicit state information at the previous moment; the intercepting time period comprises the current time, and each time corresponds to a time window in the intercepting time period.

Specifically, the forgetting gate is used to determine how much information is discarded from the target state information at the previous time, so the forgetting weight is used to indicate a weight (i.e., a weight that can be retained) by which the target state information at the previous time is not forgotten; the forgetting weight may be substantially a weight matrix. Illustratively, the history track of the time window at the current time and the implicit state information at the previous time can be encoded through an activation function for representing the forgetting gate, and mapped to a value between 0 and 1, so as to obtain the forgetting weight of the target state information at the previous time. Where 0 indicates complete discard and 1 indicates complete retention. For example, the forgetting weight f of the target state information at the previous time can be calculated according to the following formula_t：f_t＝σ(W_f·[h_t-1，S_t]+b_f). Wherein h is_t-1Indicating the implicit status information of the last moment, S_tRepresenting the historical track of the time window of the current time, sigma representing the Sigmod function of the activation function, W_fAnd b_fParameter representing Sigmod function in forgetting gate, [ h ]_t-1，S_t]Denotes a reaction of_t-1And S_tAnd (4) combining.

The input gate is used to determine how much information is important and needs to be retained in the currently input historical track. For example, the historical track of the time window at the current moment and the implicit state information at the previous moment can be encoded through an activation function for representing an input gate, so as to obtain candidate state information and an input weight of the candidate state information; the input weight of the candidate state information is used for determining how much new information in the candidate state information can be added into the target state information. For example, the waiting time can be calculated according to the following formulaSelect status information

Wherein, tanh represents that the activation function is a positive-curve double-cut function, W_cAnd b_cRepresenting the parameters of the tanh function in the input gate. And the input weight i of the candidate state information can be calculated according to the following formula_t：i_t＝σ(W_i·[h_t-1，S_t]+b_i). Where σ denotes the activation function Sigmod function, W_iAnd b_iRepresenting the parameters of the Sigmod function in the input gate.

The output gate is used to determine which information should be included in the implicit state information output to the next time window. For example, the historical track of the time window at the current time and the implicit state information at the previous time may be encoded by an activation function for characterizing the output gate, so as to obtain the output weight of the target state information at the current time. For example, the candidate state information o can be calculated according to the following formula_t：o_t＝σ(W_o·[h_t-1，S_t]+b_o)

Where σ denotes the activation function Sigmod function, W_oAnd b_oRepresenting the parameters of the Sigmod function in the output gate.

Step two: and reserving the target state information at the previous moment according to the forgetting weight to obtain first intermediate state information. For example, the obtained first intermediate state information may be

Wherein, C_t-1Indicating last conversion process target state information.

Step three: and reserving the candidate state information according to the input weight of the candidate state information to obtain second intermediate state information. For example, the obtained second intermediate state information may be

Step four: and obtaining the target state information in the current moment according to the first intermediate state information and the second intermediate state information. For example, the target status information at the current time may be

Step five: and reserving the target state information at the current moment according to the output weight of the target state information at the current moment to obtain the hidden state information at the current moment. For example, the implicit status information of the current time may be

Step six: and when executing a time window corresponding to the last time in the interception time period, determining the implicit state information output by the time window as a second feature vector corresponding to the historical track of the interception time period.

In this embodiment of the application, optionally, the manner of calculating the third feature vector corresponding to the current state of the obstacle object in the current scene may specifically be: and inputting the current state of the barrier object into the MLP, so that the MLP performs feature calculation through a plurality of hidden layers, and further outputting a third feature vector corresponding to the current state of the barrier object.

In step S120, a current feature vector corresponding to an object to be predicted in the current scene is determined according to the first feature vector, the second feature vector, and the third feature vector.

The first feature vector, the second feature vector, and the third feature vector may have the same dimension or different dimensions, and the embodiment of the present application is not limited thereto.

In this embodiment of the present application, optionally, determining a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector, and the third feature vector includes:

The splicing method of the first feature vector, the second feature vector and the third feature vector may specifically be: and connecting the first feature vector, the second feature vector and the third feature vector in series. For example, if the dimensions corresponding to the first feature vector, the second feature vector, and the third feature vector are all 1 × 3, the dimension of the current feature vector corresponding to the object to be predicted in the current scene obtained by splicing may be 3 × 3.

Therefore, by implementing the optional embodiment, the interaction relation and the game relation between the barrier objects can be extracted through the fusion of various information, so that the determination of the current feature vector corresponding to the object to be predicted in the current scene is facilitated, and the prediction accuracy of the action track of the object to be predicted is improved.

In step S130, a relationship model for representing the relationship between the object to be predicted and the obstacle object is constructed according to the current scene, and a relationship feature vector corresponding to the relationship model is calculated.

In this embodiment of the present application, optionally, a current scene includes at least one obstacle object, and a relationship model for characterizing a relationship between an object to be predicted and the obstacle object is constructed according to the current scene, where the relationship model includes:

If the current scene includes a plurality of obstacle objects, radii of influence ranges corresponding to the plurality of obstacle objects may be the same or different, and the embodiment of the present application is not limited. The intersection of the first influence range and the second influence range may be identified by an irregular area, and the radius of the first influence range may be the same as or different from the radius of the second influence range. Further, the number of target objects is equal to or less than the number of obstacle objects. The relational model may be constructed by nodes and connecting edges for connecting two nodes.

The influence range is used to define the warning area of the obstacle object, and the influence range may be identified by a circular area corresponding to a radius r, where r is speed t + λ Volume, where speed is used to indicate the speed of the obstacle object, t is the time length, λ is a constant coefficient, and Volume is used to indicate the Volume of the obstacle object. Referring to fig. 3, fig. 3 is a schematic diagram illustrating an influence range of an obstacle object according to an exemplary embodiment of the present application. As shown in fig. 3, the obstacle object is at the center of the influence range, which is represented by a circular area with radius r.

In addition, optionally, the manner of determining the first influence range of the obstacle object and the second influence range of the object to be predicted in the current scene may specifically be: determining the radius corresponding to each barrier object in the current scene and the radius corresponding to the object to be predicted; determining a first influence range of each obstacle object according to the radius corresponding to each obstacle; and determining a second influence range of the object to be predicted according to the radius corresponding to the object to be predicted.

In addition, optionally, the manner of obtaining the target object from the obstacle object by screening according to the intersection of the first influence range and the second influence range may specifically be: screening out the obstacle object corresponding to the first influence range without intersection with the second influence range to obtain a target object with intersection with the second influence range; the number of the target objects may be one or more, and the embodiments of the present application are not limited.

Referring to fig. 4, fig. 4 is a schematic diagram illustrating an influence range of each object in a current scene according to an exemplary embodiment of the present application. Specifically, the obstacle object may be represented by a rectangle, and the object to be predicted may be represented by a square, and in fig. 4, there are a plurality of obstacle objects, and each obstacle object has a first influence range and an action direction corresponding thereto, the first influence range being represented by a circular area. Referring to fig. 4, the number of the first influence ranges intersecting with the second influence range of the object to be predicted is 3, so that the number of the target objects is 3, and further, the relationship model shown in fig. 5 can be obtained. Further, referring to fig. 5, fig. 5 shows a relationship model diagram according to an exemplary embodiment of the present application. Fig. 5 is generated according to a target object having an intersection with the second influence range of the object to be predicted, in fig. 5, the object to be predicted and the target object are both represented by nodes, and the nodes having a corresponding relationship are connected by connecting edges.

Therefore, by implementing the optional embodiment, the relation model can be determined according to the intersection of the influence range of the obstacle object and the influence range of the object to be predicted, so that the analysis of the obstacle object which has little influence on the object to be predicted can be reduced, and the prediction efficiency and the prediction accuracy are further improved.

Further, calculating a relationship feature vector corresponding to the relationship model, including:

determining a current feature vector corresponding to a target object;

The method for determining the current feature vector corresponding to the target object may specifically be: determining the target object as an object to be predicted and executing the steps S110 to S120; further, v (i) may represent a current feature vector of each target object, i being a positive integer for representing the target object. Further, the manner of performing multi-layer feature extraction on the current feature vector corresponding to the target object to obtain the relationship feature vector corresponding to the relationship model may specifically be: by expression e_ij＝a(W_gatV(i),W_gatV(j))；a_ij＝softmax(e_ij)；P^ι(i)＝∑_j∈Na_ijW_gatV (j) performing multi-layer feature extraction on V (i), and extracting P^LDetermining a relation feature vector corresponding to a relation model (GAT); wherein, W_gatIs a linear transformation weight matrix, V (j) is used to represent the target object adjacent to V (i)Front feature vector, P^ιIs the output result of the third layer, layer (L) is used to represent the last layer in the multi-layer feature extraction.

Therefore, by implementing the optional embodiment, the relation feature vector corresponding to the relation model can be obtained through calculation, and the object to be predicted and the obstacle object can be fused, so that the action track prediction efficiency of the object to be predicted is improved.

In step S140, the motion trajectory of the object to be predicted is predicted according to the relationship feature vector and the current feature vector corresponding to the object to be predicted in the current scene.

Wherein the motion trajectory can be represented by a curve.

In this embodiment of the present application, optionally, predicting the motion trajectory of the object to be predicted according to the relationship feature vector and the current feature vector corresponding to the object to be predicted in the current scene includes:

respectively determining target tracks from the plurality of track sets;

training a track prediction model according to the target track;

The method for acquiring the plurality of sample tracks, performing normalization processing on the plurality of sample tracks and clustering the normalization result to obtain the plurality of track sets specifically comprises the following steps:

a historical trajectory of the sample obstacle object is collected from the database as a sample trajectory.

Further, a rotational translation operation may be performed on the plurality of sample tracks so that the plurality of sample tracks have the same starting point and are in the same coordinate system; referring to fig. 6, fig. 6 is a diagram illustrating a normalization result obtained by normalizing a plurality of sample traces according to an exemplary embodiment of the present application. In the coordinate system shown in fig. 6, it is shown that the plurality of sample tracks have the same starting point after being subjected to rotational translation.

Furthermore, a preset number of original sample tracks can be selected as initial clustering centers, an unclustered sample track is selected as a current sample track, and the distance between the current sample track and each current cluster is calculated; specifically, the number of clusters may be determined first, and the number of clusters may be determined empirically, and the most suitable number of clusters may be finally determined through further iterative tests. For example, if there are cluster a0, cluster B0, cluster C0, and cluster D0, for cluster a0, the original sample trajectory initially selected is denoted as a 1; for the cluster B0, the original sample track initially selected is marked as B1; for the clustering cluster C0, the original sample trajectory initially selected is marked as C1; for the cluster D0, the original sample trajectory initially selected is marked as D1; the initial selection may be manual selection, or random selection or other selection, which is not particularly limited in this exemplary embodiment. Suppose the number of sample tracks in the current cluster A0 is o, the number of sample tracks in the cluster B0 is p, the number of sample tracks in the cluster C0 is k, and the number of sample tracks in the cluster D0 is m. In each cluster, each sample trajectory is represented as an n-dimensional vector. The generalizations of cluster a0, cluster B0, cluster C0, cluster D0 are thus given below; wherein N represents dimension, R^NThe representation is an N-dimensional vector space: a. the₀＝{a₁,a₂,...,a_o}a_i∈R^N(i＝1,2,...,o)；B₀＝{b₁,b₂,...,b_p}b_i∈R^N(i＝1,2,...,p)；C₀＝{c₁,c₂,...,c_k}c_i∈4^N(i＝1,2,...,k)；D₀＝{d₁,d₂,...,d_m}d_i∈R^N(i ═ 1, 2.., m). After generalized representations of cluster A0, cluster B0, cluster C0 and cluster D0 are obtained, cluster centers mu of cluster A0, cluster B0, cluster C0 and cluster D0 are obtained_a、μ_b、μ_c、μ_dCan be calculated by the following formula:

that is, in the present exemplary embodiment, the cluster center of the cluster is calculated by calculating the average value of the feature vectors of all sample tracks in the cluster, and the resulting μ_a、μ_b、μ_c、μ_dAre all n-dimensional vectors. However, in other exemplary embodiments of the present disclosure, the cluster center of the cluster may be calculated in other manners, which is not limited in this exemplary embodiment. After the cluster center of each cluster is obtained through calculation, for the current sample track, the feature vector of the current sample track and the cluster centers μ of the cluster A0, the cluster B0, the cluster C0 and the cluster D0 can be calculated_a、μ_b、μ_c、μ_dDis _ a, Dis _ b, Dis _ c, Dis _ d. For example: dis _ a | | | N- μ_a||²；Dis_b＝||N-μ_b||²；Dis_c＝||N-μ_c||²；Dis_d＝||N-μ_d||². Wherein | X-Y | is the root number of the sum of squares of the components after the vector is differenced. Note that, in the present exemplary embodiment, the euclidean distance is calculated, but in other exemplary embodiments of the present disclosure, a mahalanobis distance, a cosine distance, a manhattan distance, or the like may also be calculated; these too are within the scope of the present disclosure.

Further, the manner of determining the target trajectory from the plurality of trajectory sets may specifically be: and distributing the current sample track to the nearest clustering center, recalculating the clustering center after distribution, and determining the track corresponding to the recalculated clustering center as the target track. And if the distance between the current sample track and the clustering center of a cluster is minimum, the current sample track is distributed to the cluster. For example, for the current sample trajectory, if the distance from the cluster center of the cluster a0 is the smallest, the current sample trajectory is assigned to the cluster a 0; if its distance from the cluster center of cluster B0 is minimal, then the current sample trajectory is assigned to cluster B0. After the current sample trajectory is assigned, the cluster center of the trajectory set may be recalculated. In this exemplary embodiment, the clustering centers of the sample tracks may be recalculated through the above steps and the above steps may be iterated until a clustering termination condition is satisfied, for example, the clustering termination condition may be that the homopolymerisation is completed for all sample tracks. Referring to fig. 7, fig. 7 is a diagram illustrating a target track respectively determined from a plurality of track sets according to an exemplary embodiment of the present application. Fig. 7 shows the target trajectory selected in each trajectory set.

The method for training the trajectory prediction model according to the target trajectory may specifically be: inputting the target track into a track prediction model so that the track prediction model outputs a prediction result; further, the expression may be based on

Calculating a loss function

And according to

Updating model parameters in the trajectory prediction model until

Within a preset threshold range; the model parameters may include weight parameters, bias terms, and the like, and the embodiments of the present application are not limited thereto. In addition, the following are

M and M are used to represent the upper limit time point and the lower limit time point of the truncation period, respectively, k is used to represent the target track and to distinguish different target tracks by assignment,

for the purpose of representing the function of indication,

indicates that the kth target track is closest to the real track, p_mkAnd alpha is a constant coefficient, and is used for representing the probability that the target track at the mth moment is a real track. When the k-th target track is selected,

represents the predicted track point at the mth moment

And the kth target track point τ_mkMay be represented by euclidean distances.

Therefore, by implementing the optional embodiment, the action track of the object to be predicted in the current scene can be predicted through the track prediction model obtained through training, safety early warning can be favorably carried out on the obstacle object according to the action track, so that safety accidents are avoided, and when the obstacle object is a vehicle, the implementation of the embodiment of the application is favorable for enabling a driver to know the predicted action track of a pedestrian (namely, the object to be predicted) in time, so that the driver can avoid danger.

Referring to fig. 8, fig. 8 is a schematic diagram illustrating an architecture of a method for predicting an action trajectory according to an exemplary embodiment of the present application. As shown in fig. 8, a graph feature vector corresponding to the simplified graph may be extracted through the deep learning network CNN801, and then feature calculation is performed on the graph feature vector through a plurality of hidden layers in the MLP802, so as to determine a first feature vector corresponding to an area occupied by an obstacle object. Furthermore, a second feature vector corresponding to the history trajectory of the obstacle object in the current scene may be calculated by the LSTM803, and a third feature vector corresponding to the current state of the obstacle object in the current scene may be calculated by the MLP 802. Furthermore, the first feature vector, the second feature vector and the third feature vector may be spliced, and the splicing result may be determined as a current feature vector corresponding to an object to be predicted in a current scene. Furthermore, a relationship model GAT804 for representing the relationship between the object to be predicted and the obstacle object can be constructed according to the influence range of each object in the current scene, and the current feature vector corresponding to each obstacle object in the relationship model is determined through feature extraction, so as to generate the relationship feature vector corresponding to the relationship model. Furthermore, the relationship feature vector and the current feature vector corresponding to the object to be predicted may be spliced to obtain a splicing result 805 and input into the MLP802, so that the MLP802 predicts the motion trajectory of the object to be predicted.

It can be seen that, by implementing the architecture diagram shown in fig. 8, the accuracy of predicting the action trajectory of the object to be predicted can be improved by combining the information of a plurality of obstacle objects around the object to be predicted. In addition, by constructing the relational model, the barrier objects interacting with the objects to be predicted can be integrated, so that the analysis efficiency of the barrier objects is improved, and the waste of computing resources is reduced.

Referring to fig. 9, fig. 9 is a block diagram illustrating a structure of an action trajectory prediction apparatus according to an exemplary embodiment of the present application. The action trajectory prediction apparatus includes a first feature vector calculation unit 901, a second feature vector calculation unit 902, a third feature vector calculation unit 903, and a motion trajectory prediction unit 904, where:

a first feature vector calculation unit 901, configured to calculate a first feature vector corresponding to an area occupied by an obstacle object in a current scene, a second feature vector corresponding to a historical track of the obstacle object, and a third feature vector corresponding to a current state of the obstacle object;

a second feature vector calculating unit 902, configured to determine, according to the first feature vector, the second feature vector, and the third feature vector, a current feature vector corresponding to an object to be predicted in a current scene;

a third feature vector calculation unit 903, configured to construct, according to the current scene, a relationship model for representing a relationship between an object to be predicted and an obstacle object, and calculate a relationship feature vector corresponding to the relationship model;

and a motion trajectory prediction unit 904, configured to predict a motion trajectory of the object to be predicted according to the relationship feature vector and a current feature vector corresponding to the object to be predicted in the current scene.

It can be seen that, by implementing the action trajectory prediction apparatus shown in fig. 9, the accuracy of action trajectory prediction for the object to be predicted can be improved by combining the information of a plurality of obstacle objects around the object to be predicted. In addition, by constructing the relational model, the barrier objects interacting with the objects to be predicted can be integrated, so that the analysis efficiency of the barrier objects is improved, and the waste of computing resources is reduced.

In an exemplary embodiment of the present application, the apparatus may further include a thumbnail image generation unit (not shown) and an occupied area determination unit (not shown):

a simplified diagram generating unit, configured to generate a simplified diagram according to a live-action diagram corresponding to a current scene before the first eigenvector calculating unit 901 calculates a first eigenvector corresponding to an area occupied by an obstacle object in the current scene, a second eigenvector corresponding to a historical trajectory of the obstacle object, and a third eigenvector corresponding to a current state of the obstacle object;

the first feature vector calculation unit 901 extracts a map feature vector corresponding to the simplified map through a deep learning network;

the first eigenvector calculation unit 901 calculates a first eigenvector corresponding to an area occupied by the obstacle object according to the position information and the direction information of the obstacle object in the simplified diagram.

In an exemplary embodiment of the present application, the determining, by the second feature vector calculating unit 902, a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector, and the third feature vector includes:

the second feature vector calculation unit 902 splices the first feature vector, the second feature vector, and the third feature vector, and determines a splicing result as a current feature vector corresponding to an object to be predicted in a current scene.

In an exemplary embodiment of the present application, the current scene includes at least one obstacle object, and the third feature vector calculation unit 903 constructs a relationship model for characterizing a relationship between an object to be predicted and the obstacle object according to the current scene, including:

the third feature vector calculation unit 903 determines a first influence range of an obstacle object and a second influence range of an object to be predicted in the current scene;

the third eigenvector calculation unit 903 screens the obstacle objects according to the intersection of the first influence range and the second influence range to obtain target objects;

the third feature vector calculation unit 903 constructs a relationship model for characterizing the relationship between the target object and the object to be predicted.

In an exemplary embodiment of the present application, the third feature vector calculating unit 903 calculates a relationship feature vector corresponding to the relationship model, including:

the third feature vector calculation unit 903 determines a current feature vector corresponding to the target object;

the third feature vector calculation unit 903 performs multi-layer feature extraction on the current feature vector corresponding to the target object to obtain a relationship feature vector corresponding to the relationship model.

In an exemplary embodiment of the present application, the predicting unit 904 of the motion trajectory predicts the motion trajectory of the object to be predicted according to the relation feature vector and the current feature vector corresponding to the object to be predicted in the current scene, including:

the motion trajectory prediction unit 904 collects a plurality of sample trajectories, normalizes the sample trajectories, and clusters the normalization result to obtain a plurality of trajectory sets;

the motion trajectory prediction unit 904 determines target trajectories from a plurality of trajectory sets, respectively;

the motion trajectory prediction unit 904 trains a trajectory prediction model according to the target trajectory;

the motion trajectory prediction unit 904 inputs the relationship feature vector and a current feature vector corresponding to the object to be predicted in the current scene into the trained trajectory prediction model, so that the trajectory prediction model predicts the motion trajectory of the object to be predicted.

For details that are not disclosed in the embodiments of the present application apparatus, please refer to the embodiments of the present application of the above-mentioned action trajectory prediction method for the details that are not disclosed in the embodiments of the present application apparatus.

Referring to FIG. 10, FIG. 10 is a block diagram illustrating a computer system 1000 suitable for implementing an electronic device according to an exemplary embodiment of the present application. The computer system 1000 of the electronic device shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 10, the computer system 1000 includes a Central Processing Unit (CPU)1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for system operation are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.

In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 1001.

It should be noted that the computer readable medium shown in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method for predicting an action trajectory as described in the above embodiments.

For example, the electronic device may implement the following as shown in fig. 1: step S110: calculating a first characteristic vector corresponding to an area occupied by an obstacle object in a current scene, a second characteristic vector corresponding to a historical track of the obstacle object and a third characteristic vector corresponding to a current state of the obstacle object; step S120: determining a current feature vector corresponding to an object to be predicted in a current scene according to the first feature vector, the second feature vector and the third feature vector; step S130: constructing a relation model for representing the relation between an object to be predicted and an obstacle object according to the current scene and calculating a relation characteristic vector corresponding to the relation model; step S140: and predicting the motion trail of the object to be predicted according to the relation characteristic vector and the current characteristic vector corresponding to the object to be predicted in the current scene.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for predicting a trajectory of action, comprising:

determining a current feature vector corresponding to an object to be predicted in the current scene according to the first feature vector, the second feature vector and the third feature vector;

constructing a relation model for representing the relation between the object to be predicted and the obstacle object according to the current scene and calculating a relation characteristic vector corresponding to the relation model;

and predicting the motion trail of the object to be predicted according to the relation feature vector and the current feature vector corresponding to the object to be predicted in the current scene.

2. The method according to claim 1, wherein before calculating a first eigenvector corresponding to an area occupied by an obstacle object in a current scene, a second eigenvector corresponding to a historical trajectory of the obstacle object, and a third eigenvector corresponding to a current state of the obstacle object, the method further comprises:

generating a simplified diagram according to the live-action diagram corresponding to the current scene;

and determining the occupied area of the obstacle object from the simplified diagram.

3. The method of claim 2, wherein calculating the first eigenvector corresponding to the area occupied by the obstacle object in the current scene comprises:

4. The method of claim 1, wherein determining a current feature vector corresponding to an object to be predicted in the current scene according to the first feature vector, the second feature vector and the third feature vector comprises:

and splicing the first feature vector, the second feature vector and the third feature vector, and determining a splicing result as a current feature vector corresponding to an object to be predicted in the current scene.

5. The method according to claim 1, wherein the current scene includes at least one obstacle object, and wherein constructing a relationship model for characterizing a relationship between the object to be predicted and the obstacle object according to the current scene includes:

determining a first influence range of the obstacle object and a second influence range of the object to be predicted in the current scene;

screening a target object from the obstacle objects according to the intersection of the first influence range and the second influence range;

6. The method of claim 5, wherein computing the corresponding relationship feature vector of the relationship model comprises:

determining a current feature vector corresponding to the target object;

7. The method according to claim 1, wherein predicting the motion trajectory of the object to be predicted according to the relationship feature vector and a current feature vector corresponding to the object to be predicted in the current scene comprises:

respectively determining target tracks from the plurality of track sets;

training a track prediction model according to the target track;

inputting the relationship characteristic vector and a current characteristic vector corresponding to the object to be predicted in the current scene into a trained track prediction model, so that the track prediction model predicts the motion track of the object to be predicted.

8. An action trajectory prediction device, comprising:

a second feature vector calculation unit, configured to determine, according to the first feature vector, the second feature vector, and the third feature vector, a current feature vector corresponding to an object to be predicted in the current scene;

9. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for predicting an action trajectory according to any one of claims 1 to 7.

10. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of action trajectory prediction according to any one of claims 1 to 7.