CN115366919A

CN115366919A - Trajectory prediction method, system, electronic device and storage medium

Info

Publication number: CN115366919A
Application number: CN202211052101.4A
Authority: CN
Inventors: 刘闯; 汪哲文; 杨东方; 邱利宏; 郑文
Original assignee: Chongqing Changan Automobile Co Ltd
Current assignee: Chongqing Changan Automobile Co Ltd
Priority date: 2022-08-31
Filing date: 2022-08-31
Publication date: 2022-11-22

Abstract

The application provides a track prediction method, a track prediction system, electronic equipment and a storage medium, wherein a target vehicle is obtained firstly; and a target object within a preset range with the target vehicle; then, determining the relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle and the target object relative to the geodetic coordinate system; then, based on the relative position relation, determining the moving trend of the target object relative to the target vehicle; predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle; and finally, fitting the historical moving track points, the current position points and the predicted destination of the target object to generate a predicted path track of the target object. According to the method and the device, the movement intention of the target object can be predicted firstly, then the predicted path track of the target object is generated by inference through the intention and road information, and the intelligent driving vehicle is helped to make decisions and actions better, so that the intelligent driving vehicle is more anthropomorphic as a whole.

Description

Trajectory prediction method, system, electronic device and storage medium

Technical Field

The application relates to the technical field of intelligent driving, in particular to a track prediction method, a track prediction system, electronic equipment and a storage medium.

Background

In the operation of the automatic driving system, the pedestrian target is a factor which has to be paid attention, particularly when the intelligent driving vehicle runs on a road section with more pedestrian targets, such as an urban area, a street and the like, the action of the pedestrian target has uncertainty, if the future walking track of the pedestrian target cannot be predicted, the current intelligent driving vehicle (or the vehicle) can only take measures according to the current observed state of the pedestrian target, and the measures are probably not avoided, for example, the pedestrian target crossing a lane is avoided later. Alternatively, unnecessary braking is made, such as false braking of a pedestrian target that is in the lane but is turning. Therefore, a certain prejudgment on the pedestrian target is needed, so that the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is enabled to be more anthropomorphic integrally.

Disclosure of Invention

In view of the above-mentioned shortcomings of the prior art, the present application provides a trajectory prediction method, system, electronic device and storage medium to solve the above-mentioned technical problems.

The application provides a trajectory prediction method, which comprises the following steps:

acquiring a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle;

determining the relative position relation of the target vehicle and the target object according to the position coordinates of the target vehicle relative to a geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system;

determining a movement trend of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object;

predicting a destination of the target object according to current road information and a moving trend of the target object relative to the target vehicle;

and fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate a predicted path track of the target object.

In an embodiment of the present application, the process of determining the movement tendency of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object includes:

acquiring the position of the target object relative to the target vehicle at the current moment based on the relative position relationship, and recording the position as a first position; and the number of the first and second groups,

acquiring position information of the target object relative to the target vehicle at the last moment based on the relative position relation, and recording the position information as a second position;

and determining the movement trend of the target object relative to the target vehicle according to the first position and the second position.

In an embodiment of the present application, the determining the moving trend of the target object relative to the target vehicle according to the first position and the second position includes:

calculating the distance between the first position and the target vehicle and recording as a first distance value;

calculating the distance between the second position and the target vehicle and recording as a second distance value;

judging whether the first distance value is smaller than the second distance value;

if the first distance value is smaller than the second distance value, the moving trend of the target object relative to the target vehicle is close to the target vehicle;

if the first distance value is greater than the second distance value, the moving trend of the target object relative to the target vehicle is far away from the target vehicle;

if the first distance value is equal to the second distance value, the moving trend of the target object relative to the target vehicle is static.

In an embodiment of the present application, fitting the historical movement trajectory point of the target object, the current position point of the target object, and the predicted destination of the target object, and generating the predicted path trajectory of the target object includes:

combining the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate a point set to be fitted;

performing curve fitting on the point set to be fitted for multiple times to generate a predicted path line;

taking transverse movement as a key step, extending in the tangential direction of a fitted curve when the target object moves to a preset position according to the transverse speed to obtain the driving distance of the target object, and translating the driving distance to the curve to obtain an equivalent curve moving point corresponding to the predicted path line;

and acquiring all equivalent curve moving points, and connecting all equivalent curve moving points to generate the predicted path track of the target object.

In an embodiment of the present application, after generating the to-be-fitted point set, the method further includes:

deleting historical moving track points, current position points and predicted destinations in the point set to be fitted, and adjusting the point set to be fitted so as to enable the point set to be fitted to be integrally continuous; or,

and interpolating the historical moving track points, the current position points and the predicted destination in the point set to be fitted, and adjusting the point set to be fitted so as to ensure that the point set to be fitted is integrally continuous.

In an embodiment of the present application, the target vehicle includes at least one of: the vehicle includes a level L0-driving vehicle, a level L1-driving vehicle, a level L2-driving vehicle, a level L3-driving vehicle, a level L4-driving vehicle, and a level L5-driving vehicle.

In an embodiment of the present application, the target object includes a pedestrian.

The present application further provides a trajectory prediction system, comprising:

the data acquisition module is used for acquiring a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle;

the position relation module is used for determining the relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle relative to a geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system;

the moving trend module is used for determining the moving trend of the target object relative to the target vehicle according to the relative position relation between the target vehicle and the target object;

the destination prediction module is used for predicting the destination of the target object according to current road information and the movement trend of the target object relative to the target vehicle;

and the track prediction module is used for fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate the predicted path track of the target object.

The present application further provides an electronic device, the electronic device including:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the trajectory prediction method as described in any of the above.

The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform a trajectory prediction method as defined in any one of the above.

As described above, the present application provides a trajectory prediction method, a trajectory prediction system, an electronic device, and a storage medium, which have the following advantages:

the method comprises the steps of firstly, acquiring a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle; then determining the relative position relationship between the target vehicle and the target object according to the position coordinates of the target vehicle relative to the geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system; determining the movement trend of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object; predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle; and finally, fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate the predicted path track of the target object. Therefore, the moving intention of the target object can be predicted firstly, and then the predicted path track of the target object is generated by inference through the intention and road information, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is enabled to be more anthropomorphic integrally. Meanwhile, the target historical track information and the road information can be effectively utilized, the advantages of expert rules and deep learning are integrated, and the method has the characteristics of being rapid in prototype development, capable of achieving bottom-fitting in performance, capable of achieving data closed-loop and the like.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:

FIG. 1 is a schematic diagram of an exemplary system architecture to which aspects of one or more embodiments of the present application may be applied;

FIG. 2 is a schematic flow chart illustrating a trajectory prediction method according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a predicted pedestrian trajectory according to an embodiment of the present application;

FIG. 4 is a schematic flowchart of a trajectory prediction method according to another embodiment of the present application;

FIG. 5 is a schematic diagram of a hardware structure of a trajectory prediction system according to an embodiment of the present application;

fig. 6 is a schematic hardware structure of an electronic device suitable for implementing one or more embodiments of the present application.

Detailed Description

Other advantages and effects of the present application will become apparent to those skilled in the art from the disclosure herein, wherein the embodiments of the present application will be described in detail with reference to the accompanying drawings and preferred embodiments. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It should be understood that the preferred embodiments are for purposes of illustration only and are not intended to limit the scope of the present disclosure.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation, and the type, number and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present application, however, it will be apparent to one skilled in the art that the embodiments of the present application may be practiced without these specific details, and in other embodiments, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring the embodiments of the present application.

Fig. 1 shows a schematic diagram of an exemplary system architecture to which the solution in one or more embodiments of the present application may be applied. As shown in fig. 1, system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a notebook computer, and a desktop computer. The server 130 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal device 110 and server 130, such as a wired communication link or a wireless communication link.

The system architecture in the embodiments of the present application may have any number of terminal devices, networks, and servers, according to implementation needs. For example, the server 130 may be a server group composed of a plurality of server devices. In addition, the technical solution provided in the embodiment of the present application may be applied to the terminal device 110, or may be applied to the server 130, or may be implemented by both the terminal device 110 and the server 130, which is not particularly limited in this application.

In one embodiment of the present application, the terminal device 110 or the server 130 of the present application may first acquire a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle; then determining the relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle relative to the geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system; then, based on the relative position relation between the target vehicle and the target object, determining the moving trend of the target object relative to the target vehicle; predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle; and finally, fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate the predicted path track of the target object. By using the terminal device 110 or the server 130 to execute the trajectory prediction method, the movement intention of the target object can be predicted first, and then the predicted path trajectory of the target object is generated by using the intention and road information inference, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is more personified as a whole. Meanwhile, target historical track information and road information can be effectively utilized, the advantages of expert rules and deep learning are combined, and the method has the characteristics of rapid prototype development, capability of backing up the ground, closed-loop data and the like.

The above section describes the content of an exemplary system architecture to which the technical solution of the present application is applied, and the following continues to describe the trajectory prediction method of the present application.

Fig. 2 is a schematic flowchart illustrating a trajectory prediction method according to an embodiment of the present application. Specifically, in an exemplary embodiment, as shown in fig. 2, the present embodiment provides a trajectory prediction method, which includes the following steps:

s210, acquiring a vehicle determined in advance or in real time as a target vehicle; and acquiring a target object which is within a preset range with the target vehicle. As an example, the target vehicle in the present embodiment includes at least one of: the vehicle includes a level L0-driving vehicle, a level L1-driving vehicle, a level L2-driving vehicle, a level L3-driving vehicle, a level L4-driving vehicle, and a level L5-driving vehicle. The target object in the present embodiment includes a pedestrian, an animal, and the like.

S220, determining the relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle relative to a geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system. Specifically, when determining the relative positional relationship between the target vehicle and the target object, the present application may perform coordinate synchronization conversion based on positioning in a geodetic coordinate system. The method comprises the steps of firstly obtaining the positioning of the vehicle based on a geodetic coordinate system, then converting the transverse and longitudinal distances of the pedestrian target based on the vehicle coordinate system into the position of the pedestrian target based on the geodetic coordinate system, and then converting the position of the pedestrian target based on the geodetic coordinate system into the real-time relative position relation based on the vehicle according to the requirement, so that the relative position can be calculated conveniently.

S230, determining a movement trend of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object;

s240, predicting a destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle;

and S250, fitting the historical moving track point of the target object, the current position point of the target object and the predicted destination of the target object to generate a predicted path track of the target object.

Therefore, the moving intention of the target object can be predicted firstly, and then the predicted path track of the target object is generated by inference through the intention and road information, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is enabled to be more anthropomorphic integrally. Meanwhile, the embodiment can effectively utilize the target historical track information and the road information, the advantages of expert rules and deep learning are fused, and the method has the characteristics of rapid prototype development, capability of backing up, closed-loop data and the like.

In an exemplary embodiment, the step S230 of determining the moving trend of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object includes: acquiring the position of the target object relative to the target vehicle at the current moment based on the relative position relationship, and recording the position as a first position; acquiring position information of the target object relative to the target vehicle at the last moment based on the relative position relation, and recording the position information as a second position; and determining the movement trend of the target object relative to the target vehicle according to the first position and the second position.

Wherein, according to the first position and the second position, the process of determining the movement trend of the target object relative to the target vehicle comprises: calculating the distance between the first position and the target vehicle and recording as a first distance value; calculating the distance between the second position and the target vehicle, and recording the distance as a second distance value; judging whether the first distance value is smaller than the second distance value; if the first distance value is smaller than the second distance value, the moving trend of the target object relative to the target vehicle is close to the target vehicle; if the first distance value is larger than the second distance value, the moving trend of the target object relative to the target vehicle is far away from the target vehicle; if the first distance value is equal to the second distance value, the moving trend of the target object relative to the target vehicle is static.

Therefore, in the embodiment, based on the historical track information of the pedestrian, the historical moving trend of the pedestrian target is calculated in an inference manner, first, some statistics are calculated, for example, whether the pedestrian target is closer to the vehicle or is deviated from the vehicle at each moment relative to the previous moment, the statistics are respectively counted, then, according to a certain statistical rule and a calibration value, whether the moving trend represented by the historical track information of the pedestrian target is closer to the vehicle, is deviated from the vehicle or is static is judged, and finally, a corresponding mark bit is output.

In an exemplary embodiment, the step S250 fits the historical moving trajectory point of the target object, the current position point of the target object, and the predicted destination of the target object, and the process of generating the predicted path trajectory of the target object includes: combining the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate a point set to be fitted; performing curve fitting on the point set to be fitted for multiple times to generate a predicted path line; taking transverse movement as a key step, extending in the tangential direction of a fitting curve when the target object moves to a preset position according to the transverse speed to obtain the driving distance of the target object, and translating the driving distance to the curve to obtain an equivalent curve moving point corresponding to the predicted path line; and acquiring all equivalent curve moving points, and connecting all equivalent curve moving points to generate the predicted path track of the target object. The predicted effect of the pedestrian trajectory is shown in fig. 3.

Therefore, according to the pedestrian target historical movement trend obtained by expert rule statistics, the pedestrian target destination is predicted, and the terminal point is preset for the pedestrian according to the principle idea that the pedestrian should travel outside the road, only pass through the road and not reside in the road by combining with the road information. Meanwhile, on the basis of obtaining the predicted path line, the method divides the position of the pedestrian target at each moment by using the idea of replacing a curve with a straight line, specifically, by using transverse movement as a key step, after the pedestrian moves to a certain point according to the transverse speed, the pedestrian extends in the tangential direction of the curve, the distance traveled by the pedestrian at the comprehensive speed is taken, then the pedestrian moves to the curve to obtain an equivalent curve moving point, and by analogy, a series of track points left by the pedestrian in the curve are formed, and a final pedestrian target track prediction point set is obtained.

In an exemplary embodiment, after generating the set of points to be fitted, the method further comprises: deleting historical moving track points, current position points and predicted destinations in the point set to be fitted, and adjusting the point set to be fitted so as to enable the point set to be fitted to be integrally continuous; or interpolating the historical moving track points, the current position points and the predicted destination in the point set to be fitted, and adjusting the point set to be fitted so as to ensure that the point set to be fitted is integrally continuous.

Therefore, in the embodiment, three types of points, namely the historical track position point of the pedestrian target, the current position point and the predicted destination, are combined into the point set to be fitted, the number of the three types of point sets is adjusted by deletion or interpolation, so that the whole is continuous, the historical inertia, the current accuracy and the future predictability are simultaneously achieved, and then the point set is subjected to cubic curve fitting to generate the predicted path line.

In another exemplary embodiment of the present application, as shown in fig. 4, the present application further provides a trajectory prediction method, including the steps of:

firstly, the deep learning module and the expert rule module are operated in a thread process, original images of historical frames of the pedestrian target are input, and an intention signal of the pedestrian target is continuously output according to appointed time of a 20ms period. And secondly, fusing with expert rules, continuously recording and updating the historical track of the pedestrian target, defining a static array to store state information of the pedestrian target, such as longitudinal distance, and continuously circularly shifting the static array along with a time sequence to fill the pedestrian target at a new moment. In this embodiment, when recording the position and speed information of the update action object, the scrolling update may be performed using a queued data format.

And secondly, performing coordinate synchronous conversion, namely obtaining the positioning of the vehicle based on the geodetic coordinate system, converting the transverse and longitudinal distances of the pedestrian target based on the vehicle coordinate system into the position of the pedestrian target based on the geodetic coordinate system, and converting the position of the pedestrian target based on the geodetic coordinate system into a real-time relative position relation based on the vehicle according to needs, so that the relative position can be calculated conveniently. Since the pedestrian target is perceived based on the own vehicle coordinate system, the recorded target position information is represented as the horizontal and vertical coordinates with respect to the own vehicle, but as such, the coordinate system changes with the running of the own vehicle, and information at different times is not in the same coordinate system, and the history sequence of the target cannot be described effectively, and therefore, coordinate synchronization is required. In the present embodiment, in combination with the vehicle positioning, the coordinates of the pedestrian target can be synchronized by using a geodetic coordinate system.

Thirdly, based on the historical track information of the pedestrian, the historical moving trend of the pedestrian target is calculated in an inference mode, firstly, some statistic values are calculated, for example, whether the pedestrian target is closer to the vehicle or deviates from the vehicle at each moment relative to the last moment, counting is carried out respectively, then, according to certain statistical rules and calibration values, whether the moving trend represented by the historical track information of the pedestrian target is close to the vehicle, deviates from the vehicle or is static is judged, and finally, a corresponding mark bit is output. The present embodiment may determine the historical movement tendency of the target based on the historical track information, the determination of the movement tendency is mainly based on the position information in the historical track and the relative distance change from the future driving reference line of the host vehicle, the change tendency on the historical track is counted, and the movement tendency is determined based on the statistical feature quantity, including being still, intruding into the reference line of the host vehicle, being far away from the reference line of the host vehicle, and the like.

And fourthly, predicting a pedestrian target destination according to the pedestrian target predicted movement intention output by the deep learning module and the historical movement trend of the pedestrian target obtained by expert rule statistics, and presetting a terminal point for the pedestrian according to a principle idea that the pedestrian should travel outside the road, only has a crossing purpose in the road and does not reside in the road in combination with the road information. That is, the present embodiment predicts the end point of the pedestrian target by combining the road information with the pedestrian target movement intention output by the deep learning and the pedestrian target historical movement trend obtained in the previous step. The determination of the end point may be based on some expert rule logical reasoning, for example if it is found that a pedestrian target is moving towards the host vehicle, the end point of the pedestrian target should be the edge of a lane or road edge behind the host vehicle.

And fifthly, combining three types of points including the historical track position point of the pedestrian target, the current position point and the predicted destination into a point set to be fitted, adjusting the number of the three types of point sets by deletion or interpolation to ensure that the whole is continuous, and meanwhile, the historical inertia, the current accuracy and the future predictability are achieved, and then performing cubic curve fitting on the point set to generate a predicted path line.

And sixthly, on the predicted path line obtained in the previous step, the embodiment uses the idea of replacing a curve with a straight line to divide the position of the pedestrian target at each moment, specifically, for example, taking transverse movement as a key step, firstly enabling the pedestrian to move to a certain point according to the transverse speed, then extending in the tangential direction, taking the distance traveled by the pedestrian at the comprehensive speed, then translating to the curve to obtain an equivalent curve moving point, and so on, forming a series of track points left by the pedestrian in the curve to obtain a final pedestrian target track predicted point set. That is, on the basis of obtaining the predicted trajectory line, the present embodiment may divide the position point where the pedestrian arrives at each moment according to the speed of the pedestrian by a differentiation processing manner of direct-to-curved curve, where the point where the pedestrian arrives at each moment is the position point where the time is predicted, and finally obtain the predicted trajectory point set of the pedestrian target at each 0.1s time in the future at 8s time. The predicted effect of the pedestrian trajectory is shown in fig. 3.

Therefore, the basic idea of the embodiment can be divided into two parts, the first part is to predict the movement intention of the pedestrian target by using a deep learning model, and the second part is to generate the predicted track of the pedestrian target by using the intention and combining road information in an inference mode. The method comprises the following main algorithm steps: first, the intention of movement of the pedestrian target, which may be defined as a two-classification, i.e., an intention of moving toward and away from the host vehicle, is output from the deep learning algorithm module. And secondly, an expert rule part for reasoning and generating the predicted track. Therefore, the embodiment is based on a mode of combining the expert rules and the deep learning algorithm, effectively utilizes the target historical track information and the road information, integrates the advantages of the expert rules and the deep learning, and has the characteristics of rapid prototype development, capability of entering the bottom, closed-loop data and the like.

In summary, the present application provides a trajectory prediction method, which first obtains a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle; then determining the relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle relative to the geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system; then, based on the relative position relation between the target vehicle and the target object, determining the moving trend of the target object relative to the target vehicle; predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle; and finally, fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate the predicted path track of the target object. Therefore, the method can predict the movement intention of the target object firstly, and then utilizes the intention to generate the predicted path track of the target object by inference in combination with road information, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is more anthropomorphic as a whole. Meanwhile, the method can effectively utilize the historical track information and the road information of the target, integrates the advantages of expert rules and deep learning, and has the characteristics of rapid prototype development, capability of fitting the bottom, closed-loop data and the like.

As shown in fig. 5, the present application further provides a trajectory prediction system, which includes:

a data acquisition module 510 for acquiring a predetermined or real-time determined vehicle as a target vehicle; and acquiring a target object which is within a preset range with the target vehicle. As an example, the target vehicle in the present embodiment includes at least one of: an L0-class-driven vehicle, an L1-class-driven vehicle, an L2-class-driven vehicle, an L3-class-driven vehicle, an L4-class-driven vehicle, and an L5-class-driven vehicle. The target object in the present embodiment includes a pedestrian, an animal, and the like.

A position relation module 520, configured to determine a relative position relation between the target vehicle and the target object according to the position coordinates of the target vehicle relative to a geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system. Specifically, when determining the relative positional relationship between the target vehicle and the target object, the present application may perform coordinate synchronization conversion based on positioning in a geodetic coordinate system. The method comprises the steps of firstly obtaining the positioning of the vehicle based on a geodetic coordinate system, then converting the transverse and longitudinal distances of the pedestrian target based on the vehicle coordinate system into the position of the pedestrian target based on the geodetic coordinate system, and then converting the position of the pedestrian target based on the geodetic coordinate system into the real-time relative position relation based on the vehicle according to the requirement, so that the relative position can be calculated conveniently.

A moving trend module 530, configured to determine a moving trend of the target object relative to the target vehicle according to a relative position relationship between the target vehicle and the target object;

a destination prediction module 540, configured to predict a destination of the target object according to current road information and a movement trend of the target object relative to the target vehicle;

and a trajectory prediction module 550, configured to fit the historical moving trajectory point of the target object, the current position point of the target object, and the predicted destination of the target object, and generate a predicted path trajectory of the target object.

Therefore, the movement intention of the target object can be predicted firstly, and then the predicted path track of the target object is generated through the intention and road information inference, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is enabled to be more anthropomorphic integrally. Meanwhile, the target historical track information and the road information can be effectively utilized, the advantages of expert rules and deep learning are combined, and the method has the advantages of being rapid in prototype development, capable of achieving bottom-fitting performance, capable of achieving closed-loop data and the like.

In an exemplary embodiment, the moving tendency module 530 determines the moving tendency of the target object relative to the target vehicle based on the relative position relationship between the target vehicle and the target object, and includes: acquiring the position of the target object relative to the target vehicle at the current moment based on the relative position relation, and recording the position as a first position; acquiring position information of the target object relative to the target vehicle at the last moment based on the relative position relation, and recording the position information as a second position; and determining the movement trend of the target object relative to the target vehicle according to the first position and the second position.

Wherein, according to the first position and the second position, the process of determining the movement trend of the target object relative to the target vehicle comprises: calculating the distance between the first position and the target vehicle and recording as a first distance value; calculating the distance between the second position and the target vehicle, and recording the distance as a second distance value; judging whether the first distance value is smaller than the second distance value; if the first distance value is smaller than the second distance value, the moving trend of the target object relative to the target vehicle is close to the target vehicle; if the first distance value is greater than the second distance value, the moving trend of the target object relative to the target vehicle is far away from the target vehicle; if the first distance value is equal to the second distance value, the moving trend of the target object relative to the target vehicle is static.

In an exemplary embodiment, the trajectory prediction module 550 fits the historical moving trajectory point of the target object, the current position point of the target object, and the predicted destination of the target object, and the process of generating the predicted path trajectory of the target object includes: combining the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate a point set to be fitted; performing curve fitting on the point set to be fitted for multiple times to generate a predicted path line; taking transverse movement as a key step, extending in the tangential direction of a fitting curve when the target object moves to a preset position according to the transverse speed to obtain the driving distance of the target object, and translating the driving distance to the curve to obtain an equivalent curve moving point corresponding to the predicted path line; and acquiring all equivalent curve moving points, and connecting all equivalent curve moving points to generate the predicted path track of the target object. The predicted effect of the pedestrian trajectory is shown in fig. 3.

Therefore, according to the pedestrian target historical movement trend obtained by expert rule statistics, the pedestrian target destination is predicted, and the terminal point is preset for the pedestrian according to the principle idea that the pedestrian should travel outside the road, only pass through the road and not reside in the road by combining with the road information. Meanwhile, on the basis of obtaining the predicted path line, the method and the device divide the position of the pedestrian target at each moment by using the idea of replacing a curve with a straight line, particularly, take transverse movement as a key step, extend in the tangential direction of the curve after the pedestrian moves to a certain point according to the transverse speed, take the distance traveled by the pedestrian at the comprehensive speed, then translate to the curve to obtain an equivalent curve moving point, and so on to form a series of track points left by the pedestrian in the curve to obtain a final pedestrian target track prediction point set.

In an exemplary embodiment, after generating the set of points to be fitted, the method further comprises: deleting historical moving track points, current position points and predicted destinations in the point set to be fitted, and adjusting the point set to be fitted to enable the point set to be fitted to be integrally continuous; or interpolating the historical moving track points, the current position points and the predicted destination in the point set to be fitted, and adjusting the point set to be fitted so as to ensure that the point set to be fitted is integrally continuous.

It should be noted that the trajectory prediction system provided in the foregoing embodiment and the trajectory prediction method provided in the foregoing embodiment belong to the same concept, and specific ways for each module and unit to perform operations have been described in detail in the method embodiment, and are not described herein again. In practical applications, the trajectory prediction system provided in the above embodiment may allocate the above functions to different function modules according to needs, that is, the internal structure of the system is divided into different function modules to complete all or part of the above described functions, which is not limited herein.

In another exemplary embodiment of the present application, the present application further provides a trajectory prediction system for performing the following steps:

firstly, a deep learning module and an expert rule module are operated in a thread, original images of historical frames of the pedestrian target are input, and an intention signal of the pedestrian target is continuously output according to appointed time of a 20ms period. And secondly, fusing with expert rules, continuously recording and updating the historical track of the pedestrian target, defining a static array to store state information of the pedestrian target, such as longitudinal distance, and continuously circularly shifting the static array along with a time sequence to fill the pedestrian target at a new moment. In this embodiment, when recording the position and speed information of the update action object, the scrolling update may be performed using a queued data format.

And secondly, performing coordinate synchronous conversion, wherein the coordinate synchronous conversion is performed, the positioning of the vehicle based on the geodetic coordinate system is firstly obtained, then the transverse and longitudinal distances of the pedestrian target based on the self-vehicle coordinate system are converted into the position of the pedestrian target based on the geodetic coordinate system, and then the position of the pedestrian target based on the geodetic coordinate system can be converted into a real-time relative position relation based on the self-vehicle according to needs, so that the relative position can be conveniently calculated. Since the pedestrian target is perceived based on the own vehicle coordinate system, the recorded target position information is represented as the horizontal and vertical coordinates with respect to the own vehicle, but as such, the coordinate system changes with the running of the own vehicle, and information at different times is not in the same coordinate system, and the history sequence of the target cannot be described effectively, and therefore, coordinate synchronization is required. In the present embodiment, in combination with the vehicle positioning, the coordinates of the pedestrian target can be synchronized by using a geodetic coordinate system.

Thirdly, based on the historical track information of the pedestrian, the historical moving trend of the pedestrian target is calculated in an inference mode, firstly, some statistic values are calculated, for example, whether the pedestrian target is closer to the vehicle or deviates from the vehicle at each moment relative to the previous moment, counting is carried out respectively, then, according to certain statistical rules and calibration values, whether the moving trend represented by the historical track information of the pedestrian target is close to the vehicle, deviates from the vehicle or is static is judged, and finally, a corresponding mark bit is output. The present embodiment may determine the historical movement tendency of the target based on the historical track information, the determination of the movement tendency is mainly based on the position information in the historical track and the relative distance change from the future driving reference line of the host vehicle, the change tendency on the historical track is counted, and the movement tendency is determined based on the statistical feature quantity, including being still, intruding into the reference line of the host vehicle, being far away from the reference line of the host vehicle, and the like.

And fourthly, predicting a pedestrian target destination according to the pedestrian target predicted movement intention output by the deep learning module and the historical movement trend of the pedestrian target obtained by expert rule statistics, and presetting a terminal point for the pedestrian according to a principle idea that the pedestrian should travel outside the road, only has a crossing purpose in the road and does not reside in the road in combination with the road information. That is, the present embodiment predicts the end point of the pedestrian target by combining the road information with the pedestrian target movement intention output by the deep learning and the pedestrian target historical movement trend obtained in the previous step. The determination of the endpoint may be based on some expert rule logical reasoning, for example if it is found that a pedestrian target is moving towards the host vehicle, the endpoint of the pedestrian target should be the edge of the lane or road edge behind the crossing of the host vehicle.

Therefore, the basic idea of the embodiment can be divided into two parts, the first part is to predict the movement intention of the pedestrian target by using a deep learning model, and the second part is to generate the predicted track of the pedestrian target by using the intention and combining road information in an inference mode. The method comprises the following main algorithm steps: first, the intention of movement of the pedestrian target, which may be defined as a two-classification, i.e., an intention of moving toward and away from the host vehicle, is output from the deep learning algorithm module. And secondly, an expert rule part is used for reasoning and generating a predicted track. Therefore, the embodiment is based on a mode of combining the expert rules and the deep learning algorithm, effectively utilizes the target historical track information and the road information, integrates the advantages of the expert rules and the deep learning, and has the characteristics of rapid prototype development, capability of entering the bottom, closed-loop data and the like.

In summary, the present application provides a trajectory prediction system, which first obtains a vehicle determined in advance or in real time as a target vehicle; acquiring a target object within a preset range with the target vehicle; then determining the relative position relationship between the target vehicle and the target object according to the position coordinates of the target vehicle relative to the geodetic coordinate system and the position coordinates of the target object relative to the geodetic coordinate system; then, based on the relative position relation between the target vehicle and the target object, determining the moving trend of the target object relative to the target vehicle; predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle; and finally, fitting the historical moving track points of the target object, the current position points of the target object and the predicted destination of the target object to generate the predicted path track of the target object. Therefore, the system can predict the movement intention of the target object firstly, and then generate the predicted path track of the target object by using the intention and combining road information reasoning, so that the target object can be predicted to a certain extent, the intelligent driving vehicle can be helped to make decisions and actions better, and the intelligent driving vehicle is more anthropomorphic as a whole. Meanwhile, the system can effectively utilize the target historical track information and the road information, integrates the advantages of expert rules and deep learning, and has the characteristics of rapid prototype development, capability of backing up the ground, closed-loop data and the like.

An embodiment of the present application further provides an electronic device, including: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the electronic device to implement the trajectory prediction method provided in the above-described embodiments.

FIG. 6 illustrates a schematic structural diagram of a computer system suitable for use to implement the electronic device of the embodiments of the present application. It should be noted that the computer system 1000 of the electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the application scope of the embodiments of the present application.

As shown in fig. 6, the computer system 1000 includes a Central Processing Unit (CPU) 1001 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1002 or a program loaded from a storage portion 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for system operation are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. An Input/Output (I/O) interface 1005 is also connected to the bus 1004.

The following components are connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including a Display panel such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage portion 1008 including a hard disk and the like; and a communication section 1009 including a Network interface card such as a Local Area Network (LAN) card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The driver 1010 is also connected to the I/O interface 1005 as necessary. A removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1010 as necessary, so that a computer program read out therefrom is mounted into the storage section 1008 as necessary.

In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 1009 and/or installed from the removable medium 1011. When the computer program is executed by a Central Processing Unit (CPU) 1001, various functions defined in the system of the present application are executed.

It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer-readable signal medium may comprise a propagated data signal with a computer-readable computer program embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

Yet another aspect of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to execute the trajectory prediction method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist separately without being incorporated in the electronic device.

Another aspect of the application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the trajectory prediction method provided in the above embodiments.

The above-described embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.

Claims

1. A method of trajectory prediction, the method comprising the steps of:

2. The trajectory prediction method according to claim 1, wherein the process of determining the movement tendency of the target object with respect to the target vehicle based on the relative positional relationship between the target vehicle and the target object includes:

acquiring the position of the target object relative to the target vehicle at the current moment based on the relative position relation, and recording the position as a first position; and the number of the first and second groups,

3. The trajectory prediction method of claim 2, wherein determining the trend of movement of the target object relative to the target vehicle based on the first location and the second location comprises:

4. The trajectory prediction method according to any one of claims 1 to 3, wherein fitting the historical movement trajectory points of the target object, the current position points of the target object, and the predicted destination of the target object, and generating the predicted path trajectory of the target object includes:

taking transverse movement as a key step, extending in the tangential direction of a fitting curve when the target object moves to a preset position according to the transverse speed to obtain the driving distance of the target object, and translating the driving distance to the curve to obtain an equivalent curve moving point corresponding to the predicted path line;

5. The trajectory prediction method of claim 4, after generating the set of points to be fitted, the method further comprising:

6. The trajectory prediction method of claim 1, wherein the target vehicle comprises at least one of: the vehicle includes a level L0-driving vehicle, a level L1-driving vehicle, a level L2-driving vehicle, a level L3-driving vehicle, a level L4-driving vehicle, and a level L5-driving vehicle.

7. The trajectory prediction method of claim 1, wherein the target object comprises a pedestrian.

8. A trajectory prediction system, comprising:

the destination prediction module is used for predicting the destination of the target object according to the current road information and the moving trend of the target object relative to the target vehicle;

9. An electronic device, characterized in that the electronic device comprises:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the trajectory prediction method of any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored which, when executed by a processor of a computer, causes the computer to carry out a trajectory prediction method according to any one of claims 1 to 7.