CN112078592A

CN112078592A - Method and device for predicting vehicle behavior and/or vehicle track

Info

Publication number: CN112078592A
Application number: CN201910510174.5A
Authority: CN
Inventors: 王宇舟
Original assignee: Momenta Suzhou Technology Co Ltd
Current assignee: Momenta Suzhou Technology Co Ltd
Priority date: 2019-06-13
Filing date: 2019-06-13
Publication date: 2020-12-15
Anticipated expiration: 2039-06-13
Also published as: CN112078592B

Abstract

The embodiment of the invention discloses a method and a device for predicting vehicle behaviors and/or vehicle tracks. The method comprises the following steps: determining vehicle behavior data and vehicle running tracks corresponding to other vehicles before the current moment, which are acquired by the acquisition equipment of the own vehicle; when any other vehicle corresponds to the multiple sections of vehicle running tracks, combining the multiple sections of vehicle running tracks; screening target vehicles with the movement distances reaching the preset distance requirement in the preset time period from other vehicles according to the vehicle running track corresponding to each other vehicle; and predicting the behavior category corresponding to the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle behavior data of the target vehicle, and/or predicting the future driving track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle driving track of the target vehicle. By applying the scheme provided by the embodiment of the invention, the convenience of prediction can be improved.

Description

Method and device for predicting vehicle behavior and/or vehicle track

Technical Field

The invention relates to the technical field of intelligent driving, in particular to a method and a device for predicting vehicle behaviors and/or vehicle tracks.

Background

In the automatic driving scene, the behavior and the track of other vehicles are predicted in advance to avoid the occurrence of traffic accidents, so the prediction of the behavior and the track of other vehicles is very necessary.

At present, the behaviors and the tracks of other vehicles are mainly predicted in a machine learning mode. In order to establish a behavior prediction machine learning model, a large amount of vehicle behavior sample data and model training of labeled information after the vehicle behavior sample data are labeled are required, and illustratively, when the vehicle information in the vehicle behavior sample data is left turn light flickering and the road information is left turn, the labeled information can be left turn of the vehicle.

In order to establish a track prediction machine learning model, a large number of vehicle history sample tracks and vehicle future sample tracks are required to perform model training, wherein the vehicle future sample tracks are vehicle tracks corresponding to the vehicle history sample tracks in the next time period, for example, the vehicle sample tracks within 10s are collected, the vehicle sample tracks corresponding to 1-5s are vehicle history sample tracks, and the vehicle sample tracks corresponding to 6-10s are vehicle future sample tracks.

As can be seen from the above, at present, for the prediction of the vehicle behavior and the vehicle trajectory, prediction models need to be respectively established for respective prediction, which results in poor convenience of prediction.

Disclosure of Invention

The invention provides a method and a device for predicting vehicle behaviors and/or vehicle tracks, which are used for improving the convenience of prediction. The specific technical scheme is as follows.

In a first aspect, an embodiment of the present invention provides a method for predicting vehicle behavior and/or vehicle trajectory, where the method includes:

determining vehicle behavior data and vehicle running tracks corresponding to other vehicles before the current moment, which are acquired by the acquisition equipment of the own vehicle;

when any other vehicle corresponds to a plurality of sections of vehicle running tracks, combining the plurality of sections of vehicle running tracks, wherein the plurality of sections of vehicle running tracks are formed by one vehicle running track with break points;

screening target vehicles with the movement distances reaching the preset distance requirement in the preset time period from other vehicles according to the vehicle running track corresponding to each other vehicle;

predicting the behavior category corresponding to the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle behavior data of the target vehicle, and/or predicting the future driving track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle driving track of the target vehicle, wherein the target scene type is determined according to the real-time position information of the vehicle,

wherein the target network model is: training the initial network model to obtain a network model based on the first type vehicle sample data and/or the second type vehicle sample data as model training data, wherein the target network model is for associating vehicle behavior sample data with a corresponding behavior class, and/or, correlating the vehicle history sample trajectories with corresponding vehicle future sample trajectories, the first type of vehicle sample data is real vehicle data actually driven by the vehicle, and comprises first vehicle behavior sample data, a corresponding first behavior class, a first vehicle historical sample track and a corresponding first vehicle future sample track, the second type of vehicle sample data is simulation data obtained by performing simulation driving on the vehicle based on simulation software, and comprises second vehicle behavior sample data and corresponding second behavior types, second vehicle historical sample tracks and corresponding second vehicle future sample tracks.

Optionally, when any other vehicle corresponds to multiple vehicle driving tracks, the step of combining the multiple vehicle driving tracks includes:

when any other vehicle corresponds to a plurality of sections of vehicle running tracks, obtaining track endpoints contained in the plurality of sections of vehicle running tracks;

acquiring target time points and target position information corresponding to each track endpoint;

calculating the time difference between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target time point, and calculating the distance between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target position information;

and connecting two adjacent track end points of which the time difference is smaller than a preset time difference threshold value and the distance is smaller than a preset distance threshold value.

Optionally, the vehicle behavior data includes speed information, and the step of screening out a target vehicle whose movement distance meets a preset distance requirement within a preset time period from other vehicles according to a vehicle driving track corresponding to each other vehicle includes:

for each other vehicle, selecting a preset number of track points within a preset time period from the vehicle running track corresponding to the vehicle;

calculating the standard deviation of the speed information corresponding to the preset number of track points;

and taking the vehicle with the standard deviation not less than the preset threshold value in other vehicles as a target vehicle with the movement distance reaching the preset distance requirement in the preset time period.

Optionally, the training process of the target network model includes:

acquiring vehicle sample data corresponding to the target scene type, wherein the vehicle sample data comprises first type vehicle sample data and/or second type vehicle sample data, the first type vehicle sample data is real vehicle data actually driven by a vehicle, the second type vehicle sample data is simulation data obtained by performing simulation driving on the vehicle based on simulation software, the first type vehicle sample data comprises first vehicle behavior sample data and a corresponding first behavior type, a first vehicle historical sample track and a corresponding first vehicle future sample track, and the second type vehicle sample data comprises second vehicle behavior sample data and a corresponding second behavior type, a second vehicle historical sample track and a corresponding second vehicle future sample track;

and training the initial network model by taking the first type of vehicle sample data and/or the second type of vehicle sample data as model training data to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data to be associated with the corresponding behavior class and/or enabling the vehicle historical sample track to be associated with the corresponding vehicle future sample track.

Optionally, the step of training an initial network model by using the first type of vehicle sample data and/or the second type of vehicle sample data as model training data to obtain a target network model includes:

inputting the first type of vehicle sample data and/or the second type of vehicle sample data into an initial network model, wherein the initial network model comprises a feature extraction layer, a first regression layer and a second regression layer;

for each piece of vehicle behavior sample data in the first vehicle behavior sample data and/or the second vehicle behavior sample data, determining a first feature vector corresponding to the piece of vehicle behavior sample data through a first model parameter of the feature extraction layer, performing regression on the first feature vector through a second model parameter of the first regression layer to obtain an initial behavior category, and calculating a first difference value between the initial behavior category and the behavior category corresponding to the piece of vehicle behavior sample data;

for each vehicle history sample track in the first vehicle history sample track and/or the second vehicle history sample track, determining a second feature vector corresponding to the vehicle history sample track through the first model parameter of the feature extraction layer, performing regression on the second feature vector through the third model parameter of the second regression layer to obtain an initial vehicle future track, and calculating a second difference value between the initial vehicle future track and the vehicle future sample track corresponding to the vehicle history sample track;

adjusting the first model parameter, the second model parameter, and the third model parameter based on the first difference value and/or the second difference value;

and when the iteration times reach the preset times, finishing training to obtain a target network model containing a first corresponding relation between the vehicle behavior sample data and the behavior category and/or a second corresponding relation between the vehicle historical sample track and the vehicle future sample track.

Optionally, after the step of predicting the future driving trajectory of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle driving trajectory of the target vehicle, the method further includes:

and smoothing the future driving track.

In a second aspect, an embodiment of the present invention provides an apparatus for predicting vehicle behavior and/or vehicle trajectory, the apparatus including:

the determining module is used for determining vehicle behavior data and vehicle running tracks corresponding to other vehicles before the current moment, wherein the vehicle behavior data and the vehicle running tracks are acquired by the acquiring equipment of the own vehicle;

the system comprises a merging module, a judging module and a judging module, wherein the merging module is used for merging a plurality of vehicle running tracks when any other vehicle corresponds to the plurality of vehicle running tracks, and the plurality of vehicle running tracks are formed by one vehicle running track with break points;

the screening module is used for screening target vehicles with the movement distances reaching the preset distance requirement in the preset time period from other vehicles according to the vehicle running tracks corresponding to the other vehicles;

a prediction module, configured to predict a behavior category corresponding to the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle behavior data of the target vehicle, and/or predict a future travel track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to a vehicle travel track of the target vehicle, where the target scene type is determined according to real-time location information of the vehicle,

Optionally, the merging module includes:

the first obtaining submodule is used for obtaining track end points contained in the multiple sections of vehicle running tracks when any other vehicle corresponds to the multiple sections of vehicle running tracks;

the second acquisition submodule is used for acquiring target time points and target position information corresponding to each track endpoint;

the calculation submodule is used for calculating the time difference between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target time point and calculating the distance between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target position information;

and the connecting submodule is used for connecting two adjacent track endpoints of which the time difference is smaller than a preset time difference threshold value and the distance is smaller than a preset distance threshold value.

Optionally, the vehicle behavior data includes speed information, and the screening module includes:

the selection submodule is used for selecting a preset number of track points within a preset time period from the vehicle running track corresponding to each other vehicle;

the standard deviation calculation submodule is used for calculating the standard deviation of the speed information corresponding to the preset number of track points;

and the determining submodule is used for taking the vehicle with the standard deviation not less than a preset threshold value in other vehicles as a target vehicle with the movement distance reaching the preset distance requirement in a preset time period.

Optionally, the apparatus further includes a training module, where the training module is configured to train to obtain the target network model, and the training module includes:

the vehicle sample data acquisition sub-module is used for acquiring vehicle sample data corresponding to the target scene type, wherein the vehicle sample data comprises first type vehicle sample data and/or second type vehicle sample data, the first type vehicle sample data is real vehicle data actually driven by a vehicle, the second type vehicle sample data is simulation data obtained by performing simulation driving on the vehicle based on simulation software, the first type vehicle sample data comprises first vehicle behavior sample data and a corresponding first behavior type, a first vehicle historical sample track and a corresponding first vehicle future sample track, and the second type vehicle sample data comprises second vehicle behavior sample data and a corresponding second behavior type, a second vehicle historical sample track and a corresponding second vehicle future sample track;

and the training submodule is used for training the initial network model by taking the first type of vehicle sample data and/or the second type of vehicle sample data as model training data to obtain a target network model, wherein the target network model is used for enabling the vehicle behavior sample data to be associated with the corresponding behavior type and/or enabling the vehicle historical sample track to be associated with the corresponding vehicle future sample track.

Optionally, the training submodule includes:

the input unit is used for inputting the first type of vehicle sample data and/or the second type of vehicle sample data into an initial network model, wherein the initial network model comprises a feature extraction layer, a first regression layer and a second regression layer;

a first difference value calculation unit, configured to determine, for each vehicle behavior sample data in the first vehicle behavior sample data and/or the second vehicle behavior sample data, a first feature vector corresponding to the vehicle behavior sample data through a first model parameter of the feature extraction layer, perform regression on the first feature vector through a second model parameter of the first regression layer to obtain an initial behavior category, and calculate a first difference value between the initial behavior category and a behavior category corresponding to the vehicle behavior sample data;

a second difference value calculation unit, configured to determine, for each vehicle history sample track in the first vehicle history sample track and/or the second vehicle history sample track, a second feature vector corresponding to the vehicle history sample track through the first model parameter of the feature extraction layer, perform regression on the second feature vector through a third model parameter of the second regression layer to obtain an initial vehicle future track, and calculate a second difference value between the initial vehicle future track and the vehicle future sample track corresponding to the vehicle history sample track;

an adjusting unit, configured to adjust the first model parameter, the second model parameter, and the third model parameter based on the first difference value and/or the second difference value;

and the training completion unit is used for completing training when the iteration times reach the preset times to obtain a target network model containing a first corresponding relation between the vehicle behavior sample data and the behavior category and/or a second corresponding relation between the vehicle historical sample track and the vehicle future sample track.

Optionally, the apparatus further comprises:

and the smoothing processing module is used for smoothing the future driving track of the target vehicle after the future driving track of the target vehicle is predicted based on a target network model corresponding to a target scene type established in advance according to the vehicle driving track of the target vehicle.

As can be seen from the above, the method and the device for predicting vehicle behavior and/or vehicle trajectory provided in the embodiments of the present invention can determine the vehicle behavior data and the vehicle driving trajectory corresponding to each other vehicle before the current time acquired by the acquisition device of the own vehicle, then merge the vehicle trajectories of the vehicles corresponding to the multiple segments of vehicle driving trajectories, screen out the target vehicles from the other vehicles whose movement distances meet the preset distance requirement within the preset time period, and finally predict the behavior category and/or the future driving trajectory corresponding to the target vehicle through the target network model, so that the vehicle behavior and the vehicle trajectory can be predicted simultaneously, and the vehicle behavior or the vehicle trajectory can be predicted respectively, thereby improving the convenience of prediction. Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

The innovation points of the embodiment of the invention comprise:

1. the method comprises the steps of determining vehicle behavior data and vehicle running tracks corresponding to other vehicles before the current moment and acquired by an acquisition device of the own vehicle, then merging the vehicle tracks of the vehicles corresponding to multiple sections of the vehicle running tracks, screening target vehicles with the movement distances meeting the preset distance requirement in the preset time period from the other vehicles, and finally predicting the behavior types corresponding to the target vehicles and/or the future running tracks through a target network model, so that the vehicle behavior and the vehicle tracks can be predicted at the same time, the vehicle behavior or the vehicle tracks can be predicted respectively, and the convenience of prediction is improved.

2. The method comprises the steps of calculating the time difference between two adjacent track end points of two adjacent vehicle running tracks, calculating the distance between two adjacent track end points of two adjacent vehicle running tracks, and connecting the two adjacent track end points of which the time difference is smaller than a preset time difference threshold value and the distance is smaller than a preset distance threshold value, so that the multiple sections of vehicle running tracks belonging to the same vehicle running track are combined. The vehicle running track of other vehicles is more consistent with the actual running track, and the accuracy of subsequent prediction through the target network model is improved.

3. And screening out target vehicles with the movement distances reaching the preset distance requirement in the preset time period from other vehicles in a mode of calculating the standard deviation of the speed information corresponding to the preset number of track points positioned in the preset time period in the vehicle running tracks of other vehicles. Therefore, only the future driving track of the target vehicle is predicted subsequently, and the future driving tracks of other vehicles are not predicted, so that the occupation of computing resources is reduced, and the computing efficiency is improved.

4. By training the initial network model, a target network model used for enabling vehicle behavior sample data to be associated with corresponding behavior categories and/or enabling vehicle historical sample tracks to be associated with corresponding vehicle future sample tracks can be obtained, vehicle behaviors and vehicle tracks can be predicted simultaneously through the target network model, vehicle behaviors or vehicle tracks can be predicted respectively, and convenience of prediction is improved.

5. The method for obtaining the simulation data by simulating the driving of the vehicle based on the simulation software greatly reduces the cost of data acquisition during model training.

6. Through a smoothing processing mode, the difference between the future driving track and the actual vehicle driving track is reduced, so that the future driving track is closer to the actual vehicle driving track, and the purpose of optimizing the future driving track is achieved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of some embodiments of the invention. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.

FIG. 1 is a schematic flow chart of a method for predicting vehicle behavior and/or vehicle trajectory according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of step S120 in FIG. 1;

fig. 3(a) is a schematic diagram of track points included in a vehicle driving track of a first other vehicle within a preset time period;

fig. 3(b) is a schematic diagram of track points included in the vehicle driving track of the second other vehicle within the preset time period;

FIG. 4 is a flowchart illustrating a training process of a target network model according to an embodiment of the present invention;

FIG. 5(a) is a schematic diagram of a predicted future travel path;

FIG. 5(b) is a schematic illustration of an optimized future travel path corresponding to the future travel path of FIG. 5 (a);

FIG. 5(c) is an ideal future travel path corresponding to the future travel path of FIG. 5 (a);

fig. 6 is a schematic structural diagram of a vehicle behavior and/or vehicle trajectory prediction apparatus according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the invention discloses a method and a device for predicting vehicle behaviors and/or vehicle tracks, which can predict the vehicle behaviors and/or the vehicle tracks without respectively establishing prediction models for respective prediction, and can simultaneously predict the vehicle behaviors and the vehicle tracks and respectively predict the vehicle behaviors or the vehicle tracks by only one prediction model, so that the convenience of prediction is improved. The following provides a detailed description of embodiments of the invention.

Fig. 1 is a schematic flowchart of a method for predicting vehicle behavior and/or vehicle trajectory according to an embodiment of the present invention. The method is applied to the electronic equipment. The method specifically comprises the following steps.

S110: and determining the vehicle behavior data and the vehicle running track corresponding to each other vehicle before the current moment, which are acquired by the acquisition equipment of the own vehicle.

In the automatic driving scene, if the own vehicle can predict the behavior and the track of the own vehicle in advance, the traffic accident can be avoided, and therefore, the prediction of the behavior and the track of the own vehicle is very necessary.

In order to predict the behavior and trajectory of another vehicle, it is necessary to perform prediction based on the vehicle behavior data and the vehicle travel trajectory corresponding to each other vehicle before the current time, and therefore, it is necessary to determine the vehicle behavior data and the vehicle travel trajectory corresponding to each other vehicle before the current time, which are collected by the collection device of the own vehicle.

The self vehicle is provided with a collection device and a positioning system, such as a camera and a global positioning system, so that the self vehicle can collect the collection range in real time and position the self vehicle in the driving process, and the vehicle behavior data of the self vehicle, the vehicle driving track of the self vehicle, the vehicle behavior data of other vehicles and the vehicle driving track of other vehicles can be obtained based on the images collected in real time and the real-time position information of the self vehicle.

Wherein the vehicle behavior data of each vehicle may include: vehicle information and road information.

For example, the vehicle information may include at least one of 3D coordinate information, speed information, turn signal information, brake signal information, or double jump signal information, and the speed information includes at least one of speed and acceleration.

The road information may include at least one of straight road, left turn road, right turn road, crossroads, gateway junction, parking space, zebra stripes, or red street light information.

S120: when any other vehicle corresponds to the multiple sections of vehicle running tracks, the multiple sections of vehicle running tracks are combined.

When the vehicle running tracks of other vehicles are determined, certain errors exist, so that one vehicle running track of other vehicles can have breakpoints, and a multi-section vehicle running track is formed, namely the multi-section vehicle running track is formed by one vehicle running track with breakpoints.

For the vehicle running track with the breakpoint, the prediction effect may be poor, and in order to ensure the prediction effect, in the embodiment of the invention, when any other vehicle corresponds to multiple segments of vehicle running tracks, the multiple segments of vehicle running tracks are combined.

Referring to fig. 2, step S120 may include the steps of:

s1201: when any other vehicle corresponds to multiple sections of vehicle running tracks, track end points contained in the multiple sections of vehicle running tracks are obtained.

Since the vehicle driving track is composed of track points, when any other vehicle corresponds to multiple vehicle driving tracks, in order to merge the multiple vehicle driving tracks, it is necessary to obtain track end points included in the multiple vehicle driving tracks.

S1202: and acquiring target time points and target position information corresponding to each track endpoint.

If two disconnected vehicle driving tracks belong to the same vehicle driving track, the time difference and the distance between two adjacent track endpoints of the two vehicle driving tracks should be very small, so after the track endpoints included in the multiple vehicle driving tracks are obtained, the target time point and the target position information corresponding to each track endpoint need to be obtained.

S1203: and calculating the time difference between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target time point, and calculating the distance between two adjacent track end points of two adjacent sections of vehicle running tracks based on the target position information.

After the target time point and the target position information corresponding to each track endpoint are obtained, the time difference between two adjacent track endpoints of two adjacent vehicle running tracks can be calculated based on the target time point, and the distance between two adjacent track endpoints of two adjacent vehicle running tracks can be calculated based on the target position information.

S1204: and connecting two adjacent track end points of which the time difference is smaller than a preset time difference threshold value and the distance is smaller than a preset distance threshold value.

If two adjacent track end points of two adjacent vehicle running tracks meet the condition that the time difference is smaller than the preset time difference threshold value and the distance is smaller than the preset distance threshold value, the two adjacent vehicle running tracks belong to the same vehicle running track, and at the moment, the two adjacent track end points of the two adjacent vehicle running tracks can be connected.

If the end points of two adjacent tracks of the two adjacent vehicle running tracks do not meet the condition that the time difference is smaller than the preset time difference threshold value and the distance is smaller than the preset distance threshold value, the two adjacent vehicle running tracks do not belong to the same vehicle running track, and at the moment, no processing is performed.

Therefore, the multiple sections of vehicle running tracks belonging to the same vehicle running track are combined by calculating the time difference between two adjacent track end points of two adjacent vehicle running tracks, calculating the distance between two adjacent track end points of two adjacent vehicle running tracks and connecting the two adjacent track end points of which the time difference is smaller than a preset time difference threshold value and the distance is smaller than a preset distance threshold value. The vehicle running track of other vehicles is more consistent with the actual running track, and the accuracy of subsequent prediction through the target network model is improved. This is one of the innovative points of the embodiments of the present invention.

S130: and screening out target vehicles with the movement distances reaching the preset distance requirement in the preset time period from the other vehicles according to the vehicle running tracks corresponding to the other vehicles.

In other collected vehicles, vehicles with small movement distance in a period of time may exist, and the positions of the vehicles in the period of time are basically unchanged, so that the significance of predicting the track of the vehicles is not large, and the track prediction of the vehicles also occupies computing resources, so that the other vehicles can be filtered when the track prediction is performed in order to improve the computing efficiency.

Therefore, before the trajectory prediction is performed, a target vehicle with a movement distance reaching the preset distance requirement within a preset time period needs to be screened from other vehicles according to the vehicle running trajectory corresponding to each other vehicle.

Step S130 may include the steps of:

calculating the standard deviation of the speed information corresponding to a preset number of track points;

In order to screen out other vehicles with a small movement distance in a period of time, for each other vehicle, a preset number of track points within a preset time period are selected from the vehicle running tracks corresponding to the vehicle, that is, a time window is added to the vehicle running tracks corresponding to the vehicle, so that the number of the track points within the preset time period selected from the vehicle running tracks of each other vehicle is the same.

Because the speed information is a parameter for measuring whether the vehicle moves and the movement amplitude, and the vehicle information of the vehicle comprises the speed information, the standard deviation of the speed information corresponding to the preset number of track points can be calculated after the preset number of track points are selected.

If the calculated standard deviation is smaller than a preset threshold value, the moving distance of the vehicle is smaller, and if the calculated standard deviation is not smaller than the preset threshold value, the moving distance of the vehicle is larger, so that the vehicle with the standard deviation not smaller than the preset threshold value in other vehicles is taken as a target vehicle with the moving distance reaching the preset distance requirement in a preset time period after the standard deviation is calculated.

Therefore, the target vehicles with the movement distances reaching the preset distance requirement in the preset time period are screened out from other vehicles in a mode of calculating the standard deviation of the speed information corresponding to the preset number of track points in the preset time period in the vehicle running tracks of other vehicles. Therefore, only the future driving track of the target vehicle is predicted subsequently, and the future driving tracks of other vehicles are not predicted, so that the occupation of computing resources is reduced, and the computing efficiency is improved. This is one of the innovative points of the embodiments of the present invention.

For example: referring to fig. 3(a) and 3(b), fig. 3(a) is a schematic diagram of track points included in a vehicle driving track of a first other vehicle within a preset time period, fig. 3(b) is a schematic diagram of track points included in a vehicle driving track of a second other vehicle within a preset time period, each point in the diagram represents one track point, and as can be seen from fig. 3(a) and 3(b), the moving distance of the first other vehicle in fig. 3(a) is larger, and the moving distance of the second other vehicle in fig. 3(b) is smaller, so that the first other vehicle in fig. 3(b) can be filtered out, and the second other vehicle in fig. 3(a) can be filtered out as a target vehicle.

S140: and predicting the behavior category corresponding to the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle behavior data of the target vehicle, and/or predicting the future driving track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle driving track of the target vehicle.

Since the choice of behavior categories may differ under different scene types, for example: under the type of the expressway scene, the behavior categories can comprise an entrance ramp behavior, an exit ramp behavior, a lane change behavior, a lane snatching behavior and a passing behavior; under the type of urban road scene, the behavior categories can include parking behavior, straight behavior, left turn behavior, right turn behavior, reverse behavior, traffic light behaviors such as traffic lights and obstacle avoidance behavior. Therefore, when predicting the behavior category of other vehicles, it is necessary to determine the scene type of the other vehicles.

The types of the scenes of the other vehicle and the self vehicle are the same, and the self vehicle is positioned in real time to obtain the real-time position information of the self vehicle, wherein the real-time position information of the self vehicle can reflect the position of the self vehicle at present, and the scene type of the self vehicle can be judged according to the position, so that the target scene type of the other vehicle can be determined in real time according to the real-time position information of the self vehicle, namely the target scene type is determined according to the real-time position information of the self vehicle.

When the target scene type is determined in real time, the behavior type corresponding to the target vehicle can be predicted based on the target network model corresponding to the pre-established target scene type according to the vehicle behavior data of the target vehicle, and/or the future driving track of the target vehicle can be predicted based on the target network model corresponding to the pre-established target scene type according to the vehicle driving track of the target vehicle.

Wherein, the target network model is: training the initial network model to obtain a network model based on the first type of vehicle sample data and the corresponding first behavior class and/or the second type of vehicle sample data and the corresponding second behavior class as model training data, wherein the target network model is used to associate vehicle behavior sample data with a corresponding behavior class, and/or enabling the vehicle historical sample track to be associated with the corresponding vehicle future sample track, wherein the first type of vehicle sample data is real vehicle data actually driven by the vehicle and comprises first vehicle behavior sample data, the first vehicle historical sample track and the corresponding first vehicle future sample track, and the second type of vehicle sample data is simulation data obtained by performing simulation driving on the vehicle on the basis of simulation software and comprises second vehicle behavior sample data, the second vehicle historical sample track and the corresponding second vehicle future sample track.

Referring to fig. 4, the training process of the target network model is as follows:

s401: the method comprises the steps of obtaining vehicle sample data corresponding to a target scene type, wherein the vehicle sample data comprise first type vehicle sample data and/or second type vehicle sample data, the first type vehicle sample data are real vehicle data of actual driving of a vehicle, and the second type vehicle sample data are simulation data obtained by simulating driving of the vehicle based on simulation software.

Because the target network model is established to perform behavior prediction and/or trajectory prediction on the vehicle sample data corresponding to the target scene type, the vehicle sample data corresponding to the target scene type needs to be acquired when the target network model is established.

Since the data acquisition cost is high if radar and other high-cost devices are used when the real vehicle data of the actual driving of the vehicle are acquired, in order to reduce the cost, in the embodiment of the invention, the data can be acquired through low-cost devices or software such as a camera or a global positioning system, and the data acquisition cost is reduced.

In addition, in the embodiment of the invention, the vehicle can be driven in a simulation mode based on simulation software to obtain simulation data, so that the cost of data acquisition is greatly reduced. Because the simulation data is different from the real vehicle data, after the simulation data is obtained, the simulation data needs to be adjusted to some extent, for example, speed information is smoothed, so that a speed information difference between two adjacent estimation points of a vehicle driving track is within a preset range, or noise is added to 3D coordinate information to simulate an error generated in an actual acquisition process, that is, an error is added to the 3D coordinate information to make the simulation data closer to the real vehicle data.

As can be seen from the above, the vehicle sample data may include first type vehicle sample data and/or second type vehicle sample data. The first type of vehicle sample data is real vehicle data of actual driving of the vehicle, and the second type of vehicle sample data is simulation data obtained by simulation driving of the vehicle based on simulation software.

The first type of vehicle sample data comprises first vehicle behavior sample data and a corresponding first behavior type, a first vehicle historical sample track and a corresponding first vehicle future sample track, and the second type of vehicle sample data comprises second vehicle behavior sample data and a corresponding second behavior type, a second vehicle historical sample track and a corresponding second vehicle future sample track.

In order to predict the behavior class corresponding to the vehicle behavior data, when a target network model is established, the behavior class corresponding to the vehicle behavior sample data needs to be obtained, wherein the behavior class can be determined in various ways, such as manually labeled or based on labeling software.

There are various ways of determining the behavior category based on the annotation software, including but not limited to the following two:

the first mode is as follows:

acquiring target vehicle behavior data to be labeled, wherein the target vehicle behavior data comprises vehicle information and road information;

calculating statistical characteristics of vehicle information based on a pre-established target mathematical statistical model, wherein the statistical characteristics reflect behavior categories corresponding to target vehicle behavior data;

determining a target statistical rule met by the statistical characteristics, and obtaining a behavior category corresponding to target vehicle behavior data according to the target statistical rule, a preset corresponding relation between the statistical rule and the statistical behavior and road information;

and labeling the behavior data of the target vehicle based on the obtained behavior category to obtain corresponding labeling information.

The second mode is as follows:

acquiring target vehicle behavior data to be labeled, determining a behavior class corresponding to the target vehicle behavior data based on a pre-established first target network model, labeling the target vehicle behavior data based on the determined behavior class to obtain corresponding labeling information,

wherein the first target network model is: and training the first initial network model to obtain a network model based on the first vehicle behavior sample data and the corresponding behavior class as model training data, wherein the first target network model is used for enabling the first vehicle behavior sample data to be correlated with the corresponding behavior class, and the behavior class corresponding to the first vehicle behavior sample data is input by a marker in advance aiming at the first vehicle behavior sample data.

It should be noted that, if the behavior category corresponding to the vehicle behavior sample data is determined in the two ways based on the labeling software, the vehicle behavior sample data may be used as the target vehicle behavior data to be labeled in the above ways, and finally, the labeling information corresponding to the target vehicle behavior data may be obtained, that is, the behavior category corresponding to the vehicle behavior sample data is obtained.

S402: and training the initial network model by taking the first type of vehicle sample data and/or the second type of vehicle sample data as model training data to obtain a target network model.

After vehicle sample data corresponding to the target scene type is obtained, the first type of vehicle sample data and/or the second type of vehicle sample data are/is used as model training data, and the initial model is trained to obtain a target network model. Wherein the target network model is used to associate vehicle behaviour sample data with corresponding behaviour categories and/or to associate vehicle historical sample trajectories with corresponding vehicle future sample trajectories.

Step S402 may specifically include:

inputting first type vehicle sample data and/or second type vehicle sample data into an initial network model, wherein the initial network model comprises a feature extraction layer, a first regression layer and a second regression layer;

for each vehicle behavior sample data in the first vehicle behavior sample data and/or the second vehicle behavior sample data, determining a first feature vector corresponding to the vehicle behavior sample data through a first model parameter of a feature extraction layer, performing regression on the first feature vector through a second model parameter of a first regression layer to obtain an initial behavior category, and calculating a first difference value between the initial behavior category and the behavior category corresponding to the vehicle behavior sample data;

for each vehicle history sample track in the first vehicle history sample track and/or the second vehicle history sample track, determining a second feature vector corresponding to the vehicle history sample track through a first model parameter of a feature extraction layer, performing regression on the second feature vector through a third model parameter of a second regression layer to obtain an initial vehicle future track, and calculating a second difference value between the initial vehicle future track and the vehicle future sample track corresponding to the vehicle history sample track;

adjusting the first model parameter, the second model parameter and the third model parameter based on the first difference value and/or the second difference value;

It can be understood that the electronic device first needs to construct an initial network model and then train the initial network model to obtain the target network model. In one implementation, an initial network model including a feature extraction layer, a first regression layer, and a second regression layer may be constructed using a caffe tool. For example, the initial Network model may be a SVM (Support Vector Machine) RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory Network), or GRU (Gated Recurrent Unit), etc.

After the initial network model is built, the first type of vehicle sample data and/or the second type of vehicle sample data are input into the initial network model for training.

Specifically, first vehicle behavior sample data and/or second vehicle behavior sample data are input into a feature extraction layer, and a first feature vector corresponding to each vehicle behavior sample data in the first vehicle behavior sample data and/or the second vehicle behavior sample data is determined through first model parameters of the feature extraction layer. And then inputting the determined first feature vector into a first regression layer, and performing regression on the first feature vector through a second model parameter of the first regression layer to obtain an initial behavior category.

And for each vehicle behavior sample data, after the initial behavior type is obtained, comparing the initial behavior type with the behavior class corresponding to the vehicle behavior sample data, namely calculating a first difference value between the initial behavior type and the behavior class.

And inputting the first vehicle history sample track and/or the second vehicle history sample track into a feature extraction layer, and determining a second feature vector corresponding to each vehicle history sample track in the first vehicle history sample track and/or the second vehicle history sample track through the first model parameters of the feature extraction layer and the first model parameters of the feature extraction layer. And then inputting the determined second feature vector into a second regression layer, and performing regression on the second feature vector through a third model parameter of the second regression layer to obtain an initial future vehicle track.

And for each vehicle history sample track, after the initial vehicle future track is obtained, comparing the initial vehicle future track with the vehicle future sample track corresponding to the vehicle history sample track, namely calculating a second difference value between the initial vehicle future track and the vehicle future sample track.

It should be noted that, the above-mentioned processes of inputting the first vehicle behavior sample data and/or the second vehicle behavior sample data into the feature extraction layer and inputting the first vehicle history sample trajectory and/or the second vehicle history sample trajectory into the feature extraction layer are not in sequence, and the embodiment of the present invention is only for convenience of description, and is described above.

After the first difference value and the second difference value are obtained, the first model parameter, the second model parameter and the third model parameter can be adjusted through an evaluation function in a back propagation method based on the first difference value and/or the second difference value.

The evaluation function in the back propagation method may be:

where loss is the evaluation function, k₁As a vehicle behavior parameter, B_outBehavior classes for model output, B_labelFor the behavior class of the input model, k₂As vehicle trajectory parameter, P_out,iCoordinates of the ith track point for the future track, P_label,iThe coordinate of the ith track point of the future sample track of the vehicle is shown, and N is the Nth track point in the track of the vehicle.

When model training is carried out, k is used₂And setting the number to be 0, finishing training when the number of iterations reaches a preset number, and obtaining a target network model containing a first corresponding relation between the vehicle behavior sample data and the behavior category. Will k₁And if the iteration times reach the preset times, finishing training to obtain a target network model containing a second corresponding relation between the vehicle historical sample track and the vehicle future sample track. When k is₁And k₂And when the iteration times reach the preset times, finishing training to obtain a target network model containing a first corresponding relation between the vehicle behavior sample data and the behavior category and a second corresponding relation between the vehicle historical sample track and the vehicle future sample track.

Therefore, the initial network model is trained through the training mode, the target network model used for enabling the vehicle behavior sample data to be associated with the corresponding behavior type and/or enabling the vehicle historical sample track to be associated with the corresponding vehicle future sample track can be obtained, the vehicle behavior and the vehicle track can be predicted simultaneously through the target network model, the vehicle behavior or the vehicle track can be predicted respectively, and the prediction convenience is improved. This is one of the innovative points of the embodiments of the present invention.

As can be seen from the above, in this embodiment, the vehicle behavior data and the vehicle travel tracks corresponding to each other vehicle before the current time, which are acquired by the acquisition device of the own vehicle, may be determined, then the vehicle tracks of the vehicles corresponding to the multiple segments of vehicle travel tracks in the other vehicles may be merged, then the target vehicle whose movement distance meets the preset distance requirement within the preset time period may be screened out from the other vehicles, and finally the behavior category and/or the future travel track corresponding to the target vehicle may be predicted by the target network model, so that the vehicle behavior and the vehicle track may be predicted simultaneously, and the vehicle behavior or the vehicle track may be predicted separately, thereby improving the convenience of prediction.

In another embodiment of the present invention, on the basis of the method shown in fig. 1, after the step of predicting a future travel track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle travel track of the target vehicle, the method for predicting vehicle behavior and/or vehicle track provided by the embodiment of the present invention may further include:

and smoothing the future driving track.

Since the predicted future travel track is not actual vehicle data, it has a certain difference from the actual vehicle travel track, and in order to reduce this difference and make it closer to the actual vehicle travel track, it is necessary to smooth the future travel track.

The method for smoothing the future driving track may be as follows:

and for every adjacent three track points on the future driving track, determining a straight line based on every adjacent two track points in the three track points, determining whether the included angle between the two straight lines is within the vehicle motion included angle range constrained by the vehicle dynamics model, and if not, adjusting the coordinates of the track point positioned in the middle position in the three track points according to a preset adjusting algorithm until the included angle between the two straight lines is within the vehicle motion included angle range constrained by the vehicle dynamics model.

Because the vehicle has certain motion constraint in the actual driving process, in order to enable the future driving track to be closer to the actual vehicle driving track, the future driving track needs to be made to conform to the constraint of a vehicle dynamic model, namely for every adjacent three track points on the future driving track, a straight line is determined based on every adjacent two track points in the three track points, whether an included angle between the two straight lines is within the vehicle motion included angle range constrained by the vehicle dynamic model is determined, if yes, no processing is performed, and if not, the coordinates of the track point located in the middle position in the three track points are adjusted according to a preset adjusting algorithm until the included angle between the two straight lines is within the vehicle motion included angle range constrained by the vehicle dynamic model.

Therefore, the difference between the future driving track and the actual vehicle driving track is reduced in a smoothing processing mode, so that the future driving track is closer to the actual vehicle driving track, and the purpose of optimizing the future driving track is achieved.

Referring to fig. 5(a) -5 (c), fig. 5(a) shows a predicted future travel trajectory, fig. 5(b) shows an optimized future travel trajectory obtained by smoothing the future travel trajectory in fig. 5(a), and fig. 5(c) shows an optimal ideal future travel trajectory obtained by optimizing the future travel trajectory in fig. 5(a), and generally, an ideal future travel trajectory cannot be obtained.

Fig. 6 is a schematic structural diagram of a vehicle behavior and/or vehicle trajectory prediction apparatus according to an embodiment of the present invention. The apparatus may include:

the determining module 610 is configured to determine vehicle behavior data and a vehicle driving track corresponding to each other vehicle before the current time, which are acquired by the acquiring device of the own vehicle;

the merging module 620 is configured to merge multiple vehicle driving tracks when any other vehicle corresponds to the multiple vehicle driving tracks, where the multiple vehicle driving tracks are formed by one vehicle driving track having a break point;

the screening module 630 is configured to screen, from other vehicles, a target vehicle whose movement distance meets a preset distance requirement within a preset time period according to a vehicle running track corresponding to each other vehicle;

a prediction module 640, configured to predict, according to vehicle behavior data of the target vehicle, a behavior category corresponding to the target vehicle based on a target network model corresponding to a pre-established target scene type, and/or predict, according to a vehicle travel track of the target vehicle, a future travel track of the target vehicle based on a target network model corresponding to a pre-established target scene type, where the target scene type is determined according to real-time location information of the vehicle,

In another embodiment of the present invention, the merging module 620 may include:

In another embodiment of the present invention, the vehicle behavior data includes speed information, and the filtering module 630 may include:

In another embodiment of the present invention, the apparatus may further include a training module, where the training module is configured to train to obtain the target network model, and the training module may include:

In another embodiment of the present invention, the training submodule may include:

In another embodiment of the present invention, the apparatus may further include:

The above device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment, and for the specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and for specific description, reference may be made to the method embodiment section, which is not described herein again.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method of predicting vehicle behaviour and/or vehicle trajectory, comprising:

2. The method of claim 1, wherein the step of combining the plurality of vehicle driving tracks when any other vehicle corresponds to the plurality of vehicle driving tracks comprises:

3. The method of claim 1, wherein the vehicle behavior data includes speed information, and the step of screening the other vehicles for a target vehicle with a movement distance reaching a preset distance requirement within a preset time period according to the vehicle running track corresponding to each other vehicle comprises:

4. The method of claim 1, wherein the training process of the target network model is:

5. The method according to claim 4, wherein the step of training an initial network model to obtain a target network model by using the first type of vehicle sample data and/or the second type of vehicle sample data as model training data comprises:

6. The method according to claim 1, wherein after the step of predicting the future travel track of the target vehicle based on a target network model corresponding to a pre-established target scene type according to the vehicle travel track of the target vehicle, the method further comprises:

and smoothing the future driving track.

7. A vehicle behavior and/or vehicle trajectory prediction device, comprising:

8. The apparatus of claim 7, wherein the merging module comprises:

9. The apparatus of claim 7, wherein the vehicle behavior data includes speed information, the filtering module comprising:

10. The apparatus of claim 7, further comprising a training module for training the target network model, the training module comprising: