WO2023050749A1 - Track prediction method and apparatus, electronic device, and storage medium - Google Patents

Abstract

A track prediction method, comprising: obtaining running information of a target object at a first moment; determining a preset running grid matching the running mode of the target object, the preset running grid comprising at least one running sub-grid, the running sub-grid comprising position points of at least one reference object at a plurality of future reference moments, and each future reference moment being a moment after a first preset time period starting from a sample moment; determining, on the basis of the running information, a target running sub-grid where the target object is located at a second moment, the second moment being a moment after the first preset time period from the first moment; and determining a running track of the target object from the first moment to the second moment on the basis of the target running sub-grid and the running information. The track prediction method is quick and simple. Also provided are a track prediction apparatus, an electronic device, and a computer-readable storage medium storing the track prediction method.

Description

Trajectory prediction method, device, electronic equipment and storage medium

Cross References to Related Applications

This disclosure is based on the Chinese patent application with the application number 202111160491.2, the application date is September 30, 2021, and the application name is "trajectory prediction method, device, electronic equipment and storage medium", and claims the priority of the above-mentioned Chinese patent application, The entire contents of the above-mentioned Chinese patent applications are hereby incorporated by reference into this disclosure.

technical field

The present disclosure relates to the technical field of prediction, and relates to a trajectory prediction method, device, electronic equipment, and storage medium.

Background technique

Object behavior prediction is an important issue in the field of automatic driving. Specifically, it can be expressed as the prediction of the running trajectory of the objects perceived by the automatic driving system. The above-mentioned objects include but are not limited to cars, two-wheeled electric vehicles, bicycles, and pedestrians. Object behavior prediction is an important sub-module of the automatic driving system, and it is an important basis for the automatic driving system to perform scene understanding, behavior decision-making and motion planning.

Properly modeling the future trajectory of an object is complex. Because different objects run at different speeds and methods, in order to improve the pertinence and accuracy of predictions, it is necessary to customize the modeling method for each type of object, but this method is quite inefficient.

Contents of the invention

Embodiments of the present disclosure at least provide a trajectory prediction method, device, electronic equipment, and storage medium.

In a first aspect, an embodiment of the present disclosure provides a trajectory prediction method, the method is executed by an electronic device, and the method includes:

Obtain the running information of the target object at the first moment;

Determining a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one reference object at a plurality of future reference times The position point; the future reference time is the time after the sample time as the starting point, after the first preset duration; the reference object and the target object have the same operation mode;

Based on the operation information, determine the target operation sub-grid where the target object is located at a second moment; wherein, the second moment is a moment after the first preset time period elapses at the first moment;

Based on the target running sub-grid and the running information, the running trajectory of the target object from the first moment to the second moment is determined.

In this embodiment, the running trajectory of the target object is predicted by using the preset running grid that matches the running mode of the target object, which can fully consider the characteristics such as the running speed and mode of the target object, and improve the pertinence and accuracy of trajectory prediction. Accuracy; In addition, different objects use preset running grids for trajectory prediction, so the scheme in this aspect is generalizable, and there is no need to set different models or methods for different objects, making the scheme in this aspect High efficiency; at the same time, this aspect is not based on the point set when predicting the trajectory, but uses the grid. Compared with the point set, the dimension of the processed data is reduced, so the amount of data processed by the solution in this aspect is effectively reduced. Efficiency is improved.

In the second aspect, the embodiment of the present disclosure also provides a trajectory prediction device, including:

The information acquisition part is configured to acquire the operation information of the target object at the first moment;

The positioning preprocessing part is configured to determine a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one The position points of the reference object at a plurality of future reference moments; the future reference moment is the moment after the first preset time period starting from the sample moment; the reference object has the same operating mode as the target object;

The positioning part is configured to determine the target operation sub-grid where the target object is located at a second moment based on the operation information; wherein, the second moment is after the first preset time period elapses at the first moment moment;

The trajectory prediction part is configured to determine the trajectory of the target object from the first moment to the second moment based on the target operation sub-grid and the operation information.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.

In a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned first aspect, or any of the first aspects of the first aspect, may be executed. Steps in one possible implementation.

In the fifth aspect, an embodiment of the present disclosure further provides a computer program product, the computer program product includes a computer program or instruction, and when the computer program or instruction is run on a computer, the computer executes the above-mentioned first aspect , or a step in any possible implementation manner in the first aspect.

For the effect description of the above-mentioned trajectory prediction device, electronic equipment, and computer-readable storage medium, refer to the description of the above-mentioned image processing method.

In order to make the above objects, features and advantages of the embodiments of the present disclosure more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show embodiments consistent with the present disclosure, and are used together with the specification to illustrate the technical solutions of the embodiments of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 shows a flowchart of a trajectory prediction method provided by an embodiment of the present disclosure;

Fig. 2 shows a schematic diagram of road condition information provided by an embodiment of the present disclosure;

Fig. 3 shows a flow chart of determining a target running sub-grid provided by an embodiment of the present disclosure;

FIG. 4 shows a flow chart of generating a matching preset operation grid for a certain operation mode provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of a preset running grid provided by an embodiment of the present disclosure;

FIG. 6 shows a flow chart of determining a dithering sub-grid provided by an embodiment of the present disclosure;

Fig. 7 shows a schematic diagram of a trajectory prediction device provided by an embodiment of the present disclosure;

Fig. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

After research, it is found that when performing trajectory prediction, reasonable modeling is very complicated, and it is difficult to balance the generalization of the model, the specificity for different operating modes, and the post-processing of prediction stability, resulting in the current trajectory prediction accuracy and low efficiency. , The defect of poor stability.

The above defects are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present disclosure below should be the results of the inventor's work in this disclosure. Contributions made to the embodiments of the present disclosure during the embodiments.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In view of the above technical problems, the present disclosure provides a trajectory prediction method, device, electronic equipment, and storage medium. The disclosure utilizes a preset operating grid that matches the operating mode of the target object to predict the operating trajectory of the target object, which can fully Considering the characteristics of the target object's running speed and mode, the pertinence and accuracy of trajectory prediction are improved; in addition, different objects use the preset running grid for trajectory prediction, so the scheme in this aspect has generalization In addition, there is no need to set different models or methods for different objects, so that the solution in this aspect is more efficient; at the same time, the present disclosure is not based on point sets when predicting trajectory, but uses grids, which are processed relative to point sets The data dimension of is reduced, so the amount of data processed by the solution in this aspect is effectively reduced and the efficiency is improved.

The trajectory prediction method provided by the embodiments of the present disclosure will be described below by taking the executing subject as a device capable of computing as an example.

As shown in Figure 1, the embodiment of the present disclosure provides a trajectory prediction method, which may include the following steps:

S110. Obtain running information of the target object at the first moment.

The above-mentioned target object is the object that needs trajectory prediction. The running information may include road condition information of the location of the target object at the first moment and movement state information of the target object within a period of time from the first moment.

As shown in FIG. 2 , the above road condition information may include lane lines 201 around the target object, zebra crossing outline 202 , intersection outline 203 and other information. The above road condition information may exist in the form of pictures.

The above-mentioned motion state information may include information such as the position, speed, and orientation of the target object within a period of time. For example, when the first moment is the current moment and the aforementioned period of time is 3s, the motion state information includes information such as the position, speed, and orientation of the target object in the past 3s.

For example, the position of the target object is recorded as (x, y), the heading is recorded as heading, and the speed is recorded as speed. The latest historical 16-frame data is taken every 0.2s, and each row records the corresponding motion state information of one frame. Then A 16×4 matrix is obtained, which is the motion state information within 3s.

Exemplarily, the following steps can be used to obtain the running information of the target object at the first moment:

Firstly, it is determined that starting from the first moment, the moment of the second preset duration is passed forward to obtain the third moment; after that, acquiring the motion state information of the target object from the third moment to the first moment , and use the acquired running state information as the motion state information of the target object at the first moment; finally, based on the motion state information of the target object at the first moment and the target object at the first The road condition information of the location at the time is used to determine the running information of the target object at the first time.

Here, the motion state information of the target object at the first moment and the road condition information of the location of the target object at the first moment may be directly used as the running information of the target object at the first moment.

The above-mentioned third moment is earlier than the first moment, and the motion state information between the third moment and the first moment is used as the motion state information of the target object at the first moment for trajectory prediction, which can not only increase the number of participants in trajectory prediction The amount of information can improve the accuracy of trajectory prediction, and the operating status information during the period from the third moment to the first moment can reflect the operation law of the target object to a certain extent. The trajectory prediction can further improve the accuracy of trajectory prediction.

S120. Determine a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one reference object in multiple futures The position point of the reference time; the future reference time is the time after the first preset time period starting from the sample time. Wherein, the reference object and the target object have the same running mode.

Before this step is performed, a variety of preset operating grids are preset, and different preset operating grids correspond to different operating modes. A preset operation grid may correspond to at least one operation mode, and one operation mode may correspond to at least one type of object. The operating modes of different types of objects can be different, for example, a car has a turning radius, and a pedestrian has no turning radius and can turn freely.

The above-mentioned reference object and the target object have the same operation mode, so the location points corresponding to the reference object can be used to form a preset operation grid to predict the location information and operation trajectory of the target object.

Based on the above description, it can be seen that when performing this step, the operation mode of the target object can be determined first and multiple preset operation grids can be obtained; and then from the multiple preset operation grids, the selection matches the operation mode of the target object The default run grid.

Different objects have different characteristics such as running speed and mode, so different objects have different running modes, and different running modes directly affect the trajectory prediction, so the target is based on the preset running grid that matches the running mode of the target object Objects can be used for trajectory prediction, which can effectively improve the pertinence and accuracy of prediction.

The first preset duration may be determined according to the speed of different objects, the first preset duration corresponding to a faster object may be shorter, and the second preset duration corresponding to a smaller speed object may be longer. Exemplarily, when the target object is a car, the first preset duration may be set to 5s.

S130. Based on the operation information, determine the target operation sub-grid where the target object is located at a second moment; wherein, the second moment is a time after the first preset time period elapses from the first moment.

Each position point in the preset running grid is the reference object with the sample moment as the movement start time, and the movement end point at the future reference time, the target object takes the first moment as the movement start time, and the second moment as the movement end time, The position of the target object at the second moment is the movement end point of the target object. The duration between the sample moment and the future reference moment is equal to the duration between the first moment and the second moment, both of which are the above-mentioned first preset duration.

Since the running modes of the target object and the reference object are the same, the reference image can be used as a sample to predict the position or running trajectory of the target image. At the same time, since the prediction time interval corresponding to the position point in the preset operation grid (that is, the first preset duration) is equal to the prediction time interval corresponding to the target object, the preset operation grid can be used to predict the elapsed prediction time of the target object The position after the interval, and the running trajectory during it.

Exemplarily, the following steps may be used to determine the target operation sub-grid where the target object is located at the second moment: first, based on the operation information, determine the position information of the target object at the second moment; then, based on The location information determines the target operation sub-grid where the target object is located at the second moment.

The step of determining the target running sub-grid in the above steps can be realized by using a pre-trained neural network, as shown in Figure 3, the movement state information of the target object in the running information at the first moment and the target object at the first moment The road condition information of the location is input into the trained neural network, and the neural network processes the input data to determine the target operation sub-grid where the target object is at the second moment, and output the target operation sub-grid identification symbol.

The above-mentioned running state information may include information such as the speed, position, and orientation of the target object, and the above-mentioned road condition information may include information such as lane lines and zebra crossing outlines around the target object. The neural network can more accurately determine the location information of the target object at the second moment. According to the above-mentioned embodiment, it can be seen that the running sub-grid includes the position points of multiple reference objects, and the position points here can be considered as a kind of sample points. According to the position points in the running sub-grid or according to the position of the running sub-grid, combined The position information of the target object at the second moment can more accurately determine the target running sub-grid where the target object is located.

S140. Based on the target running sub-grid and the running information, determine the running trajectory of the target object from the first moment to the second moment.

The target operation sub-grid includes a plurality of position points. Based on the position information of each position point in the target operation sub-grid, the position information corresponding to the target operation sub-grid can be determined. The position information can be considered as the target object in the second Time-point location information. Afterwards, based on the terminal position information of the target object at the second moment and the running information of the target object at the first moment, the running trajectory of the target object from the first moment to the second moment can be predicted.

Exemplarily, according to the position information of each position point in the target operation sub-grid, the coordinate mean value corresponding to each position point may be determined, and the determined coordinate mean value may be used as the position information corresponding to the target operation sub-grid. Alternatively, the center point of the target running sub-grid is calculated, and the position information of the center point is used as the position information corresponding to the target running sub-grid.

Exemplarily, the destination position information of the target object at the second moment and the running information of the target object at the first moment can be input into a pre-trained neural network, and after processing by the neural network, the target object can be output from the first moment to the The running trajectory between the second moments.

Based on the position information of each position point in the target running sub-grid, the terminal position information of the target object at the second moment can be determined more accurately, and then based on the terminal position information and running information, the target object can be determined more accurately from the first The running trajectory between the second moment and the second moment.

In the above-mentioned embodiment, the running trajectory of the target object is predicted by using the preset running grid that matches the target object, which can fully consider the characteristics of the target object’s running speed, mode, etc., and improve the pertinence and accuracy of trajectory prediction; in addition, Different objects use preset running grids for trajectory prediction, so the scheme in this aspect is generalized, and there is no need to set different models or methods for different objects, making the scheme in this aspect more efficient; at the same time In this aspect, the trajectory prediction is not based on the point set, but the grid is used. Compared with the point set, the dimension of the processed data is reduced. Therefore, the amount of data processed by the solution in this aspect is effectively reduced and the efficiency is improved.

In some embodiments, as shown in Figure 4, the present disclosure also provides a step of generating a matching preset operation grid for a certain operation mode:

S410. Obtain a sample trajectory of each reference object among the multiple reference objects having the operation mode.

The above sample trajectories are the real running trajectories of the reference objects, and the perception system can be used to record the running trajectories of the reference objects in different scenarios. Reference objects can be cars, pedestrians, etc.

A running mode may correspond to a reference object, or may correspond to multiple reference objects.

S420. For each sample trajectory, determine a plurality of sample moments corresponding to the sample trajectory, determine a future reference moment corresponding to each sample moment based on the first preset duration, and determine a reference time in each future based on the sample trajectory At the reference moment, the sample position information of the reference object corresponding to the sample track.

For a certain sample trajectory, multiple sample moments can be selected by sliding. After the sample time is selected, a future reference time that is a future time relative to the sample time can be determined according to the first preset duration. After the future reference time is determined, the position information on the sample trajectory corresponding to the future reference time is used as the sample position information of the reference object.

S430. Based on the determined sample position information, determine the position point of each reference object at each corresponding future reference moment.

Exemplarily, the position point corresponding to the sample position information may be used as the position point of the reference object at the corresponding future reference moment.

S440. Based on each determined location point, generate a preset operation grid matching the operation mode.

Exemplarily, each location point can be transformed into the same coordinate system and drawn on a picture, and then the area with the location point in the picture is divided to obtain a preset running grid including at least one running sub-grid.

In this embodiment, using the sample trajectory of the reference object with the operation mode, a plurality of position points as possible samples can be determined more accurately, and the preset positions matching the operation mode can be generated more accurately by using the position points. Run grid.

In some embodiments, the above image can also be drawn in combination with the speed of the reference object at each position point. Combined with the speed at each position point, a preset running grid with more comprehensive and rich information can be generated, which in turn helps to improve the accuracy of estimation and prediction.

Exemplarily, the following steps can be used to combine the speed to generate a preset running grid matching a certain running mode:

First, the color of each location point is determined based on the velocity of the corresponding reference object at each location point.

Exemplarily, a plurality of speed intervals can be set in advance, and the color corresponding to each speed interval can be determined, and then, for each location point, according to the speed of the location point, the speed interval where the location point is located can be determined, and the speed interval The corresponding color is used as the color of the position point.

Afterwards, based on the position information and the color of each position point, a picture including each position point is generated.

Exemplarily, each location point is drawn according to the location information and color of each location point, and a picture including each location point is obtained.

Finally, based on the color and position information of each position point, according to preset conditions, the position points on the picture are segmented to obtain a preset running grid including at least one running sub-grid; wherein, the preset conditions At least one of the following is included; the number of location points in any running sub-grid is within a preset value range; the ratio of the location points in the running sub-grid to all the location points is greater than the preset ratio; The speed difference corresponding to different position points in the same running sub-grid is smaller than the preset value.

The above condition that the number of location points in any running sub-grid is within a preset value range can make the number of location points in different running sub-grids close to each other without being far apart. The above preset proportion can be set according to the actual application scenario, for example, it can be set to 90%. The preset proportion is set to be relatively large, so that most of the position points are within the preset operating grid, which can be more comprehensive and accurately predict where the target object is located to run the subraster. The speed difference corresponding to different position points in the same running sub-grid is smaller than the preset value, and the position points with the same color or similar colors can be divided into the same running sub-grid. Among them, the positions with similar speeds have similar colors.

As shown in Figure 5, the preset operating grid in the figure is obtained by segmenting the car as a reference object. The number of position points included in different operating sub-grids 501 in the figure is similar, and the position points with similar colors can be located in the same position. Run within the subgrid. The grid may be fan-shaped, and of course may also be in other shapes, which is not limited in this embodiment of the present disclosure.

The speeds of objects in different operating modes are different, so the number of position points included in the operating sub-grids matching different operating modes may be different, that is, the preset value ranges are different. For example, the following steps can be used to determine a The preset value range corresponding to the running sub-grid matching the running mode:

Determine the average speed of the object corresponding to the operation mode; and determine the preset value range corresponding to the operation mode based on the average speed of the object.

Using the preset value range determined by the average speed of the object can make the position of each position point in the same running sub-grid close, and using the same position to represent the position of each position point in the entire running sub-grid will not affect The accuracy of the position. Based on this, a position corresponding to a running sub-grid is used to represent the position of each position point in the running sub-grid. Compared with the point set, in trajectory prediction, the data processing dimension is reduced, which is conducive to prediction Stability postprocessing.

The foregoing embodiments can generate a preset operating grid matching the operating mode based on the position information, speed, and preset conditions of each position point, thereby improving the accuracy of trajectory prediction.

Since jitter may occur in the process of determining the target running sub-grid, the modeling method also needs to be conducive to the post-processing of prediction stability, because the object information obtained from the perception system is noisy, and the noise is easy to cause the prediction of the object’s trajectory. "Jitter" (recovery after an unreasonable change in the predicted trajectory), which can negatively affect the comfort and safety of an automated driving system. Therefore, in some embodiments, after the target running sub-grid is determined, it is also necessary to detect whether the target running sub-grid is a dithering sub-grid. When the target running sub-grid is a dithering sub-grid, the most recently determined non-jittering sub-grid is used as the final target running sub-grid and participates in trajectory prediction, and the target running sub-grid is a non-jittering sub-grid In the case of , take the target running sub-grid as the final target running sub-grid and participate in trajectory prediction.

Exemplarily, the following steps can be used to detect whether the target running sub-grid is a dithering sub-grid:

Firstly, a plurality of historical operation sub-grids determined before the target operation sub-grid is determined; a first historical operation sub-grid among the plurality of historical operation sub-grids is a non-jittering sub-grid.

The above-mentioned multiple historical operation sub-grids may be a plurality of consecutive target operation sub-grids determined before the target operation sub-grid is determined.

Afterwards, based on the plurality of historical running sub-grids, it is determined whether the target running sub-grid is a dithering sub-grid.

Exemplarily, in a case where the target running sub-grid is the same sub-grid as the first historical running sub-grid, it is determined that the target running sub-grid is a non-jittering sub-grid. When the target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of jittering sub-grids among the multiple historical running sub-grids is greater than a preset number , determine that the target running sub-grid is a non-jittering sub-grid. The target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of dithering sub-grids among the multiple historical running sub-grids is less than or equal to a preset number of In this case, it is determined that the target running sub-grid is a dithering sub-grid.

The target running sub-grid is the same sub-grid as the first historical running sub-grid, indicating that the running status of the target object has not changed. At this time, the determined target running sub-grid is considered accurate and is a non-jittering sub-grid. grid; when the number of jittering sub-grids in multiple historical running sub-grids is large, it means that the running state of the target object has changed, which is a change of normal motion state, not jittering. is a non-dithered subraster. When the number of jittering sub-grids in multiple historical running sub-grids is small, it means that the running status of the target object has not changed. If the target running sub-grid is not the same sub-grid as the first historical running A raster indicating that the target running subraster is determined to be inaccurate, and the subraster is dithered.

The first identified target running subraster is preset to a non-dithered subraster.

As shown in FIG. 6 , in some embodiments, the following steps can be used to achieve the above-mentioned purpose of determining whether the target operating sub-grid is a dithering sub-grid, and determining the final target operating sub-grid:

Step 1. After determining a target running sub-grid, judge whether the queue used to store the identifier of the target running sub-grid is empty; if it is empty, it means that the determined target running sub-grid is the first determined target Run the sub-grid, consider the target running sub-grid to be a non-jittering sub-grid, store the identifier of the target running sub-grid in the queue, and use the target running sub-grid as the final target running sub-grid .

Step 2. When the above queue is not empty, judge whether the identifier of the target running subgrid is the same as the first identifier stored in the queue; if they are the same, it indicates that the target running subgrid is the same as the first identifier stored in the queue. The running sub-grid corresponding to the identifier is the same running sub-grid. At this time, the target running sub-grid is a non-jittering sub-grid. There is no need to store the identifier of the target running sub-grid in the queue. This target run subraster can be used as the final target run subraster.

Step 3. If the identifier of the target running sub-grid is not the same as the first identifier stored in the queue, store the identifier of the target running sub-grid at the end of the queue, and determine the current stored in the queue. Whether the number of identifiers is greater than the given value; if it is not greater than the given value, it indicates that the target operating sub-grid is a jittering sub-grid. At this time, the operating sub-grid corresponding to the first identifier stored in the queue needs to be used as The final goal is to run subrasters.

Step 4. The format of the identifier stored in the current queue is greater than the given value, indicating that the motion state of the target object has changed, and the target running sub-grid is a non-jittering sub-grid. Identifiers other than the tail identifier are cleared. At this time, the identifier at the end of the queue becomes the identifier at the head of the queue, and the target running sub-grid is used as the final target running sub-grid.

The running sub-grid corresponding to the identifier stored in the queue is the historical running sub-grid in the above embodiment.

In the above-mentioned embodiment, before determining the target operation sub-grid, a plurality of historical operation sub-grids are determined, and it is possible to more accurately determine whether the target operation sub-grid is a jittering sub-grid; In the case of , taking the first historical operation sub-grid as the final target operation sub-grid can effectively reduce the impact of jitter on the trajectory prediction results. The dimension of data processing is improved, which can reduce the difficulty of post-processing of prediction stability and improve the efficiency of post-processing of prediction stability.

In the above-mentioned embodiment, it is considered that the possible future motion end points of the object (i.e. the end points of the running track) are distributed in an area composed of several grids, and each grid represents a possible running track, and accordingly, the future running of the object Trajectories can be modeled as a grid of preset runs. The size and number of grids and the trajectory represented by the grid can be customized according to the characteristics of the motion mode of different objects, so that the predictions of different objects are targeted and help to improve the accuracy of trajectory prediction. In the above-mentioned embodiment, the object trajectory prediction task is defined as a grid classification task. A grid can contain multiple possible motion end points, thereby reducing the task dimension of object behavior prediction, helping to suppress the jitter of the predicted trajectory, and improve the stability of the prediction. sex. The stability post-processing of the above-mentioned embodiment is quite simple and direct. It is only necessary to suppress the jitter of the fan-shaped grid classification prediction to stabilize the prediction of the trajectory of the object. Compared with the method of modeling the predicted trajectory with a point set, the post-processing difficulty Relatively smaller.

In the above embodiments, the pre-trajectory prediction of the object is changed to grid prediction, thereby transforming the high-dimensional regression task into a low-dimensional classification task, reducing task difficulty and improving prediction stability. At the same time, the above embodiments associate the prediction mode with the motion mode of the object while the scheme is generalizable, so that the prediction mode can be adaptively adjusted for different objects, which helps to improve the prediction accuracy. In addition, the prediction method in the above embodiment can be well connected with various stability post-processing methods, and the stability of the grid classification prediction can be improved through post-processing, thereby improving the stability of the object trajectory prediction.

Based on the same inventive concept, the embodiment of the present disclosure also provides a trajectory prediction device corresponding to the trajectory prediction method. Since the problem-solving principle of the device in the embodiment of the disclosure is similar to the above-mentioned trajectory method in the embodiment of the disclosure, the implementation of the device can be See method implementation.

As shown in FIG. 7 , it is a schematic diagram of the structure of a trajectory prediction device provided by an embodiment of the present disclosure, including:

The information acquiring part 701 is configured to acquire the running information of the target object at the first moment.

The positioning preprocessing part 702 is configured to determine a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one A position point of a reference object at multiple future reference moments; the future reference moment is a moment after a first preset time period starting from the sample moment; the reference object has the same operation mode as the target object.

The positioning part 703 is configured to determine the target operation sub-grid where the target object is located at a second moment based on the operation information; wherein, the second moment is the first preset time after the first moment later moments;

The trajectory prediction part 704 is configured to determine the running trajectory of the target object from the first moment to the second moment based on the target running sub-grid and the running information.

In some embodiments, when the positioning preprocessing part 702 determines the preset operating grid matching the operating mode of the target object, it is configured to:

Get multiple preset running grids;

From the plurality of preset operation grids, a preset operation grid matching the operation mode of the target object is screened; wherein, different preset operation grids correspond to different operation modes.

In some embodiments, when determining the target operation sub-grid where the target object is located at the second moment based on the operation information, the positioning part 703 is configured to:

determining position information of the target object at the second moment based on the running information;

Based on the location information, determine the target operation sub-grid where the target object is located at the second moment.

In some embodiments, the running information includes motion state information of the target object at the first moment and road condition information of the location of the target object at the first moment.

In some embodiments, when the information acquiring part 701 acquires the running information of the target object at the first moment, it is configured to:

Before determining the first moment, the second preset time interval from the first moment is the third moment;

Obtaining the movement state information of the target object from the third moment to the first moment, and using the obtained running state information as the movement state information of the target object at the first moment;

Based on the movement state information of the target object at the first moment and the road condition information of the location of the target object at the first moment, the running information of the target object at the first moment is determined.

In some embodiments, the positioning preprocessing part 702 is also configured to generate a matching preset operating grid for any operating mode:

Acquiring a sample trajectory of each reference object among the plurality of reference objects having the operating mode;

For each sample trajectory, a plurality of sample moments corresponding to the sample trajectory are determined, based on the first preset duration, a future reference moment corresponding to each sample moment is determined, and based on the sample trajectory, it is determined that at each future reference moment , the sample position information of the reference object corresponding to the sample trajectory;

Based on the determined sample position information, determine the position point of each reference object at each corresponding future reference moment;

Based on the determined position points, a preset running grid matching the running mode is generated.

In some embodiments, the positioning preprocessing part 702 is configured to:

A preset operating grid matching the operating mode is determined based on each determined position point and the speed of the corresponding reference object at each position point.

In some embodiments, when the positioning preprocessing part 702 determines a preset operating grid matching the operating mode based on each determined position point and the velocity of the corresponding reference object at each position point, it is configured to :

determining the color of each location point based on the velocity of the corresponding reference object at each location point;

Generate a picture including each location point based on the location information and color of each location point;

Based on the color and position information of each position point, according to preset conditions, the position points on the picture are segmented to obtain a preset running grid including at least one running sub-grid; wherein the preset conditions include the following At least one item; the number of location points in any running sub-grid is within the preset value range; the proportion of the location points in the running sub-grid in all the location points is greater than the preset proportion; the same running The speed difference corresponding to different position points in the sub-grid is smaller than the preset value.

In some embodiments, the positioning preprocessing part 702 is further configured to determine the preset value range:

determining an average velocity of the object corresponding to the mode of operation;

Based on the average speed of the object, the preset value range corresponding to the operation mode is determined.

In some embodiments, when determining the running trajectory of the target object from the first moment to the second moment based on the target running sub-grid and the running information, the trajectory prediction part 704 is configured to :

Based on the position information of each position point in the target running sub-grid, determine the terminal position information of the target object at the second moment;

Based on the terminal location information and the running information, the running trajectory of the target object from the first moment to the second moment is determined.

In some embodiments, after the trajectory prediction part 704 determines the target operation sub-grid where the target object is located at the second moment, and determines that the target object is located between the first moment and the second moment Before running the track, it is also configured as:

Obtaining multiple historical running sub-grids determined before determining the target running sub-grid; the first historical running sub-grid among the multiple historical running sub-grids is a non-jittering sub-grid;

determining whether the target operating sub-grid is a dithering sub-grid based on the plurality of historical operating sub-grids;

When it is determined that the target running sub-grid is a jittering sub-grid, the first historical running sub-grid is used as the final target running sub-grid.

In some embodiments, after determining the target running sub-grid and before determining the running track of the target object, the trajectory prediction part 704 is further configured to:

If it is determined that the target operating sub-grid is a non-jittering sub-grid, the determined target operating sub-grid is used as the final target operating sub-grid.

In some embodiments, the trajectory prediction part 704 is configured to:

If the target running sub-grid is the same sub-grid as the first historical running sub-grid, it is determined that the target running sub-grid is a non-jittering sub-grid.

In some embodiments, the trajectory prediction part 704 is configured to:

When the target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of jittering sub-grids among the multiple historical running sub-grids is greater than a preset number , determine that the target running sub-grid is a non-jittering sub-grid.

In some embodiments, when determining whether the target running sub-grid is a jittering sub-grid based on the plurality of historical running sub-grids, the trajectory prediction part 704 is further configured to:

The target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of dithering sub-grids among the multiple historical running sub-grids is less than or equal to a preset number of In this case, it is determined that the target running sub-grid is a dithering sub-grid.

For the description of the processing flow of each part in the device and the interaction flow between each part, reference may be made to the relevant description in the above method embodiment, and details are not described here again.

Based on the same technical idea, an embodiment of the present disclosure also provides an electronic device. Referring to FIG. 8 , it is a schematic structural diagram of an electronic device 800 provided by an embodiment of the present disclosure, including a processor 81 , a memory 82 , and a bus 83 . Wherein, the memory 82 is used to store execution instructions, including a memory 821 and an external memory 822; the memory 821 here is also called an internal memory, and is used to temporarily store calculation data in the processor 81 and exchange data with external memory 822 such as a hard disk. The processor 81 exchanges data with the external memory 822 through the memory 821. When the electronic device 800 is running, the processor 81 communicates with the memory 82 through the bus 83, so that the processor 81 executes the following instructions:

Obtain the operation information of the target object at the first moment; determine the preset operation grid matching the operation mode of the target object; the preset operation grid includes at least one operation sub-grid; the operation sub-grid Including the position point of at least one reference object at a plurality of future reference moments; the future reference moment is a moment after a first preset time period starting from the sample moment; based on the operation information, it is determined that the target object is at The target operation sub-grid at the second moment; wherein, the second moment is the moment after the first preset duration at the first moment; the reference object and the target object have the same operation mode; based on The target running sub-grid and the running information determine the running track of the target object from the first moment to the second moment.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the trajectory prediction method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the trajectory prediction method provided by the embodiments of the present disclosure includes a computer program or an instruction, and when the computer program or instruction is run on a computer, the computer executes the trajectory described in the above method embodiment For the steps of the prediction method, refer to the above method embodiments.

The computer program product can be realized by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

Those skilled in the art can clearly understand that for the convenience and brevity of description, for the working process of the above-described system and device, reference may be made to the corresponding process in the foregoing method embodiments. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure.

The aforementioned computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by the instruction execution device, and may be a volatile storage medium or a non-volatile storage medium. A computer readable storage medium may be, for example but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disk, hard disk, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), erasable Type programmable read-only memory (Erasable Programmable Read Only Memory, EPROM or flash memory), static random-access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disk Read Only Memory, CD-ROM) , Digital versatile discs (Digital versatile Disc, DVD), memory sticks, floppy disks, mechanically encoded devices, such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

Finally, it should be noted that the above-mentioned embodiments are only specific implementation modes of the present disclosure, and are used to illustrate the technical solutions of the embodiments of the present disclosure, rather than limiting them, and the protection scope of the embodiments of the present disclosure is not limited thereto Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those of ordinary skill in the art should understand that: within the technical scope disclosed in the embodiments of the present disclosure, any person skilled in the art can still understand the aforementioned embodiments Modifications or changes can be easily imagined in the technical solutions recorded, or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure. All should be covered within the scope of protection of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure should be based on the protection scope of the claims.

Industrial Applicability

The embodiment of the present disclosure acquires the operation information of the target object at the first moment; determines the preset operation grid matching the operation mode of the target object; the preset operation grid includes at least one operation sub-grid; the The running sub-grid includes the position points of at least one reference object at a plurality of future reference moments; the future reference moment is the moment after the sample moment as a starting point and after a first preset duration; the reference object and the target object Have the same running mode; use the preset running grid that matches the running mode of the target object to predict the running trajectory of the target object, which can fully consider the running speed and mode of the target object, and improve the pertinence and accuracy of trajectory prediction. accuracy.

Claims (18)

1. A trajectory prediction method, comprising:

   Obtain the running information of the target object at the first moment;

   Determining a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one reference object at a plurality of future reference times The position point; the future reference time is the time after the sample time as the starting point, after the first preset duration; the reference object and the target object have the same operation mode;

   Based on the operation information, determine the target operation sub-grid where the target object is located at a second moment; wherein, the second moment is a moment after the first preset time period elapses from the first moment;

   Based on the target running sub-grid and the running information, the running trajectory of the target object from the first moment to the second moment is determined.
2. The method according to claim 1, wherein said determining a preset operating grid matching the operating model of the target object comprises:

   Get multiple preset running grids;

   From the plurality of preset operation grids, a preset operation grid matching the operation mode of the target object is screened; wherein, different preset operation grids correspond to different operation modes.
3. The method according to claim 1 or 2, wherein said determining the target operation sub-grid where the target object is located at the second moment based on the operation information comprises:

   determining position information of the target object at the second moment based on the running information;

   Based on the location information, determine the target operation sub-grid where the target object is located at the second moment.
4. The method according to any one of claims 1 to 3, wherein the running information includes movement state information of the target object at the first moment and road condition information of the location of the target object at the first moment;

   The acquisition of the running information of the target object at the first moment includes:

   Determining that a moment separated from the first moment by a second preset duration before the first moment is a third moment;

   Obtaining the movement state information of the target object from the third moment to the first moment, and using the obtained running state information as the movement state information of the target object at the first moment;

   Based on the movement state information of the target object at the first moment and the road condition information of the location of the target object at the first moment, the running information of the target object at the first moment is determined.
5. The method according to claim 1, wherein the method further comprises: generating a matching preset operation grid for any operation mode;

   The generating a matching preset operating grid for any operation mode includes: acquiring a sample trajectory of each reference object among a plurality of reference objects having the operation mode;

   For each sample trajectory, determine a plurality of sample moments corresponding to the sample trajectory, determine a future reference moment corresponding to each sample moment based on the first preset duration, and determine a reference time in each future based on the sample trajectory At the reference moment, the sample position information of the reference object corresponding to the sample trajectory;

   Based on the determined sample position information, determine the position point of each reference object at each corresponding future reference moment;

   Based on each determined position point, a preset operation grid matching the operation mode is generated.
6. The method according to claim 5, wherein said determining a preset operation grid matching said operation mode based on each determined position point comprises:

   Based on each determined position point and the speed of the corresponding reference object at each position point, a preset operation grid matching the operation mode is determined.
7. The method according to claim 6, wherein, based on each determined position point and the speed of the corresponding reference object at each position point, determining a preset operation grid matching the operation mode comprises :

   determining the color of each location point based on the velocity of the corresponding reference object at each location point;

   Generate a picture including each location point based on the location information and color of each location point;

   Based on the color and position information of each position point, according to preset conditions, the position points on the picture are segmented to obtain a preset running grid including at least one running sub-grid; wherein the preset conditions include At least one of the following: the number of location points in any running sub-grid is within a preset value range; the ratio of the location points in the running sub-grid to all the location points is greater than the preset ratio; the same The speed difference corresponding to different position points in the running sub-grid is smaller than the preset value.
8. The method according to claim 7, wherein the method further comprises: determining the preset value range;

   The determination of the preset value range includes:

   determining an average speed of an object corresponding to the mode of operation;

   Based on the average speed of the object, the preset value range corresponding to the operation mode is determined.
9. The method according to any one of claims 1 to 8, wherein the operation of the target object from the first moment to the second moment is determined based on the target operation sub-grid and the operation information track, including:

   determining the terminal position information of the target object at the second moment based on the position information of each position point in the target running sub-grid;

   Based on the terminal location information and the running information, the running trajectory of the target object from the first moment to the second moment is determined.
10. The method according to any one of claims 1 to 9, wherein after determining the target operation sub-grid where the target object is located at the second moment, and after determining that the target object is from the first moment to the second moment Before the running trajectory between the two moments, it also includes:

    Before obtaining the target operation sub-grid, determine a plurality of historical operation sub-grids; the first historical operation sub-grid in the plurality of historical operation sub-grids is a non-jittering sub-grid;

    determining whether the target operating sub-grid is a dithering sub-grid based on the plurality of historical operating sub-grids;

    When it is determined that the target running sub-grid is a jittering sub-grid, the first historical running sub-grid is used as the final target running sub-grid.
11. The method according to claim 10, wherein the method further comprises:

    If it is determined that the target operating sub-grid is a non-jittering sub-grid, the determined target operating sub-grid is used as the final target operating sub-grid.
12. The method according to claim 10 or 11, wherein the determining whether the target running sub-grid is a jittering sub-grid based on the multiple historical running sub-grids comprises:

    If the target running sub-grid is the same sub-grid as the first historical running sub-grid, it is determined that the target running sub-grid is a non-jittering sub-grid.
13. The method according to claim 10 or 11, wherein the determining whether the target running sub-grid is a jittering sub-grid based on the multiple historical running sub-grids comprises:

    When the target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of jittering sub-grids among the multiple historical running sub-grids is greater than a preset number , determine that the target running sub-grid is a non-jittering sub-grid.
14. The method according to claim 10 or 11, wherein the determining whether the target running sub-grid is a jittering sub-grid based on the plurality of historical running sub-grids further comprises:

    The target running sub-grid is not the same sub-grid as the first historical running sub-grid, and the number of dithering sub-grids among the multiple historical running sub-grids is less than or equal to a preset number of In this case, it is determined that the target running sub-grid is a dithering sub-grid.
15. A trajectory prediction device, comprising:

    The information acquisition part is configured to acquire the operation information of the target object at the first moment;

    The positioning preprocessing part is configured to determine a preset running grid that matches the running mode of the target object; the preset running grid includes at least one running sub-grid; the running sub-grid includes at least one The position points of the reference object at a plurality of future reference moments; the future reference moment is the moment after the first preset time period starting from the sample moment; the reference object has the same operating mode as the target object;

    The positioning part is configured to determine the target operation sub-grid where the target object is located at a second moment based on the operation information; wherein the second moment is when the first moment passes through the first preset time after

    The trajectory prediction part is configured to determine the trajectory of the target object from the first moment to the second moment based on the target operation sub-grid and the operation information.
16. An electronic device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the steps of the trajectory prediction method according to any one of claims 1 to 14 are executed.
17. A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the trajectory prediction method according to any one of claims 1 to 14 are executed.
18. A computer program product comprising a computer program or instructions which, when run on a computer, causes the computer to carry out the locus of any one of claims 1 to 14 The steps of the prediction method.

Images