CN116811883A

CN116811883A - Obstacle movement track generation method, device and equipment and automatic driving vehicle

Info

Publication number: CN116811883A
Application number: CN202310868911.5A
Authority: CN
Inventors: 孙灏
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-07-14
Filing date: 2023-07-14
Publication date: 2023-09-29

Abstract

The disclosure provides a method, a device and equipment for generating a movement track of an obstacle and an automatic driving vehicle, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, the technical field of automatic driving and the technical field of intelligent transportation. The specific implementation scheme is as follows: respectively extracting characteristics of obstacle information and running environment information related to the vehicle to obtain obstacle characteristics and environment characteristics; fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics; carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic, and generating the movement track of the obstacle related to the obstacle. The method provided by the disclosure can improve the prediction accuracy of the movement track of the obstacle.

Description

Obstacle movement track generation method, device and equipment and automatic driving vehicle

Technical Field

The present disclosure relates to the field of artificial intelligence, and more particularly to the field of deep learning, automatic driving, and intelligent transportation.

Background

With the rapid development of the automatic driving technology, a vehicle with an automatic driving function can detect obstacles such as a vehicle surrounding running environment and other vehicles surrounding the vehicle through detection devices such as millimeter wave radars and depth cameras, and predict the behavior state of the obstacles in the future according to detection results, so that the vehicle is helpful to timely adjust the running state of the vehicle.

Disclosure of Invention

The present disclosure provides a method, apparatus, electronic device, storage medium, computer program product, and autonomous vehicle for obstacle motion trajectory generation.

According to an aspect of the present disclosure, there is provided a method for generating a motion trajectory of an obstacle, including: respectively extracting characteristics of obstacle information and running environment information related to the vehicle to obtain obstacle characteristics and environment characteristics; fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics; carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic, and generating the movement track of the obstacle related to the obstacle.

According to another aspect of the present disclosure, there is provided an obstacle movement trajectory generation device including: the feature extraction module is used for respectively extracting features of obstacle information and driving environment information related to the vehicle to obtain obstacle features and environment features; the first fusion module is used for fusing the barrier characteristics and the environment characteristics to obtain fusion characteristics; the second fusion module is used for carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and the obstacle movement track generation module is used for predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic and generating the obstacle movement track related to the obstacle.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method provided in accordance with an embodiment of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform a method provided according to an embodiment of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method provided according to embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided an autonomous vehicle including an electronic device provided according to an embodiment of the present disclosure.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 schematically illustrates an exemplary system architecture to which the obstacle movement trajectory generation method and apparatus may be applied according to an embodiment of the present disclosure.

Fig. 2A schematically illustrates a flowchart of an obstacle movement trajectory generation method according to an embodiment of the disclosure.

Fig. 2B schematically illustrates a schematic diagram of a deep learning model according to an embodiment of the present disclosure.

Fig. 3 schematically illustrates a schematic diagram of an encoder according to an embodiment of the present disclosure.

Fig. 4A schematically illustrates a schematic diagram of a first feature interaction network according to an embodiment of the present disclosure.

Fig. 4B schematically illustrates a schematic diagram of a first feature interaction network according to another embodiment of the present disclosure.

Fig. 5 schematically illustrates a schematic diagram of an intent feature interaction network in accordance with an embodiment of the present disclosure.

Fig. 6 schematically illustrates a schematic diagram of a second feature interaction network according to an embodiment of the present disclosure.

Fig. 7 schematically illustrates a schematic diagram of a decoder according to an embodiment of the present disclosure.

Fig. 8 schematically illustrates a block diagram of an obstacle movement trajectory generation device according to an embodiment of the disclosure.

Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement the obstacle motion trajectory generation methods provided by embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are taken, and the public order harmony is not violated.

The inventors found that a vehicle having an automatic driving function has low prediction accuracy for future behavior states of obstacles around the vehicle, resulting in difficulty in timely adjusting the running state of the vehicle, which would have a certain adverse effect on running safety and running efficiency of the vehicle.

Embodiments of the present disclosure provide a method, apparatus, electronic device, storage medium, computer program product, and autonomous vehicle for obstacle movement trajectory generation. The obstacle movement track generation method comprises the following steps: respectively extracting characteristics of obstacle information and running environment information related to the vehicle to obtain obstacle characteristics and environment characteristics; fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics; carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic, and generating the movement track of the obstacle related to the obstacle.

According to the embodiment of the disclosure, the fusion characteristic is obtained by fusing the obstacle characteristic and the driving environment characteristic, the obstacle intention characteristic and the fusion characteristic obtained from the preset initial obstacle intention information, and the obstacle characteristic and the driving environment characteristic are subjected to characteristic fusion to obtain the intention fusion characteristic related to the obstacle, so that the intention fusion characteristic can fully represent the obstacle information and the driving environment information related to the vehicle, and further the prediction precision of the movement track of the obstacle is improved. Meanwhile, the motion track related to the obstacle can be obtained under the condition that the pre-marked obstacle behavior track is not used, the dependence on the mark of the obstacle track is reduced, and the generation efficiency of the obstacle track is improved.

It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios. For example, in another embodiment, an exemplary system architecture to which the obstacle movement trajectory generation method and apparatus may be applied may include a vehicle, but the vehicle may implement the obstacle movement trajectory generation method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

As shown in fig. 1, a system architecture 100 according to this embodiment may include vehicles 101, 102, 103, a network 104, and a server 105. The network 104 is the medium used to provide communication links between the vehicles 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.

A user may interact with the server 105 over the network 104 using the vehicles 101, 102, 103 to receive or send messages, etc. The vehicles 101, 102, 103 may be equipped with detection devices such as millimeter wave radar, data processing devices such as image processing chips, and communication devices for information interaction with the server 105.

The vehicles 101, 102, 103 may be vehicles with autopilot capabilities, including but not limited to cars, coaches, vans, or drones, among others.

The server 105 may be a server that provides various services, such as a background management server (by way of example only) that provides support for a user to perform driving functions with the vehicles 101, 102, 103. The background management server can analyze and process the received data such as the user request and the like, and feed back the processing result to the vehicle.

It should be noted that, the obstacle movement trajectory generation method provided by the embodiment of the present disclosure may be generally executed by the vehicle 101, 102, or 103. Accordingly, the obstacle movement trajectory generation device provided by the embodiment of the present disclosure may also be provided in the vehicle 101, 102, or 103.

Alternatively, the obstacle movement trajectory generation method provided by the embodiments of the present disclosure may be generally executed by the server 105. Accordingly, the obstacle movement trajectory generation device provided by the embodiments of the present disclosure may be generally provided in the server 105. The obstacle movement trajectory generation method provided by the embodiments of the present disclosure may also be performed by a server or cluster of servers that are different from the server 105 and that are capable of communicating with the vehicles 101, 102, 103 and/or the server 105. Accordingly, the obstacle movement trajectory generation device provided by the embodiments of the disclosure may also be provided in a server or server cluster that is different from the server 105 and is capable of communicating with the vehicles 101, 102, 103 and/or the server 105.

It should be understood that the number of vehicles, networks, and servers in fig. 1 are merely illustrative. There may be any number of vehicles, networks, and servers, as desired for implementation.

As shown in fig. 2A, the obstacle movement trajectory generation method includes operations S210 to S240.

In operation S210, feature extraction is performed on obstacle information and travel environment information related to the vehicle, respectively, to obtain an obstacle feature and an environment feature.

And S220, fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics.

And S230, carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information.

In operation S240, the movement track of the obstacle related to the vehicle is predicted according to the intention fusion feature, and the movement track of the obstacle related to the obstacle is generated.

According to embodiments of the present disclosure, the obstacle may be another motor vehicle, a non-motor vehicle, a pedestrian, or the like around the vehicle. The obstacle information may include attribute information of a movement state, a position, and the like of the obstacle. Obstacle information may be perceived from a driving environment by a millimeter wave radar, a monocular camera, a depth camera, or the like, which are mounted on a vehicle, in-vehicle sensors. The driving environment information may include static environment information such as lane lines, stop lines, traffic lights, road topology, and the like. The information types of the obstacle information and the traveling environment information may include images, videos, point cloud data, and the like. The embodiment of the present disclosure does not limit the information types of each of the obstacle information and the travel environment information.

According to the embodiment of the present disclosure, the obstacle information and the driving environment information related to the vehicle may be respectively feature-extracted based on a neural network algorithm, and the obstacle feature and the environment feature may be extracted based on CNN (Convolutional Neural Networks, convolutional neural network), for example. But not limited thereto, the obstacle features and the environmental features may be extracted based on other types of neural network algorithms, as embodiments of the present disclosure are not limited thereto.

According to the embodiment of the disclosure, the obstacle characteristics and the environment characteristics can be fused based on the attention network algorithm so as to fully interact the obstacle characteristics and the environment characteristics, so that the interaction relevance of the information of the fused characteristics to the obstacle information and the driving environment information is improved, and the characterization precision of the obstacle information and the driving environment information is improved.

According to an embodiment of the present disclosure, the preset initial obstacle intention information may be obtained based on an arbitrary manner, for example, the initial obstacle intention information may be obtained based on a random generation manner, or the initial obstacle intention information may also be obtained based on an obstacle trajectory collected during a history period.

In one embodiment of the disclosure, the initial obstacle intention information may be obtained based on a random generation manner, and the intention fusion feature may be extracted based on a neural network algorithm, so as to reduce the degree of dependence on the data collected in the history stage, and improve the generation efficiency of the obstacle movement track.

According to the embodiment of the disclosure, the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature can be fused based on the attention mechanism, so that the full interaction between the driving environment information and the obstacle information and the behavior intention of the obstacle is realized, the obtained intention fusion feature can fully represent the correlation between the behavior intention and the driving environment information and the obstacle information, the subsequently generated obstacle movement track is improved, and the representing precision of the behavior intention of the obstacle is aimed at.

As shown in fig. 2B, the deep learning model 200 may be adapted to process obstacle information 201, travel environment information 202, and initial obstacle intention information 203, resulting in an obstacle motion trajectory 204 of an obstacle associated with the vehicle.

The deep learning model 200 may include an encoder 210, a first feature interaction network 220, a second feature interaction network 230, a decoder 240, and an intent feature interaction network 250. The obstacle information 201 and the driving environment information 202 may be input to the encoder 210, the obstacle characteristics and the environment characteristics may be output, and the encoder 210 may be constructed based on an attention network algorithm (e.g., a transform algorithm).

As shown in fig. 2B, the initial obstacle intent information 203 may also be input to the intent feature interaction network 250, outputting the obstacle intent features 203'. The obstacle intent feature 203' may characterize one or more behavioral intents of the obstacle, and accordingly, the initial obstacle intent information 203 may also include one or more initial obstacle intent information, which may be related to the behavioral intent of the obstacle.

The obstacle intent feature 203' may be input to the second feature interaction network 230 with the obstacle and environmental features output by the encoder 210, outputting intent fusion features. The intent fusion feature may be input to the decoder 240, outputting the obstacle movement trajectory 240.

It should be noted that, the deep learning model 200 shown in fig. 2B may be obtained after training according to a training method, for example, an initial deep learning model may be trained based on a supervised training method to adjust model parameters of the initial deep learning model, so as to obtain the trained deep learning model 200.

According to an embodiment of the present disclosure, the driving environment information may include road topology information and driving control signal information. The road topology information may be map information related to a driving environment of the vehicle, and the map information may include lane lines, traffic indication marks, main road and lane indication marks, and the like. The driving control signal information may be a control signal of a traffic signal lamp; such as a left turn traffic signal, a straight traffic signal, etc. The travel control signal information may also be a knowledge flag for indicating stopping or movement of the obstacle, such as a stop line flag or the like. It should be understood that the road topology information may be extracted to obtain a road feature, and the driving control signal information may be extracted to obtain a driving control signal feature.

As shown in fig. 3, the encoder 300 may include three feature extraction branches, the first feature extraction branch may include a first feature embedding layer 311, a first convolution layer 312, and a first attention network layer 313. The second feature extraction branch may include a second feature embedding layer 321, a second convolution layer 322, and a second attention network layer 323. The third feature extraction branch may include a third feature embedding layer 331, a third convolution layer 332, and a third attention network layer 333.

The obstacle information 301, the road topology information 302, and the travel control signal information 303 may be input to the first feature embedding layer 311, the second feature embedding layer 321, and the third feature embedding layer 331, respectively, and the first embedded feature, the second embedded feature, and the third embedded feature may be output. The first, second, and third embedded features are input to the first, second, and third convolution layers 312, 322, and 332, respectively, outputting an initial obstacle feature, an initial road feature, and an initial travel control signal feature. The initial obstacle feature, the initial road feature, and the initial travel control signal feature are input to the first, second, and third attention network layers 313, 323, and 333, respectively, and the obstacle feature 301', the road feature 302', and the travel control signal feature 303' are output.

According to the embodiment of the disclosure, by inputting the obstacle information 301, the road topology information 302 and the driving control signal information 303 to different feature extraction branches respectively, it is possible to implement the region differentiation feature extraction for different types of information perceived by the obstacle in the driving process, so as to improve the characterization accuracy of the obstacle feature, the road feature and the driving control signal feature for the respective information respectively, thereby improving the prediction accuracy of the following obstacle movement track.

According to an embodiment of the present disclosure, the obstacle information may include at least one of: obstacle size (e.g., length, width, etc.), obstacle type, obstacle identification, obstacle turn signal (e.g., left turn signal, right turn signal, turn signal double flashing signal, etc.), speed, direction, position, etc.

According to an embodiment of the present disclosure, the road topology information may include at least one of: road type, road steering type, whether it is an entry road, whether it is a bifurcation road, main road identification, auxiliary road identification, traffic prohibition identification, etc.

According to an embodiment of the present disclosure, the travel control signal information may include at least one of: signal light color, signal light indication direction (e.g., straight indication direction, left turn indication direction, right turn indication direction, turn around indication direction), stop line position, etc.

According to an embodiment of the present disclosure, performing feature extraction on obstacle information and running environment information related to a vehicle, respectively, obtaining an obstacle feature and an environment feature may include: determining a plurality of initial continuous sub-information with a target type from the obstacle information, wherein the plurality of initial continuous sub-information has a time sequence relation; performing differential processing on the plurality of initial continuous sub-information according to the time sequence relation among the plurality of initial continuous sub-information to obtain a plurality of differential sub-information; and extracting the characteristics of the plurality of differential sub-information and the obstacle information based on the attention mechanism to obtain the characteristics of the obstacle.

According to an embodiment of the present disclosure, the initial continuous sub-information having the target type may be, for example, sub-information having a time-series relationship among the speed, direction, position, etc. of the obstacle. The timing relationship between the plurality of initial continuous sub-information may be, for example, a position of the obstacle acquired at time t1 and a position of the obstacle acquired at time t2, where time t1 is earlier than time t 2. The differential processing is performed on the plurality of initial continuous sub-information, for example, the position of the obstacle acquired at the time t1 and the position of the obstacle acquired at the time t2 may be processed based on a differential algorithm, so as to obtain differential sub-information.

According to an embodiment of the present disclosure, the obstacle information may further include discretized initial discrete sub-information, and the same type of initial discrete sub-information may not have a timing relationship therebetween. The initial discrete sub-information may be, for example, an obstacle size (e.g., length, width, etc.), an obstacle type, an obstacle identification, an obstacle turn signal, etc.

According to an embodiment of the present disclosure, feature extraction may be performed on the plurality of differential sub-information and the obstacle information based on an attention mechanism, where the plurality of differential sub-information, the initial discrete sub-information, and the initial continuous sub-information are respectively input to different feature embedding sub-layers in the first embedding layer, so that feature embedding (embedding) is performed on the plurality of differential sub-information, the initial discrete sub-information, and the initial continuous sub-information respectively, to obtain first embedded sub-features corresponding to the plurality of differential sub-information, the initial discrete sub-information, and the initial continuous sub-information, and after the plurality of first embedded sub-features are spliced, first embedded features output by the first feature embedding layer are obtained. The state change condition of the information related to the time sequence relationship in the obstacle information can be fully captured by the deep learning model through carrying out differential processing on the plurality of initial continuous sub-information, the extraction capability of the state change rule in the obstacle information is improved, and the prediction accuracy of the follow-up obstacle movement track is further improved.

The characteristic embedding is respectively carried out on a plurality of differential sub-information, initial discrete sub-information and initial continuous sub-information in the barrier information, so that the fine granularity distinction of the barrier information can be realized, key characteristics in the barrier information can be effectively extracted, and the expression capability of the barrier characteristics can be improved.

According to an embodiment of the present disclosure, the travel control signal information may include initial continuous sub-information having a time sequence relationship, for example, may be a stop line position, a differential stop line position is obtained by performing differential processing on a plurality of stop line positions having a time sequence relationship, the differential stop line position, the stop line position and the signal indication direction are respectively input to different feature embedding sub-layers, and second embedding sub-features corresponding to the differential stop line position, the stop line position and the signal indication direction are obtained, and further the second embedding features may be obtained according to the second embedding sub-features.

According to the embodiment of the disclosure, the feature embedding can be further performed on the road position (post) and the road direction (head) in the road topology information, so as to obtain third embedded sub-features corresponding to the road position (post) and the road direction (head), and the third embedded features are obtained according to the third embedded sub-features.

According to an embodiment of the present disclosure, the obstacle features include a plurality of obstacle features including at least one target obstacle feature.

According to embodiments of the present disclosure, the target obstacle characteristic may be a characteristic characterizing an attribute of the target obstacle, and the target obstacle may be related to an obstacle movement trajectory, i.e. the vehicle needs to predict a future movement trajectory for the target obstacle.

According to an embodiment of the present disclosure, fusing obstacle features and environmental features, the obtaining the fused features includes: performing feature fusion on the target obstacle features and the road features to obtain first intermediate fusion features; performing feature fusion on the target obstacle feature and the driving control signal feature to obtain a second intermediate fusion feature; and obtaining the fusion characteristic according to the first intermediate fusion characteristic and the second intermediate fusion characteristic.

As shown in fig. 4A, the first feature interaction network 410 may include a first feature interaction layer 411 and a second feature interaction layer 412. The first feature interaction layer 411 and the second feature interaction layer 412 may each be constructed based on an attention network algorithm.

As shown in fig. 4A, the feature fusion may be performed by inputting the target obstacle feature 401 and the road feature 402 to the first feature interaction layer 411, so that the target obstacle feature 401 and the road feature 402 perform feature information interaction, and a first intermediate fusion feature 421 is obtained. The target obstacle feature 401 and the driving control signal feature 403 may be input to the second feature interaction layer 412 by feature fusion of the target obstacle feature and the driving control signal feature, so that the target obstacle feature 401 and the road feature 402 perform feature information interaction, and a second intermediate fusion feature 422 is obtained. The fusion feature may be obtained by splicing the first intermediate fusion feature 421 and the second intermediate fusion feature 422 to obtain the fusion feature 430.

According to the embodiment of the disclosure, according to the first intermediate fusion feature and the second intermediate fusion feature, the obtained fusion feature may also be based on a neural network algorithm, for example, a Multi-Layer Perceptron (MLP) to process the first intermediate fusion feature and the second intermediate fusion feature, so as to perform deep fusion on the first intermediate fusion feature and the second intermediate fusion feature, and improve the characterization accuracy of the fusion feature.

In accordance with an embodiment of the present disclosure, a plurality of obstacle features further includes adjacent obstacle features associated with the target obstacle feature.

According to embodiments of the present disclosure, the proximity obstacle feature may characterize an obstacle attribute of a proximity obstacle associated with the target obstacle. The adjacent obstacle may be an obstacle whose distance from the target obstacle is equal to or less than a preset distance threshold value, or may also be an obstacle in the same lane as the target obstacle. Embodiments of the present disclosure are not limited in the particular manner in which an approaching obstruction is determined.

According to an embodiment of the present disclosure, fusing the obstacle feature and the environmental feature, the obtaining the fused feature further includes: and carrying out feature fusion on the target obstacle features and the adjacent obstacle features to obtain a third intermediate fusion feature.

As shown in fig. 4B, the first feature interaction network 410' may include a first feature interaction layer 411, a second feature interaction layer 412, and a third feature interaction layer 413. Each of the first feature interaction layer 411, the second feature interaction layer 412, and the third feature interaction layer 413 may be constructed based on an attention network algorithm.

As shown in fig. 4B, the target obstacle feature 401 and the adjacent obstacle feature 403 may also be input to the third feature interaction layer 413, so that the target obstacle feature 401 and the adjacent obstacle feature 403 perform feature information interaction, and a third intermediate fusion feature 423 is obtained.

According to an embodiment of the present disclosure, obtaining the fusion feature may include: and obtaining the fusion characteristic according to the first intermediate fusion characteristic, the second intermediate fusion characteristic and the third intermediate fusion characteristic.

As shown in fig. 4B, the first intermediate fusion feature 421, the second intermediate fusion feature 422, and the third intermediate fusion feature 423 may also be stitched to obtain a fusion feature 430'.

According to the embodiment of the disclosure, according to the first intermediate fusion feature, the second intermediate fusion feature and the third intermediate fusion feature, the obtained fusion feature may also be based on a neural network algorithm, for example, a Multi-Layer Perceptron (MLP), to process the first intermediate fusion feature, the second intermediate fusion feature and the third intermediate fusion feature, so as to perform deep fusion on the first intermediate fusion feature, the second intermediate fusion feature and the third intermediate fusion feature, thereby improving the characterization accuracy of the fusion features.

According to an embodiment of the present disclosure, the initial obstacle intent information is an N-dimensional initial obstacle intent vector, the initial obstacle intent vector characterizing a behavioral intent of the obstacle, N being an integer greater than 1.

According to an embodiment of the present disclosure, the initial obstacle intention information may be a matrix obtained after the N-dimensional initial obstacle intention vector is spliced.

The obstacle movement track generation method may further include: and carrying out feature fusion on the N-dimensional initial obstacle intention vectors based on the attention mechanism to obtain the obstacle intention features.

According to embodiments of the present disclosure, N-dimensional initial obstacle intent vectors may be processed based on a multi-level attention network to facilitate deep fusion of the N-dimensional initial obstacle intent vectors such that the resulting obstacle intent features are able to fully characterize the behavioral intent of the obstacle.

It should be noted that, the initial obstacle intention vector may be a vector generated based on a random manner, the initial obstacle intention information is extracted by a training-derived intention feature interaction network, the derived obstacle intention feature may be an N-dimensional obstacle intention feature vector, and the N-dimensional obstacle intention feature vector may further accurately represent N behavioral intentions of the obstacle.

And carrying out feature fusion on the obtained obstacle intention feature, the fusion feature, the obstacle feature and the environment feature, so that obstacle intention feature vectors of all dimensions in the obtained intention fusion feature can fully fuse obstacle information and driving environment information, thereby improving the prediction accuracy of the obstacle movement track.

As shown in FIG. 5, the intent feature interaction network 510 may include a first intent feature embedding sub-layer 511, an attention fusion sub-layer 512, and a second intent feature embedding sub-layer 513. The initial obstacle intent information 501 may be a 4-dimensional initial obstacle intent vector, i.e., the initial obstacle intent information 501 includes initial obstacle intent vectors 5011, 5012, 5013, and 5014.

As shown in fig. 5, inputting the initial obstacle intent information 501 into the intent feature interaction network 510 first intent feature embedding sub-layer 511 may map the initial obstacle intent vectors 5011, 5012, 5013, and 5014 to each into a high dimensional space, resulting in initial obstacle intent features. The initial obstacle intent features are then input to the attention fusion sublayer 512, which may enable the initial obstacle intent vectors 5011, 5012, 5013, and 5014 to interact sufficiently with each other to learn the relationships between different behavioral intentions, outputting intermediate obstacle intent features. The intermediate obstacle intention feature is input to the second intention feature embedding sub-layer 513, so that the intermediate obstacle intention feature can be hidden into a low-dimensional space, the number of model parameters of the whole deep learning model is reduced, and the obstacle intention feature 502 is obtained.

According to an embodiment of the present disclosure, feature fusion of an obstacle intent feature with a fusion feature, an obstacle feature, and an environmental feature, the obtaining of the intent fusion feature includes: performing feature fusion on the obstacle intention feature and the fusion feature based on an attention mechanism to obtain a first intermediate intention feature; performing feature fusion on the barrier intention feature and a splicing result of the barrier feature and the environment feature based on an attention mechanism to obtain a second intermediate intention feature; and obtaining an intention fusion feature according to the first intermediate intention feature and the second intermediate intention feature.

According to the embodiment of the disclosure, the obstacle intention feature and the fusion feature can be fused based on the attention network algorithm to obtain a first intermediate intention feature. Correspondingly, feature fusion can be performed on the barrier intention feature and the splicing result of the barrier feature and the environment feature based on the attention network algorithm, so that a second intermediate intention feature is obtained.

As shown in fig. 6, the second feature interaction network 610 may include a first intent feature interaction layer 611 and a second intent feature interaction layer 612. The first intent feature interaction layer 611 and the second intent feature interaction layer 612 may each be constructed based on an attention network algorithm.

Feature fusion of the obstacle intent feature with the fusion feature based on the attention mechanism may be by inputting the obstacle intent feature 601 and the fusion feature 602 to the first intent feature interaction layer 611, outputting the first intermediate intent feature 621. Feature fusion is performed on the barrier intent feature and the splicing result of the barrier feature and the environment feature based on the attention mechanism, that is, the splicing result of the barrier feature 6031 and the environment feature 6032 and the barrier intent feature 601 are input to the second intent feature interaction layer 612, and the second intermediate intent feature 622 is output. The intent fusion feature may be obtained by stitching the first intermediate intent feature 621 and the second intermediate intent feature 622 to obtain the intent fusion feature 620, based on the first intermediate intent feature and the second intermediate intent feature.

According to the embodiment of the disclosure, the intention fusion feature is generated according to the obtained first intermediate intention feature and the second intermediate intention feature, so that the intention fusion feature can be fully learned into the driving environment information and the obstacle information, and the prediction accuracy of the subsequent movement track of the obstacle is improved.

According to an embodiment of the present disclosure, according to an intent fusion feature, performing motion trajectory prediction for an obstacle related to a vehicle, generating an obstacle motion trajectory related to the obstacle includes: determining an initial obstacle movement track related to the obstacle according to the intention fusion characteristic; processing the initial obstacle movement track and the intention fusion characteristic based on the attention mechanism to obtain track weight related to the initial obstacle movement track; and determining the obstacle movement track according to the initial obstacle movement track and the track weight.

According to the embodiment of the disclosure, the intention fusion characteristic can be processed through the track regression branch constructed based on the neural network algorithm so as to decode the behavior intention of the obstacle and obtain the initial obstacle movement track.

According to the embodiment of the disclosure, the initial obstacle movement track and the intention fusion feature can be processed through the weight prediction branch constructed based on the attention network algorithm, so that the track weight of the initial obstacle movement track can be determined under the condition that the behavior intention of the obstacle is further fully learned, the obstacle movement track can be obtained according to the track weight, and the prediction accuracy is improved.

According to an embodiment of the present disclosure, determining an initial obstacle motion trajectory related to an obstacle according to an intent fusion feature may include: respectively processing the intention fusion characteristics according to M track point detection modes to obtain initial track point information subsets corresponding to the M track point detection modes respectively, wherein the initial track point information subsets comprise at least one piece of initial track point information, and M is an integer greater than 1; and obtaining an initial obstacle movement track according to the M initial track point information subsets.

According to the embodiment of the disclosure, the M track point detection manners respectively process the intent fusion features, which may be based on M track point regression sub-branches obtained after training, so as to obtain M initial track point information subsets. The initial track point information may be initial track point coordinates of the obstacle movement track, and the initial track point information in the M initial track point information subsets is arranged according to time attribute information associated with the initial track point information, so as to obtain the initial obstacle movement track.

According to the embodiment of the disclosure, the intention fusion features are respectively processed through the M track point detection modes, so that the problem that the prediction precision of initial track point information with a later time sequence is lower in the obtained initial track point information subset can be avoided, the prediction precision of the initial track point information can be improved, and the prediction precision of the obstacle movement track obtained later can be further improved.

According to the embodiment of the disclosure, the initial obstacle motion trail includes M sub-motion trail, and the M initial trail point information subsets are in one-to-one correspondence with the M sub-motion trail.

According to embodiments of the present disclosure, the sub-motion trajectories may be generated based on the corresponding initial trajectory point information subsets, which may be processed based on a linear regression algorithm, for example, resulting in the corresponding sub-motion trajectories.

As shown in fig. 7, the decoder 700 may include a trajectory regression branch 710 and a weight prediction branch 720. The trace regression branch 710 may include a first trace point regression branch 711, a second trace point regression branch 712, a third trace point regression branch 713, and a fourth trace point regression branch 714. The trace point regression branch may be constructed based on the full connection layer (Fully Connected Layer).

The intent fusion feature 701 may be input to a first trajectory point regression branch 711, a second trajectory point regression branch 712, a third trajectory point regression branch 713, and a fourth trajectory point regression branch 714, respectively, to facilitate predicting an initial trajectory point information subset corresponding to each of the different sub-motion trajectories. And splicing the initial track point information according to the prediction time corresponding to each initial track point information in the 4 initial track point information subsets, so as to obtain an initial obstacle movement track 702.

For example, the first trajectory point regression branch 711, the second trajectory point regression branch 712, the third trajectory point regression branch 713, and the fourth trajectory point regression branch 714 may output initial trajectory point information from the 1 st prediction time to the 5 th prediction time, initial trajectory point information 6 to 10 from the 6 th prediction time to the 10 th prediction time, initial trajectory point information from the 11 th prediction time to the 15 th prediction time, and initial trajectory point information from the 16 th prediction time to the 20 th prediction time, respectively.

As shown in fig. 7, the initial obstacle movement trajectory and the intent fusion feature are processed based on the attention mechanism to obtain the trajectory weight related to the initial obstacle movement trajectory, which may be the trajectory fusion sub-layer 721 that inputs the initial obstacle movement trajectory 702 and the intent fusion feature 701 into the weight prediction branch 720, and outputs the trajectory weight feature. The trajectory fusion sublayer 721 may be constructed based on an attention network algorithm. The track weight feature is input to a weight prediction sublayer 722 constructed based on a multi-layer perceptron algorithm, and track weight 703 is output.

According to an embodiment of the present disclosure, the output trajectory weight 703 may be a predicted probability related to the initial obstacle movement trajectory 702, and the initial obstacle movement trajectory 702 may be determined as an obstacle movement trajectory in case the trajectory weight 703 is greater than a preset threshold.

According to the embodiment of the disclosure, in the case that the intent fusion feature is an N-dimensional intent fusion feature vector, the trajectory regression branch may output initial obstacle motion trajectories corresponding to the N-dimensional intent fusion feature vector, and process the N-dimensional intent fusion feature vector and the N initial obstacle motion trajectories according to the weight prediction branch to obtain trajectory weights corresponding to the N initial obstacle motion trajectories. At least one obstacle motion trajectory is determined from the N initial obstacle motion trajectories by trajectory weights.

As shown in fig. 8, the obstacle movement trajectory generation device 800 may include: the device comprises a feature extraction module 810, a first fusion module 820, a second fusion module 830 and an obstacle movement track generation module 840.

The feature extraction module 810 is configured to perform feature extraction on obstacle information and driving environment information related to the vehicle, respectively, to obtain an obstacle feature and an environment feature.

The first fusion module 820 is configured to fuse the obstacle feature and the environmental feature to obtain a fused feature.

The second fusion module 830 is configured to perform feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature, and the environmental feature to obtain an intention fusion feature, where the obstacle intention feature is determined according to preset initial obstacle intention information.

The obstacle movement track generation module 840 is configured to predict a movement track of an obstacle related to the vehicle according to the intent fusion feature, and generate an obstacle movement track related to the obstacle.

According to an embodiment of the present disclosure, the environmental features include road features and travel control signal features, the obstacle features include a plurality, and the plurality of obstacle features includes at least one target obstacle feature.

According to an embodiment of the present disclosure, a first fusion module includes: the device comprises a first intermediate fusion feature obtaining unit, a second intermediate fusion feature obtaining unit and a fusion feature obtaining unit.

The first intermediate fusion feature obtaining unit is used for carrying out feature fusion on the target obstacle feature and the road feature to obtain a first intermediate fusion feature.

And the second intermediate fusion characteristic obtaining unit is used for carrying out characteristic fusion on the target obstacle characteristic and the driving control signal characteristic to obtain a second intermediate fusion characteristic.

And the fusion characteristic obtaining unit is used for obtaining the fusion characteristic according to the first intermediate fusion characteristic and the second intermediate fusion characteristic.

According to an embodiment of the present disclosure, the first fusion module further comprises a third intermediate fusion feature obtaining unit.

And the third intermediate fusion feature obtaining unit is used for carrying out feature fusion on the target obstacle feature and the adjacent obstacle feature to obtain a third intermediate fusion feature.

According to an embodiment of the present disclosure, the fusion feature obtaining unit includes a fusion feature obtaining subunit.

The fusion feature obtaining subunit is configured to obtain a fusion feature according to the first intermediate fusion feature, the second intermediate fusion feature, and the third intermediate fusion feature.

According to an embodiment of the present disclosure, the second fusion module includes: a first intermediate intention feature obtaining unit, a second intermediate intention feature obtaining unit, and an intention fusion feature obtaining unit.

The first intermediate intention feature obtaining unit is used for carrying out feature fusion on the obstacle intention feature and the fusion feature based on the attention mechanism to obtain a first intermediate intention feature.

The second intermediate intention feature obtaining unit is used for carrying out feature fusion on the obstacle intention feature and the splicing result of the obstacle feature and the environment feature based on the attention mechanism to obtain a second intermediate intention feature.

The intention fusion feature obtaining unit is used for obtaining an intention fusion feature according to the first intermediate intention feature and the second intermediate intention feature.

According to an embodiment of the present disclosure, wherein the initial obstacle intention information is an N-dimensional initial obstacle intention vector, the initial obstacle intention vector characterizing a behavioral intention of the obstacle, N being an integer greater than 1.

The obstacle movement trajectory generation device further includes an obstacle intention feature acquisition module.

The obstacle intention feature obtaining module is used for carrying out feature fusion on the N-dimensional initial obstacle intention vectors based on the attention mechanism to obtain obstacle intention features.

According to an embodiment of the present disclosure, the obstacle movement trajectory generation module includes: an initial obstacle movement locus obtaining unit, a locus weight obtaining unit, and an obstacle movement locus obtaining unit.

And the initial obstacle movement track obtaining unit is used for determining an initial obstacle movement track related to the obstacle according to the intention fusion characteristic.

The track weight obtaining unit is used for processing the initial obstacle movement track and the intention fusion characteristic based on the attention mechanism to obtain track weights related to the initial obstacle movement track.

And the obstacle movement track obtaining unit is used for determining the obstacle movement track according to the initial obstacle movement track and the track weight.

According to an embodiment of the present disclosure, the initial obstacle movement trajectory obtaining unit includes an initial trajectory point information subset obtaining subunit and an initial obstacle movement trajectory obtaining subunit.

The initial track point information subset obtaining subunit is used for respectively processing the intention fusion characteristics according to the M track point detection modes to obtain initial track point information subsets corresponding to the M track point detection modes respectively, wherein the initial track point information subsets comprise at least one initial track point information, and M is an integer larger than 1.

The initial obstacle movement track obtaining subunit is used for obtaining an initial obstacle movement track according to the M initial track point information subsets.

According to an embodiment of the present disclosure, the feature extraction module includes an initial continuous sub-information determination unit, a differential sub-information obtaining unit, and a feature extraction unit.

An initial continuous sub-information determining unit for determining a plurality of initial continuous sub-information having a target type from the obstacle information, the plurality of initial continuous sub-information having a timing relationship therebetween.

The difference sub information obtaining unit is used for carrying out difference processing on the plurality of initial continuous sub information according to the time sequence relation among the plurality of initial continuous sub information to obtain a plurality of difference sub information.

And the feature extraction unit is used for carrying out feature extraction on the plurality of differential sub-information and the obstacle information based on the attention mechanism to obtain the obstacle feature.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the method as described above.

According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described above.

According to an embodiment of the present disclosure, a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.

According to an embodiment of the present disclosure, an autonomous vehicle includes an electronic device provided according to an embodiment of the present disclosure.

According to the embodiments of the present disclosure, an autonomous vehicle may be mounted with an electronic device capable of performing the obstacle movement trajectory generation method provided by the embodiments of the present disclosure to improve movement prediction accuracy for obstacles around the vehicle.

Fig. 9 shows a schematic block diagram of an example electronic device 900 that may be used to implement the obstacle motion trajectory generation methods provided by embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Various components in device 900 are connected to I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, or the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, an optical disk, or the like; and a communication unit 909 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 901 performs the respective methods and processes described above, for example, the obstacle movement trajectory generation method. For example, in some embodiments, the obstacle-motion trajectory generation method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 900 via the ROM 902 and/or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the obstacle movement trajectory generation method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the obstacle movement trajectory generation method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method for generating a motion trajectory of an obstacle, comprising:

respectively extracting characteristics of obstacle information and running environment information related to the vehicle to obtain obstacle characteristics and environment characteristics;

fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics;

carrying out feature fusion on the obstacle intention feature and the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and

And predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic, and generating the movement track of the obstacle related to the obstacle.

2. The method of claim 1, wherein the environmental features include road features and travel control signal features, the obstacle features include a plurality, and the plurality of obstacle features include at least one target obstacle feature;

wherein the fusing the obstacle feature and the environmental feature to obtain a fused feature includes:

performing feature fusion on the target obstacle feature and the road feature to obtain a first intermediate fusion feature;

performing feature fusion on the target obstacle feature and the driving control signal feature to obtain a second intermediate fusion feature;

and obtaining the fusion characteristic according to the first intermediate fusion characteristic and the second intermediate fusion characteristic.

3. The method of claim 2, wherein a plurality of the obstacle features further comprise adjacent obstacle features associated with the target obstacle feature;

wherein the fusing the obstacle feature and the environmental feature to obtain a fused feature further includes:

Performing feature fusion on the target obstacle feature and the adjacent obstacle feature to obtain a third intermediate fusion feature;

wherein, according to the first intermediate fusion feature and the second intermediate fusion feature, obtaining the fusion feature includes:

and obtaining the fusion characteristic according to the first intermediate fusion characteristic, the second intermediate fusion characteristic and the third intermediate fusion characteristic.

4. The method of claim 1, wherein the feature fusing the obstacle intent feature with the fusion feature, the obstacle feature, and the environmental feature, resulting in an intent fusion feature comprises:

performing feature fusion on the obstacle intention feature and the fusion feature based on an attention mechanism to obtain a first intermediate intention feature;

performing feature fusion on the obstacle intention feature and a splicing result of the obstacle feature and the environment feature based on an attention mechanism to obtain a second intermediate intention feature; and

and obtaining the intention fusion characteristic according to the first intermediate intention characteristic and the second intermediate intention characteristic.

5. The method of any of claims 1-4, wherein the initial obstacle intent information is an N-dimensional initial obstacle intent vector, the initial obstacle intent vector characterizing a behavioral intent of an obstacle, N being an integer greater than 1;

The obstacle movement track generation method further comprises the following steps:

and carrying out feature fusion on the N-dimensional initial obstacle intention vectors based on an attention mechanism to obtain the obstacle intention features.

6. The method of claim 1, wherein the predicting a motion trajectory of an obstacle associated with the vehicle based on the intent fusion feature, generating an obstacle motion trajectory associated with the obstacle comprises:

determining an initial obstacle movement track related to the obstacle according to the intention fusion characteristic;

processing the initial obstacle movement track and the intention fusion characteristic based on an attention mechanism to obtain track weights related to the initial obstacle movement track; and

and determining the obstacle movement track according to the initial obstacle movement track and the track weight.

7. The method of claim 6, wherein the determining an initial obstacle motion trajectory associated with the obstacle from the intent fusion feature comprises:

respectively processing the intention fusion characteristics according to M track point detection modes to obtain initial track point information subsets corresponding to the M track point detection modes respectively, wherein the initial track point information subsets comprise at least one piece of initial track point information, and M is an integer greater than 1; and

And obtaining the initial obstacle movement track according to the M initial track point information subsets.

8. The method of claim 7, wherein the initial obstacle motion trajectory comprises M sub-motion trajectories, the M initial trajectory point information subsets being in one-to-one correspondence with the M sub-motion trajectories.

9. The method of claim 1, wherein the feature extracting the obstacle information and the driving environment information related to the vehicle, respectively, to obtain the obstacle feature and the environment feature includes:

determining a plurality of initial continuous sub-information with a target type from the obstacle information, wherein a time sequence relation exists among the plurality of initial continuous sub-information;

performing differential processing on the plurality of initial continuous sub-information according to the time sequence relation among the plurality of initial continuous sub-information to obtain a plurality of differential sub-information; and

and extracting the characteristics of the plurality of differential sub-information and the obstacle information based on an attention mechanism to obtain the characteristics of the obstacle.

10. An obstacle movement trajectory generation device, comprising:

the feature extraction module is used for respectively extracting features of obstacle information and driving environment information related to the vehicle to obtain obstacle features and environment features;

The first fusion module is used for fusing the obstacle characteristics and the environment characteristics to obtain fusion characteristics;

the second fusion module is used for carrying out feature fusion on the obstacle intention feature, the fusion feature, the obstacle feature and the environment feature to obtain an intention fusion feature, wherein the obstacle intention feature is determined according to preset initial obstacle intention information; and

and the obstacle movement track generation module is used for predicting the movement track of the obstacle related to the vehicle according to the intention fusion characteristic and generating the obstacle movement track related to the obstacle.

11. The apparatus of claim 10, wherein the environmental features comprise road features and travel control signal features, the obstacle features comprise a plurality, the plurality of obstacle features comprise at least one target obstacle feature;

wherein, the first fusion module includes:

the first intermediate fusion feature obtaining unit is used for carrying out feature fusion on the target obstacle feature and the road feature to obtain a first intermediate fusion feature;

the second intermediate fusion feature obtaining unit is used for carrying out feature fusion on the target obstacle feature and the driving control signal feature to obtain a second intermediate fusion feature;

12. The apparatus of claim 11, wherein a plurality of the obstacle features further comprise adjacent obstacle features associated with the target obstacle feature;

wherein, the first fusion module further includes:

the third intermediate fusion feature obtaining unit is used for carrying out feature fusion on the target obstacle feature and the adjacent obstacle feature to obtain a third intermediate fusion feature;

wherein the fusion feature obtaining unit includes:

and the fusion feature obtaining subunit is used for obtaining the fusion feature according to the first intermediate fusion feature, the second intermediate fusion feature and the third intermediate fusion feature.

13. The apparatus of claim 10, wherein the second fusion module comprises:

the first intermediate intention feature obtaining unit is used for carrying out feature fusion on the obstacle intention feature and the fusion feature based on an attention mechanism to obtain a first intermediate intention feature;

the second intermediate intention feature obtaining unit is used for carrying out feature fusion on the obstacle intention feature and the splicing result of the obstacle feature and the environment feature based on an attention mechanism to obtain a second intermediate intention feature; and

An intention fusion feature obtaining unit, configured to obtain the intention fusion feature according to the first intermediate intention feature and the second intermediate intention feature.

14. The apparatus of any of claims 10-13, wherein the initial obstacle intent information is an N-dimensional initial obstacle intent vector, the initial obstacle intent vector characterizing a behavioral intent of an obstacle, N being an integer greater than 1;

the obstacle movement locus generation device further includes:

and the obstacle intention characteristic obtaining module is used for carrying out characteristic fusion on the N-dimensional initial obstacle intention vectors based on an attention mechanism to obtain the obstacle intention characteristics.

15. The apparatus of claim 10, wherein the obstacle motion trajectory generation module comprises:

an initial obstacle movement track obtaining unit, configured to determine an initial obstacle movement track related to the obstacle according to the intent fusion feature;

the track weight obtaining unit is used for processing the initial obstacle movement track and the intention fusion characteristic based on an attention mechanism to obtain track weights related to the initial obstacle movement track; and

16. The apparatus of claim 15, wherein the initial obstacle movement trajectory obtaining unit comprises:

an initial track point information subset obtaining subunit, configured to process the intent fusion features according to M track point detection modes respectively, to obtain initial track point information subsets corresponding to the M track point detection modes respectively, where the initial track point information subsets include at least one piece of initial track point information, and M is an integer greater than 1; and

and the initial obstacle movement track obtaining subunit is used for obtaining the initial obstacle movement track according to the M initial track point information subsets.

17. The apparatus of claim 16, wherein the initial obstacle motion trajectory comprises M sub-motion trajectories, the M initial trajectory point information subsets being in one-to-one correspondence with the M sub-motion trajectories.

18. The apparatus of claim 10, wherein the feature extraction module comprises:

an initial continuous sub-information determining unit configured to determine, from the obstacle information, a plurality of initial continuous sub-information having a target type, the plurality of initial continuous sub-information having a timing relationship therebetween;

The difference molecule information obtaining unit is used for carrying out difference processing on the plurality of initial continuous sub-information according to the time sequence relation among the plurality of initial continuous sub-information to obtain a plurality of difference sub-information; and

and the feature extraction unit is used for carrying out feature extraction on the plurality of differential sub-information and the obstacle information based on an attention mechanism to obtain the obstacle features.

19. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 10.

20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 10.

21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

22. An autonomous vehicle comprising the electronic device of claim 19.