WO2022178858A1 - 一种车辆行驶意图预测方法、装置、终端及存储介质 - Google Patents

一种车辆行驶意图预测方法、装置、终端及存储介质 Download PDF

Info

Publication number
WO2022178858A1
WO2022178858A1 PCT/CN2021/078239 CN2021078239W WO2022178858A1 WO 2022178858 A1 WO2022178858 A1 WO 2022178858A1 CN 2021078239 W CN2021078239 W CN 2021078239W WO 2022178858 A1 WO2022178858 A1 WO 2022178858A1
Authority
WO
WIPO (PCT)
Prior art keywords
target vehicle
lane
driving
road
lanes
Prior art date
Application number
PCT/CN2021/078239
Other languages
English (en)
French (fr)
Inventor
李飞
范时伟
李向旭
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2021/078239 priority Critical patent/WO2022178858A1/zh
Priority to CN202180055239.7A priority patent/CN116261649A/zh
Priority to EP21927294.5A priority patent/EP4286972A4/en
Publication of WO2022178858A1 publication Critical patent/WO2022178858A1/zh
Priority to US18/456,681 priority patent/US20230399023A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B60W60/0015—Planning or execution of driving tasks specially adapted for safety
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18154—Approaching an intersection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18159—Traversing an intersection
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18163—Lane change; Overtaking manoeuvres
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0097—Predicting future conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • B60W60/0027—Planning or execution of driving tasks using trajectory prediction for other traffic participants
    • B60W60/00274—Planning or execution of driving tasks using trajectory prediction for other traffic participants considering possible movement changes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/09626—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B60W2552/10—Number of lanes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B60W2552/53—Road markings, e.g. lane marker or crosswalk
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/35—Data fusion
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/40—High definition maps

Definitions

  • the present application relates to the field of automatic driving, and in particular, to a method, device, terminal and storage medium for predicting the driving intention of a vehicle.
  • the driving intention of the vehicle determines the driving route of the vehicle, which can avoid possible dangers to a certain extent.
  • the driving intention of the vehicle is relatively easy to predict because there are physical lane lines on the road to constrain the vehicle.
  • For the intersection scene due to the complex and changeable road structure, there is no clear entity lane driving line constraint.
  • the behavioral differences of vehicles are more obvious; due to the large differences in the structure of different intersections, the semantic-level intentions of fixed categories, such as going straight, turning left, turning right, and U-turn, are difficult to accurately describe the behavioral intention of the target vehicle; Lanes may be missing or inaccurate, so it is particularly difficult to accurately predict the driving intention of vehicles in intersection scenarios.
  • the embodiments of the present application provide a method, device, terminal, and storage medium for predicting the driving intention of a vehicle, which solve the problem of predicting the driving intention of a vehicle in an intersection scene.
  • an embodiment of the present application provides a vehicle driving intention prediction method, including: when a target vehicle drives to an intersection, acquiring map information of the intersection; wherein the map information of the intersection includes road layer information and lane information layer information, the road layer information includes a plurality of roads connected to the intersection, the lane layer information includes a plurality of lanes, and the lane is a part of the road in the intersection connected to the road; obtain the driving of the target vehicle information; based on the driving information of the target vehicle and the lane layer information, determine the driving characteristics of the target vehicle relative to each of the plurality of lanes; based on the driving information of the target vehicle and the road layer information to determine the driving characteristics of the target vehicle relative to each of the plurality of roads; based on the driving characteristics of the surrounding vehicles relative to the target vehicle, the target vehicle relative to each road in the plurality of roads The driving characteristics of the target vehicle and the driving characteristics of the target vehicle relative to each of the plurality of lanes, at least determine the lane intention of the target vehicle;
  • the driving characteristics based on the surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to each of the plurality of roads, and the driving characteristics of the target vehicle relative to each of the plurality of lanes Driving characteristics, at least determining the lane intention of the target vehicle, including:
  • the road intention of the target vehicle represents the probability distribution of the target vehicle leaving the intersection through each of the plurality of roads ;
  • the determining, at least based on the lane intention of the target vehicle, the driving intention of the target vehicle includes:
  • the driving intention of the target vehicle is determined based on the road intention of the target vehicle and the lane intention of the target vehicle.
  • the road intention and lane intention of the target vehicle are determined based on the driving characteristics of the surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to the road, and the driving characteristics of the target vehicle relative to the lane, and then based on the road intention of the target vehicle. and lane intention to determine the driving intention of the target vehicle.
  • the driving intention of the target vehicle can be determined, which can effectively and accurately represent the driving behavior of the target vehicle, and can adapt to the intersection scene of different structures and avoid predefined. Inaccuracy and ambiguity in the description of the target vehicle's driving behavior by fixed-category intents (eg, left turn, right turn, U-turn, etc.).
  • the driving characteristics of the surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to each road in the plurality of roads, and the relative and determining the road intention of the target vehicle and the lane intention of the target vehicle comprising: inputting the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle into a first
  • the interaction feature extraction network determines the interaction feature vector between the surrounding vehicle and the target vehicle, and the interaction feature vector between the surrounding vehicle and the target vehicle represents the influence of the surrounding vehicle on the target vehicle; an interaction feature vector of the vehicle with the target vehicle, driving characteristics of the target vehicle relative to each of the plurality of lanes, and the target vehicle relative to a road associated with each of the plurality of lanes
  • the first interactive feature extraction network includes at least a plurality of first feature extraction sub-networks and an interactive feature vector prediction network;
  • the driving characteristics of the vehicle are input into the first interactive feature extraction network, and the interaction feature vector between the surrounding vehicles and the target vehicle is determined, including: inputting the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle into the A plurality of first feature extraction sub-networks, determining the driving feature vector of each vehicle in the surrounding vehicles relative to the target vehicle; inputting the driving feature vector of each vehicle in the surrounding vehicles relative to the target vehicle into the interactive
  • the feature vector prediction network determines the interaction feature vector between the surrounding vehicle and the target vehicle.
  • the lane intention prediction network includes at least a first lane feature extraction sub-network, a second lane feature extraction sub-network, a first road feature extraction sub-network, a second road feature extraction sub-network and lane intention predicting a sub-network; the interaction feature vector of the surrounding vehicle and the target vehicle, the driving characteristics of the target vehicle relative to each of the plurality of lanes, and the target vehicle relative to the plurality of lanes
  • the driving characteristics of the road related to each lane in the input lane intention prediction network determine the lane intention of the target vehicle and the driving characteristic latent vector of the target vehicle relative to each lane of the plurality of lanes, including:
  • the driving characteristics of the target vehicle relative to each of the plurality of lanes are input into the first lane feature extraction sub-network, and the driving characteristic vector of the target vehicle relative to each of the plurality of lanes is determined;
  • the target vehicle is input to the second lane feature extraction sub-network with respect to the driving feature vector of each lane in the plurality
  • the feature latent state vector is input to the lane intention prediction sub-network to determine the lane intention of the target vehicle.
  • the driving feature vector of the target vehicle relative to each of the plurality of lanes is input into the second lane feature extraction sub-network, and the target vehicle is determined relative to the plurality of lanes.
  • the latent state vector of the driving feature of each lane in the lane including: a plurality of feature extraction windows in the second lane feature extraction sub-network compare the driving feature vector of the target vehicle relative to each of the plurality of lanes according to the following: Feature extraction is performed sequentially at the driving time; according to the driving feature vector of the target vehicle corresponding to the current feature extraction window relative to each lane in the multiple lanes and the latent state vector output by the previous feature extraction window, the target vehicle is determined.
  • a latent state vector with respect to the driving feature of each of the plurality of lanes is input into the second lane feature extraction sub-network, and the target vehicle is determined relative to the plurality of lanes.
  • the driving feature vector of the target vehicle relative to the road associated with each of the plurality of lanes is input into the second road feature extraction sub-network, and the relative relative to the target vehicle is determined.
  • the driving feature latent state vector of the road associated with each of the plurality of lanes includes: a plurality of feature extraction windows in the second road feature extraction sub-network
  • the driving feature vector of the road associated with each lane in the lanes is extracted according to the driving time sequence; the target vehicle corresponding to the current feature extraction window is relative to the road associated with each lane in the multiple lanes.
  • the driving feature vector and the latent state vector output from the previous feature extraction window determine the driving feature latent state vector of the target vehicle relative to the road associated with each of the plurality of lanes.
  • the road intention prediction network includes at least: the first road feature extraction sub-network, the second road feature extraction sub-network and a road intention prediction sub-network; travel of the target vehicle relative to each of the plurality of lanes corresponding to a plurality of lanes associated with each of the plurality of roads, relative to the travel characteristics of each of the plurality of roads
  • the feature latent state vector and the lane intention of the target vehicle corresponding to the lane associated with each road in the plurality of roads are input into a road intention prediction network, and the road intention of the target vehicle is determined, including: converting the target vehicle Input the first road feature extraction sub-network with respect to the driving characteristics of each road in the plurality of roads, and determine the driving characteristic vector of the target vehicle relative to each road in the plurality of roads; Input the second road feature extraction sub-network into the driving feature vector of each road in the plurality of roads, and determine the driving feature latent state vector of the target vehicle relative to each road in the plurality of roads; Lane intentions corresponding to la
  • the first feature extraction sub-network is constructed based on at least a multilayer perceptron network and a recurrent neural network; the first lane feature extraction sub-network and the first road feature extraction sub-network are based on at least a multilayer A perceptron network is constructed, the second lane feature extraction sub-network and the second road feature extraction sub-network are constructed based on at least a recurrent neural network; the interaction feature vector prediction network, the lane intention prediction sub-network and the road intention prediction The sub-networks are all constructed at least based on the attention mechanism network.
  • the construction of the prediction network in the embodiment of the present application does not require a CNN network with high complexity, but only needs a simple MLP and RNN network, the input data dimension is small, and the calculation efficiency is high.
  • determining the driving intention of the target vehicle based on the road intention of the target vehicle and the lane intention of the target vehicle includes: taking a probability maximum value from the road intention of the target vehicle The corresponding road is determined as the target road; the lane corresponding to the maximum probability in the lane intention of the target vehicle corresponding to the lane associated with the target road is determined as the target lane; based on the target road and the target lane, determine the lane. Describe the driving intention of the target vehicle.
  • the target vehicle intent category determined according to the map structure where the target vehicle is located in the embodiment of the present application is not fixed, and the proposed intent can effectively improve the accuracy of the behavior description of the target vehicle.
  • the target vehicle intention prediction is converted into the matching of the target vehicle motion state and map information, which is different from the fixed category intention classification in the existing method, and the target vehicle's lane intention and road intention assist each other, which improves the target vehicle's driving intention prediction. generalizability and accuracy.
  • the driving characteristics of the surrounding vehicle relative to the target vehicle include: one or more of a position characteristic, a speed characteristic and a head-facing characteristic of the surrounding vehicle in the first coordinate system,
  • the origin of the first coordinate system is the current position of the target vehicle
  • the first coordinate system is a Cartesian coordinate system
  • the y-axis of the first coordinate system is parallel to the length direction of the body of the target vehicle , and the positive direction of the y-axis is consistent with the front direction of the target vehicle.
  • the driving characteristics of the target vehicle relative to each of the plurality of roads include: a position characteristic of the target vehicle in the second coordinate system, and a distance characteristic of the target vehicle to the origin , the head-facing feature of the target vehicle, and one or more of the position of the target vehicle in the second coordinate system, the distance between the target vehicle and the origin, and the variation characteristics of the head-facing of the target vehicle with the driving moment, wherein all
  • the second coordinate system is a Cartesian coordinate system
  • the origin of the second coordinate system is determined based on the position of the exit of each road in the plurality of roads
  • the x-axis direction is based on the driving direction of each road in the plurality of roads.
  • the driving characteristics of the target vehicle relative to each of the plurality of lanes include: a position characteristic of the target vehicle in the third coordinate system, the head orientation of the target vehicle and the lane.
  • the plurality of lanes are determined based on a topology analysis of the plurality of roads.
  • the lane is automatically generated based on the topology analysis of multiple roads, and does not depend on the high-precision map, and the driving intention of the target vehicle can still be accurately predicted when the high-precision map information is missing.
  • an embodiment of the present application further provides a vehicle driving intention prediction device, the device includes: a first acquisition module, configured to acquire map information of the intersection when the target vehicle travels to the intersection;
  • the map information of the intersection includes road layer information and lane layer information, the road layer information includes a plurality of roads connected with the intersection, the lane layer information includes a plurality of lanes, and the lanes are connected to the intersection in the intersection.
  • the second acquisition module is used to acquire the driving information of the target vehicle;
  • the first feature extraction module is used to determine the relative position of the target vehicle based on the driving information of the target vehicle and the lane layer information.
  • the driving characteristics of each lane in the plurality of lanes; the second feature extraction module is configured to determine, based on the driving information of the target vehicle and the road layer information, that the target vehicle is relative to each of the plurality of roads.
  • the driving characteristics of the roads; the prediction module is configured to be based on the driving characteristics of surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to each of the plurality of roads, and the relative driving characteristics of the target vehicle relative to the target vehicle.
  • the driving characteristics of each lane in the plurality of lanes are used to determine at least the lane intention of the target vehicle; wherein, the surrounding vehicles are vehicles within the preset range of the target vehicle, and the lane intention of the target vehicle represents the target vehicle.
  • the probability distribution of the target vehicle leaving the intersection through each of the plurality of lanes; the determining module is configured to determine the driving intention of the target vehicle at least based on the lane intention of the target vehicle.
  • the prediction module is specifically configured to: based on the driving characteristics of surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to each of the plurality of roads, and the target vehicle relative to the plurality of roads.
  • the driving characteristics of each lane in the lanes are used to determine the road intention of the target vehicle and the lane intention of the target vehicle; wherein, the road intention of the target vehicle represents that the target vehicle passes through each road in the plurality of roads
  • the probability distribution of leaving the intersection; the determining module is specifically configured to: determine the driving intention of the target vehicle based on the road intention of the target vehicle and the lane intention of the target vehicle.
  • the prediction module is specifically configured to: input the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle into a first interactive feature extraction network, and determine the relationship between the surrounding vehicles and the target vehicle.
  • the interaction feature vector of the vehicle, the interaction feature vector of the surrounding vehicle and the target vehicle represents the influence of the surrounding vehicle on the target vehicle; the interaction feature vector of the surrounding vehicle and the target vehicle, the target vehicle.
  • the driving characteristics of the vehicle relative to each of the plurality of lanes and the driving characteristics of the target vehicle relative to the road associated with each lane of the plurality of lanes are input to a lane intent prediction network to determine the target vehicle the lane intention of the target vehicle and the driving feature latent vector of the target vehicle relative to each of the multiple lanes; compare the driving characteristics of the target vehicle relative to each of the multiple roads and the multiple lanes with the multiple lanes.
  • the driving feature latent vector of the target vehicle corresponding to the plurality of lanes associated with each of the plurality of roads with respect to each of the plurality of lanes and the lanes associated with each of the plurality of roads
  • the corresponding lane intention of the target vehicle is input into a road intention prediction network to determine the road intention of the target vehicle.
  • the first interactive feature extraction network includes at least a plurality of first feature extraction sub-networks and an interactive feature vector prediction network;
  • the driving characteristics of the vehicle are input into the first interactive feature extraction network, and the interaction feature vector between the surrounding vehicles and the target vehicle is determined, including: inputting the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle into the A plurality of first feature extraction sub-networks, determining the driving feature vector of each vehicle in the surrounding vehicles relative to the target vehicle; inputting the driving feature vector of each vehicle in the surrounding vehicles relative to the target vehicle into the interactive
  • the feature vector prediction network determines the interaction feature vector between the surrounding vehicle and the target vehicle.
  • the lane intention prediction network includes at least a first lane feature extraction sub-network, a second lane feature extraction sub-network, a first road feature extraction sub-network, a second road feature extraction sub-network and lane intention predicting a sub-network; the interaction feature vector of the surrounding vehicle and the target vehicle, the driving characteristics of the target vehicle relative to each of the plurality of lanes, and the target vehicle relative to the plurality of lanes
  • the driving characteristics of the road related to each lane in the input lane intention prediction network determine the lane intention of the target vehicle and the driving characteristic latent vector of the target vehicle relative to each lane of the plurality of lanes, including:
  • the driving characteristics of the target vehicle relative to each of the plurality of lanes are input into the first lane feature extraction sub-network, and the driving characteristic vector of the target vehicle relative to each of the plurality of lanes is determined;
  • the target vehicle is input to the second lane feature extraction sub-network with respect to the driving feature vector of each lane in the plurality
  • the feature latent state vector is input to the lane intention prediction sub-network to determine the lane intention of the target vehicle.
  • the driving feature vector of the target vehicle relative to each of the plurality of lanes is input into the second lane feature extraction sub-network, and the target vehicle is determined relative to the plurality of lanes.
  • the latent state vector of the driving feature of each lane in the lane including: a plurality of feature extraction windows in the second lane feature extraction sub-network compare the driving feature vector of the target vehicle relative to each of the plurality of lanes according to the following: Feature extraction is performed sequentially at the driving time; according to the driving feature vector of the target vehicle corresponding to the current feature extraction window relative to each lane in the multiple lanes and the latent state vector output by the previous feature extraction window, the target vehicle is determined.
  • a latent state vector with respect to the driving feature of each of the plurality of lanes is input into the second lane feature extraction sub-network, and the target vehicle is determined relative to the plurality of lanes.
  • the driving feature vector of the target vehicle relative to the road associated with each of the plurality of lanes is input into the second road feature extraction sub-network, and the relative relative to the target vehicle is determined.
  • the driving feature latent state vector of the road associated with each of the plurality of lanes includes: a plurality of feature extraction windows in the second road feature extraction sub-network
  • the driving feature vector of the road associated with each lane in the lanes is extracted according to the driving time sequence; the target vehicle corresponding to the current feature extraction window is relative to the road associated with each lane in the multiple lanes.
  • the driving feature vector and the latent state vector output from the previous feature extraction window determine the driving feature latent state vector of the target vehicle relative to the road associated with each of the plurality of lanes.
  • the road intention prediction network includes at least: the first road feature extraction sub-network, the second road feature extraction sub-network and a road intention prediction sub-network; travel of the target vehicle relative to each of the plurality of lanes corresponding to a plurality of lanes associated with each of the plurality of roads, relative to the travel characteristics of each of the plurality of roads
  • the feature latent state vector and the lane intention of the target vehicle corresponding to the lane associated with each road in the plurality of roads are input into a road intention prediction network, and the road intention of the target vehicle is determined, including: converting the target vehicle Input the first road feature extraction sub-network with respect to the driving characteristics of each road in the plurality of roads, and determine the driving characteristic vector of the target vehicle relative to each road in the plurality of roads; Input the second road feature extraction sub-network into the driving feature vector of each road in the plurality of roads, and determine the driving feature latent state vector of the target vehicle relative to each road in the plurality of roads; Lane intentions corresponding to la
  • the first feature extraction sub-network is constructed based on at least a multilayer perceptron network and a recurrent neural network; the first lane feature extraction sub-network and the first road feature extraction sub-network are based on at least a multilayer A perceptron network is constructed, the second lane feature extraction sub-network and the second road feature extraction sub-network are constructed based on at least a recurrent neural network; the interaction feature vector prediction network, the lane intention prediction sub-network and the road intention prediction The sub-networks are all constructed at least based on the attention mechanism network.
  • the determining module is specifically configured to: determine the road corresponding to the maximum probability in the road intention of the target vehicle as the target road; determine the road corresponding to the lane associated with the target road as the target road; The lane corresponding to the maximum probability in the lane intention of the target vehicle is determined as the target lane; based on the target road and the target lane, the driving intention of the target vehicle is determined.
  • the driving characteristics of the surrounding vehicle relative to the target vehicle include: one or more of a position characteristic, a speed characteristic and a head-facing characteristic of the surrounding vehicle in the first coordinate system,
  • the origin of the first coordinate system is the current position of the target vehicle
  • the first coordinate system is a Cartesian coordinate system
  • the y-axis of the first coordinate system is parallel to the length direction of the body of the target vehicle , and the positive direction of the y-axis is consistent with the front direction of the target vehicle.
  • the driving characteristics of the target vehicle relative to each of the plurality of roads include: position characteristics of the target vehicle in the second coordinate system, and distance characteristics between the target vehicle and the origin , the head-facing feature of the target vehicle, and one or more of the position of the target vehicle in the second coordinate system, the distance between the target vehicle and the origin, and the variation characteristics of the head-facing of the target vehicle with the driving moment, wherein all
  • the second coordinate system is a Cartesian coordinate system
  • the origin of the second coordinate system is determined based on the position of the exit of each road in the plurality of roads
  • the x-axis direction is based on the driving direction of each road in the plurality of roads.
  • the driving characteristics of the target vehicle relative to each of the plurality of lanes include: a position characteristic of the target vehicle in the third coordinate system, the head orientation of the target vehicle and the lane.
  • the plurality of lanes are determined based on a topology analysis of the plurality of roads.
  • the present application further provides a vehicle terminal, including a memory and a processor, where executable code is stored in the memory, and the processor executes the executable code to implement the first aspect or any of the first aspect.
  • a vehicle terminal including a memory and a processor, where executable code is stored in the memory, and the processor executes the executable code to implement the first aspect or any of the first aspect.
  • the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute the first aspect or any one of the first aspects may be implemented. method described in the method.
  • the present application also provides a computer program or computer program product, wherein the computer program or computer program product includes instructions, and when the instructions are executed, the above first aspect or any one of the first aspects may be implemented. method described in the method.
  • the present application may further combine to provide more implementation manners.
  • FIG. 1 is a functional block diagram of a vehicle provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a computer system provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a chip hardware structure provided by an embodiment of the present application.
  • FIG. 4 is a frame diagram of an automatic driving system provided by an embodiment of the present application.
  • FIG. 5 is a flowchart of a method for predicting a vehicle driving intention provided by an embodiment of the present application
  • Fig. 6 is a map schematic diagram of the intersection when the target vehicle travels to the intersection;
  • FIG. 7 is a schematic diagram of a third coordinate system determined based on the center line of the lane as a reference line;
  • FIG. 8 is a schematic structural diagram of an interaction feature vector extraction network between a target vehicle and other vehicles
  • FIG. 9 is a schematic structural diagram of a lane intention prediction network of a target vehicle.
  • FIG. 10 is a schematic structural diagram of a road intent prediction network of a target vehicle
  • FIG. 11 is a schematic structural diagram of a vehicle driving intention prediction device provided by an embodiment of the present application.
  • FIG. 1 is a functional block diagram of a vehicle provided by an embodiment of the present application.
  • vehicle 100 includes various subsystems, such as travel system 102 , sensor system 104 , control system 106 , one or more peripherals 108 and power supply 110 , computer system 112 , and user interface 116 .
  • vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. Additionally, each of the subsystems and elements of the vehicle 100 may be interconnected by wire or wirelessly.
  • the travel system 102 includes components that provide powered motion for the vehicle 100 .
  • travel system 102 may include engine 118 , energy source 119 , transmission 120 , and wheels 121 .
  • the engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a gasoline engine and electric motor hybrid engine, an internal combustion engine and an air compression engine hybrid engine.
  • Engine 118 converts energy source 119 into mechanical energy.
  • Examples of energy sources include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity, among others.
  • the energy source 119 may also provide energy to other systems of the vehicle 100 .
  • Transmission 120 may transmit mechanical power from engine 118 to wheels 121 .
  • Transmissions may include gearboxes, differentials, and driveshafts.
  • the transmission 120 may also include other devices, such as clutches.
  • the drive shaft may include one or more axles that may be coupled to one or more wheels 121 .
  • the sensor system 104 may include several sensors that sense information about the environment surrounding the vehicle 100 .
  • the sensor system 104 may include a positioning system 122 (which may be a GPS system, a Beidou system or other positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and camera 130.
  • the sensor system 104 may also include sensors of internal systems of the vehicle 100 (eg, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and corresponding characteristics (position, shape, orientation, velocity, etc.). This detection and identification is a critical function for the safe operation of the vehicle 100 .
  • a positioning system may be used to estimate the geographic location of the vehicle 100 .
  • the IMU 124 is used to sense position and orientation changes of the vehicle 100 based on inertial acceleration.
  • IMU 124 may be a combination of an accelerometer and a gyroscope.
  • Radar 126 may utilize radio signals to sense objects within the surrounding environment of vehicle 100 .
  • radar 126 may be used to sense the speed and/or heading of objects.
  • the laser rangefinder 128 may utilize laser light to sense objects in the environment in which the vehicle 100 is located.
  • the laser rangefinder 128 may include one or more laser sources, laser scanners, and one or more detectors, among other system components.
  • Camera 130 may be used to capture multiple images of the surrounding environment of vehicle 100 .
  • Camera 130 may be a still camera or a video camera.
  • Control system 106 controls the operation of the vehicle 100 and its components.
  • Control system 106 may include various elements including steering system 132 , throttle 134 , braking unit 136 , sensor fusion algorithms 138 , computer vision system 140 , route control system 142 , and obstacle avoidance system 144 .
  • the steering system 132 is operable to adjust the heading of the vehicle 100 .
  • steering system 132 may include a steering wheel system in one example.
  • the throttle 134 is used to control the operating speed of the engine 118 and thus the speed of the vehicle 100 .
  • the braking unit 136 is used to control the deceleration of the vehicle 100 .
  • the braking unit 136 may use friction to slow the wheels 121 .
  • the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical energy.
  • the braking unit 136 may also take other forms to slow the wheels 121 to control the speed of the vehicle 100 .
  • Computer vision system 140 may be operative to process and analyze images captured by camera 130 in order to identify objects and/or features in the environment surrounding vehicle 100 .
  • the objects and/or features may include traffic signals, road boundaries and obstacles, etc., and the computer vision system 140 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques.
  • SFM structure from motion
  • computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, and the like.
  • the route control system 142 is used to determine the travel route of the vehicle 100 .
  • the route control system 142 may combine data from the sensors 138 , the positioning system 122 , and one or more predetermined maps to determine a driving route for the vehicle 100 .
  • the obstacle avoidance system 144 is used to identify, evaluate and avoid or otherwise traverse potential obstacles in the environment of the vehicle 100 .
  • control system 106 may add or substitute components other than those shown and described. Alternatively, some of the components shown above may be reduced.
  • Peripherals 108 may include a wireless communication system 146 , an onboard computer 148 , a microphone 150 and/or a speaker 152 .
  • peripherals 108 provide a means for a user of vehicle 100 to interact with user interface 116 .
  • the onboard computer 148 may provide information to the user of the vehicle 100 .
  • User interface 116 may also operate on-board computer 148 to receive user input.
  • the onboard computer 148 can be operated via a touch screen.
  • peripheral devices 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle.
  • microphone 150 may receive audio (eg, voice commands or other audio input) from a user of vehicle 100 .
  • speakers 152 may output audio to a user of vehicle 100 .
  • Wireless communication system 146 may wirelessly communicate with one or more devices, either directly or via a communication network.
  • wireless communication system 146 may use 3G cellular communications, such as CDMA, EVDO, GSM/GPRS, or a 4G cellular network, such as LTE. Or 5G cellular communications.
  • the wireless communication system 146 may communicate with a wireless local area network (WLAN) using WiFi.
  • WLAN wireless local area network
  • the wireless communication system 146 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee.
  • Other wireless protocols, such as various vehicle communication systems, for example, wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include communication between vehicles and/or roadside stations public and/or private data communications.
  • DSRC dedicated short range communications
  • the power supply 110 may provide power to various components of the vehicle 100 .
  • the power source 110 may be a rechargeable lithium-ion or lead-acid battery.
  • One or more battery packs of such a battery may be configured as a power source to provide power to various components of the vehicle 100 .
  • power source 110 and energy source 119 may be implemented together, such as in some all-electric vehicles.
  • Computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable storage medium such as memory 114.
  • Computer system 112 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed fashion.
  • the processor 113 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor may be a dedicated device such as an ASIC or other hardware-based processor.
  • FIG. 1 functionally illustrates the processor, memory, and other elements of the computer 110 in the same block, one of ordinary skill in the art will understand that the processor, computer or memory may actually include storage in the same/different Multiple processors, computers or memories within a physical enclosure.
  • the memory may be a hard drive or other storage medium located within the housing of a different computer 110 .
  • reference to a processor or computer will be understood to include reference to a collection of processors or computers or memories that may or may not operate in parallel.
  • some components such as the steering and deceleration components, may each have their own processor that only performs calculations related to the function of a particular component.
  • a processor may be located remotely from the vehicle and communicate wirelessly with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking steps necessary to perform a single operation.
  • memory 114 may contain instructions 115 (eg, program logic) executable by processor 113 to perform various functions of vehicle 100 , including those described above.
  • Memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and/or control one or more of travel system 102 , sensor system 104 , control system 106 , and peripherals 108 . instruction.
  • memory 114 may store data such as road maps, route information, vehicle location, direction, speed, and other similar vehicle data, among other information. Such information may be used by the vehicle 100 and the computer system 112 during operation of the vehicle 100 in autonomous, semi-autonomous and/or manual modes.
  • a user interface 116 for providing information to or receiving information from a user of the vehicle 100 .
  • user interface 116 may include one or more input/output devices within the set of peripheral devices 108, such as wireless communication system 146, onboard computer 148, microphone 150, and speakers.
  • Computer system 112 may control functions of vehicle 100 based on input received from various subsystems (eg, travel system 102 , sensor system 104 , and control system 106 ) and from user interface 116 .
  • the computer system may utilize input from the control system 106 in order to control the steering unit 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144 .
  • computer system 112 is operable to provide control over various aspects of vehicle 100 and its subsystems.
  • one or more of these components described above may be installed or associated with the vehicle 100 separately.
  • memory 114 may exist partially or completely separate from vehicle 100 .
  • the above-described components may be communicatively coupled together in a wired and/or wireless manner.
  • FIG. 1 should not be construed as a limitation on the embodiments of the present application.
  • An autonomous vehicle traveling on a road can recognize objects in its surroundings to determine the current speed.
  • the objects may be other vehicles, traffic control equipment, or other types of objects.
  • each identified object may be considered independently, and based on the object's respective characteristics, such as its current speed, acceleration, distance from the vehicle, etc., may be used to determine the speed at which the autonomous vehicle is to adjust.
  • autonomous vehicle 100 or a computing device associated with autonomous vehicle 100 eg, computer system 112, computer vision system 140, memory 114 of FIG.
  • autonomous vehicle 100 For example, traffic, rain and snow, ice on the road, etc.
  • each identified object depends on the behavior of each other, so it is also possible to predict the behavior of a single identified object by considering all identified objects together.
  • the vehicle 100 can adjust its speed based on the predicted behavior of the identified objects.
  • an autonomous vehicle can determine what state the vehicle will need to adjust to (eg, accelerate, decelerate, or stop, etc.) based on the predicted behavior of the object.
  • other factors may also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road on which it is traveling, the curvature of the road, the proximity of static and dynamic objects, and the like.
  • the computing device may also provide instructions to modify the steering angle of the vehicle 100 to cause the autonomous vehicle to follow a specified trajectory and/or to maintain contact with objects in the vicinity of the autonomous vehicle (eg, safe lateral and longitudinal distances for vehicles in adjacent lanes on the road).
  • objects in the vicinity of the autonomous vehicle eg, safe lateral and longitudinal distances for vehicles in adjacent lanes on the road.
  • the above-mentioned vehicle 100 can be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, a recreational vehicle, an amusement park vehicle, a construction equipment, a tram, a golf cart, a train, a cart, etc.
  • the present application The embodiment is not particularly limited.
  • FIG. 2 is a schematic diagram of a computer system provided by an embodiment of the present application.
  • the computer system 112 shown in FIG. 2 includes a processor 113 coupled to a system bus.
  • the processor 113 may be one or more processors, each of which may include one or more processor cores.
  • a video adapter 107 which can drive a display 109, is coupled to the system bus 105.
  • System bus 105 is coupled to input-output (I/O) bus 113 through bus bridge 111 .
  • I/O interface 115 is coupled to the I/O bus.
  • I/O interface 115 communicates with various devices such as input device 117 (eg, keyboard, mouse, touch screen, etc.), media tray 121, eg, CD-ROM, multimedia interface, and the like.
  • Transceiver 123 (which can transmit and/or receive radio communication signals), camera 155 (which can capture still and moving digital video images) and external USB interface 125 .
  • the interface connected to the I/O interface 115 may be a USB interface.
  • the processor 113 may be any conventional processor, including a reduced instruction set computing (reduced instruction set computer, RISC) processor, a complex instruction set computing (complex instruction set computer, CISC) processor, or a combination thereof.
  • the processor may be a dedicated device such as an application specific integrated circuit (ASIC).
  • the processor 113 may be a neural network processor or a combination of the neural network processor and the above conventional processors.
  • the computer system 112 may be located remotely from the autonomous vehicle and may communicate wirelessly with the autonomous vehicle.
  • some of the processes described herein are performed on a processor disposed within the autonomous vehicle, others are performed by a remote processor, including taking actions required to perform a single operation.
  • Network interface 129 is a hardware network interface, such as a network card.
  • the network 127 may be an external network, such as the Internet, or an internal network, such as an Ethernet network or a virtual private natwork (VPN).
  • the network 127 may also be a wireless network, such as a WiFi network, a cellular network, and the like.
  • the hard disk drive interface is coupled to the system bus 105 .
  • the hard drive interface is connected to the hard drive.
  • Memory 114 is coupled to system bus 105 .
  • Data running on memory 114 may include operating system 137 and application programs 143 of computer system 112 .
  • the operating system includes a parser (shell) 139 and a kernel (kernel) 141 .
  • Parser 139 is an interface between the user and the operating system kernel.
  • the parser is the outermost layer of the operating system. The parser manages the interaction between the user and the operating system, waits for the user's input, interprets the user's input to the operating system, and processes various operating system output results.
  • Kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources. Interacting directly with hardware, the operating system kernel usually runs processes and provides inter-process communication, providing CPU time slice management, interrupts, memory management, IO management, etc.
  • the application program 143 includes programs related to controlling the autonomous driving of the vehicle, for example, programs that manage the interaction between the autonomous vehicle and obstacles on the road, programs that control the route or speed of the autonomous vehicle, and programs that control the interaction between the autonomous vehicle and other autonomous vehicles on the road.
  • Application 143 also exists on the system of software deploying server 149 .
  • computer system 112 may download application 143 from software deployment server 149 when application 147 needs to be executed.
  • the application program 141 may also be a program for controlling the self-driving vehicle to avoid collision with other vehicles and pass the intersection safely.
  • Sensor 153 is associated with computer system 112 .
  • Sensor 153 is used to detect the environment around computer system 112 .
  • the sensor 153 can detect animals, vehicles, obstacles, and pedestrian crossings, etc., and further sensors can detect the environment around the above-mentioned objects, such as animals, vehicles, obstacles, and pedestrian crossings.
  • the sensors may be cameras, infrared sensors, chemical detectors, microphones, and the like.
  • the processor 113 may predict the travel trajectories of other vehicles according to the surrounding road conditions and other vehicle conditions detected by the sensor 153 .
  • the processor 113 may input current driving information and current road information of other vehicles through a pre-trained neural network to obtain predicted driving trajectories of other vehicles.
  • the pre-trained neural network may be obtained through a large amount of training sample data.
  • the training data may include current driving information and road information of other vehicles detected, and driving information of other vehicles after a preset period of time.
  • the processor 113 can train a target model based on the training data, and the target model can be used to determine the predicted travel information of the vehicle according to the current travel information and road information of the vehicle.
  • the processor 113 processes the input current driving information and road information of the vehicle, and compares the output predicted driving information with the actual driving information of the vehicle after the preset time period, until the predicted driving information output by the processor 113 is consistent with the actual driving information of the vehicle.
  • the difference of the driving information is less than a certain threshold, so as to complete the training of the target model.
  • FIG. 3 is a schematic diagram of a chip hardware structure according to an embodiment of the present application.
  • the chip includes a neural network processor (network process units, NPU) 30 .
  • the chip can be set in the processor 113 as shown in FIG. 2 to complete the determination of the predicted travel trajectories of other vehicles.
  • the algorithms of each layer in the pre-trained neural network can be implemented in the chip as shown in Figure 3.
  • the vehicle driving intention prediction method in the embodiment of the present application may be executed in the arithmetic circuit 303 and/or the vector calculation unit 307 in the neural network processor 30, so as to obtain the driving intention of the target vehicle.
  • Each module and unit in the neural network processor 30 is briefly introduced below.
  • the neural network processor 30 is mounted on the main CPU (Host CPU) as a co-processor, and the main CPU assigns tasks.
  • the core part of the neural network processor 30 is the operation circuit 303.
  • the controller 304 in the neural network processor 30 can control the operation circuit 303 to extract the matrix data in the memory and perform multiplication operations.
  • the operation circuit 303 includes multiple processing units (Process Engine, PE).
  • arithmetic circuit 303 is a two-dimensional systolic array.
  • the arithmetic circuit 303 may also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition.
  • arithmetic circuit 303 is a general-purpose matrix processor.
  • the operation circuit fetches the data corresponding to the matrix B from the weight memory 302 and buffers it on each PE in the operation circuit.
  • the arithmetic circuit fetches the data of matrix A and matrix B from the input memory 301 to perform matrix operation, and stores the partial result or final result of the matrix in the accumulator 308 .
  • the vector calculation unit 307 can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and the like.
  • the vector computing unit 307 can be used for network computation of non-convolutional/non-fully connected layers (FC) layers in the neural network, such as pooling, batch normalization, local response Normalization (local response normalization), etc.
  • FC non-convolutional/non-fully connected layers
  • vector computation unit 307 can store the processed output vectors to unified buffer 306 .
  • the vector calculation unit 307 may apply a nonlinear function to the output of the arithmetic circuit 303, such as a vector of accumulated values, to generate activation values.
  • the vector computation unit 307 generates normalized values, merged values, or both.
  • the vector of processed outputs can be used as activation input to the arithmetic circuit 303, eg, for use in subsequent layers in a neural network.
  • Unified memory 306 is used to store input data and output data.
  • the weight data directly transfers the input data in the external memory to the input memory 301 and/or the unified memory 306 through the storage unit access controller (Direct Memory Access Controller, DMAC) 305, and stores the weight data in the external memory into the weight memory 302 , and store the data in the unified memory 306 into the external memory.
  • DMAC Direct Memory Access Controller
  • the BIU is the Bus Interface Unit, that is, the bus interface unit 310, which is used to realize the interaction between the main CPU, the DMAC and the instruction fetch memory 309 through the bus.
  • the instruction fetch memory (instruction fetch buffer) 309 connected with the controller 304 is used to store the instructions used by the controller 304;
  • the controller 304 is used for invoking the instructions cached in the memory 309 to control the working process of the operation accelerator.
  • unified memory 306, input memory 301, weight memory 302, and instruction fetch memory 309 may all be on-chip memories.
  • the external memory of the NPU can be the memory outside the NPU, and the external memory can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM) or other readable and writable memory.
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • HBM high bandwidth memory
  • FIG. 3 is only an exemplary illustration, and the present application is not limited thereto.
  • FIG. 4 is a frame diagram of an automatic driving system provided by the present application.
  • the automatic driving system includes a perception unit 41, a target fusion unit 42, a prediction unit 43, a planning unit 44, a control unit 45, a map unit 46 and a navigation unit.
  • the prediction unit 43 undertakes the sensing unit 41 , the target fusion unit 42 and the map unit 46 .
  • the prediction algorithm Through the prediction algorithm, the future behavior intention and trajectory of the obstacle can be given, and then output to the downstream planning module. It can help the self-vehicle to predict the future trajectory of the target of other vehicles and judge the importance of the target, which is conducive to planning and control in dangerous scenarios to take emergency safety measures to ensure the safety of the vehicle and avoid the occurrence of collisions.
  • the function of the sensing unit 41 is implemented based on the sensor system 104 in FIG. 1 or the sensor 153 in FIG. 2 , and senses environmental information around the vehicle 100 , such as obstacles (eg, other vehicles, pedestrians, animals, etc.), road information (eg, , pedestrian crossings, lane lines, traffic lights, etc.) and other information affecting driving, the target fusion unit 42 processes the environmental information around the vehicle sensed by the sensing unit 41, and outputs obstacle target information.
  • the map unit is stored in the memory 114 in FIG. 1 or FIG. 2 , and the prediction unit 43 predicts the behavior intention and future trajectory of the target vehicle according to the current map information and the target information sensed by the perception unit.
  • the planning unit 44 plans the travel route of the vehicle according to the prediction result of the prediction unit and/or the output information of the navigation unit 47
  • the control unit 45 controls the vehicle to travel according to the planned travel route according to the travel route planned by the planning unit.
  • the target fusion unit 42 , the prediction unit 43 , the planning unit 43 and the control unit 45 are all implemented in the processor in FIG. 1 or FIG. 2 .
  • real-time, accurate and reliable prediction of the intentions of other vehicles can help the vehicle to predict the traffic situation ahead, establish the traffic situation around the vehicle, and help to judge the importance of the targets of other vehicles around it, and screen the interaction.
  • the key goal is to facilitate the self-vehicle to plan the path in advance and safely pass through complex road conditions.
  • one scheme is a rule-based trajectory prediction algorithm, which mainly uses a motion model such as a uniform speed model to calculate the possible driving in a future period of time according to the position and motion speed information of the predicted target. trajectory.
  • the information used by this method is often limited, and there are few factors to be considered.
  • the prediction results often deviate greatly from the actual situation.
  • Another solution is the probability theory prediction method based on data statistics, such as DBN, HMM, etc.
  • data statistics such as DBN, HMM, etc.
  • the advantage is that a large amount of actual data is used for regular statistics. However, such methods often require enough samples to obtain more reliable statistical probability values.
  • the increase in the number of parameters during algorithm design will lead to a significant increase in computational complexity.
  • Another solution is the CNN-based intent prediction method, which needs to accumulate target observation data for a certain length of time, and use rasterized images to represent the target motion state (target position), high-precision map information (mainly including virtual lane lines), and then The CNN network is used to extract image features, and then predict the target intent and trajectory.
  • This kind of method converts the target state and high-precision map information into high-dimensional image data, which has high computational complexity and relies on observation data of a certain length of time, resulting in poor real-time and timeliness of such methods.
  • a high-precision map is a map that is accurate to the lane level.
  • auxiliary lines namely virtual lanes
  • High-precision maps are used in high-level autonomous driving systems as auxiliary tools or prior information to help autonomous vehicles better predict the behavioral intentions of other target vehicles.
  • the differences in vehicle behavior are more obvious; due to the large differences in the structure of different intersections, semantic-level intentions with fixed categories, such as going straight and turning left , right turn and U-turn, it is difficult to accurately describe the behavior intention of the target vehicle.
  • the virtual lanes in the HD map may be missing or inaccurate, so it is particularly difficult to accurately predict the high-level intention of the vehicle.
  • the intersection is divided into twelve intersections in a coordinate system with the target vehicle position as the origin and the target vehicle orientation as the y-axis.
  • Fan-shaped area which converts target intent prediction into a twelve-element classification problem, and the fan-shaped area containing the target real exit is the target real category.
  • this scheme uses rasterized images to represent target vehicle trajectories, high-precision map information and other vehicle target trajectories, of which the high-precision map information mainly includes lane line information;
  • the softmax classifier calculates the probability of twelve fan-shaped areas, and the exit in the fan-shaped area with the highest probability is the predicted target vehicle exit, that is, the target intent.
  • this scheme exists: because this scheme extracts the correlation features between the target vehicle and the virtual lane line information in the high-precision map information through the CNN network, when the virtual lane information is inaccurate or even missing, the intention of the target vehicle cannot be effectively predicted, so it has strong The problem of relying on high-definition map information.
  • this scheme converts the target vehicle intention prediction problem into a twelve-element classification problem.
  • the twelve-element category corresponds to different sectors in the target vehicle coordinate system. In practical applications, the target vehicle exit does not necessarily fall in a certain sector. Therefore, , the performance varies greatly in different structural intersection scenarios, and there is a lack of generalization performance.
  • the present application provides a vehicle driving intention prediction method, which firstly determines the road intention and lane intention of the target vehicle based on the driving characteristics of surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to the road, and the driving characteristics of the target vehicle relative to the lane. , and then determine the driving intention of the target vehicle based on the road intention and lane intention of the target vehicle.
  • the driving intention of the target vehicle can be determined, which can effectively and accurately represent the driving behavior of the target vehicle, and can adapt to the intersection scene of different structures and avoid predefined. Inaccuracy and ambiguity in the description of the target vehicle's driving behavior by fixed-category intents (eg, left turn, right turn, U-turn, etc.).
  • a method for predicting the driving intention of a vehicle will be described in detail below with reference to FIG. 5 .
  • the method may be performed by the vehicle in FIG. 1 .
  • the trained target model is configured in the vehicle.
  • the computer system of FIG. 2 and the chip structure of FIG. 3 are arranged in this vehicle.
  • FIG. 5 provides a flowchart of a method for predicting a driving intention of a vehicle according to an embodiment of the present application. As shown in FIG. 5 , a vehicle driving intention prediction method provided by the present application includes steps S501-S506.
  • step S501 the map information of the intersection is acquired.
  • the map information may be image information depicting spatial information such as roads, traffic conditions, and administrative areas in the real world, or may be map data of a customized virtual world. Map information can be used for ground traffic control, vehicle navigation, vehicle route planning, and the like.
  • the map information includes at least road layer information and lane layer information
  • the road layer information includes a plurality of roads connected to the intersection
  • the lane layer includes a plurality of lanes
  • the lanes are part of the road connected to the intersection in the intersection.
  • intersection refers to the area formed by the intersection of multiple roads
  • intersection represents the area formed by the intersection of multiple roads
  • the lane is the area formed by the intersection of multiple roads.
  • a road segment is used to constrain a vehicle to travel to one of several roads.
  • the road layer information includes a plurality of roads connected to the intersection, for example, road 1, road 2, road 3, and road 4.
  • Lane layer information includes multiple lanes connected to the road in the intersection, such as lane 11 and lane 12 connected to road 1, lane 21 and lane 22 connected to road 2, lane 31 and lane 32 connected to road 3, and lane 32 connected to road 3. 4 Connected lanes 41 and 42. It is easy to understand that Fig. 6 only shows the lanes where the current position of the target vehicle (the outer lane of road 2) may travel, that is to say, the lane starting from the current position of the target vehicle does not represent the map of the intersection. All lanes in the information.
  • the map information includes a high-precision map
  • the high-precision map may include a static high-precision map layer and a dynamic high-precision map layer.
  • the static high-precision map layer includes a road component layer, a lane layer, a road attribute layer and other map layers containing static information.
  • the lane layer may contain road detail information, such as lane line, lane center line, lane width, curvature, slope, heading, lane rules and other information.
  • the road layer component layer can contain road components such as traffic signs and pavement signs, such as recording the precise location and height of traffic lights, and so on.
  • the dynamic high-precision map layer may include a road congestion layer, a construction situation layer, a traffic accident layer, a traffic control layer, a weather layer, and other map layers containing dynamic traffic information.
  • the Construction Situation layer may contain information such as renovations, road marking wear and repainting, and traffic sign changes.
  • the way to obtain the map information of the intersection can be obtained by directly retrieving/downloading. For example, based on the location information of the target vehicle, the map information of the intersection corresponding to the location information stored in the memory is retrieved. Or, based on the position information of the target vehicle, the map information of the intersection corresponding to the position information is downloaded from the cloud server.
  • the map information does not include a high-precision map
  • the map information includes a conventional map, including road layer information, but does not include virtual lane lines in the intersection scene in the high-precision map.
  • the way to obtain the map information is to firstly retrieve the map information of the intersection corresponding to the location information stored in the memory based on the location information of the target vehicle, or download the location information corresponding to the location information from the cloud server based on the location information of the target vehicle. map information of the intersection. Then, use the method of topology analysis to determine the lane information of the lane layer. For example, referring to FIG.
  • the topological analysis obtains the start and end positions of lane 11, lane 12, lane 21, lane 22, lane 31, lane 32, lane 41, and lane 42, and then determines lane 11 based on cubic spline interpolation , Lane 12, Lane 21, Lane 22, Lane 31, Lane 32, Lane 41, Lane 42 lane lines.
  • the method of acquiring map information may also be: sensors collect environmental information around the vehicle, such as multiple road information at intersections, construct road layer information based on the multiple road information, and then determine the lane information of the lane layer based on topology analysis, and finally Build the map information of the intersection by yourself.
  • sensors collect environmental information around the vehicle, such as multiple road information at intersections, construct road layer information based on the multiple road information, and then determine the lane information of the lane layer based on topology analysis, and finally Build the map information of the intersection by yourself.
  • This application does not limit the acquisition method of map information, as long as the acquisition of map information can be achieved.
  • the vehicle driving intention prediction method of the embodiment of the present application can still be executed smoothly, does not depend on the high-precision map, and has better generalization.
  • step S502 travel information of the target vehicle and travel information of surrounding vehicles of the target vehicle are acquired.
  • the target vehicle can be understood as a vehicle that has a great influence on the driving of the own vehicle, for example, a vehicle in front of the own vehicle.
  • the surrounding vehicles of the target vehicle can be understood as other vehicles with a certain distance around the target vehicle, and the distance can be set by the user or by a technician, and can also be related to the sensing distance of the sensor of the vehicle.
  • the target vehicle and the surrounding vehicles of the target vehicle can be any vehicle in an autonomous driving vehicle, a non-autonomous driving vehicle, a new energy vehicle, a fuel vehicle, etc., which is not limited in this application.
  • the driving information of the target vehicle or surrounding vehicles includes information that can be sensed by the host vehicle and affects the driving intention of the target vehicle, for example, the location information, driving speed information, driving direction information, vehicle head of the target vehicle and surrounding vehicles orientation information, etc.
  • the driving characteristics of the surrounding vehicles of the target vehicle relative to the target vehicle can be determined according to the driving information of the target vehicle and the driving information of the surrounding vehicles of the target vehicle, that is, the target vehicle.
  • the interaction characteristics of the vehicle with other vehicles can be determined according to the driving information of the target vehicle and the driving information of the surrounding vehicles of the target vehicle, that is, the target vehicle.
  • the method for acquiring the interaction feature between the target vehicle and other vehicles may be extracted according to the following rule: extracting one of the position feature, speed feature, and head-facing feature of each vehicle in the surrounding vehicle in the first coordinate system, or
  • the origin of the first coordinate system is the current position of the target vehicle
  • the first coordinate system is a Cartesian coordinate system
  • the y-axis of the first coordinate system is parallel to the length direction of the body of the target vehicle
  • the positive direction of the y-axis is Consistent with the heading of the target vehicle.
  • step S503 based on the driving information of the target vehicle and the lane layer information, the driving characteristics of the target vehicle relative to each lane are determined, that is, the interaction characteristics between the target vehicle and each lane.
  • the interaction features of the target vehicle and each lane can be extracted according to the following rules: extracting the position features of the target vehicle in each third coordinate system, the angle formed by the head orientation of the target vehicle and the driving direction of the lane one or more of the characteristics and the position of the target vehicle in each third coordinate system, the angle formed by the head orientation of the target vehicle and the driving direction of the lane with the driving moment, wherein each of the The third coordinate system is the frenet coordinate system, the reference line of each third coordinate system is determined based on the center line of each lane, and the origin of each third coordinate system is based on the center line of each lane end point is determined.
  • the center line of the lane refers to a line formed by sequentially connecting the center points in the width direction of the lane from the start point to the end point of the lane.
  • the calculation method is as follows:
  • is the vector length.
  • step S504 based on the driving information of the target vehicle and the road layer information, the driving characteristics of the target vehicle relative to each road are determined, that is, the interaction characteristics between the target vehicle and each road.
  • the interaction features of the target vehicle and each road can be extracted according to the following rules: the position feature of the target vehicle in each second coordinate system, the distance feature of the target vehicle and the origin, the head facing feature of the target vehicle, and one or more of the position of the target vehicle in each second coordinate system, the distance between the target vehicle and the origin, and the change characteristics of the head orientation of the target vehicle with the driving moment, wherein each second coordinate system is a Cartesian coordinate
  • the origin of each second coordinate system is determined based on the position of the exit of each road, and the x-axis direction is determined based on the travel direction of each road.
  • the above-mentioned extraction rules for the interaction characteristics between the target vehicle and other vehicles, the interaction characteristics between the target vehicle and each lane, and the interaction characteristics between the target vehicle and each road are only examples of extraction rules, and other extraction rules can also be used.
  • the rules extract the interaction features between the target vehicle and other vehicles, the interaction features between the target vehicle and each lane, and the interaction features between the target vehicle and each road, which are not limited in this embodiment of the present application.
  • step S505 the road intention of the target vehicle and the lane intention of the target vehicle are determined based on the interaction characteristics between the target vehicle and other vehicles, the interaction characteristics between the target vehicle and each lane, and the interaction characteristics between the target vehicle and each road.
  • the road intention of the target vehicle represents the probability distribution of the target vehicle leaving the intersection through each road
  • the lane intention of the target vehicle represents the probability distribution of the target vehicle leaving the intersection through each lane
  • the interaction feature of each vehicle in the target vehicle and other vehicles is input into the interaction feature vector extraction network between the target vehicle and other vehicles, and the interaction feature vector between the target vehicle and other vehicles is extracted, and the vector represents the interaction between the target vehicle and the target vehicle. Influence of driving intention.
  • the interaction feature vector extraction network of the target vehicle and other vehicles includes a plurality of feature extraction sub-networks and an interaction feature vector prediction network.
  • the driving characteristics of each vehicle of other vehicles relative to the target vehicle are respectively input into multiple feature extraction sub-networks to obtain the driving feature vector of each vehicle in the other vehicles relative to the target vehicle; each vehicle in the other vehicles is relative to the target vehicle.
  • the driving feature vector of the target vehicle is input into the interactive feature vector prediction network to obtain the interactive feature vector between other vehicles and the target vehicle.
  • the feature extraction sub-network is constructed based on multi-layer perceptron (MLP) and recurrent neural network (RNN), and the interactive feature vector prediction network is based on attention mechanism network (Attention in Neural Network). Networks, ANN) construction.
  • MLP multi-layer perceptron
  • RNN recurrent neural network
  • ANN attention mechanism network
  • a j RNN(MLP(a j ))
  • a j represents the driving characteristics of any vehicle in other vehicles relative to the target vehicle
  • a j represents the driving characteristics of any vehicle in other vehicles relative to the target vehicle
  • the driving feature vector of the target vehicle, ⁇ j represents the weighted coefficient obtained by the normalized classification of the driving feature vector of any other vehicle relative to the target vehicle through the attention mechanism network, A feature vector that characterizes the interaction of the target vehicle with other vehicles.
  • the calculated interaction characteristic vector of the target vehicle and other vehicles needs to be taken into consideration for the prediction of the lane intention of the target vehicle.
  • the influence of the interaction characteristics between the target vehicle and the lane and the interaction characteristics between the target vehicle and the road associated with the lane on the lane intention prediction is also considered.
  • a lane-intent prediction network is constructed by inputting the interaction feature vector of the target vehicle with other vehicles, the interaction feature of the target vehicle with each lane, and the interaction feature of the target vehicle with the road associated with each lane as input trained
  • the lane intent prediction network determines the lane intent of the target vehicle and the latent state vector of the interaction feature of the target vehicle with each lane.
  • the road associated with each lane can be understood as the road connected to the lane, in other words, the road associated with each lane is the road to which the lane leads, for example, in FIG. 6, the road associated with the lane 11 The associated road is Road 1, and the road associated with Lane 21 is Road 2.
  • the interaction feature vector between the target vehicle and other vehicles, the interaction feature between the target vehicle and lane 11, and the interaction feature between the target vehicle and road 1 are input into the trained lane intent prediction network to get The probability of the target vehicle leaving the intersection through the lane 11 and the latent state vector of the interaction feature between the target vehicle and the lane 11.
  • the same method is used to obtain the probability of the target vehicle leaving the intersection through other lanes and the hidden state vector of the interaction feature between the target vehicle and other lanes.
  • the lane intention prediction network includes at least a first lane feature extraction sub-network, a second lane extraction sub-network, a first road feature extraction sub-network, a second road feature extraction sub-network and a lane intention prediction sub-network.
  • the interaction feature vector of the associated road is input into the second road feature extraction sub-network, and the hidden state vector of the interaction feature between the target vehicle and the road associated with each lane is extracted.
  • the target vehicle interaction feature vector with each lane, the target vehicle interaction feature vector with each lane, the target vehicle interaction feature latent vector with each lane, and the target vehicle interaction feature with the road associated with each lane The latent state vector is input to the lane intent prediction sub-network to determine the lane intent of the target vehicle.
  • the first lane feature extraction sub-network and the first road feature extraction sub-network are both constructed based on MLP
  • the second lane extraction sub-network and the second road feature extraction sub-network are both constructed based on RNN
  • the lane intent prediction sub-network is constructed based on the attention mechanism network.
  • the lane intent prediction algorithm of the target vehicle is as follows:
  • the interaction feature vector representing the interaction between the target vehicle at the current moment and the i-th lane is extracted by the MLP network and the interaction feature vector between the target vehicle at the current moment and the i-th lane,
  • the latent state vector representing the interaction feature between the target vehicle and the i-th lane at the previous moment The latent state vector representing the interaction feature between the target vehicle at the current moment and the i-th lane, Characterize the interaction characteristics between the target vehicle at the current moment and the jth road (the jth road is the road associated with the ith lane),
  • the latent state vector representing the interaction feature between the target vehicle at the current moment and the jth road, ⁇ ji represents The probability of the target vehicle leaving the intersection from the ith lane obtained after inputting the lane intention prediction sub-network, that is, the lane intention of
  • a road intent prediction network is constructed, and the interaction features of the target vehicle and each road, the interaction features of the target vehicle and each lane corresponding to the multiple lanes associated with each road, and the latent state vectors of each lane are combined with each road.
  • the lane intention of the target vehicle corresponding to the lane associated with each road is input into the road intention prediction network to determine the road intention of the target vehicle.
  • the lanes associated with each road can be understood as the lanes connected to each road, that is, the lanes leading to the road.
  • the lanes associated with road 1 are at least lane 11 and lane 12
  • the lanes associated with road 2 are at least lane 21 and lane 22 .
  • the road intent prediction network includes at least a first road feature extraction sub-network, a second road feature extraction sub-network and a road intent prediction sub-network.
  • the lane associated with the jth road has a total of k lanes, including lane 1, lane 2, and lane k, and the interaction features of lane 1, lane 2, and lane k are obtained from the lane intent network.
  • the state vector is weighted, and the result of the weighting process is spliced to obtain the latent state fusion vector of the lane associated with the jth road.
  • the latent fusion vectors of lanes associated with other roads are obtained in a similar way.
  • the interaction feature vector of the target vehicle and each road and the latent fusion vector of the lanes associated with each road are input into the road intent prediction sub-network to determine the road intent of the target vehicle.
  • the first road feature extraction sub-network is constructed based on MLP
  • the second road feature extraction sub-network is constructed based on RNN
  • the road intent prediction sub-network is constructed based on the attention mechanism network.
  • the road intent prediction algorithm of the target vehicle is as follows:
  • step S506 the driving intention of the target vehicle is determined based on the road intention of the target vehicle and the lane intention of the target vehicle.
  • the final driving intention of the target vehicle is determined, and the future driving trajectory of the target vehicle is characterized, that is, the driving trajectory of the target vehicle leaving the intersection.
  • the target road is determined according to the road corresponding to the maximum probability in the road intention of the target vehicle; then the lane corresponding to the maximum probability in the lane intention of the target vehicle corresponding to the lane associated with the target road is determined as the target Lane; finally, based on the target road and the target lane, the driving intention of the target vehicle is determined. That is, the target lane is taken as the driving trajectory of the target vehicle, and the target road is taken as the road the target vehicle will drive to.
  • step S502 may be performed before step S501, that is, first obtain the map information of the intersection, and then obtain the travel information of the target vehicle and the travel information of the surrounding vehicles of the target vehicle.
  • the driving intention of the target vehicle is determined by predicting the lane intention and road intention of the target vehicle. Inaccuracy and ambiguity in the description of the target vehicle's driving behavior by the intent of the fixed category. On the other hand, the mutual auxiliary correction of road intention and lane intention improves the accuracy of the final predicted driving intention of the target vehicle.
  • the target model in this application includes a first interaction feature extraction network, a lane intention prediction network, and a road prediction network, which may be the first interaction feature extraction network, lane intention prediction network, and road prediction network that are trained and completed by training with a large amount of training sample data.
  • the training data may include the current driving information and intersection map information of the detected target vehicle and other vehicles, and the driving information of the target vehicle after a preset period of time.
  • the target model is obtained by training based on the training data, and the target model can be used to predict the driving intention of the vehicle according to the current driving information of the target vehicle and other vehicles and the map information of the intersection.
  • the MLP in the target model is a single-layer fully connected network
  • the hidden unit dimension is 64
  • the activation function is Relu
  • the RNN in the target model uses GRU
  • the hidden unit dimension is 128.
  • the dimension of the MLP hidden unit that calculates the weight coefficient in the attention mechanism module is 1, and after softmax, it is the normalized target intent probability.
  • the open-source deep learning framework is used to realize the vehicle high-level intent prediction network.
  • the network training adopts the multi-task learning method.
  • the loss function includes road-level intent cross-entropy and lane-level intent cross-entropy.
  • the batch size of training data is 512, and the initial learning rate is 0.001.
  • the learning rate is changed in the form of exponential decreasing, the decay step size is set to 10 rounds, and the total number of training rounds is 50 rounds.
  • One training round refers to traversing all training data once.
  • FIG. 11 is a schematic structural diagram of a vehicle driving intention prediction device provided by an embodiment of the present application. As shown in FIG. 11 , the vehicle driving intention prediction device 1100 at least includes:
  • the first obtaining module 1101 is configured to obtain map information of the intersection when the target vehicle travels to the intersection; wherein the map information of the intersection includes road layer information and lane layer information, and the road layer information includes a plurality of The road connected by the intersection, the lane layer information includes a plurality of lanes, and the lane is a part of the road in the intersection that communicates with the road;
  • a second acquisition module 1102 configured to acquire the driving information of the target vehicle
  • a first feature extraction module 1103, configured to determine the driving characteristics of the target vehicle relative to each lane based on the driving information of the target vehicle and the lane layer information;
  • the second feature extraction module 1104 is configured to determine the driving characteristics of the target vehicle relative to each road based on the driving information of the target vehicle and the road layer information;
  • the prediction module 1105 is configured to determine, based on the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle, the driving characteristics of the target vehicle relative to each road, and the driving characteristics of the target vehicle relative to each lane The road intention of the target vehicle and the lane intention of the target vehicle; wherein, the surrounding vehicles are vehicles within a preset range of the target vehicle, and the road intention of the target vehicle indicates that the target vehicle passes the The probability distribution of each road leaving the intersection, the lane intention of the target vehicle represents the probability distribution of the target vehicle leaving the intersection through each lane;
  • a determination module 1106, configured to determine the driving intention of the target vehicle based on the road intention of the target vehicle and the lane intention of the target vehicle.
  • the predicting module 1105 is specifically configured to: input the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle into the first interactive feature extraction network, and determine the relationship between the surrounding vehicles and the target vehicle.
  • the interaction feature vector of the target vehicle, the interaction feature vector between the surrounding vehicle and the target vehicle represents the influence of the surrounding vehicle on the target vehicle;
  • the driving characteristics of the target vehicle relative to each road, the driving characteristics of the target vehicle corresponding to the plurality of lanes associated with each road, and the hidden state vectors of the driving characteristics of the target vehicle relative to each lane are combined with each road.
  • the lane intention of the target vehicle corresponding to the associated lane is input to a road intention prediction network to determine the road intention of the target vehicle.
  • the first interactive feature extraction network includes at least a plurality of first feature extraction sub-networks and an interactive feature vector prediction network;
  • the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle are respectively input into the plurality of first feature extraction sub-networks, and the driving characteristic vector of each vehicle in the surrounding vehicles relative to the target vehicle is determined. ;
  • the driving feature vector of each vehicle in the surrounding vehicles relative to the target vehicle is input into the interaction feature vector prediction network, and the interaction feature vector between the surrounding vehicle and the target vehicle is determined.
  • the lane intention prediction network includes at least a first lane feature extraction sub-network, a second lane feature extraction sub-network, a first road feature extraction sub-network, a second road feature extraction sub-network and lane intention prediction subnet;
  • the intent prediction network determines the lane intent of the target vehicle and the latent state vector of the target vehicle's driving characteristics relative to each lane, including:
  • the interaction feature vector of the surrounding vehicle and the target vehicle, the driving feature vector of the target vehicle relative to each lane, the hidden state vector of the target vehicle's driving feature relative to each lane are relative to the target vehicle.
  • the lane intention prediction sub-network is input to the driving feature latent state vector of the road associated with each lane to determine the lane intention of the target vehicle.
  • the driving feature vector of the target vehicle relative to each lane is input into the second lane feature extraction sub-network, and the driving feature latent vector of the target vehicle relative to each lane is determined, include:
  • a plurality of feature extraction windows in the second lane feature extraction sub-network perform feature extraction on the driving feature vector of the target vehicle relative to each lane in the order of driving time;
  • the driving feature latent vector of the target vehicle relative to each lane is determined.
  • the driving feature vector of the target vehicle relative to the road associated with each lane is input into the second road feature extraction sub-network, and the target vehicle relative to each lane is determined to be relative to each lane.
  • the hidden state vector of the driving feature of the connected road including:
  • a plurality of feature extraction windows in the second road feature extraction sub-network perform feature extraction on the driving feature vector of the target vehicle relative to the road associated with each lane in the order of driving time;
  • the driving feature vector of the target vehicle corresponding to the current feature extraction window relative to the road associated with each lane and the latent state vector output by the previous feature extraction window, it is determined that the target vehicle is relative to each lane.
  • the latent state vector of the driving feature of the road is determined that the target vehicle is relative to each lane.
  • the road intention prediction network includes at least: the first road feature extraction sub-network, the second road feature extraction sub-network and a road intention prediction sub-network;
  • the driving characteristics of the target vehicle relative to each road, the driving characteristics of the target vehicle relative to each lane corresponding to the plurality of lanes associated with each road, and the hidden state vectors of each lane are combined with the each road.
  • the lane intention of the target vehicle corresponding to the lane associated with each road is input into the road intention prediction network, and the road intention of the target vehicle is determined, including:
  • the driving feature of the target vehicle relative to each road is input into the first road feature extraction sub-network to determine the driving feature vector of the target vehicle relative to each road;
  • weighting is performed on the latent vector of the driving feature of the target vehicle corresponding to the lane associated with each road relative to each lane, and the The weighted processing results are spliced to obtain a latent fusion vector of the lanes associated with each road;
  • the driving feature vector of the target vehicle relative to each road and the latent fusion vector of the lane associated with each road are input into the road intention prediction sub-network to determine the road intention of the target vehicle.
  • the first feature extraction sub-network is constructed based on at least a multilayer perceptron network and a recurrent neural network;
  • the first lane feature extraction sub-network and the first road feature extraction sub-network are constructed at least based on a multilayer perceptron network, and the second lane feature extraction sub-network and the second road feature extraction sub-network are constructed at least based on a recurrent neural network;
  • the interaction feature vector prediction network, the lane intention prediction sub-network and the road intention prediction sub-network are all constructed based on at least an attention mechanism network.
  • the determining module 1106 is specifically configured to:
  • the lane corresponding to the maximum probability in the lane intention of the target vehicle corresponding to the lane associated with the target road is determined as the target lane; based on the target road and the target lane, the driving intention of the target vehicle is determined.
  • the driving characteristics of each vehicle in the surrounding vehicles relative to the target vehicle include:
  • the first coordinate system is a Cartesian coordinate system
  • the y-axis of the first coordinate system is parallel to the longitudinal direction of the body of the target vehicle
  • the positive direction of the y-axis is consistent with the front direction of the target vehicle.
  • the driving characteristics of the target vehicle relative to each road include:
  • each second coordinate system is a Cartesian coordinate system
  • the origin of each second coordinate system is based on the The location of the exit of each road is determined, and the x-axis direction is determined based on the direction of travel of each road.
  • the driving characteristics of the target vehicle relative to each lane include:
  • each third coordinate system is a frenet coordinate system
  • the reference line of each third coordinate system The origin of each third coordinate system is determined based on the centerline of each lane, and the origin of each third coordinate system is determined based on the end point of the centerline of each lane.
  • each lane is determined based on a topology analysis of the plurality of roads.
  • the vehicle driving intention prediction apparatus 1100 may correspond to executing the method described in the embodiment of the present application, and the above-mentioned and other operations and/or functions of each module in the vehicle driving intention prediction apparatus 1100 are respectively for the purpose of realizing FIG.
  • the corresponding flow of each method in 4-10 is not repeated here for brevity.
  • connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • the present application further provides a vehicle driving intention prediction terminal, including a memory and a processor, where executable code is stored in the memory, and the processor executes the executable code to implement any of the above methods.
  • the vehicle driving intention prediction terminal may be an in-vehicle terminal with a vehicle driving intention prediction function, or may be a vehicle with a vehicle driving intention prediction function.
  • the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute any one of the above methods.
  • the present application also provides a computer program or computer program product, the computer program or computer program product including instructions, when the instructions are executed, make a computer perform any one of the above methods.
  • the present application further provides an electronic device, including a memory and a processor, where executable codes are stored in the memory, and the processor executes the executable codes to implement any of the foregoing methods.
  • non-transitory English: non-transitory
  • the storage medium is non-transitory ( English: non-transitory) media, such as random access memory, read only memory, flash memory, hard disk, solid state disk, magnetic tape (English: magnetic tape), floppy disk (English: floppy disk), optical disc (English: optical disc) and any combination thereof.

Abstract

一种车辆行驶意图预测方法、装置、终端及存储介质。该方法基于周边车辆相对于目标车辆的行驶特征、目标车辆相对于道路的行驶特征和目标车辆相对于车道的行驶特征，确定目标车辆的道路意图和车道意图，再基于目标车辆的道路意图和车道意图，确定目标车辆的行驶意图。通过对目标车辆的多级意图的预测，例如车道意图和道路意图，来确定目标车辆的行驶意图，能够有效、精确表示目标车辆的行驶行为，能够适应不同结构的路口场景，避免预先定义的固定类别的意图对目标车辆行驶行为描述的不准确和模糊性。

Description

一种车辆行驶意图预测方法、装置、终端及存储介质 技术领域
本申请涉及自动驾驶领域，尤其涉及一种车辆行驶意图预测方法、装置、终端及存储介质。
背景技术
人们通常需要对未来的事情进行预测，例如，预测明天的天气、预测收成等。同样的，人们在开车时也会下意识地预测其他车辆的行驶意图(即其他车辆未来的行驶行为，例如，刹车、变道、加速、直行、左转、右转、掉头等)，以根据其他车辆的行驶意图确定自车的行车路线，这样能够一定程度地规避可能发生的危险。
随着科技的进步与发展，人们开始研究可以应用于自动驾驶领域的车辆自动驾驶技术，使得车辆能够实现自动驾驶，解放人们的双手。车辆实现自动驾驶不仅需要自动的启动、行驶、刹车以及停车等，还需要在驾驶过程中模拟人类的预测行为，准确的识别出周围目标车辆的行驶意图，进而可以通过周围目标车辆的行驶意图调整行驶路线，降低交通事故的发生概率。
在非路口场景中，由于道路上有实体车道线对车辆进行约束，车辆的行驶意图相对来说比较容易预测，而对于路口场景，由于道路结构复杂多变，没有明确的实体车道行驶线约束，车辆的行为差异性更加明显；由于不同路口结构差异较大，类别固定的语义级意图，如直行、左转、右转以及调头，难以准确描述目标车辆的行为意图；此外高精地图中的虚拟车道有可能缺失或者不准确，因此，在路口场景中准确预测车辆的行驶意图尤为困难。
发明内容
本申请实施例提供了一种车辆行驶意图预测方法、装置、终端及存储介质，解决了在路口场景中的车辆行驶意图预测的问题。
第一方面，本申请实施例提供了一种车辆行驶意图预测方法，包括：当目标车辆行驶至路口时，获取所述路口的地图信息；其中，所述路口的地图信息包括道路层信息和车道层信息，所述道路层信息包括与所述路口连通的多条道路，所述车道层信息包括多条车道，所述车道为所述路口中连通所述道路的部分路段；获取目标车辆的行驶信息；基于所述目标车辆的行驶信息和所述车道层信息，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征；基于所述目标车辆的行驶信息和所述道路层信息，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征；基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图；其中，所述周边车辆为所述目标车辆的预设范围内的车辆，所述目标车辆的车道意图表征所述目标车辆通过所述多条车道中每条车道驶离所述路口的概 率分布；至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
所述基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图，包括：
基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图；其中，所述目标车辆的道路意图表征所述目标车辆通过所述多条道路中每条道路驶离所述路口的概率分布；
所述至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图，包括：
基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
本申请实施例通过周边车辆相对于目标车辆的行驶特征、目标车辆相对于道路的行驶特征和目标车辆相对于车道的行驶特征，确定目标车辆的道路意图和车道意图，再基于目标车辆的道路意图和车道意图，确定目标车辆的行驶意图。通过对目标车辆的多级意图的预测(即车道意图和道路意图)，来确定目标车辆的行驶意图，能够有效、精确表示目标车辆的行驶行为，能够适应不同结构的路口场景，避免预先定义的固定类别的意图(例如，左转、右转、掉头等)对目标车辆行驶行为描述的不准确和模糊性。
在一个可能的实现中，所述基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图，包括：将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，所述周边车辆与所述目标车辆的交互特征向量表征所述周边车辆对所述目标车辆的影响；将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图。
本申请实施例在目标车辆的车道意图预测中，不仅考虑目标车辆相对于车道的行驶特征，而且考虑周边车辆相对于目标车辆的行驶特征和目标车辆与部分道路的行驶特征；在目标车辆的道路意图预测中，不仅考虑目标车辆相对于道路的行驶特征，而且考虑目标车辆与部分车道的行驶特征。提高了本方案的精度、泛化性以及对噪声的鲁棒性。
在另一个可能的实现中，所述第一交互特征提取网络至少包括多个第一特征提取子网络和交互特征向量预测网络；所述将所述周边车辆中每个车辆相对于所述目标车辆 的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，包括：将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征分别输入所述多个第一特征提取子网络，确定所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量；将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量输入交互特征向量预测网络，确定所述周边车辆与所述目标车辆的交互特征向量。
在另一个可能的实现中，所述车道意图预测网络至少包括第一车道特征提取子网络、第二车道特征提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网；将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：将所述目标车辆相对于所述多条车道中每条车道的行驶特征输入第一车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征向量；将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量；将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量；将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量输入车道意图预测子网络，确定所述目标车辆的车道意图。
在另一个可能的实现中，所述将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：所述第二车道特征提取子网络中多个特征提取窗口对所述目标车辆相对于所述多条车道中每条车道的行驶特征向量按照驾驶时刻顺序进行特征提取；根据当前特征提取窗口对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量。
在另一个可能的实现中，所述将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量，包括：所述第二道路特征提取子网络中多个特征提取窗口对所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量按照驾驶时刻顺序进行特征提取；根据当前特征提取窗口对应的所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量。
在另一个可能的实现中，所述道路意图预测网络至少包括：所述第一道路特征提取子网络、所述第二道路特征提取子网络和道路意图预测子网络；所述将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图，包括：将所述目标车辆相对于所述多条道路中每条道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征隐态向量；根据与所述多条道路中每条道路相关联的车道对应的车道意图，对与所述多条道路中每条道路相关联的车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与所述多条道路中每条道路相关联的车道的隐态融合向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量和所述与所述多条道路中每条道路相关联的车道的隐态融合向量输入所述道路意图预测子网络，确定所述目标车辆的道路意图。
在另一个可能的实现中，所述第一特征提取子网络至少基于多层感知机网络和循环神经网络构建；所述第一车道特征提取子网络和第一道路特征提取子网络至少基于多层感知机网络构建，所述第二车道特征提取子网络和第二道路特征提取子网络至少基于循环神经网络构建；所述交互特征向量预测网络、所述车道意图预测子网络和所述道路意图预测子网络均至少基于注意力机制网络构建。
本申请实施例中预测网络的构建不需要复杂度较高的CNN网络，只需简单的MLP和RNN网络，输入数据维度小，计算效率高。
在另一个可能的实现中，所述基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图，包括：将所述目标车辆的道路意图中概率最大值对应的道路，确定为目标道路；将与所述目标道路关联的车道对应的所述目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；基于目标道路和目标车道，确定所述目标车辆的行驶意图。
同现有方法中采用的固定类别的语意意图相比，本申请实施例根据目标车辆所处地图结构确定的目标车辆意图类别不固定，所提意图能有效提高对目标车辆行为描述的准确性。此外将目标车辆意图预测转换为目标车辆运动状态与地图信息的匹配，不同于现有方法中的固定类别意图分类，而且目标车辆的车道意图和道路意图互相辅助，提高了目标车辆的行驶意图预测的泛化性和精准性。
在另一个可能的实现中，所述周边车辆相对于所述目标车辆的行驶特征，包括：所述周边车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，所述第一坐标系的原点为所述目标车辆的当前位置、所述第一坐标系为直角坐标系，所述第一坐标系的y轴与所述目标车辆的车身的长度方向平行，且y轴的正向与所述目标车辆的车头朝向一致。
在另一个可能的实现中，所述目标车辆相对于所述多条道路中每条道路的行驶特征， 包括：所述目标车辆在第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及所述目标车辆在第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，所述第二坐标系为直角坐标系，所述第二坐标系的原点基于所述多条道路中每条道路的出口的位置确定，x轴方向基于所述多条道路中每条道路的行驶方向确定。
在另一个可能的实现中，所述目标车辆相对于所述多条车道中每条车道的行驶特征，包括：所述目标车辆在第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述第三坐标系为frenet坐标系，所述第三坐标系的参考线基于所述多条车道中每条车道的中心线确定，所述第三坐标系的原点基于所述多条车道中每条车道的中心线的终点确定。
在另一个可能的实现中，所述多条车道基于对所述多条道路进行拓扑分析确定。
本申请实施例中，车道是基于对于多条道路的拓扑分析自动生成的，不依赖于高精地图，在高精地图信息缺失的情况下依然可以准确预测目标车辆的行驶意图。
第二方面，本申请实施例还提供了一种车辆行驶意图预测装置，所述装置包括：第一获取模块,用于当目标车辆行驶至路口时，获取所述路口的地图信息；其中，所述路口的地图信息包括道路层信息和车道层信息，所述道路层信息包括与所述路口连通的多条道路，所述车道层信息包括多条车道，所述车道为所述路口中连通所述道路的部分路段；第二获取模块，用于获取目标车辆的行驶信息；第一特征提取模块，用于基于所述目标车辆的行驶信息和所述车道层信息，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征；第二特征提取模块，用于基于所述目标车辆的行驶信息和所述道路层信息，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征；预测模块，用于基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图；其中，所述周边车辆为所述目标车辆的预设范围内的车辆，所述目标车辆的车道意图表征所述目标车辆通过所述多条车道中每条车道驶离所述路口的概率分布；确定模块，用于至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
所述预测模块具体用于：基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图；其中，所述目标车辆的道路意图表征所述目标车辆通过所述多条道路中每条道路驶离所述路口的概率分布；所述确定模块具体用于：基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
在一个可能的实现中，所述预测模块具体用于：将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，所述周边车辆与所述目标车辆的交互特征向量表征所述周边车辆对所述目标车辆的影响；将所述周边车辆与所述目标车辆的交互特征向量、所述目 标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图。
在另一个可能的实现中，所述第一交互特征提取网络至少包括多个第一特征提取子网络和交互特征向量预测网络；所述将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，包括：将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征分别输入所述多个第一特征提取子网络，确定所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量；将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量输入交互特征向量预测网络，确定所述周边车辆与所述目标车辆的交互特征向量。
在另一个可能的实现中，所述车道意图预测网络至少包括第一车道特征提取子网络、第二车道特征提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网；将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：将所述目标车辆相对于所述多条车道中每条车道的行驶特征输入第一车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征向量；将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量；将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量；将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量输入车道意图预测子网络，确定所述目标车辆的车道意图。
在另一个可能的实现中，所述将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：所述第二车道特征提取子网络中多个特征提取窗口对所述目标车辆相对于所述多条车道中每条车道的行驶特征向量按照驾驶时刻顺序进行特征提取；根据当前特征提取窗口对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相 对于所述多条车道中每条车道的行驶特征隐态向量。
在另一个可能的实现中，所述将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量，包括：所述第二道路特征提取子网络中多个特征提取窗口对所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量按照驾驶时刻顺序进行特征提取；根据当前特征提取窗口对应的所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量。
在另一个可能的实现中，所述道路意图预测网络至少包括：所述第一道路特征提取子网络、所述第二道路特征提取子网络和道路意图预测子网络；所述将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图，包括：将所述目标车辆相对于所述多条道路中每条道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征隐态向量；根据与所述多条道路中每条道路相关联的车道对应的车道意图，对与所述多条道路中每条道路相关联的车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与所述多条道路中每条道路相关联的车道的隐态融合向量；将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量和所述与所述多条道路中每条道路相关联的车道的隐态融合向量输入所述道路意图预测子网络，确定所述目标车辆的道路意图。
在另一个可能的实现中，所述第一特征提取子网络至少基于多层感知机网络和循环神经网络构建；所述第一车道特征提取子网络和第一道路特征提取子网络至少基于多层感知机网络构建，所述第二车道特征提取子网络和第二道路特征提取子网络至少基于循环神经网络构建；所述交互特征向量预测网络、所述车道意图预测子网络和所述道路意图预测子网络均至少基于注意力机制网络构建。
在另一个可能的实现中，所述确定模块具体用于：将所述目标车辆的道路意图中概率最大值对应的道路，确定为目标道路；将与所述目标道路关联的车道对应的所述目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；基于目标道路和目标车道，确定所述目标车辆的行驶意图。
在另一个可能的实现中，所述周边车辆相对于所述目标车辆的行驶特征，包括：所述周边车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，所述第一坐标系的原点为所述目标车辆的当前位置、所述第一坐标系为直角坐标系，所述第一坐标系的y轴与所述目标车辆的车身的长度方向平行，且y轴的正向与所述目标车辆的车头朝向一致。
在另一个可能的实现中，所述目标车辆相对于所述多条道路中每条道路的行驶特征，包括：所述目标车辆在第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及所述目标车辆在第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，所述第二坐标系为直角坐标系，所述第二坐标系的原点基于所述多条道路中每条道路的出口的位置确定，x轴方向基于所述多条道路中每条道路的行驶方向确定。
在另一个可能的实现中，所述目标车辆相对于所述多条车道中每条车道的行驶特征，包括：所述目标车辆在第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述第三坐标系为frenet坐标系，所述第三坐标系的参考线基于所述多条车道中每条车道的中心线确定，所述第三坐标系的原点基于所述多条车道中每条车道的中心线的终点确定。
在另一个可能的实现中，所述多条车道基于对所述多条道路进行拓扑分析确定。
第三方面，本申请还提供一种车辆终端，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码，实现上述第一方面或第一方面任一种可能实现方式中所述的方法。
第四方面，本申请还提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行上述第一方面或第一方面任一种可能实现方式中所述的方法。
第五方面，本申请还提供了一种计算机程序或计算机程序产品，所述计算机程序或计算机程序产品包括指令，当所述指令执行时，实现上述第一方面或第一方面任一种可能实现方式中所述的方法。
本申请在上述各方面提供的实现方式的基础上，还可以进行进一步组合以提供更多实现方式。
附图说明
图1为本申请实施例提供的车辆的功能框图；
图2为本申请实施例提供的计算机系统的结构示意图；
图3为本申请实施例提供的一种芯片硬件结构示意图；
图4为本申请实施例提供的一种自动驾驶系统的框架图；
图5为本申请实施例提供的一种车辆行驶意图预测方法的流程图；
图6为当目标车辆行驶至路口时该路口的地图示意图；
图7为基于车道的中心线作为参考线确定的第三坐标系的示意图；
图8为目标车辆与其他车辆的交互特征向量提取网络的结构示意图；
图9为目标车辆的车道意图预测网络的结构示意图；
图10为目标车辆的道路意图预测网络的结构示意图；
图11本申请实施例提供的一种车辆行驶意图预测装置的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。
图1为本申请实施例提供的车辆的功能框图。如图1所示，车辆100包括各种子系统，例如行进系统102、传感器系统104、控制系统106、一个或多个外围设备108以及电源110、计算机系统112和用户接口116。
可选地，车辆100可包括更多或更少的子系统，并且每个子系统可包括多个元件。另外，车辆100的每个子系统和元件可以通过有线或者无线互连。
行进系统102包括为车辆100提供动力运动的组件。在一个示例中，行进系统102可包括引擎118、能量源119、传动装置120和车轮121。引擎118可以是内燃引擎、电动机、空气压缩引擎或其他类型的引擎组合，例如，汽油发动机和电动机组成的混动引擎，内燃引擎和空气压缩引擎组成的混动引擎。引擎118将能量源119转换成机械能量。
能量源的示例包括汽油、柴油、其他基于石油的燃料、丙烷、其他基于压缩气体的燃料、乙醇、太阳能电池板、电池和其他电力来源等。能量源119也可以为车辆100的其他系统提供能量。
传动装置120可以将来自引擎118的机械动力传送到车轮121。传动装置可包括变速箱、差速器和驱动轴。
在一个示例中，传动装置120还可以包括其他器件，例如离合器。其中，驱动轴可包括可耦合到一个或多个车轮121的一个或多个轴。
传感器系统104可包括感测关于车辆100周边的环境信息的若干传感器。
例如，传感器系统104可包括定位系统122(定位系统可以是GPS系统，也可以是北斗系统或其他定位系统)、惯性测量单元(inertial measurement unit，IMU)124、雷达126、激光测距仪128以及摄像头130。传感器系统104还可包括车辆100的内部系统的传感器(例如，车内空气质量监测器、燃油量表、机油温度表等)。来自这些传感器中的一个或多个的传感器数据可用于检测对象及相应特性(位置、形状、方向、速度等)。这种检测和识别是车辆100的安全操作的关键功能。
定位系统可用于估计车辆100的地理位置。IMU 124用于基于惯性加速度来感测车辆100的位置和朝向变化。在一个示例中，IMU 124可以是加速度计和陀螺仪的组合。
雷达126可利用无线电信号来感测车辆100的周边环境内的物体。在一个示例中，除了感测物体以外，雷达126还可用于感测物体的速度和/或前进方向。
激光测距仪128可利用激光来感测车辆100所位于的环境中的物体。在一个示例中，激光测距仪128可包括一个或多个激光源、激光扫描器以及一个或多个检测器，以及其他系统组件。
摄像头130可用于捕捉车辆100的周边环境的多个图像。摄像头130可以是静态摄像头或视频摄像头。
控制系统106为控制车辆100及其组件的操作。控制系统106可包括各种元件，其中包括转向系统132、油门134、制动单元136、传感器融合算法138、计算机视觉系统140、路线控制系统142以及障碍规避系统144。
转向系统132可操作来调整车辆100的前进方向。例如，在一个示例中转向系统 132可以包括方向盘系统。
油门134用于控制引擎118的操作速度并进而控制车辆100的速度。
制动单元136用于控制车辆100减速。制动单元136可使用摩擦力来减慢车轮121。在其他示例中，制动单元136可将车轮121的动能转换为电能。制动单元136也可采取其他形式来减慢车轮121转速从而控制车辆100的速度。
计算机视觉系统140可以操作处理和分析由摄像头130捕捉的图像以便识别车辆100周边环境中的物体和/或特征。所述物体和/或特征可包括交通信号、道路边界和障碍物等、计算机视觉系统140可使用物体识别算法、运动中恢复结构(structure from motion，SFM)算法、视频跟踪和其他计算机视觉技术。在一些示例中，计算机视觉系统140可以用于为环境绘制地图、跟踪物体、估计物体的速度等。
路线控制系统142用于确定车辆100的行驶路线。在一些示例中，路线控制系统142可结合来自传感器138、定位系统122和一个或多个预定地图的数据为车辆100确定行驶路线。
障碍规避系统144用于识别、评估和避免或者以其他方式越过车辆100的环境中的潜在障碍物。
当然，在一个示例中，控制系统106可以增加或替换包括除了所示出和描述的那些以外的组件。或者也可以减少一部分上述示出的组件。
车辆100通过外围设备108与外部传感器、其他车辆、其他计算机系统或用户之间进行交互。外围设备108可包括无线通信系统146、车载电脑148、麦克风150和/或扬声器152。
在一些示例中，外围设备108提供车辆100的用户与用户接口116交互的手段。例如，车载电脑148可向车辆100的用户提供信息。用户接口116还可操作车载电脑148来接收用户的输入。车载电脑148可以通过触摸屏进行操作。在其他情况中，外围设备108可提供用于车辆100与位于车内的其他设备通信的手段。例如，麦克风150可从车辆100的用户接收音频(例如，语音命令或其他音频输入)。类似的，扬声器152可向车辆100的用户输出音频。
无线通信系统146可以直接地或者经由通信网络来与一个或多个设备无线通信。例如，无线通信系统146可使用3G蜂窝通信，例如CDMA、EVDO、GSM/GPRS，或者4G蜂窝网络，例如LTE。或者5G蜂窝通信。无线通信系统146可利用WiFi与无线局域网(wireless local area network，WLAN)通信。在一个示例中，无线通信系统146可利用红外链路、蓝牙或ZigBee与设备直接通信。其他无线协议，例如各种车辆通信系统，例如，无线通信系统146可包括一个或多个专用短程通信(dedicated short range communications，DSRC)设备，这些设备可包括车辆和/或路边台站之间的公共和/或私有数据通信。
电源110可向车辆100的各种组件提供电力。在一个示例中，电源110可以为可再充电锂离子或铅酸电池。这种电池的一个或多个电池组可被配置为电源为车辆100的各种组件提供电力。在一些示例中，电源110和能量源119可一起实现，例如一些全电动车中那样。
车辆100的部分或所有功能受计算机系统112控制。计算机系统112可包括至少 一个处理器113，处理器113执行存储在例如存储器114这样的非暂态计算机可读存储介质中的指令115。计算机系统112还可以是采用分布式方式控制车辆100的个体组件或子系统的多个计算设备。
处理器113可以是任何常规的处理器，诸如商业可获得的CPU。可选地，该处理器可以是诸如ASIC或其他基于硬件的处理器的专用设备。尽管图1功能性地图示了处理器、存储器和在相同块中的计算机110的其他元件，但是本领域的普通技术人员应该理解该处理器、计算机或存储器实际上可以包括存储在相同/不同的物理外壳内的多个处理器、计算机或存储器。例如，存储器可以是硬盘驱动器或位于不同计算机110的外壳内的其他存储介质。因此，对处理器或计算机的引用将被理解为包括对可以或者可以不并行操作的处理器或计算机或存储器的集合的引用。不同于使用单一的处理器来执行此处所描述的步骤，诸如转向组件和减速组件的一些组件每个都可以具有其自己的处理器，所述处理器只执行与特定组件的功能相关的计算。
在此处所描述的每个方面中，处理器可以位于远离该车辆并且与该车辆进行无线通信。在其他方面中，此处所描述的过程中的一些在布置于车辆内的处理器上执行而其它则由远程处理器执行，包括采取执行单一操作的必要步骤。
在一些示例中，存储器114可包含指令115(例如，程序逻辑)，指令115可被处理器113执行来执行车辆100的各种功能，包括以上描述的那些功能。存储器114也可包含额外的指令，包括向行进系统102、传感器系统104、控制系统106和外围设备108中的一个或多个发送数据、从其接收数据、与其交互和/或对其进行控制的指令。
除了指令115以外，存储器114还可存储数据，例如道路地图、路线信息，车辆的位置、方向、速度以及其他类似的车辆数据，以及其他信息。这种信息可在车辆100在自主、半自主和/或手动模式中操作期间被车辆100和计算机系统112使用。
用户接口116，用于向车辆100的用户提供信息或从其接收信息。可选地，用户接口116可包括在外围设备108的集合内的一个或多个输入/输出设备，例如无线通信系统146、车载电脑148、麦克风150和扬声器。
计算机系统112可基于从各种子系统(例如，行进系统102、传感器系统104和控制系统106)以及从用户接口116接收的输入来控制车辆100的功能。例如，计算机系统可利用来自控制系统106的输入以便控制转向单元132来避免由传感器系统104和障碍规避系统144检测到的障碍物。在一些示例中，计算机系统112可操作来对车辆100及其子系统的多方面提供控制。
可选地，上述这些组件中的一个或多个可与车辆100分开安装或关联。例如，存储器114可以部分或完全地与车辆100分开存在。上述组件可以按有线和/或无线方式来通信地耦合在一起。
可选地，上述组件只是一个示例，实际应用中，上述每个模块中的组件有可能根据实际需要增添或删减，图1不应理解为对本申请实施例的限制。
在道路行进的自动驾驶车辆，如上面的车辆100，可以识别其周围环境内的物体以确定对当前速度。所述物体可以是其他车辆、交通控制设备、或者其他类型的物体。在一些示例中，可以独立地考虑每个识别的物体，并且基于物体的各自的特性，诸如它的当前速度、加速度、与车辆的间距等，可以用来确定自动驾驶汽车所要调整的速 度。
可选地，自动驾驶车辆100或者与自动驾驶车辆100相关联的计算设备(如图1的计算机系统112、计算机视觉系统140、存储器114)可以基于所识别的物体的特性和周围环境的状态(例如，交通、雨雪、道路上的冰等)来预测所述识别的物体的行为。可选的，每一个所识别的物体都依赖于彼此的行为，因此还可以将所识别的所有物体全部一起考虑来预测单个识别的物体的行为。车辆100能够基于预测的识别的物体的行为来调整自身的速度。换句话说，自动驾驶车辆能够基于所预测的物体的行为来确定车辆将需要调整到(例如，加速、减速或停止等)什么状态。在这个过程中，也可以考虑其他因素来确定车辆100的速度，诸如，车辆100在行驶的道路中的横向位置、道路的曲率、静态和动态的物体的接近度等。
除了提供调整自动驾驶车辆的速度的指令之外，计算设备还可以提供修改车辆100的转向角的指令，以使自动驾驶车辆遵循指定的轨迹和/或维持与自动驾驶车辆附近的物体(例如，道路上的相邻车道中的车辆)的安全横向和纵向距离。
上述车辆100可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车辆、游乐场车辆、施工设备、电车、高尔夫球车、火车和手推车等，本申请实施例不做特别的限定。
图2是本申请实施例提供的计算机系统的示意图。
如图2所示的计算机系统112包括处理器113，处理器113和系统总线耦合。处理器113可以是一个或多个处理器，其中每个处理器都可以包括一个或多个处理器核。显示适配器(video adapter)107，显示适配器可以驱动显示器109，显示器109和系统总线105耦合。系统总线105通过总线桥111和输入输出(I/O)总线113耦合。I/O接口115和I/O总线耦合。I/O接口115和多种设备进行通信，比如输入设备117(例如，键盘、鼠标、触摸屏等)，多媒体盘(media tray)121，例如，CD-ROM，多媒体接口等。收发器123(可以发送和/或接收无线电通信信号)，摄像头155(可以捕捉静态和动态数字视频图像)和外部USB接口125。其中，可选的，和I/O接口115相连接的接口可以是USB接口。
其中，处理器113可以是任何传统处理器，包括精简指令集计算(reduced instruction set computer，RISC)处理器、复杂指令集计算(complex instruction set computer,CISC)处理器或上述组合。可选的，处理器可以是诸如专用集成电路(application specific integrated circuit,ASIC)的专用装置。可选的，处理器113可以是神经网络处理器或是神经网络处理器和上述传统处理器的组合。
可选的，在本文所述的各实施例中，计算机系统112可位于远离自动驾驶车辆的地方，并且可与自动驾驶车辆无线通信。在其他方面，本文所述的一些过程在设置在自动驾驶车辆内的处理器上执行，其他由远程处理器执行，包括采取执行单个操作所需的动作。
计算机系统112可以通过网络接口129和软件部署服务器149通信。网络接口129是硬件网络接口，比如，网卡。网络127可以是外部网络，比如因特网，也可以是内部网络，比如以太网或虚拟私人网络(virtual private natwork，VPN)。可选的，网络127还可以是无线网络，比如WiFi网络，蜂窝网络等。
硬盘驱动接口和系统总线105耦合。硬件驱动接口和硬盘驱动器相连接。存储器114和系统总线105耦合。运行在存储器114的数据可以包括计算机系统112的操作系统137和应用程序143。
操作系统包括解析器(shell)139和内核(kernel)141。解析器139是介于使用者和操作系统内核之间的一个接口。解析器是操作系统最外面的一层。解析器管理使用者与操作系统之间的交互，等待使用者的输入，向操作系统解释使用者的输入，并且处理各种各样的操作系统的输出结果。
内核141由操作系统中用于管理存储器、文件、外设和系统资源的那些部分组成。直接与硬件交互，操作系统内核通常运行进程，并提供进程间的通信，提供CPU时间片管理、中断、内存管理、IO管理等。
应用程序143包括控制车辆自动驾驶相关的程序，例如，管理自动驾驶车辆和路上障碍物交互的程序，控制自动驾驶车辆路线或者速度的程序，控制自动驾驶车辆和路上其他自动驾驶车辆交互的程序。应用程序143也存在于软件部署服务器(deploying server)149的系统上。在一个示例中，在需要执行应用程序147时，计算机系统112可以从软件部署服务器149下载应用程序143。
例如，应用程序141还可以是控制自动驾驶车辆避免与其他车辆碰撞，安全通过路口的程序。
传感器153和计算机系统112关联。传感器153用于探测计算机系统112周围的环境。举例来说，传感器153可以探测动物、车辆、障碍物和人行横道等，进一步传感器还可以探测上述动物、车辆、障碍物和人行横道等物体周围的环境。可选地，如果计算机系统112位于自动驾驶车辆上，传感器可以是摄像头、红外线感应器、化学检测器、麦克风等。
示例性的，处理器113可以根据传感器153探测到的周围道路情况和其他车辆情况，预测其他车辆的行驶轨迹。
处理器113可以通过预先训练的神经网络输入当前其他车辆的行驶信息和当前道路信息，得到其他车辆的预测行驶轨迹。预先训练的神经网络可以是通过大量训练样本数据获得，例如，训练数据可以包括探测到其他车辆当前的行驶信息和道路信息，以及预设时段后其他车辆的行驶信息。处理器113可以基于训练数据训练得到目标模型，该目标模型可以用于根据车辆当前的行驶信息和道路信息，确定车辆的预测行驶信息。处理器113对输入的车辆当前的行驶信息和道路信息进行处理，将输出的预测行驶信息与预设时段后车辆实际的行驶信息进行比对，直到处理器113输出的预测行驶信息与车辆实际的行驶信息的差值小于一定阈值，从而完成目标模型的训练。
图3为本申请实施例提供的一种芯片硬件结构示意图。该芯片包括神经网络处理器(network process units，NPU)30。该芯片可以被设置在如图2所示的处理器113中，用于完成确定其他车辆的预测行驶轨迹。预训练的神经网络中各层的算法均可在如图3所示的芯片中得以实现。
本申请实施例的车辆行驶意图预测方法可以在神经网络处理器30中的运算电路303和/或向量计算单元307中执行，从而得到目标车辆的行驶意图。
下面对神经网络处理器30中的每个模块和单元进行简单的介绍。
神经网络处理器30作为协处理器挂载到主CPU(Host CPU)上，由主CPU分配任务。神经网络处理器30的核心部分为运算电路303，在神经网络处理器30工作时，神经网络处理器30中的控制器304可以控制运算电路303提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路303内部包括多个处理单元(Process Engine,PE)。在一些实现中，运算电路303是二维脉动阵列。运算电路303还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路303是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器302中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器301中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)308中。
向量计算单元307可以对运算电路的输出做进一步处理，如向量乘、向量加、指数运算、对数运算、大小比较等等。例如，向量计算单元307可以用于神经网络中非卷积/非全连接层(fully connected layers，FC)层的网络计算，如池化(pooling)，批归一化(batch normalization)，局部响应归一化(local response normalization)等。
在一些实现中，向量计算单元307能将经处理的输出的向量存储到统一缓存器306。例如，向量计算单元307可以将非线性函数应用到运算电路303的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元307生成归一化的值、合并值、或者二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路303的激活输入，例如用于在神经网络中的后续层中的使用。
统一存储器306用于存放输入数据以及输出数据。
权重数据直接通过存储单元访问控制器(Direct Memory Access Controller，DMAC)305将外部存储器中的输入数据搬运到输入存储器301和/或统一存储器306、将外部存储器中的权重数据存入权重存储器302中，以及将统一存储器306中的数据存入外部存储器。
BIU为Bus Interface Unit即，总线接口单元310，用于通过总线实现主CPU、DMAC和取指存储器309之间进行交互。
与控制器304连接的取指存储器(instruction fetch buffer)309，用于存储控制器304使用的指令；
控制器304，用于调用指存储器309中缓存的指令，实现控制该运算加速器的工作过程。
一般地，统一存储器306、输入存储器301、权重存储器302以及取指存储器309均可以为片上(on-chip)存储器。NPU的外部存储器可以为该NPU外部的存储器，该外部存储器可以为双倍数据率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)、高带宽存储器(high bandwidth memory，HBM)或其他可读可写的存储器。应理解，图3示出的芯片硬件结构仅为示例性说明，本申请并未限定于此。
下面结合图4至图11对本申请的实施例进行详细的说明。
图4为本申请提供的自动驾驶系统的框架图。如图4所示，自动驾驶系统包括感 知单元41、目标融合单元42、预测单元43、规划单元44、控制单元45、地图单元46和导航单元。预测单元43承接感知单元41、目标融合单元42和地图单元46，通过预测算法，可以给出障碍物未来的行为意图以及轨迹，然后将其输出给下游的规划模块。可以帮助自车预知他车目标的未来轨迹以及目标重要性判断，在危险场景下有利于规划控制采取紧急安全措施保障车辆安全，避免碰撞的发生。
感知单元41的功能基于图1中的传感器系统104或图2中的传感器153实现，感测车辆100周边的环境信息，例如障碍物(例如，其他车辆、行人、动物等)、道路信息(例如，人行横道、车道线、交通信号灯等)等影响行车的信息，目标融合单元42对感知单元41感知到的车辆周边的环境信息进行处理，输出障碍物目标信息。地图单元存储于图1或图2中的存储器114中，预测单元43根据当前的地图信息和感知单元感测到的目标信息，对目标车辆的行为意图及未来轨迹进行预测。规划单元44根据预测单元的预测结果和/或导航单元47的输出信息规划车辆的行驶路线，控制单元45根据规划单元规划的行驶路线控制车辆按照规划的行驶路线行驶。目标融合单元42、预测单元43、规划单元43和控制单元45均在图1或图2中的处理器中实现。在车辆行驶过程中，实时、准确、可靠地对其他车辆意图预测，可以帮助自车预知前方的交通状况，建立自车周围的交通态势，有助于对周围他车目标重要性判断，筛选交互的关键目标，便于自车提前进行路径规划，安全通过复杂路况场景。
对车辆轨迹的预测方法有多种方案，例如，一种方案是基于于规则的轨迹预测算法，主要利用运动模型如匀速模型，根据预测目标的位置和运动速度信息计算未来一段时间内的可能行驶轨迹。这种方法利用的信息往往比较有限，考虑的要素较少，当前端感知输出的车辆运动信息不准确时预测结果往往与实际偏差较大。
另一种方案是基于数据统计的概率理论预测方法，如DBN、HMM等，优势是使用了大量的实际数据进行规律统计。但这类方法往往需要足够多的样本才能得到更可靠的统计概率值，除此之外，在算法设计时参数个数增加会导致计算复杂度大幅增加。
另一种方案是基于CNN的意图预测方法，需要积累一定时间长度的目标观测数据，并采用栅格化图像表示目标运动状态(目标位置)、高精地图信息(主要包括虚拟车道线)，然后利用CNN网络提取图像特征，然后预测目标意图与轨迹。这类方法将目标状态与高精地图信息转换为高维图像数据，运算复杂度较高，而且依赖于一定时间长度的观测数据，导致此类方法的实时性与时效性较差。
可以理解的是，高精度地图是一种精确到车道级的地图，在路口这种无实车道线的区域会增加辅助线即虚拟车道便于约束自动驾驶车辆行驶。高级别自动驾驶系统中用到高精地图作为辅助工具或者先验信息，帮助自动驾驶车辆更好的对其它目标车辆的行为意图进行预测。对于城市交通路口场景，由于道路结构复杂多变，没有明确的实体车道行驶线约束，车辆的行为差异性更加明显；由于不同路口结构差异较大，类别固定的语义级意图，如直行、左转、右转以及调头，难以准确描述目标车辆的行为意图。此外高精地图中的虚拟车道有可能缺失或者不准确，因此准确预测车辆的高阶意图尤为困难。
上述方案在复杂的路口场景下时，预测准确性较差，针对此问题，提出一种方案：以目标车辆位置为原点、目标车辆朝向为y轴的坐标系下，将路口划分为十二个扇形 区域，将目标意图预测转换为十二元分类问题，包含目标真实出口的扇形区域为目标真实类别。此方案积累特定时间长度的观测数据后，利用栅格化图像表示目标车辆轨迹、高精地图信息以及其他车辆目标轨迹，其中高精地图信息主要包括车道线信息；然后通过CNN网络提取特征，利用softmax分类器计算十二个扇形区域的概率，概率最大的扇形区域内的出口为预测的目标车辆出口，即目标意图。
但是该方案存在着：由于该方案是通过CNN网络提取目标车辆与高精地图信息中的虚拟车道线信息关联特征，当虚拟车道信息不准确甚至缺失情况下无法有效预测目标车辆意图，因此具有强依赖高精地图信息的问题。另外，该方案将目标车辆意图预测问题转换为十二元分类问题，十二元类别对应目标车辆坐标系下的不同扇区，实际应用中目标车辆出口并不一定落在某一扇区，因此，在不同结构路口场景下性能差异较大，存在着泛化性能不足。另一方面，由于利用高维栅格化图像表示目标车辆与其他车辆轨迹以及高精地图信息，导致输入数据维度大大增加，此外深层CNN网络运算复杂度较高，对平台运算能力有一定要求，存在着计算复杂度高的问题。另一方面，该方案需要积累特定时间长度观测数据才能对目标车辆意图进行预测。当目标车辆意图变化或处于跟踪起始阶段时，无法及时预测目标意图，存在着时效性差的问题。另一方面，意图预测问题中存在一对多的可能，即多模态问题，该方案将多模态意图预测问题转换为特定的十二元分类问题，因此，存在着无法解决意图预测中的多模态问题。
本申请提供一种车辆行驶意图预测方法，先基于周边车辆相对于目标车辆的行驶特征、目标车辆相对于道路的行驶特征和目标车辆相对于车道的行驶特征，确定目标车辆的道路意图和车道意图，再基于目标车辆的道路意图和车道意图，确定目标车辆的行驶意图。通过对目标车辆的多级意图的预测(即车道意图和道路意图)，来确定目标车辆的行驶意图，能够有效、精确表示目标车辆的行驶行为，能够适应不同结构的路口场景，避免预先定义的固定类别的意图(例如，左转、右转、掉头等)对目标车辆行驶行为描述的不准确和模糊性。
下面结合图5对本申请实施例的一种车辆行驶意图预测方法，进行详细介绍。该方法可以由图1中的车辆执行。该车辆中配置训练完成的目标模型。该车辆中配置图2的计算机系统和图3的芯片结构。
图5为本申请实施例提供一种车辆行驶意图预测方法的流程图。如图5所示，本申请提供的一种车辆行驶意图预测方法包括步骤S501-S506。
当目标车辆行驶至路口时，执行该方法，在步骤S501中，获取路口的地图信息。
其中，地图信息可以是描绘了现实世界中的道路、交通状况、行政区域等空间信息的图像信息，也可以是自定义的虚拟世界的地图数据。地图信息可以用于地面交通管制、车辆导航、车辆行车路线规划等。
在一些示例中，地图信息至少包括道路层信息和车道层信息，道路层信息包括多条与路口连通的道路，车道层包括多条车道，车道为路口中连通道路的部分路段。
可以理解的是，路口指多条道路交叉汇合所形成的区域，路口中则表示多条道路交叉汇合所形成的区域范围内，则车道为多条道路交叉汇合形成的区域内连通多条道路的路段，用于约束车辆行驶向多条道路中的某一条道路。
例如，图6中示出了一种路口的地图。道路层信息包括多条与路口连通的道路，例如，道路1、道路2、道路3、道路4。车道层信息包括路口中多条与道路连通的车道，例如与道路1连通的车道11、车道12，与道路2连通的车道21、车道22，与道路3连通的车道31、车道32，与道路4连通的车道41、车道42。容易理解的是，图6仅示出了目标车辆当前所处位置(道路2的外侧车道)可能会行驶的车道，也就是说，以目标车辆当前位置为起点的车道，并不代表路口的地图信息中的所有车道。
在一个示例中，地图信息包括高精地图，该高精地图可以包括静态高精地图层和动态高精地图层。
其中，静态高精地图层包括道路部件层、车道层、道路属性层等包含静态信息的地图层。具体的，车道层中可以包含道路细节信息，如车道线、车道中心线、车道宽度、曲率、坡度、航向、车道规则等信息。道路层部件层可以包含交通标志牌、路面标志等道路部件，比如记录交通信号灯的精确位置以及高度等等。
其中，动态高精地图层可以包括道路拥堵层、施工情况层、交通事故层、交通管制层、天气层等包含动态交通信息的地图层。比如，施工情况层中可以包含如整修、道路标识线磨损及重漆、交通标示改变等信息。
当地图信息包括高精地图时，获取路口的地图信息的方式可以通过直接调取/下载的方式获得。例如，基于目标车辆的位置信息，调取存储器中存储的该位置信息对应的路口的地图信息。或者，基于目标车辆的位置信息，从云端服务器下载该位置信息对应的路口的地图信息。
在另一个示例中，地图信息不包括高精地图，地图信息包括常规地图，包括道路层信息，但是不包括高精地图中路口场景中的虚拟车道线。此时，获取地图信息的方式为，先基于目标车辆的位置信息，调取存储器中存储的该位置信息对应的路口的地图信息，或者基于目标车辆的位置信息，从云端服务器下载该位置信息对应的路口的地图信息。然后，利用拓扑分析的方法确定车道层的车道信息。例如，参见图6中，拓扑分析得到车道车道11、车道12、车道21、车道22、车道31、车道32、车道41、车道42的起点和终点位置，再基于三次样条插值法确定车道11、车道12、车道21、车道22、车道31、车道32、车道41、车道42的车道线。
或者，获取地图信息的方式还可以为：传感器采集车辆周围的环境信息，例如路口的多条道路信息，基于该多条道路信息构建道路层信息，再基于拓扑分析确定车道层的车道信息，最终自行构建得到路口的地图信息。本申请并不对地图信息的获取方式做限定，只要可以实现获取地图信息即可。
如此，当无法获取高精地图，或高精地图中的车道层信息缺失时，本申请实施例的车辆行驶意图预测方法依然可以顺利执行，不依赖于高精地图，泛化性更好。
在步骤S502中，获取目标车辆的行驶信息和目标车辆的周边车辆的行驶信息。
在本实施例中，目标车辆可以理解为对本车行驶有较大影响的车辆，例如，在本车的车前的车辆。目标车辆的周围车辆可以理解为对目标车辆周围一定距离的其他车辆，该距离可以由用户设置，也可以由技术人员设置，还可以与本车传感器感应距离有关。
容易理解的是，目标车辆和目标车辆的周边车辆可以为自动驾驶车辆、非自动驾 驶车辆、新能源车辆、燃油车辆等中的任意车辆，本申请不做限定。
在一个示例中，目标车辆或周边车辆的行驶信息包括可被本车感测到的影响目标车辆行驶意图的信息，例如，目标车辆和周边车辆的位置信息、行驶速度信息、行驶方向信息、车头朝向信息等。
获取目标车辆的行驶信息和目标车辆的周边车辆的行驶信息后，可根据目标车辆的行驶信息和目标车辆的周边车辆的行驶信息，确定目标车辆的周边车辆相对于目标车辆的行驶特征，即目标车辆与其他车辆的交互特征。
在一个示例中，目标车辆与其他车辆的交互特征的获取方法可根据下述规则提取：提取周边车辆中的每个车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，第一坐标系的原点为目标车辆的当前位置、第一坐标系为直角坐标系，第一坐标系的y轴与目标车辆的车身的长度方向平行，且y轴的正向与目标车辆的车头朝向一致。
在步骤S503中，基于目标车辆的行驶信息和车道层信息，确定目标车辆相对于每条车道的行驶特征，即目标车辆与每条车道的交互特征。
在一个示例中，目标车辆与每条车道的交互特征可根据下列规则进行提取：提取目标车辆在每个第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在每个第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述每个第三坐标系为frenet坐标系，所述每个第三坐标系的参考线基于所述每条车道的中心线确定，所述每个第三坐标系的原点基于所述每条车道的中心线的终点确定。
容易理解的是，车道的中心线是指从车道的起点到终点，由车道的宽度方向上的中心点依次连接而成的线。
以图7为例，说明在第三坐标系下目标位置坐标的计算方法。该计算方法如下所示：
Figure PCTCN2021078239-appb-000001
x＝|PR|
其中L j为虚拟车道线线段长度，j＝0,1,2,3,4；L 5为前一段线段终点与目标投影点的距离；|·|为向量长度。
在步骤S504中，基于目标车辆的行驶信息和道路层信息，确定目标车辆相对于每条道路的行驶特征，即目标车辆与每条道路的交互特征。
在一个示例中，目标车辆与每条道路的交互特征可根据下列规则进行提取：目标车辆在每个第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及目标车辆在每个第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，每个第二坐标系为直角坐标系，每个第二坐标系的原点基于每条道路的出口的位置确定，x轴方向基于每条道路的行驶方向确定。
可以理解的是，上述目标车辆与其他车辆的交互特征、目标车辆与每条车道的交互特征和目标车辆与每条道路的交互特征的提取规则仅为一种提取规则举例，也可以 采用其他提取规则分别提取目标车辆与其他车辆的交互特征、目标车辆与每条车道的交互特征和目标车辆与每条道路的交互特征，本申请实施例不做限定。
在步骤S505中，基于目标车辆与其他车辆的交互特征、目标车辆与每条车道的交互特征和目标车辆与每条道路的交互特征，确定目标车辆的道路意图和目标车辆的车道意图。
其中，所述目标车辆的道路意图表征目标车辆通过每条道路驶离路口的概率分布，目标车辆的车道意图表征目标车辆通过每条车道驶离路口的概率分布。
具体的，将目标车辆与其他车辆中每个车辆的交互特征输入目标车辆与其他车辆的交互特征向量提取网络，提取得到目标车辆与其他车辆的交互特征向量，该向量表征其他车辆对目标车辆的行驶意图的影响。
在一个示例中，目标车辆与其他车辆的交互特征向量提取网络包括多个特征提取子网络和交互特征向量预测网络。将其他车辆的每个车辆相对于目标车辆的行驶特征分别输入多个特征提取子网络，得到其他车辆中每个车辆相对于目标车辆的行驶特征向量；将所述其他车辆中每个车辆相对于目标车辆的行驶特征向量输入交互特征向量预测网络，得到其他车辆与目标车辆的交互特征向量。
如图8所示，特征提取子网络基于多层感知机网络(multi layer perceptron，MLP)和循环神经网络(recurrent neural network，RNN)构建，交互特征向量预测网络基于注意力机制网络(Attention in Neural Networks，ANN)构建。目标车辆与其他车辆的交互特征向量算法如下：
Figure PCTCN2021078239-appb-000002
A j＝RNN(MLP(a j))
β j＝softmax(MLP(A j))
其中a j表示其他车辆中任一车辆相对于目标车辆的行驶特征，A j表征其他车辆中任一车辆相对于目标车辆的行驶特征经过MLP和RNN提取特征得到的其他车辆中任一车辆相对于目标车辆的行驶特征向量，β j表征其他车辆中任一车辆相对于目标车辆的行驶特征向量经过注意力机制网络归一化分类得到的加权系数，
Figure PCTCN2021078239-appb-000003
表征目标车辆与其他车辆的交互特征向量。
由于目标车辆与其他车辆的交互特征可能会影响目标车辆的车道意图的预测，因此，需要将计算得出的目标车辆与其他车辆的交互特征向量作为目标车辆的车道意图预测的考虑因素。
另外为了增加车道意图预测的准确性，还考虑目标车辆与车道的交互特征和目标车辆与该车道相关联的道路的交互特征对车道意图预测的影响。
在一个示例中，构建车道意图预测网络，将目标车辆与其他车辆的交互特征向量、目标车辆与每条车道的交互特征和目标车辆与与每条车道相关联的道路的交互特征输入训练好的车道意图预测网络，确定目标车辆的车道意图和目标车辆与每条车道的交互特征隐态向量。
其中，与每条车道相关联的道路可以理解为该车道连通的道路，换句话说，与每条车道相关联的道路为该车道通向的道路，例如，在图6中，与车道11相关联的道路为道路1，与车道21相关联的道路为道路2。
例如，将目标车辆与其他车辆的交互特征向量，目标车辆与车道11的交互特征和目标车辆与道路1(即与车道11相关的道路)的交互特征，输入训练好的车道意图预测网络，得到目标车辆通过车道11驶离路口的概率和目标车辆与车道11的交互特征隐态向量。类似的，采用相同的方法得到目标车辆通过其他车道驶离路口的概率和目标车辆与其他车道的交互特征隐态向量。
在一个示例中，车道意图预测网络至少包括第一车道特征提取子网络、第二车道提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网络。
将目标车辆与每条车道的交互特征输入第一车道特征提取子网络，提取得到目标车辆与每条车道的交互特征向量；将目标车辆与每条车道的交互特征向量继续输入至第二车道特征提取子网络，提取得到目标车辆与每条车道的交互特征隐态向量。
将目标车辆与与每条车道相关联的道路的交互特征输入第一道路特征提取子网络，提取得到目标车辆与与每条车道相关联的道路的交互特征向量；将目标车辆与与每条车道相关联的道路的交互特征向量输入第二道路特征提取子网络，提取得到目标车辆与与每条车道相关联的道路的交互特征隐态向量。
将目标车辆与每条车道的交互特征向量、目标车辆与每条车道的交互特征向量、目标车辆与每条车道的交互特征隐态向量和目标车辆与与每条车道相关联的道路的交互特征隐态向量输入车道意图预测子网络，确定目标车辆的车道意图。
在一个示例中，如图9所示，第一车道特征提取子网络、第一道路特征提取子网络均基于MLP构建，第二车道提取子网络、第二道路特征提取子网络均基于RNN构建，车道意图预测子网络基于注意力机制网络构建。目标车辆的车道意图预测算法如下所示：
Figure PCTCN2021078239-appb-000004
Figure PCTCN2021078239-appb-000005
Figure PCTCN2021078239-appb-000006
Figure PCTCN2021078239-appb-000007
Figure PCTCN2021078239-appb-000008
其中，
Figure PCTCN2021078239-appb-000009
表征当前时刻的目标车辆与第i条车道的交互特征，
Figure PCTCN2021078239-appb-000010
表征当前时刻的目标车辆与第i条车道的交互特征经过MLP网络提取得到的当前时刻的目标车辆与第i条车道的交互特征向量，
Figure PCTCN2021078239-appb-000011
表征前一时刻的目标车辆与第i条车道的交互特征隐态向量，
Figure PCTCN2021078239-appb-000012
表征当前时刻的目标车辆与第i条车道的交互特征隐态向量，
Figure PCTCN2021078239-appb-000013
表征当前时刻的目标车辆与第j条道路的交互特征(第j条道路为与第i条车道相关联的道路)，
Figure PCTCN2021078239-appb-000014
表征前一时刻的目标车辆与第j条道路的交互特征隐态向量，
Figure PCTCN2021078239-appb-000015
表征当前时刻的目标车辆与第j条道路的交互特征隐态向量，α ji表征
Figure PCTCN2021078239-appb-000016
输入车道意图预测子网络后得到的目标车辆从第i条车道驶离路口的概率，也就是说第i条车道对应的目标车辆的车道意图。
在另一示例中，构建道路意图预测网络，将目标车辆与每条道路的交互特征、与每条道路相关联的多个车道对应的目标车辆与每条车道的交互特征隐态向量和与每条道路相关联的车道对应的目标车辆的车道意图输入道路意图预测网络，确定目标车辆的道路意图。
其中，与每条道路相关联的车道可以理解为与每条道路连通的车道，也就是说通向该道路的车道。例如，在图6中，与道路1相关联的车道至少为车道11和车道12，与道路2相关联的车道至少为车道21，车道22。
在一个示例中，道路意图预测网络至少包括第一道路特征提取子网络、第二道路特征提取子网络和道路意图预测子网络。
将所述目标车辆与每条道路的交互特征输入第一道路特征提取子网络，提取得到目标车辆与每条道路的交互特征向量；
将目标车辆与每条道路的交互特征向量输入第二道路特征提取子网络，提取得到目标车辆与每条道路的交互特征隐态向量；
根据与每条道路相关联的车道对应的车道意图，对所有与每条道路相关联的车道对应的目标车辆与每条车道的交互特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与每条道路相关联的车道的隐态融合向量。例如，与第j条道路相关联的车道一共有k条车道，包括车道1、车道2、〃〃〃车道k，从车道意图网络中得到车道1、车道2、〃〃〃车道k的交互特征隐态向量以及车道1、车道2、〃〃〃车道k的车道意图，根据车道1、车道2、〃〃〃车道k的车道意图对车道1、车道2、〃〃〃车道k的交互特征隐态向量进行加权处理，将加权处理结果进行拼接得到与第j条道路相关联的车道的隐态融合向量。与其他道路相关联的车道的隐态融合向量的获取方法与之类似。
将目标车辆与每条道路的交互特征向量和与每条道路相关联的车道的隐态融合向量输入道路意图预测子网络，确定目标车辆的道路意图。
在一个示例中，如图10所示，第一道路特征提取子网络基于MLP构建，第二道路特征提取子网络基于RNN构建，道路意图预测子网络基于注意力机制网络构建。目标车辆的道路意图预测算法如下所示：
Figure PCTCN2021078239-appb-000017
Figure PCTCN2021078239-appb-000018
其中，
Figure PCTCN2021078239-appb-000019
表征表征当前时刻的目标车辆与第j条道路的交互特征隐态向量，
Figure PCTCN2021078239-appb-000020
表征目标车辆与第i条车道的交互特征隐态向量，α ji表征第i条车道对应的目标车辆的车道意图，
Figure PCTCN2021078239-appb-000021
表征所有与第j条道路相关联的车道对应的目标车辆与第i条车道的交互特征隐态向量的融合向量，β j表征
Figure PCTCN2021078239-appb-000022
输入道路意图预测子网络后得到的标车辆从第j条道路驶离路口的概率，也就是说第j条道路对应的目标车辆的道路意图。
回到图在步骤5，在步骤S506中，基于目标车辆的道路意图和目标车辆的车道意图，确定目标车辆的行驶意图。
通过目标车辆的道路意图和目标车辆的车道意图，确定最终的目标车辆的行驶意图，表征目标车辆未来的行驶轨迹，也就是说，目标车辆驶离路口的行车轨迹。
具体的，首先，根据目标车辆的道路意图中概率最大值对应的道路，确定目标道路；然后将与目标道路关联的车道对应的目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；最后基于目标道路和目标车道，确定目标车辆的行驶意图。即将目标车道作为目标车辆的行驶轨迹，目标道路作为目标车辆将要驶向的道路。
应理解的，上述各步骤的序号的大小并不意味着执行顺序的先后，各步骤的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定，例如步骤S502可以在步骤S501之前执行，即先获取路口的地图信息，再获取目标车辆的行驶信息和目标车辆的周边车辆的行驶信息。
本申请实施通过对目标车辆的车道意图和道路意图的预测，来确定目标车辆的行驶意图，一方面，能够有效、精确表示目标车辆的行驶行为，能够适应不同结构的路口场景，避免预先定义的固定类别的意图对目标车辆行驶行为描述的不准确和模糊性。另一方面，道路意图和车道意图的互相辅助修正，提高最终预测的目标车辆的行驶意图的准确性。
本申请中的目标模型包括，第一交互特征提取网络、车道意图预测网络、道路预测网络，可以是通过大量训练样本数据训练得到训练完成的第一交互特征提取网络、车道意图预测网络、道路预测网络。例如，训练数据可以包括探测到目标车辆和其他车辆当前的行驶信息和路口地图信息，以及预设时段后目标车辆的行驶信息。对输入的目标车辆和其他车辆当前的行驶信息和路口地图信息进行处理，将输出的目标车辆的行驶意图与预设时段后目标车辆实际的行驶信息进行比对，直到输出的目标车辆的行驶意图与目标车辆实际的行驶信息的差值小于一定阈值，从而完成目标模型的训练。基于训练数据训练得到目标模型，该目标模型可以用于根据目标车辆和其他车辆当前的行驶信息和路口的地图信息，预测车辆的行驶意图。
或者，当训练到预设次数时，判断目标模型训练完成。例如，目标模型中的MLP为单层全连接网络，隐单元维度为64，激活函数为Relu，目标模型中的RNN采用GRU，隐单元维度为128。注意力机制模块中计算权重系数的MLP隐单元维度为1，经过softmax后为归一化后的目标意图概率。利用开源深度学习框架实现车辆高阶意图预测网络，网络训练采用多任务学习方式，损失函数包括道路级意图交叉熵以及车道级意图交叉熵，训练数据的批大小为512，初始学习率为0.001，学习率采用指数递减形式变化，衰减步长设置为10轮，总训练轮数为50轮，训练一轮指的是所有训练数据遍历一次。
图11为本申请实施例提供的一种车辆行驶意图预测装置的结构示意图。如图11所示，该车辆行驶意图预测装置1100至少包括：
第一获取模块1101,用于当目标车辆行驶至路口时，获取所述路口的地图信息；其中，所述路口的地图信息包括道路层信息和车道层信息，所述道路层信息包括多条与所述路口连通的道路，所述车道层信息包括多条车道，所述车道为所述路口中连通所 述道路的部分路段；
第二获取模块1102，用于获取目标车辆的行驶信息；
第一特征提取模块1103，用于基于所述目标车辆的行驶信息和所述车道层信息，确定所述目标车辆相对于每条车道的行驶特征；
第二特征提取模块1104，用于基于所述目标车辆的行驶信息和所述道路层信息，确定所述目标车辆相对于每条道路的行驶特征；
预测模块1105，用于基于周边车辆中每个车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于每条道路的行驶特征和所述目标车辆相对于每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图；其中，所述周边车辆为所述目标车辆的预设范围内的车辆，所述目标车辆的道路意图表征所述目标车辆通过所述每条道路驶离所述路口的概率分布，所述目标车辆的车道意图表征所述目标车辆通过所述每条车道驶离所述路口的概率分布；
确定模块1106，用于基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
在一个可能的实现中，所述预测模块1105具体用于：将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，所述周边车辆与所述目标车辆的交互特征向量表征所述周边车辆对所述目标车辆的影响；
将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于每条车道的行驶特征和所述目标车辆相对于与所述每条车道相关联的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于每条车道的行驶特征隐态向量；
将所述目标车辆相对于每条道路的行驶特征、与所述每条道路相关联的多个车道对应的所述目标车辆相对于每条车道的行驶特征隐态向量和与所述每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图。
在另一个可能的实现中，所述第一交互特征提取网络至少包括多个第一特征提取子网络和交互特征向量预测网络；
所述将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，包括：
将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征分别输入所述多个第一特征提取子网络，确定所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量；
将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量输入交互特征向量预测网络，确定所述周边车辆与所述目标车辆的交互特征向量。
在另一个可能的实现中，所述车道意图预测网络至少包括第一车道特征提取子网络、第二车道特征提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网；
所述将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于每条 车道的行驶特征和所述目标车辆相对于与所述每条车道相关的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于每条车道的行驶特征隐态向量，包括：
将所述目标车辆相对于每条车道的行驶特征输入第一车道特征提取子网络，确定所述目标车辆相对于每条车道的行驶特征向量；
将所述目标车辆相对于每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于每条车道的行驶特征隐态向量；
将所述目标车辆相对于与每条车道相关联的道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于与每条车道相关联的道路的行驶特征向量；
将所述目标车辆相对于与每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与每条车道相关联的道路的行驶特征隐态向量；
将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于每条车道的行驶特征向量、所述目标车辆相对于每条车道的行驶特征隐态向量和所述目标车辆相对于与每条车道相关联的道路的行驶特征隐态向量输入车道意图预测子网络，确定所述目标车辆的车道意图。
在另一个可能的实现中，所述将所述目标车辆相对于每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于每条车道的行驶特征隐态向量，包括：
所述第二车道特征提取子网络中多个特征提取窗口对所述目标车辆相对于每条车道的行驶特征向量按照驾驶时刻顺序进行特征提取；
根据当前特征提取窗口对应的所述目标车辆相对于每条车道的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于每条车道的行驶特征隐态向量。
在另一个可能的实现中，所述将所述目标车辆相对于与每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与每条车道相关联的道路的行驶特征隐态向量，包括：
所述第二道路特征提取子网络中多个特征提取窗口对所述目标车辆相对于与每条车道相关联的道路的行驶特征向量按照驾驶时刻顺序进行特征提取；
根据当前特征提取窗口对应的所述目标车辆相对于与每条车道相关联的道路的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于与每条车道相关联的道路的行驶特征隐态向量。
在另一个可能的实现中，所述道路意图预测网络至少包括：所述第一道路特征提取子网络、所述第二道路特征提取子网络和道路意图预测子网络；
所述将所述目标车辆相对于每条道路的行驶特征、与所述每条道路相关联的多个车道对应的所述目标车辆相对于每条车道的行驶特征隐态向量和与所述每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图，包括：
将所述目标车辆相对于每条道路的行驶特征输入第一道路特征提取子网络，确定 所述目标车辆相对于每条道路的行驶特征向量；
将所述目标车辆相对于每条道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于每条道路的行驶特征隐态向量；
根据与所述每条道路相关联的车道对应的车道意图，对与所述每条道路相关联的车道对应的所述目标车辆相对于每条车道的行驶特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与所述每条道路相关联的车道的隐态融合向量；
将所述目标车辆相对于每条道路的行驶特征向量和所述与所述每条道路相关联的车道的隐态融合向量输入所述道路意图预测子网络，确定所述目标车辆的道路意图。
在另一个可能的实现中，所述第一特征提取子网络至少基于多层感知机网络和循环神经网络构建；
所述第一车道特征提取子网络和第一道路特征提取子网络至少基于多层感知机网络构建，所述第二车道特征提取子网络和第二道路特征提取子网络至少基于循环神经网络构建；
所述交互特征向量预测网络、所述车道意图预测子网络和所述道路意图预测子网络均至少基于注意力机制网络构建。
在另一个可能的实现中，所述确定模块1106具体用于：
将所述目标车辆的道路意图中概率最大值对应的道路，确定为目标道路；
将与所述目标道路关联的车道对应的所述目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；基于目标道路和目标车道，确定所述目标车辆的行驶意图。
在另一个可能的实现中，所述周边车辆中每个车辆相对于所述目标车辆的行驶特征，包括：
所述周边车辆中的每个车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，所述第一坐标系的原点为所述目标车辆的当前位置、所述第一坐标系为直角坐标系，所述第一坐标系的y轴与所述目标车辆的车身的长度方向平行，且y轴的正向与所述目标车辆的车头朝向一致。
在另一个可能的实现中，所述目标车辆相对于每条道路的行驶特征，包括：
所述目标车辆在每个第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及所述目标车辆在每个第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，所述每个第二坐标系为直角坐标系，所述每个第二坐标系的原点基于所述每条道路的出口的位置确定，x轴方向基于所述每条道路的行驶方向确定。
在另一个可能的实现中，所述目标车辆相对于每条车道的行驶特征，包括：
所述目标车辆在每个第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在每个第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述每个第三坐标系为frenet坐标系，所述每个第三坐标系的参考线基于所述每条车道的中心线确定，所述每个第三坐标系的原点基于所述每条车道的中心线的终点确定。
在另一个可能的实现中，所述每条车道基于对所述多条道路进行拓扑分析确定。
根据本申请实施例的车辆行驶意图预测装置1100可对应于执行本申请实施例中描述的方法，并且车辆行驶意图预测装置1100中的每个模块的上述和其它操作和/或功能分别为了实现图4-10中的每个方法的相应流程，为了简洁，在此不再赘述。
另外需说明的是，以上所描述的实施例仅仅是示意性的，其中所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的设备实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
本申请还提供一种车辆行驶意图预测终端，包括存储器和处理器，存储器中存储有可执行代码，处理器执行该可执行代码，实现上述任一项方法。
可以理解的是，车辆行驶意图预测终端可以为具有车辆行驶意图预测功能的车载终端，也可以为具有车辆行驶意图预测功能的车辆。
本申请还提供一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序在计算机中执行时，令计算机执行上述任一项方法。
本申请还提供一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括指令，当该指令执行时，令计算机执行上述任一项方法。
本申请还提供一种电子设备，包括存储器和处理器，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码，实现上述任一项方法。
本领域普通技术人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令处理器完成，所述的程序可以存储于计算机可读存储介质中，所述存储介质是非短暂性(英文：non-transitory)介质，例如随机存取存储器，只读存储器，快闪存储器，硬盘，固态硬盘，磁带(英文：magnetic tape)，软盘(英文：floppy disk)，光盘(英文：optical disc)及其任意组合。
以上所述，仅为本申请较佳的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。

Claims (30)

  1. 一种车辆行驶意图预测方法，其特征在于，所述方法包括：
    当目标车辆行驶至路口时，获取所述路口的地图信息；其中，所述路口的地图信息包括道路层信息和车道层信息，所述道路层信息包括与所述路口连通的多条道路，所述车道层信息包括多条车道，所述车道为所述路口中连通所述道路的部分路段；
    获取目标车辆的行驶信息；
    基于所述目标车辆的行驶信息和所述车道层信息，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征；
    基于所述目标车辆的行驶信息和所述道路层信息，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征；
    基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图；其中，所述周边车辆为所述目标车辆的预设范围内的车辆，所述目标车辆的车道意图表征所述目标车辆通过所述多条车道中每条车道驶离所述路口的概率分布；
    至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
  2. 根据权利要求1所述的方法，其特征在于，所述基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图，包括：
    基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图；其中，所述目标车辆的道路意图表征所述目标车辆通过所述多条道路中每条道路驶离所述路口的概率分布；
    所述至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图，包括：
    基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
  3. 根据权利要求2所述的方法，其特征在于，所述基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图，包括：
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，所述周边车辆与所述目标车辆的交互特征向量表征所述周边车辆对所述目标车辆的影响；
    将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中 每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图。
  4. 根据权利要求3所述的方法，其特征在于，所述第一交互特征提取网络至少包括多个第一特征提取子网络和交互特征向量预测网络；
    所述将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，包括：
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征分别输入所述多个第一特征提取子网络，确定所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量；
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量输入交互特征向量预测网络，确定所述周边车辆与所述目标车辆的交互特征向量。
  5. 根据权利要求3或4任一所述的方法，其特征在于，所述车道意图预测网络至少包括第一车道特征提取子网络、第二车道特征提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网；
    将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：
    将所述目标车辆相对于所述多条车道中每条车道的行驶特征输入第一车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征向量；
    将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；
    将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量；
    将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量；
    将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量输入车道意图预测子网络，确定所述目标车辆的车道意图。
  6. 根据权利要求5所述的方法，其特征在于，所述将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：
    所述第二车道特征提取子网络中多个特征提取窗口对所述目标车辆相对于所述多条车道中每条车道的行驶特征向量按照驾驶时刻顺序进行特征提取；
    根据当前特征提取窗口对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量。
  7. 根据权利要求5或6所述的方法，其特征在于，所述将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量，包括：
    所述第二道路特征提取子网络中多个特征提取窗口对所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量按照驾驶时刻顺序进行特征提取；
    根据当前特征提取窗口对应的所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量。
  8. 根据权利要求5-7任一所述的方法，其特征在于，所述道路意图预测网络至少包括：所述第一道路特征提取子网络、所述第二道路特征提取子网络和道路意图预测子网络；
    所述将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图，包括：
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征隐态向量；
    根据与所述多条道路中每条道路相关联的车道对应的车道意图，对与所述多条道路中每条道路相关联的车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与所述多条道路中每条道路相关联的车道的隐态融合向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量和所述与所述多条道路中每条道路相关联的车道的隐态融合向量输入所述道路意图预测子网络，确定所述目标车辆的道路意图。
  9. 根据权利要求8所述的方法，其特征在于，所述第一特征提取子网络至少基于多层感知机网络和循环神经网络构建；
    所述第一车道特征提取子网络和第一道路特征提取子网络至少基于多层感知机网络构建，所述第二车道特征提取子网络和第二道路特征提取子网络至少基于循环神经网络构建；
    所述交互特征向量预测网络、所述车道意图预测子网络和所述道路意图预测子网络均至少基于注意力机制网络构建。
  10. 根据权利要求2-9任一所述的方法，其特征在于，所述基于所述目标车辆的 道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图，包括：
    将所述目标车辆的道路意图中概率最大值对应的道路，确定为目标道路；
    将与所述目标道路关联的车道对应的所述目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；
    基于目标道路和目标车道，确定所述目标车辆的行驶意图。
  11. 根据权利要求1-10任一所述的方法，其特征在于，所述周边车辆相对于所述目标车辆的行驶特征，包括：
    所述周边车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，所述第一坐标系的原点为所述目标车辆的当前位置、所述第一坐标系为直角坐标系，所述第一坐标系的y轴与所述目标车辆的车身的长度方向平行，且y轴的正向与所述目标车辆的车头朝向一致。
  12. 根据权利要求1-11任一所述的方法，其特征在于，所述目标车辆相对于所述多条道路中每条道路的行驶特征，包括：
    所述目标车辆在第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及所述目标车辆在第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，所述第二坐标系为直角坐标系，所述第二坐标系的原点基于所述多条道路中每条道路的出口的位置确定，x轴方向基于所述多条道路中每条道路的行驶方向确定。
  13. 根据权利要求1-12任一所述的方法，其特征在于，所述目标车辆相对于所述多条车道中每条车道的行驶特征，包括：
    所述目标车辆在第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述第三坐标系为frenet坐标系，所述第三坐标系的参考线基于所述多条车道中每条车道的中心线确定，所述第三坐标系的原点基于所述多条车道中每条车道的中心线的终点确定。
  14. 根据权利要求1-13任一所述的方法，其特征在于，所述多条车道基于对所述多条道路进行拓扑分析确定。
  15. 一种车辆行驶意图预测装置，其特征在于，所述装置包括：
    第一获取模块,用于当目标车辆行驶至路口时，获取所述路口的地图信息；其中，所述路口的地图信息包括道路层信息和车道层信息，所述道路层信息包括与所述路口连通的多条道路，所述车道层信息包括多条车道，所述车道为所述路口中连通所述道路的部分路段；
    第二获取模块，用于获取目标车辆的行驶信息；
    第一特征提取模块，用于基于所述目标车辆的行驶信息和所述车道层信息，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征；
    第二特征提取模块，用于基于所述目标车辆的行驶信息和所述道路层信息，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征；
    预测模块，用于基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相 对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，至少确定所述目标车辆的车道意图；其中，所述周边车辆为所述目标车辆的预设范围内的车辆，所述目标车辆的车道意图表征所述目标车辆通过所述多条车道中每条车道驶离所述路口的概率分布；
    确定模块，用于至少基于所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
  16. 根据权利要求15所述的装置，其特征在于，所述预测模块具体用于：
    基于周边车辆相对于所述目标车辆的行驶特征、所述目标车辆相对于所述多条道路中每条道路的行驶特征和所述目标车辆相对于所述多条车道中每条车道的行驶特征，确定所述目标车辆的道路意图和所述目标车辆的车道意图；其中，所述目标车辆的道路意图表征所述目标车辆通过所述多条道路中每条道路驶离所述路口的概率分布；
    所述确定模块具体用于：
    基于所述目标车辆的道路意图和所述目标车辆的车道意图，确定所述目标车辆的行驶意图。
  17. 根据权利要求16所述的装置，其特征在于，所述预测模块具体用于：
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，所述周边车辆与所述目标车辆的交互特征向量表征所述周边车辆对所述目标车辆的影响；
    将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图。
  18. 根据权利要求17所述的装置，其特征在于，所述第一交互特征提取网络至少包括多个第一特征提取子网络和交互特征向量预测网络；
    所述将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征输入第一交互特征提取网络，确定所述周边车辆与所述目标车辆的交互特征向量，包括：
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征分别输入所述多个第一特征提取子网络，确定所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量；
    将所述周边车辆中每个车辆相对于所述目标车辆的行驶特征向量输入交互特征向量预测网络，确定所述周边车辆与所述目标车辆的交互特征向量。
  19. 根据权利要求17或18所述的装置，其特征在于，所述车道意图预测网络至少包括第一车道特征提取子网络、第二车道特征提取子网络、第一道路特征提取子网络、第二道路特征提取子网络和车道意图预测子网；
    所述将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述 多条车道中每条车道的行驶特征和所述目标车辆相对于与所述多条车道中每条车道相关的道路的行驶特征输入车道意图预测网络，确定所述目标车辆的车道意图和所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：
    将所述目标车辆相对于所述多条车道中每条车道的行驶特征输入第一车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征向量；
    将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量；
    将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量；
    将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量；
    将所述周边车辆与所述目标车辆的交互特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征向量、所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量输入车道意图预测子网络，确定所述目标车辆的车道意图。
  20. 根据权利要求19所述的装置，其特征在于，所述将所述目标车辆相对于所述多条车道中每条车道的行驶特征向量输入第二车道特征提取子网络，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量，包括：
    所述第二车道特征提取子网络中多个特征提取窗口对所述目标车辆相对于所述多条车道中每条车道的行驶特征向量按照驾驶时刻顺序进行特征提取；
    根据当前特征提取窗口对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量。
  21. 根据权利要求19或20所述的装置，其特征在于，所述将所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量，包括：
    所述第二道路特征提取子网络中多个特征提取窗口对所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量按照驾驶时刻顺序进行特征提取；
    根据当前特征提取窗口对应的所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征向量和前一特征提取窗口输出的隐态向量，确定所述目标车辆相对于与所述多条车道中每条车道相关联的道路的行驶特征隐态向量。
  22. 根据权利要求19-21任一所述的装置，其特征在于，所述道路意图预测网络至少包括：所述第一道路特征提取子网络、所述第二道路特征提取子网络和道路意图预测子网络；
    所述将所述目标车辆相对于所述多条道路中每条道路的行驶特征、与所述多条道 路中每条道路相关联的多个车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量和与所述多条道路中每条道路相关联的车道对应的所述目标车辆的车道意图输入道路意图预测网络，确定所述目标车辆的道路意图，包括：
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征输入第一道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量输入第二道路特征提取子网络，确定所述目标车辆相对于所述多条道路中每条道路的行驶特征隐态向量；
    根据与所述多条道路中每条道路相关联的车道对应的车道意图，对与所述多条道路中每条道路相关联的车道对应的所述目标车辆相对于所述多条车道中每条车道的行驶特征隐态向量进行加权处理，并将加权处理结果进行拼接，以获得与所述多条道路中每条道路相关联的车道的隐态融合向量；
    将所述目标车辆相对于所述多条道路中每条道路的行驶特征向量和所述与所述多条道路中每条道路相关联的车道的隐态融合向量输入所述道路意图预测子网络，确定所述目标车辆的道路意图。
  23. 根据权利要求22所述的装置，其特征在于，所述第一特征提取子网络至少基于多层感知机网络和循环神经网络构建；
    所述第一车道特征提取子网络和第一道路特征提取子网络至少基于多层感知机网络构建，所述第二车道特征提取子网络和第二道路特征提取子网络至少基于循环神经网络构建；
    所述交互特征向量预测网络、所述车道意图预测子网络和所述道路意图预测子网络均至少基于注意力机制网络构建。
  24. 根据权利要求16-23任一所述的装置，其特征在于，所述确定模块具体用于：将所述目标车辆的道路意图中概率最大值对应的道路，确定为目标道路；
    将与所述目标道路关联的车道对应的所述目标车辆的车道意图中的概率最大值对应的车道，确定为目标车道；
    基于目标道路和目标车道，确定所述目标车辆的行驶意图。
  25. 根据权利要求15-24任一所述的装置，其特征在于，所述周边车辆相对于所述目标车辆的行驶特征，包括：
    所述周边车辆在第一坐标系的位置特征、速度特征和车头朝向特征中的一个或多个，其中，所述第一坐标系的原点为所述目标车辆的当前位置、所述第一坐标系为直角坐标系，所述第一坐标系的y轴与所述目标车辆的车身的长度方向平行，且y轴的正向与所述目标车辆的车头朝向一致。
  26. 根据权利要求15-25任一所述的装置，其特征在于，所述目标车辆相对于所述多条道路中每条道路的行驶特征，包括：
    所述目标车辆在第二坐标系中的位置特征、目标车辆与原点的距离特征、目标车辆的车头朝向特征、以及所述目标车辆在第二坐标系中的位置、目标车辆与原点的距离、目标车辆的车头朝向随驾驶时刻的变化特征中的一个或多个，其中，所述第二坐标系为直角坐标系，所述第二坐标系的原点基于所述多条道路中每条道路的出口的位 置确定，x轴方向基于所述多条道路中每条道路的行驶方向确定。
  27. 根据权利要求15-26任一所述的装置，其特征在于，所述目标车辆相对于所述多条车道中每条车道的行驶特征，包括：
    所述目标车辆在第三坐标系中的位置特征、目标车辆的车头朝向与车道的行驶方向的形成的角度特征以及所述目标车辆在第三坐标系中的位置、目标车辆的车头朝向与车道的行驶方向的形成的角度随驾驶时刻的变化特征中的一个或多个，其中，所述第三坐标系为frenet坐标系，所述第三坐标系的参考线基于所述多条车道中每条车道的中心线确定，所述第三坐标系的原点基于所述多条车道中每条车道的中心线的终点确定。
  28. 根据权利要求15-27任一所述的装置，其特征在于，所述多条车道基于对所述多条道路进行拓扑分析确定。
  29. 一种车辆行驶意图预测终端，包括存储器和处理器，其特征在于，所述存储器中存储有可执行代码，所述处理器执行所述可执行代码，实现权利要求1-14任一项所述的方法。
  30. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，当所述计算机程序在计算机中执行时，令计算机执行权利要求1-14任一项所述的方法。
PCT/CN2021/078239 2021-02-26 2021-02-26 一种车辆行驶意图预测方法、装置、终端及存储介质 WO2022178858A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2021/078239 WO2022178858A1 (zh) 2021-02-26 2021-02-26 一种车辆行驶意图预测方法、装置、终端及存储介质
CN202180055239.7A CN116261649A (zh) 2021-02-26 2021-02-26 一种车辆行驶意图预测方法、装置、终端及存储介质
EP21927294.5A EP4286972A4 (en) 2021-02-26 2021-02-26 METHOD AND DEVICE FOR PREDICTING THE DRIVING INTENTION OF A VEHICLE, TERMINAL AND STORAGE MEDIUM
US18/456,681 US20230399023A1 (en) 2021-02-26 2023-08-28 Vehicle Driving Intention Prediction Method, Apparatus, and Terminal, and Storage Medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/078239 WO2022178858A1 (zh) 2021-02-26 2021-02-26 一种车辆行驶意图预测方法、装置、终端及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/456,681 Continuation US20230399023A1 (en) 2021-02-26 2023-08-28 Vehicle Driving Intention Prediction Method, Apparatus, and Terminal, and Storage Medium

Publications (1)

Publication Number Publication Date
WO2022178858A1 true WO2022178858A1 (zh) 2022-09-01

Family

ID=83047686

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/078239 WO2022178858A1 (zh) 2021-02-26 2021-02-26 一种车辆行驶意图预测方法、装置、终端及存储介质

Country Status (4)

Country Link
US (1) US20230399023A1 (zh)
EP (1) EP4286972A4 (zh)
CN (1) CN116261649A (zh)
WO (1) WO2022178858A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115906265A (zh) * 2022-12-27 2023-04-04 中交第二公路勘察设计研究院有限公司 一种基于换道行为特征的近主线出口标线优化方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116541715B (zh) * 2023-07-05 2023-09-29 苏州浪潮智能科技有限公司 目标检测方法、模型的训练方法、目标检测系统及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108550279A (zh) * 2018-04-03 2018-09-18 同济大学 基于机器学习的车辆驾驶行为预测方法
CN110060467A (zh) * 2018-01-19 2019-07-26 本田技研工业株式会社 预测装置、预测方法及存储介质
US20190333373A1 (en) * 2017-01-20 2019-10-31 Nissan Motor Co., Ltd. Vehicle Behavior Prediction Method and Vehicle Behavior Prediction Apparatus
CN110400490A (zh) * 2019-08-08 2019-11-01 腾讯科技（深圳）有限公司 轨迹预测方法和装置
CN110843789A (zh) * 2019-11-19 2020-02-28 苏州智加科技有限公司 一种基于时序卷积网络的车辆换道意图预测方法
CN111653114A (zh) * 2020-06-12 2020-09-11 北京百度网讯科技有限公司 信息提醒方法、装置、电子设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9248834B1 (en) * 2014-10-02 2016-02-02 Google Inc. Predicting trajectories of objects based on contextual information

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190333373A1 (en) * 2017-01-20 2019-10-31 Nissan Motor Co., Ltd. Vehicle Behavior Prediction Method and Vehicle Behavior Prediction Apparatus
CN110060467A (zh) * 2018-01-19 2019-07-26 本田技研工业株式会社 预测装置、预测方法及存储介质
CN108550279A (zh) * 2018-04-03 2018-09-18 同济大学 基于机器学习的车辆驾驶行为预测方法
CN110400490A (zh) * 2019-08-08 2019-11-01 腾讯科技（深圳）有限公司 轨迹预测方法和装置
CN110843789A (zh) * 2019-11-19 2020-02-28 苏州智加科技有限公司 一种基于时序卷积网络的车辆换道意图预测方法
CN111653114A (zh) * 2020-06-12 2020-09-11 北京百度网讯科技有限公司 信息提醒方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4286972A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115906265A (zh) * 2022-12-27 2023-04-04 中交第二公路勘察设计研究院有限公司 一种基于换道行为特征的近主线出口标线优化方法

Also Published As

Publication number Publication date
EP4286972A4 (en) 2024-03-27
EP4286972A1 (en) 2023-12-06
CN116261649A (zh) 2023-06-13
US20230399023A1 (en) 2023-12-14

Similar Documents

Publication Publication Date Title
CN109901574B (zh) 自动驾驶方法及装置
KR102070527B1 (ko) 자율 주행 차량 교통 예측에서 예측되는 궤적에 대한 평가 프레임 워크
CN110379193B (zh) 自动驾驶车辆的行为规划方法及行为规划装置
WO2022001773A1 (zh) 轨迹预测方法及装置
EP3218890B1 (en) Hyper-class augmented and regularized deep learning for fine-grained image classification
CN109901572B (zh) 自动驾驶方法、训练方法及相关装置
WO2021102955A1 (zh) 车辆的路径规划方法以及车辆的路径规划装置
CN110371132B (zh) 驾驶员接管评估方法及装置
WO2021000800A1 (zh) 道路可行驶区域推理方法及装置
WO2021212379A1 (zh) 车道线检测方法及装置
WO2021217420A1 (zh) 车道线跟踪方法和装置
CN112512887B (zh) 一种行驶决策选择方法以及装置
CN112534483B (zh) 预测车辆驶出口的方法和装置
US20230399023A1 (en) Vehicle Driving Intention Prediction Method, Apparatus, and Terminal, and Storage Medium
WO2022142839A1 (zh) 一种图像处理方法、装置以及智能汽车
WO2022051951A1 (zh) 车道线检测方法、相关设备及计算机可读存储介质
US20230048680A1 (en) Method and apparatus for passing through barrier gate crossbar by vehicle
CN113631452A (zh) 一种变道区域获取方法以及装置
WO2022017307A1 (zh) 自动驾驶场景生成方法、装置及系统
CN112810603A (zh) 定位方法和相关产品
CN114440908A (zh) 一种规划车辆驾驶路径的方法、装置、智能车以及存储介质
CN114261404A (zh) 一种自动驾驶方法及相关装置
CN114167404A (zh) 目标跟踪方法及装置
WO2022001432A1 (zh) 推理车道的方法、训练车道推理模型的方法及装置
CN114556251B (zh) 用于确定车辆可通行空间的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21927294

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021927294

Country of ref document: EP

Effective date: 20230901

NENP Non-entry into the national phase

Ref country code: DE