WO2021093335A1 - Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle - Google Patents

Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle Download PDF

Info

Publication number
WO2021093335A1
WO2021093335A1 PCT/CN2020/098164 CN2020098164W WO2021093335A1 WO 2021093335 A1 WO2021093335 A1 WO 2021093335A1 CN 2020098164 W CN2020098164 W CN 2020098164W WO 2021093335 A1 WO2021093335 A1 WO 2021093335A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
lane
frame
lane change
trajectory data
Prior art date
Application number
PCT/CN2020/098164
Other languages
French (fr)
Inventor
Xiaofeng Zhang
Di Jiang
Chen Zhao
Pingliang HAN
Wei Wang
Original Assignee
Suzhou Zhijia Science & Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Zhijia Science & Technologies Co., Ltd. filed Critical Suzhou Zhijia Science & Technologies Co., Ltd.
Publication of WO2021093335A1 publication Critical patent/WO2021093335A1/en

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34Route searching; Route guidance
    • G01C21/36Input/output arrangements for on-board computers
    • G01C21/3626Details of the output of route guidance instructions
    • G01C21/3658Lane guidance

Definitions

  • the present disclosure belongs to the technical field of unmanned driving vehicles, and in particular relates to a method for automatically labeling a lane changing intention based on trajectory data of a vehicle.
  • Autonomous vehicles also referred to as unmanned vehicles, may be driven partly, or entirely, without the assistance of a human driver.
  • the autonomous vehicle needs to be able to predict the future intentions and trajectories of surrounding vehicles.
  • the traveling intentions of the surrounding vehicles have an important impact on decision-making and planning of the autonomous vehicle, and driving intentions of them may affect each other.
  • Human drivers can make predictions about the traveling intentions and future trajectories of surrounding vehicles based on the driver’s own experience and observations. In this way, based upon these predictions, human drivers can make important traveling decisions such as overtaking, decelerating, or changing lanes.
  • Existing driving assistance systems usually lack such a predictive capability, which leaves traveling decisions entirely to human drivers.
  • it is desirable for unmanned driving vehicles to have such decision-making capability which requires the ability to predict the future intention and trajectory of surrounding vehicles.
  • a rule-based algorithm is the “gap acceptance model” which assumes that a lane changing motivation of a driver is based on lead and lag gaps of a target lane. In this method, it is assumed that when the gap reaches the minimum acceptable value, the driver tends to make a lane change.
  • the method is characterized by simple and convenient determination of a vehicle intention, such a method requires a large amount of cumbersome and time-consuming fine-tuning of parameters.
  • learning-based algorithms use a function or a neural network which may be trained or modeled using a large amount of data. Learning based algorithms require a large amount of training data in order to perfect the model.
  • One approach is to manually label public data sets (such as NGSIM US101) , but this requires a large amount of labor, and such public data sets are generally collected by using fixed sensors on the road which is very different from the type of sensor data collected by the sensors of an autonomous vehicle in practice.
  • an actual angle of the vehicle is very small during a lane change, and the change of angle is difficult to recognize in a noisy data set; and 2. in this method, a threshold needs to be fixed for determining the change of the angle, but vehicles have different positions and speeds during lane changes, so the method is not suitable for changeable road conditions.
  • the present application proposes a method for automatically labeling a lane change intention of a vehicle based on the vehicle’s trajectory data.
  • the vehicle whose lane change intention is to be labeled may be a vehicle in the surroundings of an autonomous vehicle and thus may be referred to as a surrounding vehicle.
  • the method comprises:
  • the labeled frame is labeled as a lane changing frame or a lane keeping frame.
  • the tracking trajectory data is collected by a sensor system of an autonomous vehicle in a test drive of the autonomous vehicle. This way the tracking and trajectory data may accurately reflect the type of data an autonomous vehicle will receive in practice.
  • This method may be performed for a plurality of surrounding vehicles. In this way tracking trajectory data of a plurality of vehicles can be identified and classified or labeled, thereby generating training data for a learning based vehicle intention and trajectory prediction model.
  • the output of the method may comprise labeled tracking and trajectory data including a plurality of frames, with each frame relating to a surrounding vehicle at a point in time, and each frame being labeled as a lane changing frame or a lane keeping frame.
  • the approach of this method may be able to cope with noise and is fairly robust as it is able to detect an essence of lane changing behaviors which may then be labeled.
  • the method may be implemented by a computing device or processor executing machine readable instructions.
  • a second aspect of the present disclosure provides a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform the method of the first aspect of the present disclosure.
  • preprocessing of the tracking trajectory data comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data: data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
  • data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
  • the method comprises, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle.
  • verifying the effective lane changing trajectory may comprise discarding lane change trajectories in which a period of time, or number of frames, between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
  • finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1) th frame where the surrounding vehicle is in a first lane and a nth frame where the surrounding vehicle is in a second lane and defining the nth frame as the point of lane change of the surrounding vehicle.
  • the method calculates an average value of a distance between a vehicle and the road center over time and determines the start of a lane change based on changes in this average value. This use of an average value may further enhance the ability of the method to cope with noise.
  • determining the starting point of lane change comprises:
  • the present disclosure is extremely robust to extracting lane change trajectories from noisy tracking trajectory data of vehicles, a generalized definition is proposed for lane changing behaviors of vehicles, and labeling validity is verified during a labeling process, so that a large number of correct and effective lane change frames and lane keeping frames can be labeled without human intervention.
  • Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention of the present disclosure.
  • Fig. 2 is a schematic diagram of a lane keeping trajectory and a lane change trajectory in the tracking data of the vehicle of the present disclosure, where the lane keeping trajectory is on the left side and the lane changing trajectory is on the right side.
  • Fig. 3 is a schematic diagram of a Cartesian coordinate system and a Frenet coordinate system.
  • Fig. 4 shows determination of a lane changing frame in the method for automatically labeling a lane changing intention of the present disclosure.
  • Fig. 5 shows determination of a lane keeping frame in the method for automatically labeling a lane changing intention of the present disclosure.
  • Fig. 6 is a schematic diagram of labeling intention of a vehicle on an NGSIM data set in an automatic labeling method in the prior art.
  • Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle according to the present disclosure, and the method for automatically labeling a lane changing intention includes the following blocks:
  • labeled output frames may be generated as a training set for input into a neural network, which may be then be trained to identify a lane change intention of a vehicle.
  • This method is fairly robust and may be able to label even high-noise trajectory data.
  • High noise trajectory data is data in which the trajectory of a vehicle is not smooth. In practice vehicles often meander rather than traveling in a straight line. As shown in Fig. 2, not every change in angle ends with a lane change, while sometimes a vehicle moves left and right within a lane before finally changing lane. Such data may be considered to be high-noise.
  • Example implementations of blocks S1 to S5 will now be discussed.
  • Block S1 comprises collecting and preprocessing tracking trajectory data of surrounding vehicles.
  • the tracking trajectory data of the surrounding vehicles may be generated by a sensing system of a vehicle during a test drive of the vehicle.
  • the tracking trajectory data may include one or more of a map with road information, physical information of the vehicle (for example, a vehicle type, a vehicle size such as a length, a width and a height) , a position of the vehicle (based on a Frenet road coordinate system, or a Cartesian world coordinate system) , a lane in which the vehicle is located, a speed (in one embodiment, the speed may be based on the Frenet road coordinate system) , an orientation of the vehicle (in one embodiment, the orientation may be based on the Frenet road coordinate system) , etc.; which are generated after tracking process is performed on the vehicle after data sensed by a camera, a laser radar, a millimeter wave radar, etc.
  • fuse refers to a calculation which combines data derived from separate sources.
  • camera and radar data may be fused together so that both an object (such as a surrounding vehicle) and a distance to the object may be detected.
  • the tracking trajectory data of the surrounding vehicles may be high-noise tracking trajectory data.
  • Fig. 2 shows example of high noise tracking and trajectory data, in which the left side of Fig. 2 depicts a lane keeping trajectory and the right side of Fig. 2 depicts a lane changing trajectory.
  • the Frenet road coordinate system may be used.
  • the Cartesian coordinate system is usually used to describe a position of an object, but it is not the optimal choice for unmanned driving. Unmanned driving cars usually make acceleration or lane changing decisions based on the road.
  • the Frenet coordinate system uses a center line of the road as a reference line, and uses a tangential direction and a normal direction of the reference line to define the coordinate system. As shown in Fig. 3, a distance in the tangential direction S and a lateral distance L of the vehicle relative to the center line of the road are coordinates of the vehicle in the Frenet coordinate system.
  • the preprocessing is performed on the tracking trajectory data of the surrounding vehicles.
  • the pre-processing is to screen out data and may include:
  • Block S2 comprises finding a lane changing point.
  • a time point at which the vehicle crosses different lanes may be found, e.g., the location of the vehicle at the (n-1) th frame may be defined as a lane A, and the position of the vehicle at the nth frame may be defined as a lane B, and the nth frame may be defined as the lane changing point of the vehicle.
  • a vehicle may have one or more lane changing points along the entire trajectory. In other examples, lane keeping behavior may be maintained along the entire trajectory such that no lane changing points exist.
  • a left side of Fig. 4 shows tracking trajectory data of a lane change in a road 400 between a first lane and a second lane.
  • Reference numeral 401 indicates a center line of the first lane
  • reference numeral 402 indicates a center line of the second lane.
  • a trajectory 410 of a vehicle changing lane includes a plurality of points or frames indicated by dots and reference numerals 411.
  • the point of lane change 440 is denoted by a triangle in Fig. 4 and has already been determined in block S2 of the method.
  • Block S3 comprises determining a starting point of the lane change.
  • the method of determining the starting point of lane change may comprise a basic model (a) and a modification of the basic model (b) to enable the model to better cope with noise.
  • a basic model is made for a behavior of the lane changing vehicle in the present disclosure: according to this model when the lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases.
  • a target lane i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases.
  • a rhombus 420 in Fig. 4 is an actual starting point of lane change, while positions corresponding to rectangles 430 are misjudged as starting points of lane change according to this basic model. Therefore, the basic model can be improved.
  • the improved model (b) is discussed below.
  • the present disclosure proposes a method for determining a movement trend of a vehicle.
  • An average value Lmean for a plurality of frames (e.g. 5 frames) in coordinates of direction L is used as a basis for determination, which greatly increases robustness of determination of a lane changing behavior.
  • the smoother curve 450 on the right in Fig. 4 is a trajectory graph drawn with the alternative Lmean. Compared with the original trajectory 410 adjacent to the curve, it can be seen that the new trajectory graph is more robust to noise.
  • Block S4 verifies an effective lane changing trajectory. For example, a lane changing trajectory which is too short or too long (for example, t ⁇ 10 or t > 100) is discarded and is not considered, where t corresponds to a number of frames in the lane changing trajectory. Whether the vehicle is still in the current lane is verified in each frame in (b) in block S3. If another lane change occurs during a lane changing backstepping process, the backstepping is immediately terminated and t is recorded. If t is too short or too long, the track is discarded. This verification can effectively filter out excessive noise or incomplete data.
  • a lane changing trajectory which is too short or too long for example, t ⁇ 10 or t > 100
  • t corresponds to a number of frames in the lane changing trajectory.
  • FIG. 5 shows an example of a lane changing trajectory 510 between a first lane and a second lane of a road 500, in which a center of the first lane is denoted by reference numeral 501 and a center of the second lane is denoted by a reference numeral 502
  • a triangle point 520 at the front end is an actual point of lane change
  • the other triangles 510 are points of lane change caused by noises.
  • the right side of Fig. 5 depicts a lane keeping trajectory 520 on the same road, in which triangles 510 are points of lane change which are falsely identified due to noise.
  • lane changing trajectories shown in Fig. 5 are discarded and are not considered, thus ensuring the correctness of the training data. For example, through the back stepping process which traverses frames prior to an identified point of lane change (e.g. one of the triangles) it may be found that the lane changing trajectories shown in Fig. 5 are too short, so they may be discarded.
  • an identified point of lane change e.g. one of the triangles
  • Block S5 outputs at least one of a labeled lane changing frame and labeled lane keeping frame.
  • all frames [n-t, n] from the start of the lane change of the vehicle to the point of lane change are defined as lane changing frames of the vehicle, and the others are lane keeping frames.
  • an ending point of lane change is not set, where the ending point of lane change is the point of lane change, because an actual behavior of the vehicle after the point of lane change is lane keeping until the next lane change.
  • the method may be performed on a computing device, by executing instructions by a processor of the computing device.
  • a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform any of the above methods is provided.
  • the technical solutions discussed above are robust in extracting lane changing trajectories from noisy tracking trajectory data of vehicles. They use a generalized definition for lane changing behaviors of vehicles, and validity of a labeling is verified during the labeling process, so that a large number of correct and effective lane changing frames and lane keeping frames can be labeled without human intervention.
  • a change in an average value of a distance between a vehicle and the road center over time is used as to determine the start of a lane change. In this way the influence of noise may be reduced and a robust model provided which focuses on an essence of lane changing behaviors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Abstract

A method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle is provided. The method includes: collecting and preprocessing tracking trajectory data of a surrounding vehicle (S1); finding a point of lane change (S2); determining a starting point of lane change (S3); verifying an effective trajectory (S4); and outputting at least one labeled frame of the vehicle (S5).

Description

METHOD FOR AUTOMATICALLY LABELING LANE CHANGING INTENTION BASED ON HIGH-NOISE TRAJECTORY DATA OF VEHICLE TECHNICAL FIELD
The present disclosure belongs to the technical field of unmanned driving vehicles, and in particular relates to a method for automatically labeling a lane changing intention based on trajectory data of a vehicle.
BACKGROUND
Autonomous vehicles, also referred to as unmanned vehicles, may be driven partly, or entirely, without the assistance of a human driver. In order for such unmanned or autonomous driving technologies to be able to safely and efficiently pass through complex traffic scenarios and make optimal decisions, the autonomous vehicle needs to be able to predict the future intentions and trajectories of surrounding vehicles. The traveling intentions of the surrounding vehicles have an important impact on decision-making and planning of the autonomous vehicle, and driving intentions of them may affect each other. Human drivers can make predictions about the traveling intentions and future trajectories of surrounding vehicles based on the driver’s own experience and observations. In this way, based upon these predictions, human drivers can make important traveling decisions such as overtaking, decelerating, or changing lanes. Existing driving assistance systems usually lack such a predictive capability, which leaves traveling decisions entirely to human drivers. However, it is desirable for unmanned driving vehicles to have such decision-making capability which requires the ability to predict the future intention and trajectory of surrounding vehicles.
Existing intention prediction algorithms are generally divided into two types: a rule-based algorithm and a learning-based algorithm. One example of a rule-based algorithm, is the “gap acceptance model” which assumes that a lane changing motivation of a driver is based on lead and lag gaps of a target lane. In this method, it is assumed that when the gap reaches the minimum acceptable value, the driver tends to make a lane change. Although the method is characterized by simple and convenient determination of a vehicle intention, such a method requires a large amount of cumbersome and time-consuming fine-tuning of parameters. In contrast, learning-based algorithms, use a function or a neural network which may be trained or modeled using a large amount of data. Learning based algorithms require a large amount of training data in order to perfect the model.
Training data needs to be labeled and how to obtain a large amount of labeled data is one of the difficulties of such methods. One approach is to manually label public data sets (such as NGSIM US101) , but this requires a large amount of labor, and such public data sets are generally collected by using fixed sensors on the road which is very different from the type of sensor data collected by the sensors of an autonomous vehicle in practice.
In an article published in 2018 and titled “Learning Vehicle Cooperative Lane-changing Behavior from Observed Trajectories in the NGSIM Dataset” (with authors Su, Shuang &Mülling, Katharina &Dolan, John &Palanisamy, Praveen &Mudalige, Priyantha) , an automatic labeling method is used to perform vehicle intention labeling on an NGSIM data set. As shown in Fig. 6, a change in an angle between a vehicle orientation and the road is used as a start sign of a lane change, but this is impractical in actual application,  which has the following two problems: 1. an actual angle of the vehicle is very small during a lane change, and the change of angle is difficult to recognize in a noisy data set; and 2. in this method, a threshold needs to be fixed for determining the change of the angle, but vehicles have different positions and speeds during lane changes, so the method is not suitable for changeable road conditions.
SUMMARY
Accordingly the present application proposes a method for automatically labeling a lane change intention of a vehicle based on the vehicle’s trajectory data. The vehicle whose lane change intention is to be labeled may be a vehicle in the surroundings of an autonomous vehicle and thus may be referred to as a surrounding vehicle.
In one example the method comprises:
collecting and preprocessing tracking trajectory data of a surrounding vehicle;
finding a point of lane change of the surrounding vehicle;
determining a starting point of the lane change; and
outputting at least one labeled frame of the surrounding vehicle, wherein the labeled frame is labeled as a lane changing frame or a lane keeping frame.
In one example the tracking trajectory data is collected by a sensor system of an autonomous vehicle in a test drive of the autonomous vehicle. This way the tracking and trajectory data may accurately reflect the type of data an autonomous vehicle will receive in practice. This method may be performed for a plurality of surrounding vehicles. In this way tracking trajectory data of a plurality of vehicles can be identified and classified or labeled, thereby generating training data for a learning based vehicle intention and trajectory prediction model. The output of the method may comprise labeled tracking and trajectory data including a plurality of frames, with each frame relating to a surrounding vehicle at a point in time, and each frame being labeled as a lane changing frame or a lane keeping frame.
The approach of this method may be able to cope with noise and is fairly robust as it is able to detect an essence of lane changing behaviors which may then be labeled.
Due to the large volumes of data, it may not be possible for a human to label the data accurately in a reasonable amount of time and a human labeler may miss subtle indications of lane change, which may be detectable by a computer. Accordingly the method may be implemented by a computing device or processor executing machine readable instructions.
A second aspect of the present disclosure provides a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform the method of the first aspect of the present disclosure.
In one example, preprocessing of the tracking trajectory data, comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data: data relating to a surrounding vehicle which cannot be located on a known map, data relating to a surrounding vehicle which is stationary; and data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data. In this way the amount of data to be labeled is reduced thus saving processing power and memory. Furthermore, a richer and more useful training set may be obtained as the most useful portions of the collected data are retained.
In one example, the method comprises, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle. This may help to eliminate data which is ambiguous and which may confound a training model. For example, verifying the effective lane changing trajectory may comprise discarding lane change trajectories in which a period of time, or number of frames, between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
In one example finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1) th frame where the surrounding vehicle is in a first lane and a nth frame where the surrounding vehicle is in a second lane and defining the nth frame as the point of lane change of the surrounding vehicle.
In one example, the method calculates an average value of a distance between a vehicle and the road center over time and determines the start of a lane change based on changes in this average value. This use of an average value may further enhance the ability of the method to cope with noise.
In one example, determining the starting point of lane change comprises:
a) determining, for each frame of a plurality of frames, a lane in which the surrounding vehicle is located;
b) determining, for each frame of the plurality of frames, lateral distance L of the surrounding vehicle relative to the lane in which the surrounding vehicle is located;
c) determining, for each frame, an average value of the lateral distance, Lmean, based on an average of the lateral distance L of the current frame a number of previous frames; and
d) determining the starting point of lane change, by traversing frames going backwards in time from the point of lane change, and selecting as the starting point of lane change: a first frame in which the average value of the lateral distance, Lmean, has not decreased relative to the previously traversed frame, or a first frame in which the average value of the lateral distance, Lmean, is at the center of the previous lane.
The advantages of the present disclosure lie in that: the present disclosure is extremely robust to extracting lane change trajectories from noisy tracking trajectory data of vehicles, a generalized definition is proposed for lane changing behaviors of vehicles, and labeling validity is verified during a labeling process, so that a large number of correct and effective lane change frames and lane keeping frames can be labeled without human intervention.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention of the present disclosure.
Fig. 2 is a schematic diagram of a lane keeping trajectory and a lane change trajectory in the tracking data of the vehicle of the present disclosure, where the lane keeping trajectory is on the left side and the lane changing trajectory is on the right side.
Fig. 3 is a schematic diagram of a Cartesian coordinate system and a Frenet coordinate system.
Fig. 4 shows determination of a lane changing frame in the method for automatically labeling a lane changing intention of the present disclosure.
Fig. 5 shows determination of a lane keeping frame in the method for automatically labeling a lane  changing intention of the present disclosure.
Fig. 6 is a schematic diagram of labeling intention of a vehicle on an NGSIM data set in an automatic labeling method in the prior art.
DETAILED DESCRIPTION OF PREFERRED IMPLEMENTATIONS
The following further describes a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle in conjunction with Figs. 1 to 5. It should be noted that the embodiments described below with reference to the drawings are exemplary and are used to explain the present disclosure, but cannot be construed as limiting the present disclosure.
Fig. 1 is a flowchart of a method for automatically labeling a lane changing intention based on high-noise trajectory data of a vehicle according to the present disclosure, and the method for automatically labeling a lane changing intention includes the following blocks:
S1, collecting and preprocessing tracking trajectory data of surrounding vehicles;
S2, finding a lane change point;
S3, determining a lane change starting point;
S4, verifying an effective lane change trajectory; and
S5, outputting at least one of a labeled lane change frame and a labeled lane keeping frame.
In this way labeled output frames may be generated as a training set for input into a neural network, which may be then be trained to identify a lane change intention of a vehicle. This method is fairly robust and may be able to label even high-noise trajectory data. High noise trajectory data is data in which the trajectory of a vehicle is not smooth. In practice vehicles often meander rather than traveling in a straight line. As shown in Fig. 2, not every change in angle ends with a lane change, while sometimes a vehicle moves left and right within a lane before finally changing lane. Such data may be considered to be high-noise. Example implementations of blocks S1 to S5 will now be discussed.
Block S1 comprises collecting and preprocessing tracking trajectory data of surrounding vehicles. The tracking trajectory data of the surrounding vehicles may be generated by a sensing system of a vehicle during a test drive of the vehicle. The tracking trajectory data may include one or more of a map with road information, physical information of the vehicle (for example, a vehicle type, a vehicle size such as a length, a width and a height) , a position of the vehicle (based on a Frenet road coordinate system, or a Cartesian world coordinate system) , a lane in which the vehicle is located, a speed (in one embodiment, the speed may be based on the Frenet road coordinate system) , an orientation of the vehicle (in one embodiment, the orientation may be based on the Frenet road coordinate system) , etc.; which are generated after tracking process is performed on the vehicle after data sensed by a camera, a laser radar, a millimeter wave radar, etc. is fused. As described herein and in the claims, “fuse” refers to a calculation which combines data derived from separate sources. For example, camera and radar data may be fused together so that both an object (such as a surrounding vehicle) and a distance to the object may be detected. The tracking trajectory data of the surrounding vehicles may be high-noise tracking trajectory data. Fig. 2 shows example of high noise tracking and trajectory data, in which the left side of Fig. 2 depicts a lane keeping trajectory and the right side of Fig. 2 depicts a lane changing trajectory.
In some examples of the present disclosure, the Frenet road coordinate system may be used. The  Cartesian coordinate system is usually used to describe a position of an object, but it is not the optimal choice for unmanned driving. Unmanned driving cars usually make acceleration or lane changing decisions based on the road. As an alternative solution to the Cartesian coordinate system, the Frenet coordinate system uses a center line of the road as a reference line, and uses a tangential direction and a normal direction of the reference line to define the coordinate system. As shown in Fig. 3, a distance in the tangential direction S and a lateral distance L of the vehicle relative to the center line of the road are coordinates of the vehicle in the Frenet coordinate system.
In block S1, the preprocessing is performed on the tracking trajectory data of the surrounding vehicles. In one example the pre-processing is to screen out data and may include:
(1) only selecting trajectory data with map information, wherein when a vehicle cannot be positioned on a known map, a trajectory of the vehicle is not selected;
(2) not selecting a trajectory of a stationary vehicle because a lane changing behavior and a lane keeping behavior of a vehicle are only meaningful to moving vehicles;
(3) only selecting a tracking trajectory of a vehicle within a certain distance (such as, within 100 meters) from an unmanned driving vehicle. Because noises in tracking data will be too large in the case of more than 100 meters, and the tracking data is not suitable for being used as training data.
Block S2 comprises finding a lane changing point. For example, taking the vehicle as a unit, a time point at which the vehicle crosses different lanes may be found, e.g., the location of the vehicle at the (n-1) th frame may be defined as a lane A, and the position of the vehicle at the nth frame may be defined as a lane B, and the nth frame may be defined as the lane changing point of the vehicle. A vehicle may have one or more lane changing points along the entire trajectory. In other examples, lane keeping behavior may be maintained along the entire trajectory such that no lane changing points exist.
A left side of Fig. 4 shows tracking trajectory data of a lane change in a road 400 between a first lane and a second lane. Reference numeral 401 indicates a center line of the first lane, while reference numeral 402 indicates a center line of the second lane. A trajectory 410 of a vehicle changing lane includes a plurality of points or frames indicated by dots and reference numerals 411. The point of lane change 440 is denoted by a triangle in Fig. 4 and has already been determined in block S2 of the method.
Block S3 comprises determining a starting point of the lane change. The method of determining the starting point of lane change may comprise a basic model (a) and a modification of the basic model (b) to enable the model to better cope with noise.
(a) A basic model is made for a behavior of the lane changing vehicle in the present disclosure: according to this model when the lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., a lateral distance L of the vehicle from a current lane increases and a lateral distance L of the vehicle from the target lane decreases. However, this is not necessarily true in noisy tracking data. There may be a large number of “jumps” in positioning of the vehicle in the Frenet road coordinate system, which includes jumps in both the direction S and in the direction L. A rhombus 420 in Fig. 4 is an actual starting point of lane change, while positions corresponding to rectangles 430 are misjudged as starting points of lane change according to this basic model. Therefore, the basic model can be improved. The improved model (b) is discussed below.
(b) The present disclosure proposes a method for determining a movement trend of a vehicle. An  average value Lmean for a plurality of frames (e.g. 5 frames) in coordinates of direction L is used as a basis for determination, which greatly increases robustness of determination of a lane changing behavior. Starting from the nth frame of the point of lane change of the vehicle (the triangle on the left in Fig. 4) and sequentially calculating Lmean of the (n-1) th frame, the (n-2) th frame, . . ., etc. of the vehicle relative to the current lane, wherein according to the definition in (a) , a distance of the vehicle in direction L at the (n-1) th frame relative to the current lane should be less than a distance of the vehicle in direction L at the nth frame relative to the current lane, a distance in the direction L for the (n-2) th frame should be less than a distance in direction L for the (n-1) th frame, and so on, until calculation is performed on the (n-t) th frame; and determining the (n-t) th frame as the starting point of lane change of the vehicle when Lmean no longer decreases progressively or reaches a center line of the lane in which the vehicle is located (in this time, L=0) . The smoother curve 450 on the right in Fig. 4 is a trajectory graph drawn with the alternative Lmean. Compared with the original trajectory 410 adjacent to the curve, it can be seen that the new trajectory graph is more robust to noise.
Block S4, verifies an effective lane changing trajectory. For example, a lane changing trajectory which is too short or too long (for example, t < 10 or t > 100) is discarded and is not considered, where t corresponds to a number of frames in the lane changing trajectory. Whether the vehicle is still in the current lane is verified in each frame in (b) in block S3. If another lane change occurs during a lane changing backstepping process, the backstepping is immediately terminated and t is recorded. If t is too short or too long, the track is discarded. This verification can effectively filter out excessive noise or incomplete data. The left side of Fig. 5 shows an example of a lane changing trajectory 510 between a first lane and a second lane of a road 500, in which a center of the first lane is denoted by reference numeral 501 and a center of the second lane is denoted by a reference numeral 502 As shown in the left side of Fig. 5, in the lane changing trajectory, a triangle point 520 at the front end is an actual point of lane change, and the other triangles 510 are points of lane change caused by noises. The right side of Fig. 5 depicts a lane keeping trajectory 520 on the same road, in which triangles 510 are points of lane change which are falsely identified due to noise. In block S4, both trajectories shown in Fig. 5 are discarded and are not considered, thus ensuring the correctness of the training data. For example, through the back stepping process which traverses frames prior to an identified point of lane change (e.g. one of the triangles) it may be found that the lane changing trajectories shown in Fig. 5 are too short, so they may be discarded.
Block S5, outputs at least one of a labeled lane changing frame and labeled lane keeping frame. For example, all frames [n-t, n] from the start of the lane change of the vehicle to the point of lane change are defined as lane changing frames of the vehicle, and the others are lane keeping frames. In the present disclosure, in one example, an ending point of lane change is not set, where the ending point of lane change is the point of lane change, because an actual behavior of the vehicle after the point of lane change is lane keeping until the next lane change.
The method may be performed on a computing device, by executing instructions by a processor of the computing device. In one example, a non-transitory machine readable storage medium storing instructions which are executable by a processor to perform any of the above methods is provided.
The technical solutions discussed above are robust in extracting lane changing trajectories from noisy tracking trajectory data of vehicles. They use a generalized definition for lane changing behaviors of vehicles, and validity of a labeling is verified during the labeling process, so that a large number of correct and  effective lane changing frames and lane keeping frames can be labeled without human intervention.
Specific parameters in the present disclosure can be adjusted, which is also applicable to low-noise trajectory data of vehicle and not limited to tracking data.
In the present disclosure, a change in an average value of a distance between a vehicle and the road center over time is used as to determine the start of a lane change. In this way the influence of noise may be reduced and a robust model provided which focuses on an essence of lane changing behaviors.
The embodiment described above is used to explain the present disclosure, rather than limit the present disclosure. Within the spirit of the present disclosure and the protection scope of the claims, any modifications and changes made to the present disclosure fall into the protection scope of the present disclosure.

Claims (17)

  1. A method for automatically labeling a lane changing intention of a vehicle based on trajectory data of the vehicle, the method comprising:
    collecting and preprocessing tracking trajectory data of a surrounding vehicle;
    finding a point of lane change of the surrounding vehicle;
    determining a starting point of lane change;
    and
    outputting at least one labeled frame of the vehicle, wherein the labeled frame is labeled as a lane changing frame or a lane keeping frame.
  2. The method of claim 1, wherein the tracking trajectory data of the surrounding vehicle is generated by a sensing system of an autonomous vehicle during a test drive of the autonomous vehicle, and wherein the tracking trajectory data includes one or more of: a map with road information, physical information of the surrounding vehicle, a position of the surrounding vehicle, a lane in which the surrounding vehicle is located, a speed of the surrounding vehicle, and an orientation of the surrounding vehicle.
  3. The method of claim 1 or 2, wherein the preprocessing of the tracking trajectory data comprises screening the tracking and trajectory data to remove one or more of the following types of tracking and trajectory data:
    data relating to a surrounding vehicle which cannot be located on a known map,
    data relating to a surrounding vehicle which is stationary; and
    data relating to a surrounding vehicle which is more than a predetermined distance away from an autonomous vehicle which collected the tracking and trajectory data.
  4. The method of claim 1 or 2 wherein pre-processing of the tracking and trajectory data includes selecting tracking and trajectory data which relates to a surrounding vehicle which is moving and can be positioned on a known map and is within a predetermined distance of an autonomous vehicle which collected the tracking and trajectory data.
  5. The method of any of the above claims, wherein finding the point of lane change of the surrounding vehicle comprises identifying a position of the surrounding vehicle in a plurality of frames corresponding to successive points in time, finding an (n-1)  th frame where the surrounding vehicle is in a first lane and a n th frame where the surrounding vehicle is in a second lane and defining the n th frame as the point of lane change of the surrounding vehicle.
  6. The method of any of the above claims wherein determining the starting point of lane change comprises:
    a) determining, for each frame of a plurality of frames, a lane in which the surrounding vehicle is located;
    b) determining, for each frame of the plurality of frames, lateral distance L of the surrounding vehicle relative to the lane in which the surrounding vehicle is located;
    c) determining, for each frame, an average value of the lateral distance, L mean, based on an average of the lateral distance L of the current frame a number of previous frames; and
    d) determining the starting point of lane change, by traversing frames going backwards in time from the point of lane change, and selecting as the starting point of lane change: a first frame in which the average value  of the lateral distance, L mean, has not decreased relative to the previously traversed frame, or a first frame in which the average value of the lateral distance, L mean, is at the center of the previous lane.
  7. The method of any of claims 1-5, wherein determining the starting point of lane change comprises:
    using an average value L mean which is an average of the coordinates of the lane changing vehicle relative to the current lane in a lateral direction L taken over 5 frames, as a basis for determining a starting point of the lane change; calculating L mean starting from a n th frame which is the lane changing point of the lane changing vehicle and sequentially calculating L mean of previous frames including a (n-1)  th frame, a (n-2)  th frame, ..., etc. of the lane changing vehicle, until calculation is performed on the (n-t)  th frame; and determining the (n-t)  th frame as the starting point of the lane change of the lane changing vehicle, wherein the (n-t)  th frame is a frame in which L mean no longer decreases progressively compared to the previously calculated frame or is a frame in which L mean reaches a center line of the lane in which the lane changing vehicle is located.
  8. The method of any one of the above claims further comprising, after determining a starting point of lane change and before outputting at least one labeled frame, verifying an effective lane change lane changing trajectory of the surrounding vehicle.
  9. The method of claim 8, wherein in verifying the effective lane changing trajectory of the surrounding vehicle, comprises discarding lane change trajectories in which a period of time or number of frames between the starting point of the lane change and the point of lane change is below a minimum threshold or above a maximum threshold.
  10. A non-transitory computer readable storage medium storing instructions which are executable by a processor to perform the method of any of claims 1-9.
  11. A method for automatically labeling a lane change intention based on high-noise vehicle trajectory data, comprising the following steps:
    S1, collecting and preprocessing tracking trajectory data of surrounding vehicles;
    S2, finding a lane change point;
    S3, determining a lane change starting point;
    S4, verifying an effective lane change trajectory; and
    S5, outputting a labeled lane change frame and lane keeping frame.
  12. The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S1, the tracking trajectory data of the surrounding vehicles is generated by an unmanned driving sensing system during a drive test, which comprises a map with road information, vehicle physical information, a vehicle position, a lane in which the vehicle is located, a speed, and a vehicle orientation.
  13. The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S1, the preprocessing of the tracking trajectory data comprises the following data screening:
    (1) only selecting trajectory data with map information, wherein when a vehicle cannot be positioned on a known map, a trajectory of the vehicle is not selected;
    (2) skipping selecting a trajectory of a stationary vehicle;
    (3) only selecting a tracking trajectory of a vehicle within a certain distance from an unmanned driving vehicle.
  14. The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein in step S2, taking the vehicle as a unit, a time point for the vehicle to cross different lanes is found, i.e., the vehicle at the (n-1)  th frame is defined as a lane A, and the vehicle at the n th frame is defined as a lane B, and the n th frame is the lane change point of the vehicle.
  15. The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 11, wherein step S3 further comprises:
    (a) making a setting for a behavior of the lane changing vehicle: when a lane changing behavior of the vehicle starts, the vehicle will gradually move towards a target lane, i.e., an L lateral distance of the vehicle from a current lane increases and an L lateral distance of the vehicle from the target lane decreases;
    (b) using an average value L mean for the first five frames of L direction coordinates as a basis for determination; starting from the n th frame of the lane change point of the vehicle and sequentially calculating L mean of the (n-1)  th frame, the (n-2)  th frame, ..., etc. of the vehicle relative to the current lane, wherein according to the definition in (a) , an L direction distance of the vehicle at the (n-1)  th frame relative to the current lane should be less than an L direction distance of the vehicle at the n th frame relative to the current lane, an L direction distance for the (n-2)  th frame should be less than an L direction distance for the (n-3)  th frame, and so on, until calculation is performed on the (n-t)  th frame; and determining the (n-t)  th frame as the lane change starting point of the vehicle when L mean no longer decreases progressively or reaches a center line of the lane in which the vehicle is located.
  16. The method for automatically labeling a lane change intention based on high-noise vehicle trajectory data of claim 15, wherein in step S4, a too short or too long lane change trajectory is discarded and is not considered.
  17. A non-transitory computer readable storage medium storing instructions which are executable by a processor to perform the method of any of claims 11-16.
PCT/CN2020/098164 2019-11-14 2020-06-24 Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle WO2021093335A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911113106.1A CN110986994B (en) 2019-11-14 2019-11-14 Automatic lane change intention marking method based on high-noise vehicle track data
CN201911113106.1 2019-11-14

Publications (1)

Publication Number Publication Date
WO2021093335A1 true WO2021093335A1 (en) 2021-05-20

Family

ID=70084361

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/098164 WO2021093335A1 (en) 2019-11-14 2020-06-24 Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle

Country Status (2)

Country Link
CN (1) CN110986994B (en)
WO (1) WO2021093335A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115482659A (en) * 2022-08-18 2022-12-16 浙江工商大学 Intelligent agent autonomous decision-making method based on deep reinforcement learning

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110986994B (en) * 2019-11-14 2021-08-03 苏州智加科技有限公司 Automatic lane change intention marking method based on high-noise vehicle track data
CN111599182B (en) * 2020-07-22 2020-10-20 中汽院汽车技术有限公司 Automatic lane changing behavior extraction method based on natural driving data

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130345895A1 (en) * 2012-06-20 2013-12-26 Trimble Navigation Limited Lane change monitoring
CN107967486A (en) * 2017-11-17 2018-04-27 江苏大学 A kind of nearby vehicle Activity recognition method based on V2V communications with HMM-GBDT mixed models
US20190084558A1 (en) * 2017-09-19 2019-03-21 Toyota Jidosha Kabushiki Kaisha Vehicle surrounding monitoring device
CN109886304A (en) * 2019-01-22 2019-06-14 江苏大学 A kind of complex road condition following peripheral vehicle behavior recognition methods based on HMM-SVM bilayer improved model
CN109946688A (en) * 2019-03-18 2019-06-28 中国汽车工程研究院股份有限公司 Lane-change contextual data extracting method, device and server
CN110146100A (en) * 2018-02-13 2019-08-20 华为技术有限公司 Trajectory predictions method, apparatus and storage medium
CN110986994A (en) * 2019-11-14 2020-04-10 苏州智加科技有限公司 Automatic lane change intention marking method based on high-noise vehicle track data

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823382B (en) * 2014-02-27 2016-08-03 浙江省科威工程咨询有限公司 A kind of lane-change track optimizing based on vehicle and speed and the Visual Implementation method
KR101617345B1 (en) * 2014-12-03 2016-05-02 현대모비스 주식회사 System and method for estimating driving lane of vehicle
US10077050B2 (en) * 2016-05-24 2018-09-18 GM Global Technology Operations LLC Automated driving system for evaluating lane cut-out and method of using the same
CN107545612A (en) * 2016-06-23 2018-01-05 戚艳红 Make video recording startup method and the activation system of making video recording of a kind of drive recorder
JP7006093B2 (en) * 2017-09-28 2022-02-10 トヨタ自動車株式会社 Driving support device
CN108693883B (en) * 2018-06-06 2021-12-24 苏州智加科技有限公司 High-precision parking method and system for automatic driving of vehicle
CN110103955B (en) * 2019-04-28 2021-04-23 浙江吉利控股集团有限公司 Vehicle early warning method and device and electronic equipment
CN110304075B (en) * 2019-07-04 2020-06-26 清华大学 Vehicle track prediction method based on hybrid dynamic Bayesian network and Gaussian process
CN110329263B (en) * 2019-07-05 2020-08-21 西南交通大学 Self-adaptive track changing planning method for automatic driving vehicle

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130345895A1 (en) * 2012-06-20 2013-12-26 Trimble Navigation Limited Lane change monitoring
US20190084558A1 (en) * 2017-09-19 2019-03-21 Toyota Jidosha Kabushiki Kaisha Vehicle surrounding monitoring device
CN107967486A (en) * 2017-11-17 2018-04-27 江苏大学 A kind of nearby vehicle Activity recognition method based on V2V communications with HMM-GBDT mixed models
CN110146100A (en) * 2018-02-13 2019-08-20 华为技术有限公司 Trajectory predictions method, apparatus and storage medium
CN109886304A (en) * 2019-01-22 2019-06-14 江苏大学 A kind of complex road condition following peripheral vehicle behavior recognition methods based on HMM-SVM bilayer improved model
CN109946688A (en) * 2019-03-18 2019-06-28 中国汽车工程研究院股份有限公司 Lane-change contextual data extracting method, device and server
CN110986994A (en) * 2019-11-14 2020-04-10 苏州智加科技有限公司 Automatic lane change intention marking method based on high-noise vehicle track data

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115482659A (en) * 2022-08-18 2022-12-16 浙江工商大学 Intelligent agent autonomous decision-making method based on deep reinforcement learning
CN115482659B (en) * 2022-08-18 2023-09-19 浙江工商大学 Intelligent autonomous decision-making method based on deep reinforcement learning

Also Published As

Publication number Publication date
CN110986994B (en) 2021-08-03
CN110986994A (en) 2020-04-10

Similar Documents

Publication Publication Date Title
JP7106664B2 (en) Intelligent driving control method and device, electronic device, program and medium
WO2021093335A1 (en) Method for automatically labeling lane changing intention based on high-noise trajectory data of vehicle
US20200265710A1 (en) Travelling track prediction method and device for vehicle
US10255812B2 (en) Method and apparatus for preventing collision between objects
CN110532916B (en) Motion trail determination method and device
CN106647776B (en) Method and device for judging lane changing trend of vehicle and computer storage medium
CN111907521B (en) Transverse control method and device for automatic driving vehicle and storage medium
KR102456626B1 (en) Apparatus and method for traffic lane recognition in automatic steering control of vehilcles
US20150203114A1 (en) Lane relative position estimation method and system for driver assistance systems
CN112793576B (en) Lane change decision method and system based on rule and machine learning fusion
CN112530170A (en) Vehicle driving state detection method and device, electronic equipment and storage medium
CN113127466B (en) Vehicle track data preprocessing method and computer storage medium
US11080544B1 (en) Method and device for calibrating pitch of camera on vehicle and method and device for continual learning of vanishing point estimation model to be used for calibrating the pitch
JP4937844B2 (en) Pedestrian detection device
JP2018147399A (en) Target detection device
EP3522073A1 (en) Method and apparatus for detecting road surface marking
JP5573780B2 (en) Course evaluation device and course evaluation method
JP5682302B2 (en) Traveling road estimation device, method and program
JP2020119526A (en) Main object selection for assist function or automatic driving function of driver assist system or driving system of power vehicle
US20240020964A1 (en) Method and device for improving object recognition rate of self-driving car
CN115649174B (en) Intelligent overtaking event processing method and related product
CN115641569B (en) Driving scene processing method, device, equipment and medium
CN113581168B (en) Bad-sight-distance curve vehicle-meeting safety method and system based on artificial intelligence
CN117104222B (en) Obstacle detection method and device applied to vehicle driving area
WO2021166893A1 (en) Target object recognition device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20888488

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20888488

Country of ref document: EP

Kind code of ref document: A1