Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/88—Radar or analogous systems specially adapted for specific applications
  • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
  • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
  • G01S13/50—Systems of measurement based on relative movement of target
  • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/66—Radar-tracking systems; Analogous systems
  • G01S13/72—Radar-tracking systems; Analogous systems for two-dimensional [2D] tracking, e.g. combination of angle and range tracking, track-while-scan radar
  • G01S13/723—Radar-tracking systems; Analogous systems for two-dimensional [2D] tracking, e.g. combination of angle and range tracking, track-while-scan radar by using numerical data
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
  • G01S13/874—Combination of several systems for attitude determination
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
  • G01S7/28—Details of pulse systems
  • G01S7/285—Receivers
  • G01S7/295—Means for transforming co-ordinates or for evaluating data, e.g. using computers
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
  • G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
  • G01S13/88—Radar or analogous systems specially adapted for specific applications
  • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
  • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
  • G01S2013/9327—Sensor installation details
  • G01S2013/93271—Sensor installation details in the front of the vehicles

Definitions

• Planning systems utilize information associated with objects in an environment to determine actions relative to those objects. For example, some existing planning systems for autonomous vehicles consider velocities and/or yaw rates of objects, such as other vehicles on the road, when determining maneuvers for the autonomous vehicle to traverse through the environment.
• Conventional systems may rely on different types of data, including but not limited to radar, LiDAR, vision, and/or the like to determine information about the object(s).
• conventional systems have required receiving the sensor data over multiple iterations to confidently characterize object movement. For example, determining a yaw rate of a vehicle may require multiple, successive scans of one or more sensors and/or processing of data from a number of different sensor types to determine the correct speed and/or heading of an object. Such processes may result in increased processing time and/or decreased efficiency in identifying and/or characterizing objects that may be potential obstacles to safe travel.
• FIG. 1 is a schematic representation illustrating an example method for determining a yaw rate of an object using radar data, as described herein.
• FIG. 2 includes textual and visual flowcharts to illustrate an example method for determining a yaw rate of an object from radar data generated by radar sensors on a vehicle, as described herein.
• FIG. 3 includes textual and visual flowcharts to illustrate an example method for determining a yaw rate of an object from radar data generated by radar sensors on a vehicle, as described herein.
• FIG. 4 is a block diagram illustrating an example computing system for generating an instantaneous yaw rate from radar data, as described herein.
• FIG. 5 is a flowchart illustrating an example method for determining a yaw rate from radar data, as described herein.
• FIG. 6 is a flowchart illustrating an example method for controlling a vehicle, such as an autonomous vehicle, as described herein.
• Techniques described herein are directed to characterizing movement of objects in in an environment based on radar data. For example, in implementations described herein, techniques may be used to determine radar returns that are associated with an object in an environment of the sensor, and a yaw rate and/or two-dimensional velocity for the object can be determined from the returns.
• an example system that implements the techniques of this disclosure may include an autonomous vehicle having multiple radar sensors (and/or sensors of other or different modalities).
• the autonomous vehicle can include multiple radar sensors having overlapping fields of view.
• a first sensor e.g., a first Doppler radar sensor, can capture first sensor data, e.g.
• a radar scan (which may include a collection of a number of measurements of radar returns as data points), of an environment and a second sensor, e.g., a second Doppler radar sensor, can capture second sensor data, e.g., a second radar scan (which may include a second collection of measurements of radar returns as data points) of the environment.
• a second radar scan (which may include a second collection of measurements of radar returns as data points) of the environment.
• Each of the first scan and the second scan may return a number of points, each having associated information.
• Such information may include position information, e.g., a location of the point relative to the sensor, the vehicle, and/or in a coordinate system (any or all of which may be determined based on a range and/or azimuth angle of the signal), signal strength information, e.g., a radar cross-section (RCS) value, or velocity information, e.g., a velocity of the point relative to the sensor.
• position information e.g., a location of the point relative to the sensor, the vehicle, and/or in a coordinate system (any or all of which may be determined based on a range and/or azimuth angle of the signal)
• signal strength information e.g., a radar cross-section (RCS) value
• velocity information e.g., a velocity of the point relative to the sensor.
• data association techniques can be used to associate returns in the radar scans to objects in the environment.
• clustering techniques can be used to group points in the first scan with point in the second scan according to any of the information received with the returns.
• points in a similar area e.g., having close locational proximity, may be candidates for clustering as all being related to a single object.
• the signal strength information, RCS, the velocity information, and/or other information determined by the one or more sensors may also or alternatively be used to cluster points according to implementations of this disclosure.
• the signal strength may be useful to differentiate between a person standing at a street comer and a light post upon which the person is leaning.
• points in the first scan and in the second scan having similar characteristics may be aggregated as a point cluster, to yield a robust representation of the sensed object.
• the multiple scans e.g., sequential scans of the first sensor and/or the second sensor may be aggregated in a cluster associated with an object.
• the data association techniques can include historical information about a sensed object. For instance, tracking information associated with an object can be used to determine whether radar returns are expected to be associated with particular objects.
• radar data may be combined and/or compared against data from other sensor modalities (e.g., lidar, vision (images), etc.) to associate the radar data with an object (such as may be detected in another modality and/or detected based on a combination of the modalities).
• sensor modalities e.g., lidar, vision (images), etc.
• the aggregated radar returns may be used to determine an instantaneous yaw rate and/or two-dimensional velocity (e.g., in an x-direction and a y-direction) of the object with which the returns are associated.
• techniques described herein can project individual of the returns onto a rotational center (or an estimated rotational center), e.g., to determine a yaw rate about the rotational center. In this manner, the yaw rate can be determined from a number of returns, and substantially instantaneously.
• the two-dimensional velocity can be determined using geometry associated with positions of points associated with the returns, a position of the radar sensor(s), and a position of the rotational center.
• Some examples may use an iterative estimation process, e.g., an iterative least square or random sample consensus (RANSAC) methods, though any other statistical methods that are robust to outliers are contemplated.
• RANSAC techniques can provide a robust estimation of yaw and/or two- dimensional velocity based on the returns, while rejecting outliers.
• the scans may be nearly identical in time, e.g., within about 100 milliseconds, because the scans are coming from different sensors, but with overlapping fields of view.
• more sensors with overlapping fields of view may be used to further increase the number of points associated with an object, thereby strengthening the returns of the yaw rate estimation techniques described herein.
• a single scan from a single radar device may comprise multiple subscans comprising scans at different frequencies, phase shifts, and the like.
• each of the subscans may be treated equally, as overlapping scans from multiple radar.
• techniques presented herein may be used with one or more sensors.
• Historical data of the sensor(s) (or of an object, e.g., a vehicle, on which the sensor(s) is/are mounted) and/or historical data of the sensed object may also be obtained, e.g., from sensors other than, or in addition to, the radar sensors.
• the historical data may include one or more of historical position data, historical velocity data, historical yaw rate information, historical acceleration data, or other information. The historical information may be used to extrapolate or estimate radar data captured at different times to a common time.
• position and/or velocity associated with a return generated at a first time can be used to determine updated position and/or velocity data at a second time based on a state of the vehicle (e.g., position, velocity, yaw rate, acceleration) and/or a state of the object (e.g., position, velocity, rotational center, yaw rate, acceleration) between the first time and the second time.
• a radar return can be projected to a new position and/or as having a new velocity based on a state velocity and/or state yaw rate of the vehicle and/or the object between the first time and the second time.
• an instantaneous yaw rate and/or two-dimensional velocity can be determined at the second time based on returns generated proximate the second time, as well as based on returns generated at an earlier time, e.g., the first time.
• yaw rate and/or two-dimensional velocities may have been determined using data from multiple sensor modalities and/or using multiple observations, e.g., by determining changes between vehicle poses over relatively long periods of time.
• techniques described herein may result in ayaw rate determination nearly instantaneously. For example, radar returns within tens of milliseconds can be used to accurately determine a yaw rate of objects detected by the sensors.
• the computing device of the autonomous vehicle may determine one or more trajectories for proceeding relative to the object.
• yaw data and/or velocity data generated according to techniques described herein may be combined, or fused, with data from other sensor modalities to predict a movement of the object and/or to plan a path relative to the object.
• Techniques described herein are directed to leveraging sensor and perception data to enable a vehicle, such as an autonomous vehicle, to navigate through an environment while circumventing objects in the environment.
• Techniques described herein can utilize information sensed about the objects in the environment, e.g., by radar sensors, to more accurately determine movement associated with the object.
• techniques described herein may be faster than conventional techniques, as they may alleviate the need for information from a plurality of different sensors. That is, techniques described herein provide a technological improvement over existing object detection, classification, prediction and/or navigation technology.
• techniques described herein can provide a smoother ride and improve safety outcomes by, for example, more accurately providing safe passage to an intended destination.
• techniques described herein are not limited application in autonomous vehicles.
• any system that uses radar data to navigate an environment may benefit from the yaw determination techniques described.
• techniques described herein may be used on aircrafts, e.g., to identify other aircraft and/or moving objects, and determine yaw rates associated with objects in an airspace or on the ground.
• non-autonomous vehicles could also benefit from techniques described herein, e.g., for collision detection and/or avoidance systems.
• FIG. 1 provides additional details associated with techniques described herein.
• FIG. 1 is a schematic illustration showing an example scenario 100 in which a vehicle 102 is driving on a road surface 104 at a first velocity, Vi.
• a second vehicle 106 is also travelling on the road surface 104.
• the second vehicle 106 is travelling at a second velocity, V2, in a direction along arrow 138 that is not parallel to the direction of travel of the vehicle 102.
• the second vehicle 106 may be changing lanes, e.g., to a position in front of the vehicle 102.
• the vehicle 102 can be an autonomous vehicle configured to operate according to a Level 5 classification issued by the U.S. National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions for the entire trip, with the driver (or occupant) not being expected to control the vehicle at any time.
• the vehicle 102 can be configured to control all functions from start to stop, including all parking functions, it can be unoccupied. Additional details associated with the vehicle 102 are described below.
• the vehicle 102 is merely an example, and the systems and methods described herein can be incorporated into any ground-home, airborne, or waterborne vehicle, including those ranging from vehicles that need to be manually controlled by a driver at all times, to those that are partially or fully autonomously controlled.
• techniques described herein may be useful in settings other than vehicles. The techniques described in this specification may be useful in many different applications in which sensor data is used to determine information about objects in an environment.
• the vehicle 102 may include a plurality of sensors, including a first radar sensor 108 and a second radar sensor 110.
• the first radar sensor 108 and the second radar sensor 110 are arranged to propagate waves generally in a direction of travel of the vehicle 102.
• the first radar sensor 108 and the second radar sensor 110 have overlapping fields of view. Accordingly, first emitted radio waves 112, emitted by the first radar sensor 108, will reflect off the second vehicle 106 and return to the first radar sensor 108 where they are detected via a first radar scan. Similarly, second emitted radio waves 114, emitted by the second radar sensor 110, will also reflect off the second vehicle 106 and return to the second radar sensor 110 where they are detected via a second radar scan.
• the first radar sensor 108 and the second radar sensor 110 may be substantially identical, except for their position on the vehicle 102. In other examples, however, the radar sensors 108, 110 may be differently configured.
• the radio waves 112, 114 may be emitted at different frequencies, e.g. pulse-regulated frequencies.
• the radar sensors 108, 110 may be configured such that scans at the sensors 108, 110 have a different interval, e.g. , a Doppler interval.
• features of the radar sensors 108, 110 including but not limited to the center frequency, the scan type, the scan pattern, frequency modulation, the pulse repetition frequency, pulse repetition interval, may be configured, e.g., to create the different Doppler intervals.
• the radar sensors 108, 110 may both be disposed to sense obj ects generally in the same direction relative to the vehicle 102, but the radar sensors 108, 110 can be configured differently. In other examples, however, several features and functions of the radar sensors 108, 110 may be the same or similar.
• the sensors 108, 110 may receive the radio waves 112, 114 after the waves reflect off a surface in the environment, and the radar sensors 108, 110 can generate radar data based on the reflection.
• the radar data may include diverse types of information, including but not limited to a velocity associated with each of many points representative of surfaces or objects in the environment of the sensor.
• the sensors 108, 110 are pulse-Doppler sensors, the sensors 108, 110 may be able to determine a velocity of an object relative to the respective sensor.
• FIG. 1 illustrates a plurality of first radar returns, schematically represented by points.
• first points 116(1), 116(2), 116(3), 116(4) (collectively, the first points 116) are illustrated as circles, and represent radar returns associated with the first radar sensor 108. That is, individual of the first points 116 are indicative of locations on the second vehicle 106 at which the first emitted waves 112 reflect.
• second points 118(1), 118(2), 118(3), 118(4) are illustrated as“X”s, and represent radar returns associated with the second radar sensor 110. Stated differently, individual of the second points 118 are indicative of locations on the second vehicle 106 at which the second emitted waves 114 reflect.
• the first radar sensor 108 and the second radar sensor 110 (also illustrated in the block diagram accompanying FIG. 1 as sensor systems 120) generate data from radio waves reflected from each of the points 116, 118. More specifically, the first radar sensor 108 generates first radar data 122 and the second radar sensor 110 generates second radar data 124.
• the radar data 122, 124 includes position data of the respective points 116, 118.
• information associated with radar returns from the points 116, 118 may include information indicative of a location in the environment, e.g., a location of the points 116, 118.
• a position of the second vehicle 106 can be determined.
• the location information may include a range and azimuth relative to the points 116, 118 or a position in a local or global coordinate system.
• the first radar data 122 generated by the first radar sensor 108 and/or the second radar data 124 generated by the second radar sensor 110 may include signal strength information.
• the signal strength information can indicate a type of the object. More specifically, radio waves may be reflected more strongly by objects having certain shapes and/or compositions. For example, broad, flat surfaces and/or sharp edges are more highly reflective than rounded surfaces, and metal is more highly reflective than a person.
• the signal strength may include a radar cross-section (RCS) measurement.
• RCS radar cross-section
• the first radar sensor data 122 and/or the second radar data 124 may also include velocity information.
• a velocity of each of the points 116, 118 (and/or of the second vehicle 106) may be based on a frequency of radio energy reflected from the points 116, 118 and/ or a time at which the reflected radio energy is detected.
• the first radar data 122 can include a distance of the respective one of the first points 116 from the first radar sensor 108 (e.g., a range or radial distance), a velocity (e.g., a Doppler velocity) of the respective one of the first points 116 along the distance, a strength measurement (e.g., an RCS value), and/or additional information.
• the second radar data 124 can include a distance of the respective one of the second points 118 from the second radar sensor 110 (e.g., a range or radial distance), a velocity (e.g., a Doppler velocity) of the respective one of the second points 118 along the associated distance, strength information, and/or additional information.
• the different types of information contained in the first radar data 122 and the second radar data 124 can be used to determine additional information.
• the vehicle 102 can include one or more vehicle computing device(s) 126 including a yaw rate and two-dimensional velocity determination component 128.
• the yaw rate and two-dimensional velocity determination component 128 can be configured to receive the first radar data 122 and the second radar data 124 and determine a yaw rate and two-dimensional velocity of the second vehicle 106.
• FIG. 1 also illustrates that the vehicle 102 can include one or more vehicle computing device(s) 126 including a yaw rate and two-dimensional velocity determination component 128.
• the yaw rate and two-dimensional velocity determination component 128 can be configured to receive the first radar data 122 and the second radar data 124 and determine a yaw rate and two-dimensional velocity of the second vehicle 106.
• the second vehicle 106 can have a center of rotation 130 and the yaw rate and two-dimensional velocity determination component 128 can determine a yaw rate 132 of the second vehicle 106 about the center of rotation 130.
• the yaw rate and two-dimensional velocity determination component 128 can assume that the second vehicle 106 is a fixed rigid body, and project the velocities associated with each of the first points 116 and the second points 118 onto the center of rotation 130 to estimate the yaw rate.
• the yaw rate and two-dimensional velocity determination component 128 can treat each of the velocities as dependent upon some rotation on the second vehicle 106, at a distance from the center of rotation.
• the yaw rate and two-dimensional velocity determination component 128 can compute the yaw rate 132 about the center of rotation using the velocity and relative positions. Moreover, because the yaw rate and two-dimensional velocity determination component 128 can assume that the second vehicle 106 is a fixed rigid body, the yaw rate 132 is the same regardless of the position of the center of rotation 130. Accordingly, any one of the first points 116 or the second points 118 can be used as the center of rotation. Additional details about the functioning of the yaw rate and two-dimensional velocity determination component 128 are described below in connection with FIGS. 2 and 3.
• the yaw rate and two-dimensional velocity determination component 128 can determine a two-dimensional velocity of the second vehicle 106, e.g., based on the return velocity and positions of the center of rotation 130, the sensors 108, 110, and the return.
• the yaw rate and two-dimensional velocity determination component 128 can determine the yaw rate 132 because each of the points 116, 118 is on the rigid body (e.g., associated with the second vehicle 106).
• FIG. 1 shows only the second vehicle 106, and the first points 116 and the second points 118 all are reflected off the second vehicle 106.
• the radar sensors 108, 110 generally receive reflections from a number of different objects in the environment, including but not limited to other vehicles, buildings, fixtures, surfaces, pedestrians, bicyclists, and/or the like.
• the vehicle computing device(s) 126 can further include a data association component 134.
• the data association component 134 can include functionality to associate returns with each other and/or with specific objects.
• the data association component 134 can determine that returns associated with the first points 116 and the second points 118 are associated with each other and/or with the second vehicle 106.
• the data association component 134 can also determine that other returns, e.g., in a same radar scan, are associated with other objects, for example, the road surface 104 proximate the second vehicle 106, other vehicles (not shown), or the like.
• the data association component 134 can cluster points, e.g., the first points 116 and the second points 118, based on information from those returns. For instance, the first points 116 and the second points 118 are closely situated, e.g., within a threshold distance, and in some instances the data association component 134 can estimate that those points are indicative of a single object.
• a point cluster may include a plurality of points that have some likelihood, e.g., a level and/or degree of similarity, to identify a single object or grouping of objects that should be considered together, e.g., by a planning system of the autonomous vehicle.
• information in addition to position information may be used to determine the point cluster.
• the first points 116 and the second points 118 have similar positional returns
• other points that are also close in proximity can be excluded by the data association component 134, e.g., because such points may be representative of a different object.
• the signal strength, e.g., the RCS value, associated with an additional point 136 may be significantly different than the signal strength associated with the first points 116 and/or the second points 118. This despite the additional point 136 being physically closer to the second point 118(4) than several of the first points 116 and/or the second points 118.
• the additional point 136 may be associated with the road surface 104, some debris on the road surface 104, or the like. As will be appreciated, associating the additional point 136 with the object could negatively impact functioning of the yaw rate and two-dimensional velocity determination component 128, e.g., because the velocity of the additional point 136 may be quite different from the velocity of the second vehicle 106.
• the data association component 134 can use additional or different information to determine returns associated with objects. For example, when the radar sensor is a Doppler- type sensor, velocity of the objects may be used to determine the point cluster. In the illustrated example, when the additional point 136 is associated with the road and the second vehicle 106 is moving, the velocities associated with the radar returns may be sufficiently different to allow for proper clustering of returns. More specifically, because the road surface is stationary and the vehicle is moving, points having a similar, non-zero velocity component can be clustered as the second vehicle. Other information may also or alternatively be used to cluster points in accordance with the disclosure herein, and the examples provided are for illustration only.
• historical data of the object e.g., of the second vehicle 106
• the data association component 134 can also add points to already existing clusters. For instance, later-in-time returns can be associated with object clusters based on a distance, e.g., a Euclidian distance from points already in the cluster.
• the data association component 134 can also add new returns to a cluster returns when a value, e.g., a signal strength, falls between a minimum signal strength and a maximum signal strength of points already associated with an object. Inclusion in a cluster can also be based on a proximity e.g., by location data, of later received points to points already in a cluster (e.g. a number of nearest neighbors (1, 2, or more)).
• Clustering may be performed based on any physical parameters associated with the points (including, but not limited to, velocities, signal strength, location, nearest neighbors, a time stamp the measurement was performed, etc.), as well as corresponding threshold differences in any of the aforementioned parameters. Inclusion may also be based on a combination of these and other information.
• the data association component 134 can determine association of radar returns with an object based on one or more additional sensor modalities.
• objects in the environment such as the second vehicle 106
• lidar data may be determined based on any one or more of lidar data from one or more lidar sensors (which may include time of flight sensors), image data from any number of cameras, and the like, or any combination thereof.
• the radar data may be associated with the detection from any one or more of the other modalities in order to determine which radar data is generated based on reflections from the second vehicle 106.
• previously determined positions of objects in an environment and/or their associated and determined dynamics may be used to form an estimate of where the object may likely be (e.g., using a Kalman, or other Bayesian, filter). These object tracks may then be used to perform data association with the newly collected data to determine where the object is currently.
• the data association component 134 can include functionality to determine returns associated with an object, and the yaw rate and two-dimensional velocity determination component 128 can use information (e.g., position and velocity) associated with those returns to determine a yaw rate and/or the two-dimensional velocity for that object.
• the vehicle 102 can include functionality to control the vehicle 102, and the yaw rate 132 and/or the two-dimensional velocity of the second vehicle 106 can be used by the vehicle to travel relative to the second vehicle 106.
• the yaw rate 132 and/or the two- dimensional velocity may be a factor used to predict a path or trajectory of the second vehicle 106, and a planning system of the vehicle 102 may use that prediction information to determine a drive path for the vehicle 102, to avoid the predicted path/trajectory.
• a yaw rate may provide a better indication as to whether the second vehicle 106 is entering the lane of travel of the vehicle 102, or likely to proceed in its current lane.
• Techniques described herein may improve planning system accuracy and performance by determining yaw rates of objects in the environment quickly and accurately.
• radar sensors like the radar sensor 108, 110 are among the fastest sensors on some vehicles to generate meaningful amounts of data about objects, like the second vehicle 106.
• attributes of the second vehicle 106 can be obtained more quickly and reliably, thereby improving reliability and timeliness of subsequent predictions based on such radar data.
• FIG. 2 includes textual and visual flowcharts to illustrate an example process 200 for determining a yaw rate and/or a two-dimensional velocity of an object based on sensor data.
• the sensor data may be obtained by radar sensors disposed on an autonomous vehicle.
• the process 200 uses multiple radar sensors with overlapping fields of view to determine a point cluster indicative of an object in the environment of the autonomous vehicle and then determines a yaw rate based on the clustered radar returns.
• the process 200 can include capturing radar data from one or more radar sensors disposed on an autonomous vehicle.
• any number of radar sensors having overlapping fields of view can use the techniques described herein.
• An example 204 accompanying the operation 202 illustrates a vehicle 206 having a first radar sensor 208 and a second radar sensor 210 disposed on the vehicle 206.
• the vehicle 206 may be traversing through the environment generally in a direction indicated by an arrow 212 (although in other implementations, the vehicle 206 may be stationary or moving in a different direction), such that the first radar sensor 208 and the second radar sensor 210 are disposed on the leading end of the vehicle 206, e.g., to capture data about objects in front of the vehicle 206. More specifically, the first radar sensor 208 captures first radar data 214, e.g., via first radar scans, and the second radar sensor 210 captures second radar data 216, e.g., via second radar scans. In the illustrated embodiment, the first and second radar sensors 208, 210 are generally configured next to each other, both facing in the direction of travel, and with significant overlap in their fields of view.
• the vehicle 206 may correspond to the vehicle 106 and the first radar sensor 208 and the second radar sensor 210 may correspond to the first and second radar sensors 108, 110, shown in, and discussed in connection with, FIG. 1.
• the first radar sensor 208 and the second radar sensor 210 may be radar sensors that measure the range to objects and/or the velocity of objects.
• the radar sensors may be Doppler sensors, pulse- type sensors, continuous wave frequency modulation (CWFM) sensors, or the like.
• the radar sensors 208, 210 may emit pulses of radio energy at predetermined (and in some instances, configurable) intervals.
• the intervals may be configurable, e.g., to promote enhanced detection of objects at relatively far distances or relatively close distances.
• the pulses of radio energy reflect off objects in the environment and can be received by the radar sensors 208, 210, e.g., as the first radar data 214 and the second radar data 216.
• the frequency with which the radar returns are read out may also be configurable, in some examples.
• the first radar data 214 and/or the second radar data 216 may include position information indicative of a location of objects in the environment, e.g., a range and azimuth relative to the vehicle 206 or a position in a local or global coordinate system.
• the first sensor data 214 and/or the second sensor data 216 may also include signal strength information.
• the signal strength information may be an indication of a type or composition of the surface from which the radio waves are reflected.
• the signal strength may be a radar cross-section (RCS) measurement.
• the first sensor data 214 and/or the second sensor data 216 may also include velocity information.
• velocity of the object may be based on a frequency of radio energy reflected by the object and/or a time at which the reflected radio energy is detected.
• the first and second radar sensor data 214, 216 may also include specific information about the sensor, including but not limited to an orientation of the sensor, e.g., a pose of the sensor relative to the vehicle, a pulse repetition frequency (PRF) or a pulse repetition interval (PRI) for the sensor, a field of view or detecting arc, or the like.
• PRF pulse repetition frequency
• PRI pulse repetition interval
• certain types of the radar sensor data 214, 216 may be pre-configured, e.g., the field of view, the orientation, the PRF or PRI, or the like, in which case that data may be available to the autonomous vehicle 206 computing system, and may not be transferred with radar returns, for example.
• the process 200 can determine returns associated with an object from the radar data.
• An example 220 accompanying the operation 218 includes a visualization 222 of the first radar data 214 and the second radar data 216. More specifically, the visualization 222 includes a plurality of points which may be representative of objects, surfaces, or other reflective items, in an environment of the vehicle 206. As illustrated, some circular points are labelled with reference numerals 224(1), 224(2), 224(3), 224(4), 224(5), 224(6) (hereinafter all of the circular points, whether or not labelled in FIG. 2, may be referred to, collectively, as“the first points 224”).
• each of the first points 224 may correspond to a radio wave detected at the first radar sensor 208, e.g., after reflecting off some object in front of the vehicle 206.
• some points designated by an“x” are labelled with reference numerals 226(1), 226(2), 226(3), 226(4), 226(5) (hereinafter all of the “x” points, whether or not labelled in FIG. 2, may be referred to, collectively, as“the second points 226”).
• each of the second points 226 may correspond to a radio wave detected at the second radar sensor 210, e.g., after reflecting off some object in front of the vehicle.
• the positions of the detected first points 224 and the detected second points 226 may generally correspond to a position of one or more detected objects or surfaces, e.g., objects (or surfaces) from which the emitted radio wave reflected.
• the first and second radar sensors 208, 210 may receive and/or generate several types of information about the environment, so the visualization 222 could include more or less information; the visualization 222 is merely for explanatory purposes.
• the visualization 222 illustrates that some of the points appear to be close in location, and as such, may be associated with a single object.
• the points 224(2) - 224(6) and the points 226(2) - 226(4) are closely situated, e.g., within a threshold distance, and in some instances those points may be estimated to be indicative of a single object, such as an object 228.
• the data association component 134 may determine that the first points 224(2)-224(6) and the second points 226(2)-226(4) could be identified as a point cluster representative of the object 228.
• a point cluster may include a plurality of points that have some likelihood, e.g., a level and/or degree of similarity, to identify a single object or grouping of objects that should be considered together, e.g., by a planning system of the autonomous vehicle.
• information in addition to position information may be used to determine the point cluster.
• the data association component 134 can consider historical information, e.g., track information for the vehicle 206 and/or the object 228 to properly associate the radar returns with the object 228.
• the operation 218, e.g., implemented by the data association component 134 can determine returns representative of the object 228, e.g., because the points may be clustered based on one or more diverse types of information, and because sensor data from multiple scans and/or radar sensors is used to create the cluster.
• the object 228 may be the second vehicle 106.
• the object 228 is identified using sensor data from two radar sensors 208, 210.
• fewer or more sensors may be used.
• a point cluster according to the foregoing techniques may be obtained for two scans (e.g., subscans as above) from the same sensor, e.g., in instances in which the scans are within a threshold time, e.g., 100 milliseconds or less. So long as the sensors are disposed to capture information about the same object, techniques described herein may be applied. Moreover, scans from the same sensor can be used in techniques described herein, e.g., by normalizing or synchronizing returns from multiple scans using information about the object 228 and/or the vehicle 206. For example, FIG.
• the process 200 can determine a yaw rate and/or a two- dimensional velocity of the object based on returns associated with the object.
• An example 232 accompanying the operation 230 includes a visualization 234 including the vehicle 206 (with the first radar sensor 208 and the second radar sensor 210) and three of the points 224(2), 224(5), 226(4) associated with the object 228.
• the object 228 may be the second vehicle 106 of FIG. 1 (and the vehicle 206 may be the vehicle 102).
• the visualization 234 also illustrates a coordinate system 236.
• the first point 224(2) is represented as a first detected point Odi
• the first point 224(5) is represented as a second detected point (1 ⁇ 2
• the second point 226(4) is represented as a third detected point Od3.
• the techniques described herein can use any and all returns associated with the object 228.
• the object 228 may have a center of rotation Oi, and movement of the object 228 may be characterized by a velocity in the x-direction, e.g., a velocity v x , a velocity in the y-direction, e.g., a velocity v y , and a yaw rate w about the center of rotation Oi.
• the center of rotation Oi may be selected arbitrarily.
• the object 228 can be treated as a rigid body, which means that the yaw rate will be the same at any point on the body.
• the center of rotation Oi is illustrated as generally centrally located on the object 228 (and spaced from the points 224, 226), in other examples any point can be used, including but not limited to one of the points 224, 226.
• the object 228 By considering the object 228 as a rigid body, techniques described herein, e.g., which may be implemented using the yaw rate and two-dimensional velocity determination component 128, can estimate the yaw rate and/or the of the object 228 using multiple returns.
• the position of the first detected point Odi, the position of the first radar sensor 208 (which generated the return associated with the first point Odi) and the position of the center of rotation Oi all are known.
• the positions of the second detected point Od2 and of the third return Od3 also are known, as is the position of the second radar sensor 210 (which generated the return associated with the third point Od3).
• each of the first detected point Odi, the second detected point Od2, and the third detected point Od3 has an associated velocity, e.g., a doppler velocity.
• velocities are illustrated in the example as Vdidop, Vd2dop, and Vd3dop, respectively.
• Such velocities are velocities along the direction between the point and the sensor sensing the velocity.
• Vdidop is a velocity along a direction from the first detected point Odi to the first radar sensor 208
• Vd2dop is a velocity along a direction from the second detected point Od2 to the first radar sensor 208
• Vd3dop is a velocity along a direction from the third detected point Od3 to the second radar sensor 210.
• the directions of the velocities Vdidop, Vd2dop, and Vd3dop indicate that the object 228 is moving counter-clockwise about the center of rotation Oi.
• the operation 230 can include solving Equation (1):
• Equation (1) includes three unknowns (v x , v y , co), which describe movement of the object in the x,y plane.
• the operation 230 can include simultaneously solving Equation (1) for each of n returns associated with the object and measured by in sensors. For instance, Equation (1) can be solved simultaneously for the three points included in FIG. 2 and/or any other returns associated with the object, and captured by the two sensors 208, 210.
• OR m indicates a position of the radar sensor in that captured the return n.
• Equation (1) can be parameterized to also solve for the center of rotation.
• Equation (1) for the point Odi in FIG. 2, would be re-written as Equation (2):
• Equation (2) describes a projection of the doppler velocity associated with the first detected point Odi onto the center of rotation.
• the received doppler velocity is equal to some proportion of the velocity in the x and y direction and the impact caused by the rotation about the center for rotation.
• the first term represents a velocity in the x-direction, scaled according to the angle, at the first detected point Odi
• the second term represents a velocity in the y-direction, the first detected point Odi
• the third term represents the (rotational) impact of the velocities on the center of rotation.
• the third term illustrates the product of a distance between the center of rotation Oi and the first detected point Odi and a unit vector extending from the first detected point Odi toward the position of the radar sensor.
• Equation (1) can be solved simultaneously using all points associated with the object 228, e.g., using a single matrix.
• Equation (1) can be solved using an iterative estimation technique.
• a least-squares estimation of the three unknowns can be used.
• solving a least squares estimation may require more than three returns, e.g., because three radar returns may not provide a unique solution for each of the unknowns.
• the estimation may benefit from random sample consensus (RANSAC) methods.
• RANSAC logic can perform a robust estimation by rejecting outliers in a relatively large data set.
• RANSAC methods may reject returns that were originally associated with the object 228, e.g., by the data association component 134, but that are not actually from the object 228.
• the RANSAC methods can reject returns from the ground proximate the object 228.
• this disclosure is not limited to using RANSAC for outlier rejection.
• Other techniques including but not limited to iterative least squares, can alternatively be used.
• a yaw rate e.g., an instantaneous yaw rate, and an instantaneously two-dimensional velocity
• the yaw rate and/or the two-dimensional velocity may be used to predict a trajectory or path of the object 228.
• the vehicle 206 may then plan a path that avoids the predicted trajectory/path of the object 228, e.g., for safe navigation.
• some examples can use as few as three returns to determine the yaw rate, although other examples may use as many returns as are available for the object 228.
• FIG. 3 includes textual and visual flowcharts to illustrate an example process 300 for updating sensor data, e.g. , radar sensor data, for use in yaw rate determination.
• the process 300 can be implemented on radar sensor data to prepare such radar sensor data for use in the process 200, although the process 300 need not be used in connection with the process 200, and, as described above, the process 200 may be implemented without techniques discussed in connection with the process 300.
• the process 300 can include receiving historical data about a vehicle and/or an object.
• An example 304 accompanying the operation 302 illustrates historical information about a vehicle 306, which may be an autonomous vehicle (e.g., the vehicle 102 and/or the vehicle 206), e.g., on which one or more sensors, such as the radar sensors 108, 110 may be mounted.
• the historical information may also include historical information about an object 308, which can be an object in an environment of the vehicle 306.
• the object 308 can be the second vehicle 106 and/or the object 228 described herein.
• the example 304 includes a timeline 310, indicating a plurality of times, e.g., To, T-iand illustrates that data 312 is associated with the vehicle 306 at each of the illustrated times and that data 314 is associated with the object 308 at each of the illustrated times.
• the two times are for illustration only, as the data 312, 314 can be generated and/or determined at any number of times and/or frequencies.
• the data 312 may be generated (or captured) at time intervals different from those at which the data 314 may be generated (or captured).
• the data 312 and the data 314 may include any information about the vehicle 306 and the object 308, respectively.
• the data 312 and/or the data 314 may be state data about the vehicle/object at the specific time.
• the data 312 and/or the data 314 can include position and/or orientation data, e.g., any one or more of an x,y,z position in a coordinate system and/or a roll/pitch/yaw, of the vehicle 306 and/or the object 308.
• the data 312 at time T-i may identify a position of the vehicle 306 at that time
• the data 312 associated with the time To may identify a current (or most-recently detected) position of the vehicle 306.
• the data 312 and/or the data 314 can include velocity information, acceleration information, and/or any other information associated with the vehicle 306 and/or the object 308 at the given times (e.g., including, but not limited to, an estimated center of the object, estimated extents of the object - length, width, height, predicted trajectories, associated confidence levels of any of the above, and the like).
• the data 312 and/or the data 314 can include state velocity of the vehicle at the given times.
• the data 312 and/or the data 314 can also include tracking data about the respective vehicle 306 or the object 308.
• the process 300 can include receiving radar data including timestamp information.
• the timeline 310 is illustrated, and an instance of first radar data 320 and an instance of second radar data 322 are illustrated on the timeline 310.
• the first radar data 320 can correspond to the first radar data 122 and/or the first radar data 214 and the second radar data 322 can correspond to the second radar data 124 and/or the second radar data 216.
• the first radar data 320 may be captured by a first radar sensor disposed on the vehicle 306 and the second radar data 322 may be captured by a second radar sensor disposed on the vehicle 306.
• the first radar data 320 and the second radar data 322 can include radar returns corresponding to overlapping fields of view.
• both the first radar data 320 and the second radar data 322 may include radar returns associated with the object 308.
• the first radar data 320 is associated with (e.g., generated, output, or the like) a first time TRI, which is between the times T-i and To
• the second radar data 322 is associated with (e.g., generated at, output by the sensor at, or the like) a second time TR2 which generally coincides with the time To.
• the times associated with the first radar data 320 and/or the second radar data 322 may be determined from timestamps associated with one or more returns comprising the respective data, and are for illustration only.
• the process 300 can include estimating radar data using the timestamp information and the historical data.
• the first radar data 320 generated at the time TRI, and the historical data 312, 314 associated with the time T-i and/or the time To may be used to generate estimated first radar data 328.
• the estimated first radar data may be a projection or prediction of the returns captured at TRI to a different, e.g., present or most-recent time.
• techniques described herein can determine an effect the changes would have on the radar return.
• a radar return associated with the first radar data 320 at time TRI may show a first doppler velocity at a first position.
• that velocity is no longer accurate at time To, at least because the relative velocity has changed and/or because the position of the object 308 has moved. Accordingly, when the change in velocity and/or position is known from T-i to To, the return from time TRI can be“projected” to a different time, e.g., time To, and used together with returns captured at To.
• a state velocity, a center of rotation, and/or a state yaw rate e.g., for a last known state (at To in the example) can be used as a basis for determining the projected first radar data 328.
• the second radar is substantially simultaneous with the time To at which a state of the vehicle 306 and/or the object 308 are determined, the second radar data 322 may not be projected. However, in other examples, the second radar data 322 can also be normalized or otherwise projected to a more recent time, e.g., as estimated second radar data (not shown.
• a time difference between measurements (e.g., 100ms) can be used to approximate what the velocities would be in a current time, as if the measurements were made contemporaneously .
• the process 300 can include determining an instantaneous yaw rate and/or a two-dimensional velocity using the estimated radar returns.
• the estimated first radar data 328 and the second radar data 322 can be used to determine a yaw rate and two-dimensional velocity 334 at the time To.
• the second radar data 322 is associated with (e.g., generated at) the time To and the first radar data 320 has been projected to the time To, e.g., as the estimated first radar data 328, returns associated with the second radar data 322 and estimated returns associated with the estimated first radar data 328 are normalized to the same time. Accordingly, the techniques described herein can determine the yaw rate and/or two-dimensional velocity to determine parameters associated with the vehicle.
• FIG. 3 can be used to increase a number of points for consideration in determining a yaw rate and/or two-dimensional velocity, as in the process 200 described above.
• the more points associated with an object the higher the degree of certainty that may be associated with the calculated yaw rate and/or two- dimensional velocity.
• the determination of the yaw rate and/or the two-dimensional velocity from a relatively larger set of returns can provide enhanced outlier rejection, thereby resulting in improved accuracy.
• the first radar data 320 may be a single scan by a first sensor (not shown) and the second radar data 322 may be a second scan by a second sensor (not shown).
• the scans may be the closest-in-time scans from the respective sensors, and the parameters, e.g., the yaw rate and/or the two-dimensional velocity can be determined using a single return from each sensor.
• the process 300 can be used to normalize the data from the different sensors to a common time.
• the first radar data 320 and the second radar data 322 can be scans by the same sensor. The process 300 described herein can be used to normalize one or more scans from one or more sensors
• FIG. 4 illustrates a block diagram of an example system 400 for implementing the techniques described herein.
• the system 400 can include a vehicle 402, which can be the same vehicle as the vehicle 102, the vehicle 206, and/or the vehicle 306 described above with reference to FIGS. 1, 2, and 3, respectively.
• the vehicle 402 can include one or more vehicle computing devices 404, one or more sensor systems 406, one or more emitters 408, one or more communication connections 410, at least one direct connection 412, one or more drive modules 414, and a user interface 416.
• the vehicle computing device(s) 404 can include one or more processors 418 and memory 420 communicatively coupled with the one or more processors 418.
• the vehicle 402 is an autonomous vehicle; however, the vehicle 402 could be any other type of vehicle.
• the memory 420 of the vehicle computing device 404 stores a localization component 422, a perception component 424, a planning component 426, one or more system controllers 428, a data association component 430, a yaw rate and two-dimensional velocity determination component 432, and historical data 434. Though depicted in FIG.
• the localization component 422, the perception component 424, the planning component 426, the one or more system controllers 428, the data association component 430, the yaw rate and two-dimensional velocity determination component 432, and/or the historical data 434 can additionally, or alternatively, be accessible to the vehicle 402 (e.g., stored on, or otherwise accessible by, memory remote from the vehicle 402).
• the localization component 422 can include functionality to receive data from the sensor system(s) 406 to determine a position and/or orientation of the vehicle 402 (e.g., one or more of an x-, y-, z-position, roll, pitch, or yaw).
• the localization component 422 can include and/or request/receive a map of an environment and can continuously determine a location and/or orientation of the autonomous vehicle within the map.
• the localization component 422 can utilize SLAM (simultaneous localization and mapping), calibration, localization and mapping, simultaneously techniques, relative SLAM, bundle adjustment, non-linear least squares optimization, or the like to receive image data, LiDAR data, radar data, IMU data, GPS data, wheel encoder data, and the like to accurately determine a location of the autonomous vehicle.
• the localization component 422 can provide data to various components of the vehicle 402 to determine an initial position of an autonomous vehicle for generating a candidate trajectory, as discussed herein.
• the perception component 424 can include functionality to perform object detection, segmentation, and/or classification.
• the perception component 424 can provide processed sensor data that indicates a presence of an entity that is proximate to the vehicle 402 and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, unknown, etc.).
• the perception component 424 can provide processed sensor data that indicates one or more characteristics associated with a detected entity and/or the environment in which the entity is positioned.
• characteristics associated with an entity can include, but are not limited to, an x-position (global position), a y-position (global position), a z-position (global position), an orientation (e.g., a roll, pitch, yaw), an entity type (e.g., a classification), a velocity of the entity, an acceleration of the entity, an extent of the entity (size), etc.
• Characteristics associated with the environment can include, but are not limited to, a presence of another entity in the environment, a state of another entity in the environment, a time of day, a day of a week, a season, a weather condition, an indication of darkness/light, etc.
• the perception component 424 may receive yaw rates of entities determined according to techniques described herein.
• the planning component 426 can determine a path for the vehicle 402 to follow to traverse through an environment. For example, the planning component 426 can determine various routes and trajectories and various levels of detail. For example, the planning component 426 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route can be a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 426 can generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location.
• GPS global positioning system
• the planning component 426 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints.
• the instruction can be a traj ectory , or a portion of a trajectory.
• multiple trajectories can be substantially simultaneously generated (e.g., within technical tolerances) in accordance with a receding horizon technique, wherein one of the multiple trajectories is selected for the vehicle 402 to navigate.
• the vehicle computing device 404 can include one or more system controllers 428, which can be configured to control steering, propulsion, braking, safety, emitters, communication, and other systems of the vehicle 402. These system controller(s) 428 can communicate with and/or control corresponding systems of the drive module(s) 414 and/or other components of the vehicle 402.
• the data association component 430 can be the data association component 134 detailed above.
• the data association component 430 can include functionality to associate points, e.g. from radar scans, with objects to determine a plurality of returns associated with individual objects.
• the data association component 430 may receive sensor data comprising a plurality of points and information associated with the points.
• the clustering component may receive position information, signal strength information, velocity information, or the like about points, and determine similarities between the points based on some or all of that information.
• the data association component 430 may determine points having close positional proximity and a signal strength within a threshold amount of neighboring points are indicative of an object and should be clustered.
• the data association component 430 can associate points from a number of scans and/or sensors. Accordingly, the clustering component may include points from the second scan to the cluster based on the criteria described herein. In some implementations, the clustering component may utilize algorithmic processing, e.g., DBSCAN, computer learning processing, e.g., K-means unsupervised learning, and/or additional clustering techniques.
• algorithmic processing e.g., DBSCAN
• computer learning processing e.g., K-means unsupervised learning
• the data association component 430 can also include functionality to associate points with each other and/or with objects based on the historical data 434.
• the historical data can include one or more previous states or parameters associated with an object, and based on those states/parameters, the data association component 430 can determine that radar returns are likely associated with the object.
• the yaw rate and two-dimensional velocity determination component 432 can be the yaw rate and two-dimensional velocity determination component 128, detailed further above.
• the yaw rate and two-dimensional velocity determination component 432 can include functionality to determine a yaw rate and/or two-dimensional of an object based a plurality of radar returns associated known to be associated with the object.
• the yaw rate and two-dimensional velocity determination component 432 may include functionality to use rigid body properties to estimate dynamics of the body in an x-y plane, e.g., velocity in the x- and y-directions and a rotation about the z-axis (yaw rate).
• the yaw rate and two-dimensional velocity determination component 432 can also include functionality to synchronize radar returns to a common time, e.g., such that radar returns captured at different times can be used to solve for an instantaneous yaw rate.
• the yaw rate and two-dimensional velocity determination component 432 can use the historical data 434, which may be tracking information, for example, to determine one or more previous states of the vehicle 402 and/or of a sensed object, and updated a previous radar return based on such change(s) of state.
• the historical data 434 may include a most recent state of a sensed object, which may be a most-recent velocity of the sensed object.
• Radar returns from a time previous to the most recent state can be updated based on the most recent state, e.g., by assuming a constant velocity (for instance, the most-recent state velocity) of the object from the previous state to the most recent state. Updating the previously-acquired radar return may include projecting a new position and/or a new velocity of the return, e.g., to a more recent time.
• the sensor system(s) 406 can include the radar sensors described herein. Also in examples, the sensor system(s) 406 can include LiDAR sensors, ultrasonic transducers, sonar sensors, location sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environment sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc.
• the sensor system(s) 406 can include multiple instances of each of these or other types of sensors.
• implementations of this disclosure may use multiple scans from multiple sensors, e.g., multiple radar sensors, with overlapping fields of view.
• the autonomous vehicle 402 may include a number of radar sensors.
• the LiDAR sensors can include individual LiDAR sensors located at the comers, front, back, sides, and/or top of the vehicle 402.
• the camera sensors can include multiple cameras disposed at various locations about the exterior and/or interior of the vehicle 402.
• the sensor system(s) 406 can provide input to the vehicle computing device 404.
• the sensor system(s) 406 can send sensor data, via the one or more networks 436, to the one or more computing device(s) at a particular frequency, after a lapse of a predetermined period of time, in near real-time, etc.
• the historical data 434 can be based at least in part on one or more of the sensor system(s) 406.
• the emitter(s) 408 may be configured to emit light and/or sound.
• the emitter(s) 408 in this example include interior audio and visual emitters to communicate with passengers of the vehicle 402.
• interior emitters can include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and/or force feedback), mechanical actuators (e.g., seatbelt tensioners, seat positioners, headrest positioners, etc.), and the like.
• one or more of the interior emitters may be used to signal to the passenger that the vehicle is approaching or has arrived at an unmapped region and that continued movement in the unmapped region will require permission and/or manual control.
• the interior emitters may alert the passenger(s) that a teleoperator or other external source (e.g., a passenger-in-waiting) has taken manual control of the vehicle 402.
• the emitter(s) 408 in this example can also include exterior emitters.
• the exterior emitters in this example include lights to signal a direction of travel or other indicator of vehicle action (e.g., indicator lights, signs, light arrays, etc.), and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) to audibly communicate with pedestrians or other nearby vehicles, one or more of which comprising acoustic beam steering technology.
• the communication connection(s) 410 can enable communication between the vehicle 402 and one or more other local or remote computing device(s). For instance, the communication connection(s) 410 can facilitate communication with other local computing device(s) on the vehicle 402 and/or the drive module(s) 414. Also, the communication connection(s) 410 can allow the vehicle to communicate with other nearby computing device(s) (e.g., other nearby vehicles, traffic signals, etc.). The communications connection(s) 410 also enable the vehicle 402 to communicate with a remote teleoperations computing device or other remote controllers.
• the communications connection(s) 410 can include physical and/or logical interfaces for connecting the vehicle computing device 404 to another computing device or a network, such as network(s) 436.
• the communications connection(s) 410 can enable Wi-Fi-based communication such as via frequencies defined by the IEEE 802.11 standards, short range wireless frequencies such as Bluetooth®, cellular communication (e.g., 2G, 4G, 4G, 4G LTE, 5G, etc.) or any suitable wired or wireless communications protocol that enables the respective computing device to interface with the other computing device(s).
• the vehicle 402 can include the drive module(s) 414.
• the vehicle 402 can have a single drive module 414.
• individual drive modules 414 can be positioned on opposite ends of the vehicle 402 (e.g., the front and the rear, etc.).
• the drive module(s) 414 can include one or more sensor systems to detect conditions of the drive module(s) 414 and/or the surroundings of the vehicle 402.
• the sensor system(s) can include one or more wheel encoders (e.g., rotary encoders) to sense rotation of the wheels of the drive modules, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors to acoustically detect objects in the surroundings of the drive module, LiDAR sensors, radar sensors, etc.
• Some sensors, such as the wheel encoders can be unique to the drive module(s) 414.
• the sensor system(s) 406 on the drive module(s) 414 can overlap or supplement corresponding systems of the vehicle 402 (e.g., the sensor system(s) 406).
• the drive module(s) 414 can include many vehicle systems, including a high voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and steering rack (which can be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and/or pneumatic components, a stability control system for distributing brake forces to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head/tail lights to illuminate an exterior surrounding of the vehicle), and one or more other systems (e.g., cooling system, safety systems, onboard charging system, other electrical components such as a DC/DC converter, a high voltage junction, a high voltage cable, charging system, charge port, etc.).
• vehicle systems including a high voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and steering rack (which can be
• the drive module(s) 414 can include a drive module controller which can receive and preprocess data from the sensor system(s) 406 and to control operation of the various vehicle systems.
• the drive module controller can include one or more processors and memory communicatively coupled with the one or more processors.
• the memory can store one or more modules to perform various functionalities of the drive module(s) 414.
• the drive module(s) 414 also include one or more communication connection(s) that enable communication by the respective drive module with one or more other local or remote computing device(s).
• the direct connection 412 can provide a physical interface to couple the one or more drive module(s) 414 with the body of the vehicle 402.
• the direction connection 412 can allow the transfer of energy, fluids, air, data, etc. between the drive module(s) 414 and the vehicle.
• the direct connection 412 can further releasably secure the drive module(s) 414 to the body of the vehicle 402.
• the user interface 416 may include one or more devices, buttons and/or control panels via which a passenger can communicate with the vehicle 402.
• a passenger in the vehicle 402 may control functionality of the vehicle 402 via interaction(s) with the user interface 416.
• the user interface 416 may comprise a microphone configured to receive a verbal or spoken input.
• the user interface 416 may provide a means though which a passenger can interface with the vehicle computing device(s) 404.
• the vehicle 402 may be in communication, via one or more network(s) 436, with one or more computing device(s) 438.
• the vehicle 402 can communicate with the one or more computing device(s) 438 via the network(s) 436.
• the vehicle 402 can receive control signals from the computing device(s) 438.
• the vehicle 402 can transmit information to the computing device(s) 438.
• the computing device(s) 438 may be embodied as a fleet management system.
• the computing device(s) 438 can include processor(s) 440 and memory 442 communicatively coupled with the processor(s) 440.
• the memory 442 of the computing device(s) 438 stores a data association component 444 and a yaw rate and two-dimensional velocity determination component 446.
• the data association component 444 can correspond to at least a portion of the data association component 430.
• the data association component 444 may receive radar data from multiple radar scans and determine point clusters representative of objects in the data.
• the yaw rate and two-dimensional velocity determination component 446 446 can correspond to at least a portion of the yaw rate and two-dimensional velocity determination component 432.
• the yaw rate and two-dimensional velocity determination component 446 may configure the computing device(s) 438 to determine an instantaneous yaw rate and/or two-dimensional velocity for an object based on radar returns identified by the data association component 444.
• FIGS. 2, 3, 5, and 6 illustrate example processes in accordance with embodiments of the disclosure. These processes are illustrated as logical flow graphs, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof.
• the operations represent computer- executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations.
• computer- executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
• the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
• FIG. 5 depicts an example process 500 for clustering sensor data, such as radar data from multiple radar scans.
• some or all of the process 500 can be performed by one or more components in FIG. 4, as described herein.
• some or all of the process 500 can be performed by the data association component 430 and/or the data association component 444 and/or by the yaw rate and two-dimensional velocity determination component 432 and/or the yaw rate and two-dimensional velocity determination component 446.
• the process 500 can include receiving first radar returns from a first radar sensor.
• the vehicle 402 can include sensor system(s) 406.
• the sensor system(s) 406 can include a plurality of radar sensors having overlapping fields of view, e.g., to receive radar returns associated with the same objects in an environment of the vehicle 402.
• the first radar returns can include one or more of position information of objects in an environment of the vehicle 402, velocity information (e.g., a doppler velocity) for the objects, signal strength (e.g., an RCS measurement), or the like.
• the process 500 can include receiving second radar data from a second radar sensor.
• the second radar sensor can have a field of view that overlaps with the field of view of the first radar sensor.
• the second radar sensor can generate, as the second radar data, one or more of position information, velocity information, signal strength information, or the like.
• the process 500 can include identifying first radar returns and second radar returns associated with an object.
• the first radar returns may be a portion of the first radar data and the second radar returns may be a portion of the second radar data.
• Techniques described herein may use a clustering component, e.g., the data association component 134, 430, 444, to associate returns with objects in the environment.
• the data association component(s) can group points based on common velocity, signal strength, position, or any other physical property measured by or otherwise associated with the radar returns.
• the data association component(s) can also consider historical information, such as previous states and/or tracks of the object and/or the sensors.
• the process 500 can include generating estimated retum(s) to synchronize identified returns to a common time.
• the first radar returns e.g., associated with the first radar data
• the second radar returns e.g., associated with the second radar data
• the first radar sensor and the second radar sensor may have different scan times, frequencies, or the like, such that the first radar data and the second radar data are not captured at exactly the same time.
• the first radar data and/or the second radar data can include multiple scans by the respective first radar sensor and/or the second radar sensor.
• the operation 508 can modify or update relatively older returns, e.g., to synchronize all returns at a common, e.g., most recent time.
• the operation 508 can use historical data of the vehicle 402 and/or of the object being sensed to project older-in-time returns to a more recent time.
• the historical data can include information about positions, velocities, accelerations, yaw, and/or yaw rates associated with the vehicle 402 and/or the sensed object.
• the operation 508 may include techniques like those described above in connection with FIG. 3.
• the process 500 can include determining a center of rotation for the object.
• a yaw rate can be determined for an object by projecting doppler velocity measurements onto a rotational center of the object. Accordingly, the operation 510 can identify this center of rotation. Because techniques described herein can assume the object is a rigid body, any point on the body can be used as the center of rotation. Accordingly, the center of rotation may be arbitrary. In at least some examples, a location associated with one of the first radar returns or the second radar returns can be determined to be the center of rotation.
• the process 500 can include determining a yaw rate about the center of rotation and/or a two-dimensional velocity from the first radar returns, the second radar returns, and/or the estimated retum(s).
• the yaw rate and two-dimensional velocity determination component 128, 432, 446 can determine a yaw rate of an object based on radar returns associated with the object.
• the yaw rate and two- dimensional velocity determination component 128, 432, 446 can estimate a solution of Equation (1), about the center of rotation, for all returns associated with the object.
• the inclusion of more points may provide a more robust estimation of the yaw rate and/or two- dimensional velocity.
• the yaw rate and the two-dimensional velocity can be solved using a least squares estimation.
• RANSAC logic can be used to remove outliers when solving Equation (1) simultaneously for all measured returns.
• the process 500 may transmit the yaw rate and/or the two- dimensional velocity for prediction and/or planning.
• the operation 514 can include sending the yaw rate and/or the two-dimensional velocity of the object to one or more of a perception system (e.g., the perception system 424), a planning system (e.g., the planning system 426), and/or the like.
• the yaw rate and/or the two-dimensional velocity may be used by one or more systems of the vehicle to determine one or more trajectories through the environment and relative to the object.
• FIG. 6 depicts an example process 600 for controlling an autonomous vehicle based at least in part on a yaw rate of a detected object, as discussed herein.
• some or all of the process 600 can be performed by one or more components in FIG. 4, as described herein.
• some or all of the process 600 can be performed by the localization component 422, the perception component 424, the planning component 426, and/or the system controller(s) 428.
• the process 600 can include receiving radar data associated with an object.
• the object may be characterized by a point cluster determined using the clustering techniques described herein.
• the radar data may include a plurality of radar returns from a plurality of radar sensors.
• the radar data may include at least three and in some instances preferably more than three radar returns. Of the returns, at least one may be from a first radar sensor and one or more of the returns can be from a second radar sensor. In other examples, all returns may be from a same sensor, e.g., from different scans of the same sensor.
• the process 600 can include determining an instantaneous yaw rate and/or a two-dimensional velocity of the object.
• the radar returns include position information and velocity, e.g., doppler velocity, information, and such velocities can be projected onto a center of rotation.
• the yaw rate and/or the two-dimensional velocity can be estimated using Equation (1), e.g., by simultaneously solving Equation (1) for all returns associated with the object to determine the three unknowns, as discussed above.
• Equation (1) Equation (1)
• the process 600 can include generating, based on the yaw rate, the two-dimensional velocity, and additional data, a travel path relative to the object.
• the travel path generated the operation 606 may be based at least in part on a predicted motion of the object, e.g., predicted based at least in part on the yaw rate and/or the two- dimensional velocity.
• the additional data can include sensor data from one or more of the sensor system(s) 406.
• the process 600 can include controlling an autonomous vehicle to follow the travel path.
• the commands generated in the operation 609 can be relayed to a controller onboard an autonomous vehicle to control the autonomous vehicle to drive the travel path.
• program modules include routines, programs, objects, components, data structures, etc., and define operating logic for performing particular tasks, or implement particular abstract data types.
• software can be stored and distributed in various ways and using different means, and the particular software storage and execution configurations described above can be varied in many different ways.
• software implementing the techniques described above can be distributed on various types of computer-readable media, not limited to the forms of memory that are specifically described.
• An example autonomous vehicle includes: a first radar sensor on the vehicle; a second radar sensor on the vehicle; one or more processors; and memory storing processor- executable instructions that, when executed by the one or more processors, cause to vehicle to perform acts comprising: receiving, from the first radar sensor, first radar returns; receiving, from the second radar sensor, second radar returns; determining a subset of radar returns associated with an object in an environment of the autonomous vehicle, the subset of radar returns comprising a plurality of the first radar returns and at least one of the second radar returns and being associated with a time; determining a center of rotation of the object; determining, based on the subset of radar returns, a yaw rate of the object about the center of rotation and a two-dimensional velocity of the object at the time; and generating, based at least in part on the yaw rate, a travel path through the environment relative to the object.
• A The autonomous vehicle of example A, wherein a first radar return of the first radar returns and a second radar return of the second radar returns include timestamp information, the acts further comprising: receiving historical information about at least one of the autonomous vehicle or the object; and determining, based at least in part on the historical information and the timestamp information, estimated radar data for individual of the plurality of radar returns, wherein the yaw rate is determined based at least in part on the estimated radar data.
• C The autonomous vehicle of example A or example B, wherein the time is a first time and the estimated radar data comprises a projection of a velocity from a second time prior to the first time to the first time.
• D The autonomous vehicle of any one of example A through example C, wherein: the first radar returns are associated with a first radar scan captured at the time and the time is a first time, the second radar returns are associated with a second radar scan captured at a second time prior to the first time, the historical information comprises at least one of an object velocity or an object yaw rate at the second time, and determining the estimated radar data comprises: determining an estimated track of the object from the second time to the first time; and estimating at least one of a depth or a velocity of at least one of the second radar returns based at least in part on the estimated track.
• E The autonomous vehicle of of any one of example A through example D, wherein the determining the yaw rate comprises: determining, for a return of the subset of radar returns, a first distance between a position associated with the return and the first radar sensor or the second radar sensor; and determining, for the return, a second distance between the first position and a second position associated with the center of rotation; wherein the yaw rate is based at least in part on a projection of a velocity along the second distance over the first distance.
• G An example method comprising: receiving radar returns from at least one radar sensor, the radar returns including information about a first distance to the at least one radar sensor; determining a subset of the radar returns associated with an object; for individual returns of the subset of the radar returns, determining a second distance from a location associated with the individual return to a rotational center of the object; and determining, based at least in part on the first distance and the second distance, a yaw rate of the object about the rotational center.
• J The method of any one of example G through example I, further comprising determining, based at least in part on the first distance and the second distance, a two- dimensional velocity of the object.
• K The method of any one of example G through example J, wherein the determining the subset of the radar returns associated with the object comprises generating a cluster of radar returns clustered based at least in part on at least one of positions of the radar returns, velocities of the radar returns, or signal strengths of the radar returns.
• L The method of any one of example G through example K, further comprising: receiving historical information about the object, the historical information comprising at least one of an historical velocity, an historical acceleration, a historical yaw rate, a historical orientation, or an historical position of the object, wherein the determining the subset of the radar returns comprises associating radar returns with the object based at least in part on the historical data.
• M The method of any one of example G through example L, further comprising: receiving timestamp information associated with the radar returns, wherein the determining the subset of radar returns comprises determining, based on the timestamp information, the subset of radar returns as returns within a period of time, and wherein the yaw rate is an instantaneous yaw rate determined for the period of time.
• N The method of any one of example G through example M, further comprising: receiving historical information comprising one or more of a previously determined yaw rate or velocity of the object; and determining, based at least in part on the historical information and the timestamp information, estimated radar data for one or more of the subset of radar returns, wherein the yaw rate is determined based at least in part on the estimated radar data.
• the estimated radar data comprises, for a radar return having first timestamp information identifying a first time, an estimated velocity of the radar return at a second time different from the first time.
• [00112] P One or more example non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising: receiving radar data from one or more radar sensors; determining a subset of the radar data associated with an object, the subset of the radar data including a radar return associated with a first time; receiving historical information about the object, the historical information comprising at least one of a previous velocity of the object, a previous position of the object, or a previous yaw rate of the object; determining, based at least in part on the historical information and the radar return, an estimated radar return comprising a projection of the radar return to a second time different from the first time; and determining, based at least in part on the estimated radar return and a plurality of additional radar returns associated with the subset of the radar data, a yaw rate of the object.
• R The one or more non-transitory computer-readable media of example P or example Q, the operations further comprising: receiving timestamp information associated with the radar returns, wherein the determining the subset of radar returns comprises determining, based on the timestamp information, the subset of radar returns as returns within a time range.
• S The one or more non-transitory computer-readable media of any one of example P through example R, wherein the determining the yaw rate comprises: determining, for the return, a first distance between a first position associated with the return and a radar sensor of the one or more radar sensors generating the return; and determining, for the return, a second distance between the first position and a second position associated with a center of rotation of the object; wherein the yaw rate is based at least in part on a projection of a velocity of the return along the second distance over the first distance.
• T The one or more non-transitory computer-readable media of any one of example P through example S, wherein the operations further comprise determining a two-dimensional velocity of the object based at least in part on the estimated radar return and a plurality of additional radar returns associated with the subset of the radar data.

Landscapes

• Engineering & Computer Science (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Electromagnetism (AREA)
• Radar Systems Or Details Thereof (AREA)
• Traffic Control Systems (AREA)
• Navigation (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=73456773

Family Applications (1)

Country Status (5)

Cited By (1)

Families Citing this family (23)

Citations (5)

Family Cites Families (14)

• 2019
  • 2019-05-20 US US16/416,686 patent/US11275173B2/en active Active
• 2020
  • 2020-05-19 JP JP2021569050A patent/JP7534333B2/ja active Active
  • 2020-05-19 EP EP20810322.6A patent/EP3973320A4/en active Pending
  • 2020-05-19 CN CN202080037264.8A patent/CN113853533B/zh active Active
  • 2020-05-19 WO PCT/US2020/033608 patent/WO2020236819A1/en not_active Ceased

Patent Citations (5)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events