JP7464616B2 - Recognizing radar returns using speed and position information. - Google Patents

Description

Various exemplary embodiments of the present invention relate to recognizing radar returns using velocity and position information.

This PCT International Patent Application claims the benefit of and priority to the filing date of U.S. Patent Application No. 16/288,990, filed February 28, 2019, and U.S. Patent Application No. 16/289,068, filed February 28, 2019, the disclosures of each of which are incorporated herein by reference.

Autonomous vehicles utilize various methods, devices, and systems to navigate environments that include obstacles. For example, autonomous vehicles may utilize route planning methods, devices, and systems to travel through areas that may include other vehicles, structures, pedestrians, and the like. These planning systems may rely on sensor data, including radar data, LiDAR data, image data, and the like. However, in certain instances, the presence of vehicles, structures, and/or objects in the environment may cause reflections that result in inaccurate or erroneous sensor data, including, for example, false positives. Inaccurate and/or erroneous sensor data may pose challenges to safely and comfortably navigating an environment.

The detailed description will be described with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different drawings indicates similar or identical components or features.

FIG. 1 is a schematic diagram illustrating an exemplary vehicle including a radar sensor for sensing objects in an environment, and the environment in which the vehicle operates, according to an embodiment of the present disclosure. 2 is a schematic diagram of the environment of FIG. 1 illustrating an example technique for using velocity information to distinguish radar returns associated with objects in the environment from reflected returns reflected from intervening objects, according to an embodiment of the present disclosure. FIG. 2 is another schematic diagram of the environment of FIG. 1 illustrating an example technique for using position information to distinguish radar returns associated with objects in the environment from reflected returns reflected from intervening objects, according to an embodiment of the present disclosure. FIG. 1 is a schematic block diagram illustrating an example system including a vehicle and a computing device that can be used to implement the radar reflection recognition techniques described herein, in accordance with embodiments of the present disclosure. 4 is a flowchart illustrating an example process for implementing a radar reflection recognition technique that uses velocity information, in accordance with an embodiment of the present disclosure. 4 is a flowchart illustrating an example process for implementing a radar reflection recognition technique using location information, according to an embodiment of the present disclosure.

As described above, some types of sensor data, e.g., radar data, may be susceptible to reflections, for example, from vehicles, structures, and other objects in the environment. Such reflected clutter can pose challenges to an autonomous vehicle's safe and/or comfortable traversal of the environment. Planning in response to these nonexistent or phantom "objects" can cause the vehicle to take unnecessary actions, e.g., braking, steering, etc.

This application describes techniques for identifying reflected echoes in radar sensor data captured by a radar system. In general, a radar sensor may emit wireless energy that reflects (or bounces) off objects in the environment before returning to the sensor. When the emitted energy returns directly from the object to the radar sensor, the radar sensor can capture object echoes that contain accurate data about the object (e.g., distance, position, velocity, etc.). However, in some examples, the wireless energy can reflect off a variety of objects in the environment before returning to the radar sensor. In these examples, the radar sensor can capture reflected echoes that do not accurately represent objects in the environment. In at least some examples, without knowing whether these echoes are from actual objects or are reflections, safety-critical path planning for autonomous vehicles can be affected.

In one example, the techniques described herein may determine that a radar return is a reflected return using a pairwise comparison of the return, e.g., the return being considered, to know the object return. More specifically, a radar return not associated with a known object may be compared to an object return (e.g., a return associated with a known object) to determine whether the radar return corresponds generally to a theoretical return of the object return from some hypothetical object. In one example, the object return can be determined from among all radar returns based on information relating the return to a track or other previously acquired information. As a non-limiting example, the theoretical return can be determined by projecting the object velocity (magnitude and direction from the object return that may be caused by the radar return) onto a radial direction extending from the vehicle to the return being considered. For example, the radial component of the projected velocity extending along the radial direction represents the expected velocity of the reflection of the object return along this radial direction. Thus, when the magnitude of the radial component of the projected velocity corresponds to the velocity associated with the return in question, the return may be identified as a reflected return.

In one example, the techniques described herein can use pairwise comparisons of radar returns where there is no previous knowledge available to determine that a return is reflected. For example, any two returns can be compared to determine whether it is theoretically possible for one of the returns, e.g., the more distant one of the returns, to be a return of the other of the returns. In one example, such a comparison can be based on projecting the velocity of the closer return onto the location of the more distant one of the returns. In another example, the location of the returns can be used to determine a theoretical reflection point on a line between the sensor and the more distant one of the returns.

In some examples, the techniques described herein may function to verify that an identified reflected echo is in fact a reflection. For example, the techniques described herein may determine a hypothetical reflection point, e.g., a point along the radial direction of the considered echo, from which wireless energy would be reflected if the considered echo were a reflected echo. In aspects of the present disclosure, further sensor information about the environment, e.g., LiDAR data, further radar data, time-of-flight data, SONAR data, image data, etc., may be used to verify the presence of an object proximate to the reflection point. In other words, the presence of an object at the hypothetical or calculated reflection point may further suggest (and/or confirm) that the echo is a reflected echo of another echo.

Also, in some implementations, the techniques described herein may determine a subset of all radar clutters for the search as potential reflected clutters. For example, the radar clutters may be filtered to include only radar clutters that may be reflected clutters and/or that may have a non-negligible effect on the operation of the vehicle. In some examples, clutters associated with tracks or other previously acquired information may be designated as object clutters and thus not potential reflected clutters. Furthermore, clutters that are relatively closer to the vehicle, e.g., radially closer, than the object clutters may not be possible reflected clutters and thus may be excluded from consideration using the techniques described herein. Furthermore, radar clutters having a speed below a threshold speed may be ignored, e.g., because they are not considered by the vehicle's planning system. In some examples, the threshold speed may vary with distance from the vehicle. In at least some instances, filtering such radar returns may reduce the amount of time, processing, and/or memory required to determine whether a return is a reflection of an actual object.

In one example, information about the reflected radar returns may be output to be used by a vehicle computing device of the autonomous vehicle to control the autonomous vehicle safely through the environment. For example, the vehicle computing device may exclude the reflected returns from route planning and/or trajectory planning. In this manner, the autonomous vehicle does not brake, steer, or otherwise take action in response to the phantom "object." Additionally, the vehicle computing device may not track or otherwise follow returns that are determined to be reflected returns, which may, for example, reduce processing load.

The techniques described herein can improve the functionality of a computing device in multiple ways. For example, in the context of determining control for a vehicle, the amount of data to be considered can be reduced, for example, by filtering out reflected bounces, thereby reducing excessive resources devoted to unnecessary decisions about the environment. Improved trajectory generation can improve safety outcomes and can improve the rider experience (e.g., by reducing the occurrence of unnecessary braking in response to phantom objects, sudden turns to avoid phantom objects, and the like). These and other improvements in computer functionality and/or user experience are described herein.

The techniques described herein can be implemented in multiple ways. Exemplary implementations are provided below with reference to the accompanying figures. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein can be applied to a variety of systems (e.g., robotic platforms) and are not limited to autonomous vehicles. In another example, the techniques can be utilized in an aviation or nautical context, or in any system that uses machine vision.

1 is a schematic diagram of an environment 100 in which a vehicle 102 is operating. In the illustrated example, the vehicle 102 is moving in the environment, while in other examples, the vehicle 102 may be stationary and/or parked in the environment 100. The vehicle 102 includes one or more radar sensor systems 104 that capture data representative of the environment 100. By way of example, and not by way of limitation, the vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle that can perform all safety-critical functions throughout the entire journey without a driver (or passenger) being expected to control the vehicle at any point. In such an example, the vehicle 102 may be passenger-less, as it may be configured to control all functions from start to stop, including all parking functions. This is by way of example only, and the systems and methods described herein may be incorporated into any land, air, or water vehicle, including vehicles ranging from those that must be manually controlled at all times by a driver to those that are partially or fully autonomously controlled. Further details associated with vehicle 102 are described below.

The vehicle 102 may move through the environment 100, for example, generally in a direction indicated by arrow 106 relative to one or more other objects. For example, the environment 100 may include dynamic objects, such as an additional vehicle 108 (moving generally in a direction indicated by arrow 110), and stationary objects, such as a first parked vehicle 112(1), a second parked vehicle 112(2), a third parked vehicle 112(3) (collectively, "parked vehicles 112"), a first structure 114(1), a second vehicle 114(2), and a third vehicle 114(3) (collectively, "structure 114"). The additional vehicle 106, the parked vehicle 112, and the structure 114 are merely examples of objects that may be in the environment 100, and additional and/or different objects may also or alternatively be in the environment 100, including, but not limited to, vehicles, pedestrians, bicyclists, trees, road signs, fixtures, and the like.

In at least one example, as described above, the vehicle 102 can be associated with a radar sensor 104, which can be disposed on the vehicle 102. The radar sensor 104 can be configured to measure the range to an object and/or the speed of the object. In an example system, the radar sensor 104 can include a Doppler sensor, a pulsed sensor, a continuous wave frequency modulated (CFWM) sensor, or the like. The radar sensor 104 can emit pulses of wireless energy at predetermined intervals. In an implementation, the intervals can be configurable, for example, to facilitate enhanced detection of objects at relatively long or relatively close distances. In general, the pulses of wireless energy emitted by the radar sensor 104 can reflect off objects in the environment 100 and be received by the radar sensor 104, for example, as radar data or radar returns.

In the example of FIG. 1, wireless energy emitted by the radar sensor 104 generally along the direction of the arrow 116 may come into contact with the further vehicle 108 and be reflected back toward the radar sensor 104, for example, generally along the direction of the arrow 118. In this example, the wireless energy is emitted and returned along generally the same path, which may be the object return path 120. The object return path 120 is substantially a straight line in the illustrated example. The wireless energy reflected along the object return path 120 is captured by the radar sensor 104, for example, as the object return 122. The object return 122 is illustrated as a block at a location determined based on the captured data. For example, information associated with the object return 122 may include information indicating a location in the environment, for example, the location of the further vehicle 108. The location information may include range and azimuth relative to the vehicle 102, or a location in a local or global coordinate system. Also, in an implementation, the object return 122 may include signal strength information. For example, the signal strength information can indicate the type of object. More specifically, radio waves may be reflected more strongly by objects having certain shapes and/or components. For example, broad, flat surfaces and/or sharp edges are more reflective than rounded surfaces, and metal is more reflective than people. In one example, the signal strength may include a radar cross section (RCS) measurement. The object reflection 122 may also include speed information. For example, the speed of the further vehicle 108 may be based on the frequency of the radio energy reflected by the further vehicle 108 and/or the time the reflected radio energy was detected.

The object reflection 122 may be an example of accurate data captured by the radar sensor 104, for example, corresponding to the further vehicle 108. However, it is also possible for the radar sensor 104 to capture reflections that may be less reliable. For example, FIG. 1 also illustrates that wireless energy emitted by the radar sensor 104 generally along the direction of arrow 124 may reflect off the first structure 114(1) traveling generally along the direction of arrow 126. In the illustrated example, the wireless energy reflected off the first structure 114(1) may then be reflected by the further vehicle 108 generally along the direction of arrow 128, i.e., opposite to the direction of arrow 126, back toward the first structure 114(1). Finally, the wireless energy may then reflect again from the first structure 114(1) back to the radar sensor 104 generally along the direction of arrow 130. Thus, the wireless energy reflected from the further vehicle 108 may return to the radar sensor 104 along a first reflected wave path 132 that includes a first leg 134 between the further vehicle 108 and the first structure 114(1) and a second leg 136 between the first structure 114(1) and the vehicle 102 (e.g., radar sensor 104). Such reflected energy may be captured by the radar sensor 104 as a first reflected wave 138. As illustrated in FIG. 1 , the first reflected wave 138 is received along the direction of the second leg 136, e.g., along arrows 124, 130, but has a range equal to the distance of the first reflected wave path 132, e.g., a distance that is the sum of the distance of the first leg 134 and the distance of the second leg 136. As can be seen from FIG. 1 and the above description, the first reflected wave 138 suggests the presence of an object at a location corresponding to the first reflected wave 138, i.e., along the direction of the second section 136. However, as described herein, such an "object" is a non-existent, phantom "object."

1 illustrates that the radar sensor 104 can also capture a second reflected echo 140 corresponding to wireless energy reflected from the first parked vehicle 112(1) and the further vehicle 108. More specifically, wireless energy emitted by the radar sensor 104 generally along the direction of arrow 142 may reflect from the first parked vehicle 112(1) traveling generally along the direction of arrow 144. The wireless energy may then contact the further vehicle 108 and reflect back toward the first parked vehicle 112(1) generally along the direction of arrow 146, i.e., opposite the direction of arrow 144. Finally, the wireless energy may then reflect again from the first parked vehicle 116(1) generally along the direction of arrow 148. Thus, the wireless energy corresponding to the second reflected wave 140 may travel along a second reflected wave path 154 that includes a first section 156 between the further vehicle 108 and the first parked vehicle 112(1) and a second section 158 between the first parked vehicle 112(1) and the vehicle 102 (e.g., radar sensor 104). As illustrated in FIG. 1, the second reflected wave includes information about wireless energy received along the second section 158, e.g., along the direction of the arrows 142, 148, but has a range equal to a distance of the second reflected wave path 154, e.g., a distance that is the sum of the distance of the first section 156 and the second distance of the second section 158. As can be understood from FIG. 1 and the above description, the second reflected wave 140 indicates the presence of an object at a location corresponding to the second reflected wave 140, i.e., along the direction of the second section 158. However, as explained in this specification, such "objects" are non-existent, phantom "objects."

In addition to including position information (e.g., from range and azimuth) as illustrated by FIG. 1, the object return 122, the first reflected return 138, and the second reflected return 140 (herein the first reflected return 138 and the second reflected return 140 may be referred to as "reflected returns 138, 140") may also include speed information. More specifically, the object return 122 may include information about the object velocity 160, which corresponds to, for example, the velocity (of the further vehicle 108) along the direction of the arrow 118. Similarly, the first reflected wave reflection 138 may include information about a first reflected wave velocity 162, e.g., a velocity along the direction of the second section 136 of the first reflected wave path 132, and the second reflected wave reflection 140 may include information about a second reflected wave velocity 164, e.g., a velocity along the direction of the second section 158 of the second reflected wave path 154. Thus, while the object reflection 122 provides information (e.g., position, velocity) about the additional vehicle 108, the first reflected wave reflection 138 suggests that an object (phantom object) is approaching from a position associated with that reflection at the first reflected wave velocity 162, and the second reflected wave reflection 140 suggests that an object (phantom object) is approaching from a position associated with that reflection at the second reflected wave velocity 164. As described herein, the vehicle 102 may include a planning system that, among other functions, determines a route, trajectory, and/or control for objects in the environment 100 based on received sensor data. However, a planning system that uses reflected echoes 138, 140 may plan to react to objects that are not actually present.

The techniques described herein may improve planning system accuracy and planning system performance by recognizing reflected waves such as reflected waves 138, 140 as reflected reflected waves. For example, as further illustrated in FIG. 1, the radar sensor 104 may be one of a plurality of sensor systems 164 associated with the vehicle 102. In an example, the sensor system 164 may further include one or more additional sensors 166, which may include additional radar sensors, light detection and ranging (LiDAR) sensors, ultrasonic transducers, acoustic navigation and ranging (sonar) sensors, position sensors (e.g., Global Positioning System (GPS), COMPASS, etc.), inertial sensors (e.g., inertial measurement units, accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), wheel encoders, microphones, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc.

1, the radar sensor 104 can generate radar data 168 and the additional sensor 166 can generate sensor data 170. The radar data 168 can include information about the object return 122, the first reflected return 138, and the second reflected return 140, including, but not limited to, position, velocity, and/or other information associated with these returns. In some examples, the radar data 168 can also include signal strength information, which may include RCS measurements. The radar data can also include specific information about the sensor, including, but not limited to, the orientation of the sensor, e.g., the orientation of the sensor relative to the vehicle, the pulse repetition frequency (PRF) or pulse repetition interval (PRI) for the sensor, the field of view or detection arc, etc. In some implementations, some aspects of the radar data 168, e.g., the field of view, the orientation, the PRF or PRI, etc., can be pre-configured, in which case the data can be available, e.g., in storage, and not transferred along with the radar return, for example. The sensor data 170 can include any information about the environment 100 and/or the further sensor 166. As a non-limiting example, when the further sensor 166 includes a LiDAR sensor, the sensor data 170 can include point cloud data, and when the further sensor 166 includes a camera, the sensor data 170 can include image data. The radar sensor 104 and the further sensor 166 can generate the radar data 168 and the sensor data 170, respectively, at predetermined intervals that may be the same or different for the different sensor systems 164. For example, the radar sensor 104 can have a scanning frequency at which reflected waves are captured and the radar data 168 is generated.

The radar data 168 and the sensor data 170 can be received at one or more vehicle computing devices 172 and utilized by the vehicle computing devices 172 to perform planning, for example, using a planning system (not shown). In the illustrated example, the sensor system 164 and the vehicle computing devices 172 are located on the vehicle 102, for example, and are part of the vehicle 102. However, in other examples, some or all of the sensor system 164 and/or the vehicle computing devices 172 may be separate from the vehicle 102 and/or may be located remotely from the vehicle 102. In such configurations, data capture, processing, command, and/or control may be communicated to/from the vehicle 102 by one or more remote computing devices via wired and/or wireless networks.

In at least one example, the vehicle computing device 172 can utilize the radar data 168 and the sensor data 170 captured by the sensor system 164 in a reflection recognition component 174. For example, the reflection recognition component 174 can receive the radar data 168 including the object return 122, the first reflected return 138, and the second reflected return 140, and determine that the reflected return 138, 140 are reflected returns rather than returns associated with actual objects in the environment. By making this distinction, the reflection recognition component 174 can send only the object return 122 (without sending the reflected return 138, 140) to the planning system and/or use additional returns to further refine the estimated position and/or estimated velocity of the object. Thus, control planning can be performed excluding the reflected return 138, 140. As described in more detail below with reference to FIG. 2, the reflection recognition component 174 can determine whether a reflection (e.g., a candidate reflection or an unidentified reflection) is a reflected reflection by comparing two reflections, such as the object reflection 122 and an additional reflection, such as reflection 138 and/or reflection 140. For example, the reflection recognition component 174 can "project" a known object reflection, such as the object reflection 122, onto a line that extends along a direction associated with the unidentified reflection, e.g., a direction extending radially from the radar sensor 104 and passing through the unidentified reflection. Techniques for projecting the object reflection 122 are described in more detail below with reference to FIG. 2, but the projection of the reflection has a velocity component that has the same direction as the direction of the velocity associated with the unidentified reflection (e.g., radially relative to the radar sensor 104). The reflection recognition component 174 can also determine a magnitude of velocity associated with the projection of the object reflection 122. In some implementations, if this magnitude matches the magnitude of the velocity of the unidentified return, the reflection recognition component 174 can determine that the unidentified return is (or is likely to be) a reflected return. Magnitudes as used herein may match if they are substantially equal, e.g., within some threshold range or within some error range.

The reflection recognition component 174 may utilize some a priori knowledge of the objects in the environment and/or the environment itself. For example, the reflection recognition component 174 may only project reflections that are known object reflections. In some examples, this knowledge may be by tracking and/or previously identifying an object, such as the further vehicle 108. For example, such tracking and/or identification may be based on previously captured radar data 168 sensing the object and/or previously captured sensor data 170. For example, an object, such as the further vehicle 108, may be tracked over a period of several seconds and/or at an extended distance. On the other hand, reflected reflections may be more ephemeral as they require alignment of the vehicle 102, the further vehicle 108, and intervening reflective objects/surfaces (e.g., the first parked vehicle 112(1) and the first structure 114(1)). If one or more of those objects move, the conditions (e.g., relative alignment and/or orientation) that allow for a reflected echo may disappear, causing the reflected echo to disappear from a subsequent scan of the radar sensor 104. Additionally or alternatively, map data available to the vehicle 102 (which may be downloaded from time to time based on the location of the vehicle 102 or otherwise accessible to the vehicle) may comprise a three-dimensional representation of the environment, such as, for example, a mesh of the local three-dimensional environment. In such an example, the reflection recognition component 174 may determine that the first reflected echo 138 is a non-material echo based on knowledge of the corresponding structure 114(1).

The reflection recognition component 174 may also include functionality to confirm the determination of the component 174 that the candidate return is a reflected return. In one example, the reflection recognition component 174 may attempt to track the return, for example, over subsequent scans received as radar data 168. As discussed above, relative motion of the vehicle 102 relative to the further vehicle 108 may result in weaker reflection conditions and therefore failure to receive subsequent returns corresponding to the "object" associated with the reflected return. However, ignoring a potential (approaching) object until the object's track can be verified over successively collected radar scans may slow reaction times, which may be unsafe. Thus, in other implementations, the reflection recognition component 174 may use, for example, the sensor data 170 and/or the radar data 168 to determine whether a reflective object is located along a direction toward the reflected return. Referring to FIG. 1, in one example, LiDAR data, image data, or the like may confirm the presence of the first parked vehicle 112(1) and/or the first structure 114(1). Thus, the reflection recognition component 174 may identify the first parked vehicle 112(1) and/or the first structure 114(1) and confirm a previous determination that the reflection is a reflected reflection 138 based on the location of those objects, e.g., at the junction between the segments of the respective reflected reflection paths 132, 154. Map data may be useful for identifying static, fixed objects, e.g., structures 114, terrain, road signs, utility equipment, etc. However, real-time or near real-time sensor data may be required to identify movable objects, whether static or dynamic. For example, parked vehicles 112, pedestrians, bicyclists, other moving vehicles, etc. may not be available via map data.

In some implementations, the reflection recognition component 174 can process radar returns to determine whether the radar returns are reflected returns in real time or near real time. For example, the reflection recognition component 174 can compare multiple returns to known object returns, such as object returns 122, for example, in parallel. In one example, all returns that are unknown object returns in a radar scan can be compared to known object returns to determine whether such returns are reflections. In other implementations, the reflection recognition component 174 can filter the returns, for example, to eliminate points that are unlikely to be reflections or that are physically impossible to be reflections. As a non-limiting example, a relatively closer return compared to a known object return cannot be a reflection of an object return. In other examples, the reflection recognition component 174 can eliminate returns that exhibit a speed below a certain threshold or a speed of zero. Other filtering techniques may also be used.

While the examples described herein may compare points or reflections to known object reflections 122, in other examples, the reflection recognition component 174 may also or alternatively compare pairs of reflections, e.g., without knowledge that one of the reflections is a known object reflection. More specifically, the techniques described herein may determine that two reflections are likely reflections of each other, regardless of whether one of the reflections is an object reflection. For example, when a pair of reflections is considered, the more distant reflection may be tagged as a potential reflection or otherwise designated as such. Further processing may be used as a basis for determining whether the closer reflection is associated with an object in the environment (or the more distant reflection is otherwise associated with a reflection). The reflection recognition component 174 may perform high-level filtering as described herein to identify a subset of points for pairwise comparison. As non-limiting examples, returns below a threshold velocity may be omitted, pairs with too large a difference in position and/or velocity (e.g., greater than or equal to a threshold difference) may not be compared, etc. In at least some examples, various techniques may be combined to improve the level of certainty about whether a return is a reflection.

FIG. 2 is another schematic diagram of environment 100 illustrating further aspects of the present disclosure. To avoid confusion, elements from FIG. 1 that are specifically described with reference to FIG. 2, such as environment 100, include the same reference numbers in FIG. 1 and FIG. 2. Additionally, for clarity, certain elements of environment 100 are grayed out in FIG. 2.

More specifically, FIG. 2 shows the vehicle 102, the further vehicle 108, the first parked vehicle 112(1), and the first structure 114(1). FIG. 2 also shows the object return 122, the first reflected return 138, and the second reflected return 140. As described above in connection with FIG. 1, the object return 122 and the reflected return 138, 140 may include position and speed information based on attributes of the wireless energy received at the radar sensor 104. For example, the object return 122 spaced a distance from the vehicle 102 along a line 202 passing through both the vehicle 102 and the further vehicle 108 may indicate both the position of the object return (and therefore the position of the further vehicle 108) and the object return speed 160. As will be described, since the object return 122 is a known (e.g., from previously acquired data about the environment 100 and/or the further vehicle 108, such as a truck) direct reflection from the further vehicle 108, e.g., along the line 202, the object return may be considered to be an accurate representation of a portion of the further vehicle 108. As stated above, the presence of an object, e.g., the further vehicle 108, or knowledge that the return is an object return, e.g., the object return 122, may not be required in all instances. Techniques described herein may be applied to pairwise comparisons of the returns to determine the likelihood that the returns are reflections of one another.

The reflected echoes 138, 140 also include at least position and velocity information associated with the wireless energy received at the radar sensor 104. As described above, the first reflected echo 138 includes information about the first reflected echo velocity at an illustrated position along the line 204. The distance of the first reflected echo 138 along the line 204 is a distance or range determined by the radar sensor 104 and corresponds to a distance traveled by the received wireless energy, e.g., a distance corresponding to the first reflected echo path 132 shown in FIG. 1. Similarly, the second reflected echo 140 includes information about the second reflected echo velocity at an illustrated position along the line 206. The distance of the second reflected echo 140 along the line 206 is a distance or range determined by the radar sensor 104 and corresponds to a distance traveled by the received wireless energy, e.g., a distance corresponding to the second reflected echo path 154 shown in FIG. 1.

The techniques described herein may utilize geometry in the environment 100 to determine whether a radar return is a reflected return. For example, at the time of capture at the radar sensor 104, the object return 122 and the reflected returns 138, 140 are merely returns. Only through some previously acquired knowledge of the environment 100 and/or the objects in the environment 100 can the object return 122 be assigned with certainty to an additional vehicle 108. In one example, a tracker or other component associated with the vehicle 102 may be tracking the additional vehicle 108, and the return is identified as an object return 122 if it matches an expectation associated with the track. In another example, as described herein, the object return 122 can be considered, for example, as one return in a return pair, regardless of whether the object return 122 is assigned to an additional vehicle 108. While the object return 122 is illustrated as a single point at a centrally located location in front of the further vehicle 108, the object return 122 may correspond to one or more other returns and/or one or more other locations on the further vehicle 108. As a non-limiting example, attributes of the object return, such as the object return velocity 160, may be determined based on multiple returns associated with the further vehicle 108. For example, the object return 122 may be an average of multiple returns from the further vehicle 108.

As mentioned in the previous paragraph, unlike object reflections 122, reflected reflections may be transient, occurring only when some conditions, e.g., geometric conditions, exist that may cause the reflected reflections to move erratically or to come in and out of existence. Therefore, there is no track or other a priori knowledge of the reflected reflections. However, the techniques described herein determine whether a reflection is a reflection or may directly correspond to an actual object in the environment 100.

As described above, the first reflected echo 138 corresponds to a echo located along the line 204. To characterize the first reflected echo 138 as a reflected echo, the techniques herein may pair it with the object echo 122. For example, since the direction of the line 204 is known and the position of the first reflected echo 138 relative to the radar sensor on the line 204 is known, the object echo 122 can be projected onto the position of the first reflected echo. Conceptually, projecting the object echo 122 can include determining a line 208 that connects a position associated with the object echo 122 (e.g., along the radial line 202) with a position associated with the first reflected echo 138 (e.g., along the radial line 204). As also illustrated, a line 210 perpendicular to and bisecting the line 208 intersects the line 204 at a reflection point 212, such that the distance of a line segment 214 between the reflection point 212 and the object reflected wave 122 is equal to the distance of a line segment 216 between the reflection point 212 and the first reflected reflected wave 138. Furthermore, the object reflected wave velocity 160 can be reflected about the line 210 as a first reflected velocity 218. Once determined, the first reflected velocity 218 can be resolved into two velocity components: a radial velocity component 220 generally along the line 204, and a tangential velocity component 222 perpendicular to the radial velocity component 220. The radial velocity component 220 is the projected velocity of the object reflected wave 122 along the first direction 204. In other words, the first projected velocity is the radial velocity component 220 of the first reflected velocity 218, which is the mirror image of the vector representing the object reflected wave velocity 160 about the line 210.

In the example of FIG. 2, the radial velocity component 220 (projected velocity) is the velocity (i.e., magnitude and direction) that would be sensed by the radar sensor 104 if the wireless energy bouncing off the further vehicle 108 were also reflected at the reflection point 212. As can be seen, the radial velocity component 220 of the first reflected velocity 218 is along the same direction (i.e., along the line 204) as the first reflected reflected wave velocity 162 (shown in FIG. 1 and not shown in FIG. 2). Thus, if the magnitude of the radial velocity component 220 (first projected velocity) and the magnitude of the first reflected reflected wave velocity 162 are substantially similar, e.g., within a predetermined threshold or range of each other, the first reflected echo 138 can be flagged, tagged, or otherwise identified as having a high likelihood of being a reflection.

2 shows a similar conceptualization of determining whether the second reflected wave 140 is a reflected wave. For example, the second reflected velocity 224 at the location of the second reflected wave 140 may be a mirror image of the object reflected wave velocity 160 about a line 226. The line 226 is perpendicular to and bisects a line 228 that extends between the location of the object reflected wave 122 and the location of the second reflected wave 140. The second reflected velocity 224 includes a radial velocity component 230, i.e., a component along the radial line 206, and a tangential velocity component 232, i.e., a component that is perpendicular to the radial line 206.

2, the line 226 meets the radial line 206 at a reflection point 234. As described above, the radial velocity component 230 of the second projected velocity 224 is the projected, e.g., expected, velocity (i.e., direction and magnitude) of the reflected wave associated with the wireless energy bouncing off the further vehicle 108 and reflecting back to the radar sensor 104 at the reflection point 234. In other words, the radial velocity component 230 is the projected velocity and corresponds to the expected reflected wave associated with the reflection at the reflection point 234. As with the first projected reflected wave, the direction of the radial velocity component 230 is substantially the same as the direction of the second reflected reflected wave velocity 164. Thus, both directions are along the line 206. Thus, if the magnitude of the radial velocity component 230 (second projected return) and the magnitude of the second reflected return wave velocity 164 are substantially identical, the second reflected return wave 140 can be tagged, flagged, or otherwise identified as having a high likelihood of being a return.

In an implementation of the present disclosure, as described further herein, after determining that the reflected reflections 138, 140 are reflected reflections, e.g., because the respective radial velocity components 220, 230 are substantially identical to the first reflected reflection wave velocity 162 and the second reflected reflection wave velocity 164, the vehicle 102 may ignore, e.g., exclude, the reflected reflections 138, 140 from planning. The vehicle 102 may also confirm the reflections, e.g., by determining that an object is present at each of the reflection points 212, 234. For example, the vehicle 102 may confirm the presence of the object using sensor data, map data, etc., as described herein. In at least some examples, the reflections may be flagged such that other components may be aware that there may not be any corresponding objects associated with those reflections.

2 also illustrates another exemplary implementation in which a pedestrian 236 exits a structure 238 and enters the environment 100. As illustrated, the pedestrian 236 may enter the environment at a location close to the location associated with the second reflected echo 140. The radar sensor 104 may generate radar data including information about the newly present pedestrian. Because the pedestrian 236 has not been tracked previously, i.e., the pedestrian 236 has just appeared, the implementation described herein may use techniques described herein to compare the echo associated with the pedestrian 236 to known object echoes, e.g., the object echo 122. For example, the object echo velocity 160 may be projected onto the location associated with the pedestrian 236. Because the pedestrian 236 is farther away from the radar sensor 104 than the further vehicle 108, a reflection point may also be identified. In the illustrated example, the reflection point is close to the reflection point 234. The radial component of the projected object clutter velocity at the location of the pedestrian clutter can also be determined as in the other examples. However, because the pedestrian clutter is not a reflected clutter, the velocity magnitude of the radial velocity component of the projected object clutter velocity is likely not similar. For example, the pedestrian clutter will not be (correctly) identified as a clutter unless the pedestrian is walking in a manner in which the component of the pedestrian's velocity toward the radar sensor 104 matches the velocity of the reflected clutter. Upon determining that the pedestrian clutter is not a clutter, the pedestrian's movement may be tracked and/or information about the pedestrian 236 may be used to generate control for the vehicle.

Aspects of the present disclosure are not limited to the exemplary implementation shown in FIG. 2. For example, the techniques described herein can be used to identify reflections of other objects in the environment, including reflections from the vehicle 102. In one example, wireless energy reflected by the further vehicle 108 can reflect (or bounce) off the vehicle 102, travel back to the further vehicle 108 (or other object), and reflect again from the further vehicle before being captured by the radar sensor 104. In other words, the detected wireless energy can traverse a path along the line 202 four times (twice in each direction) before being captured at the radar sensor. This "double bounce" can result in a reflected wave along the direction 202 but twice as far from the vehicle 102. In one example, the magnitude of the velocity is different from the magnitude of the object reflected wave velocity (e.g., half the speed), which can result in identification of the reflected reflected wave.

Furthermore, while FIG. 2 illustrates the intervening, reflecting objects as stationary objects, in other implementations, the techniques described herein can also determine whether a return is reflected from a moving object. For example, the velocity of the intervening object can be known (e.g., from other radar returns) and that velocity can be used to vary the radial component of the projected velocity. Other modifications are also contemplated.

Figure 3 shows another schematic diagram of the environment 100 and is used to illustrate an alternative method for determining whether a reflected wave is (or is likely to be) a reflection of another reflected wave. More specifically, Figure 3 may illustrate a technique that uses geometry to determine a reflection point, which may then be compared to environmental information, for example, to determine whether the reflection point corresponds to an object in the environment.

3 illustrates an object return 122 (from the further vehicle 108) and a first reflected return 138. For clarity, the second reflected return 140 is not illustrated, but the techniques described with reference to the return pair including the first reflected return 138 and the object return 122 can be applied to any pair of radar returns, including, for example, the first reflected return 138 and the second reflected return 140, including the object return 122 and the second reflected return 138, and/or any other pair of returns. As described herein, although a priori knowledge of the environment 100 may enable the object return 122 to be associated with the further vehicle 108, implementations of the present disclosure may be similarly applied to any pair of radar returns, regardless of known object associations and/or regardless of the availability of a priori knowledge of the environment 100. Thus, while the returns may be labeled as "object" returns and "reflected" returns, one or both of the labels may not be known until after further processing. In at least some examples, such labels may be used to help guide an autonomous vehicle. For example, a "reflected" return may still be considered for navigation, although its importance may be downweighted or otherwise taken into account during planning.

As described herein, the radar sensor 104 may receive only range, position, and/or speed information about the reflected waves. Thus, for example, the radar sensor 104 may generate sensor data indicative of the position of the object reflected wave 122 and the position of the first reflected reflected wave 138. For example, in FIG. 3, the line 302 in FIG. 3 is a line between the position of the object reflected wave 122 and the position of the radar sensor 104, and the line 304 is a line between the position of the first reflected reflected wave 138 and the position of the radar sensor 104. Based on the reflected waves 122, 138, i.e., the positions of the reflected waves 122, 138, the lengths of the lines 302, 304 and the angle 314 between the lines 302 and 304 are known (or can be easily determined). Techniques described herein can use the lines 302, 304, and the angle 314 to determine a potential, or theoretical, reflection point 308 along the line 304 from which wireless energy initially reflected by an object (e.g., a further vehicle 108) associated with the object return 122 will be reflected prior to being received at the radar sensor 104. Once the location of the theoretical reflection point 308 is determined, techniques described herein can determine whether an object is present at the theoretical reflection point 308, thereby confirming (or at least suggesting) that the first reflected return 138 is a reflection of an object return.

More specifically, the distance between the first reflected wave 138 and the object reflected wave 122, e.g., the distance of the line 310 shown in FIG. 3, can be determined using the distances of the lines 302, 304, and the angle 306. For example, the law of cosines can be used to determine the length of the line 310, although this is not required. A theoretical reflection point 308 may then be determined as the intersection of the line 304 and a line 312 that bisects the line 310 extending between the reflected wave 122 and the reflected wave 138 and is perpendicular to the line 310. For example, simple geometry can be used to determine the angle 314 between the line 304 extending between the sensor 304 and the first reflected wave 138, and the line 310 extending between the first reflected wave 138 and the object reflected wave 122. The length of the line segment 316 between the first reflected wave 138 and the reflection angle 316, which is bisected by the line 312, can then be easily determined to yield the position of the reflected wave 308 along the line 304. Furthermore, if the first reflected wave 138 is a true reflection, the distance of the line segment 316 is equal to the distance from the object wave 122 to the reflection point 308. This further information can be used (in addition or alternatively) to calculate the position of the reflection point 308.

In the implementation just described, the potential reflection point 308 can be determined using geometry alone. For example, speed information for the reflected wave is not required. However, potential reflection points can be determined for any pair of reflected waves at different distances. Thus, the techniques described herein can also determine whether an object is present at the potential reflection point to determine whether one of the reflected waves, e.g., the more distant reflected wave, is a reflection. For example, sensor data including, but not limited to, LiDAR data, image data, further radar data, etc. can be used to determine whether an object is present near the location of the theoretical reflection point 308. In the illustrated example, the structure 114(1) may be identified using image data, LiDAR data, and/or other sensor data captured by the vehicle 102. In other embodiments, map data may confirm the presence of an object at the reflection point 308. In one example, speed information can be used to determine the potential reflection based at least in part on projecting the speed from the reflected wave onto a unit vector associated with the other reflected wave. Such projected velocities can then be directly compared to the velocities of the other reflected waves.

2 and 3 (as well as FIGS. 5 and 6 below) are described with reference to the components illustrated in FIG. 1 in the environment 100 of FIG. 1, by way of example. However, the examples illustrated and described with reference to FIGS. 2, 3, 5, and 6 are not limited to being performed in the environment 100 or using the components of FIG. 1. For example, some or all of the examples described with reference to FIGS. 2, 3, 5, and 6 can be performed by one or more components of FIG. 4 or by one or more other systems or components described herein.

FIG. 4 illustrates a block diagram of an example system 400 for implementing the techniques described herein. In at least one example, the system 400 can include a vehicle 402, which may be the same as or different from the vehicle 102 shown in FIG. 1.

The vehicle 402 may include a vehicle computing device 404, one or more sensor systems 406, one or more emitters 408, one or more communication connections 410, one or more drive modules 412, and at least one direct connection 414.

The vehicle computing device 404 may include one or more processors 416 and a memory 418 communicatively coupled to the one or more processors 416. In the illustrated example, the vehicle 402 is an autonomous vehicle. However, the vehicle 402 may be any other type of vehicle having at least one sensor (e.g., a camera-enabled smartphone), or any other system. In the illustrated example, the memory 418 of the vehicle computing device 404 stores a localization component 420, a perception component 422, a prediction component 424, a planning component 426, a reflex recognition component 428, one or more system controllers 430, one or more maps 432, and a tracker component 434. Although shown in FIG. 4 as residing in memory 418 for illustrative purposes, it is contemplated that localization component 420, perception component 422, prediction component 424, planning component 426, reflex recognition component 428, system controller 430, map 432, and/or tracker component 434 may also or alternatively be accessible to vehicle 402 (e.g., stored on a memory remote from vehicle 402 or otherwise accessible by such memory). In one example, vehicle computing device 404 may correspond to or be an example of vehicle computing device 172 of FIG. 1.

In at least one example, the localization component 420 can include functionality for receiving data from the sensor system 406 to determine the position and/or orientation (e.g., one or more of x, y, z position, roll, pitch, or yaw) of the vehicle 402. For example, the localization component 420 can include and/or request/receive a map of the environment and can continually determine the position and/or orientation of the autonomous vehicle within the map. In one example, the localization component 420 can utilize SLAM (simultaneous localization and mapping), CLAMS (simultaneous calibration, localization, and mapping), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc. to receive image data, LiDAR data, radar data, IMU data, GPS data, wheel encoder data, and the like to accurately determine the position of the autonomous vehicle. In one example, the localization component 420 can provide data to various components of the vehicle 402 to determine an initial position of the autonomous vehicle 402 to generate a trajectory.

In one example, the perception component 422 may include functionality to perform object detection, segmentation, and/or classification. In one example, the perception component 422 may provide processed sensor data indicative of the presence of an object proximate to the vehicle 402 and/or the classification of the object as an object type (e.g., automobile, pedestrian, cyclist, animal, structure, tree, road surface, curb, sidewalk, unknown, etc.). In further and/or alternative examples, the perception component 422 may provide processed sensor data indicative of one or more characteristics associated with the detected object (e.g., tracked object) and/or the environment in which the object is located. In one example, the characteristics associated with the object may include, but are not limited to, x-position (global and/or local position), y-position (global and/or local position), z-position (global and/or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), object velocity, object acceleration, object extent (size), etc. Characteristics associated with the environment can include, but are not limited to, the presence of other objects in the environment, the state of other objects in the environment, the time of day, the day of the week, the season, weather conditions, darkness/light indicators, etc. In one example, the perception component 422 can use radar data to determine the object, and may receive information about reflected echoes to include/exclude sensor data, for example, as described herein.

In one example, the prediction component 424 may include functionality for generating predicted trajectories of objects in the environment. For example, the prediction component 424 may generate one or more predicted trajectories for vehicles, pedestrians, animals, and the like within a threshold distance from the vehicle 402. In one example, the prediction component 424 may measure a trace of an object and generate a trajectory for the object. In one example, the prediction component 424 may cooperate with a tracker 434 to track an object as it progresses through the environment. In one example, information from the prediction component 424 may be used in determining whether a radar return is from a known object.

In general, the planning component 426 can determine a path for the vehicle 402 to follow to move through an environment. For example, the planning component 426 can determine various routes and trajectories, as well as various levels of detail. For example, the planning component 426 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route can be a sequence of waypoints for traveling between the two locations. As non-limiting examples, the waypoints include streets, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 426 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 426 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In one example, the instructions can be a trajectory, or a portion of a trajectory. Further, in some implementations, multiple trajectories can be generated substantially simultaneously (e.g., within engineering tolerances) via receding horizon techniques, with one of the multiple trajectories being selected for the vehicle 402 to navigate. In some examples, the planning component 426 can generate one or more trajectories for the vehicle 402 based at least in part on sensor data, e.g., radar returns. For example, the planning component 426 can filter out returns that are determined to be reflected returns.

In general, the reflection recognition component 428 may include functionality to identify whether sensor data, e.g., radar returns, correspond to an actual object or are a return of an object from some intervening object. In one example, the reflection recognition component 428 may correspond to the reflection recognition component 174 of FIG. 1. As described herein, the reflection recognition component 428 may receive radar data, LiDAR data, image data, map data, and the like to determine whether a sensed object is an actual object or merely a return of an actual object. In one example, the reflection recognition component 428 may provide sensor information determined not to correspond to a return to the planning component 426 to determine when to control the vehicle 402 through the environment. In one example, the reflection recognition component 428 may filter out sensor data determined to correspond to a return to the planning component 426, e.g., filter out sensor data, e.g., radar data, that the planning component 426 has determined to be associated with a return from an intervening object to determine control. The reflection recognition component 428 can also provide sensor information to the tracker 434, for example, so that the tracker tracks dynamic objects in the environment.

The system controller 430 can be configured to control the steering system, propulsion system, braking system, safety system, emitter system, communication system, and other systems of the vehicle 402, for example, based on the control generated by and/or information from the planning component 426. The system controller 430 can communicate with and/or control corresponding systems of the drive module 414 and/or other components of the vehicle 402.

Map 432 can be used by vehicle 402 to navigate within the environment. For purposes of this description, a map can be any number of data structures modeled in two, three, or N dimensions that can provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountain ranges, roads, terrain, and the environment in general. In one example, the map can include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV/HSL color information), and the like), intensity information (e.g., LiDAR information, radar information, and the like), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and/or intensities), reflectivity information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, and the like). In one example, the map can include a three-dimensional mesh of the environment. In one example, the map can be stored in a tiled format, with each tile of the map representing a separate portion of the environment, and can be loaded into the working memory as needed. In one example, the map 432 can include at least one map (e.g., an image and/or a mesh). The vehicle 402 can be controlled at least in part based on the map 432. That is, the map 432 can be used in conjunction with the localization component 420, the perception component 422, the prediction component 424, the planning component 426, and/or the reflection recognition component 432 to determine the position of the vehicle 402, identify objects in the environment, and/or generate a route and/or trajectory for navigating within the environment. Additionally, as described herein, the map 432 can be used to verify the presence of objects, e.g., intervening objects from which wireless energy may be reflected before being received at a radar sensor.

In one example, map 432 may be stored on one or more remote computing devices (such as one or more computing devices 438) accessible via one or more networks 436. In one example, map 432 may include a variety of similar maps stored, for example, based on characteristics (e.g., type of entity, time of day, day of the week, or season of the year, etc.). Storing a variety of maps 432 in this manner may have similar memory requirements but increases the speed with which data in the maps may be accessed.

The tracker 434 may include functionality for tracking, e.g., following, the movement of an object. For example, the tracker 434 may receive sensor data from one or more of the sensor systems 406 that represent dynamic objects in the vehicle's environment. For example, an image sensor on the vehicle 402 may capture image sensor data, a LiDAR sensor may capture point cloud data, and a radar sensor may acquire echoes that indicate the object's position, orientation, etc. in the environment at various times. Based on this data, the tracker 434 may determine track information for the object. For example, the track information may provide historical position, velocity, acceleration, and the like for the associated object. Additionally, in one example, the tracker 434 may track an object within an occluded region of the environment. U.S. Patent Application No. 16/147,177, filed Sep. 28, 2018, for "Radar Spatial Estimation," the entire disclosure of which is incorporated herein by reference, describes techniques for tracking objects in occluded regions. As described herein, information from tracker 434 may be used to verify that radar returns correspond to tracked objects. In at least some examples, tracker 434 may be associated with perception component 422 such that perception component 422 performs data association to determine whether a newly identified object should be associated with a previously identified object.

As will be appreciated, the components described herein (e.g., localization component 420, perception component 422, prediction component 424, planning component 426, reflex recognition component 428, system controller 430, map 432, and tracker component 434) are described as separated for illustrative purposes. However, the operations performed by the various components can be combined or performed in any other component. As an example, reflex recognition functions may be performed by the perception component 422 and/or the planning system 426 (e.g., rather than by the reflex recognition component 428) to reduce the amount of data transferred by the system.

In some examples, aspects of some or all of the components described herein may include modules, algorithms, and/or machine learning algorithms. For example, in some examples, the components in memory 418 (and/or memory 442, described below) may be implemented as neural networks.

As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer of a neural network may comprise another neural network, or may comprise any number of layers (whether convolutional or not). As can be understood in the context of this disclosure, a neural network may utilize machine learning, which may refer to such a broad class of algorithms in which an output is generated based on learned parameters.

Although described in the context of neural networks, any type of machine learning may be used consistent with this disclosure. For example, machine learning algorithms include regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), example-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomizer 4 (ID3), chi-squared automated interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average-one dependent estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, backpropagation, etc.). , Hopfield Networks, Radial Basis Function Networks (RBFN)), deep learning algorithms (e.g., Deep Boltzmann Machines (DBM), Deep Belief Networks (DBN), Convolutional Neural Networks (CNN), Stacked Autoencoders), dimensionality reduction algorithms (e.g., Principal Component Analysis (PCA), Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Sammon Mapping, Multidimensional Scaling (MDS), Projection Pursuit, Linear Discriminant Analysis (LDA), Mixed Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analysis (FDA)), ensemble algorithms (e.g., Boosting, Bootstrap Aggregation (Bagging), AdaBoost), Stacked Generalization (Blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forests), SVM (Support Vector Machines), supervised learning, unsupervised learning, semi-supervised learning, and others, but are not limited to these.

Further examples of architectures include neural networks such as ResNet70, ResNet101, VGG, DenseNet, PointNet, and the like.

In at least one example, the sensor system 406 can include LiDAR sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, COMPASS, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc. The sensor system 406 can include various instances of each of these or other types of sensors. For example, the LiDAR sensors can include individual LiDAR sensors located at the corners, front, rear, sides, and/or top of the vehicle 402. As another example, the camera sensors can include various cameras located at various locations around the exterior and/or interior of the vehicle 402. As another example, the radar system can include various instances of the same or different radar sensors located at various locations around the vehicle 402. The sensor system 406 can provide input to the vehicle computing device 404. Additionally or alternatively, the sensor system 406 can send sensor data to the computing device 438 over one or more networks 436 at a particular frequency, such as after a predetermined period of time, in near real time, etc. In one example, the sensor system 406 can correspond to the sensor system 164 of FIG. 1, including the radar sensor 104 and/or additional sensors 166.

Emitters 408 can be configured to emit light and/or sound. Emitters 408 in this example can include internal audio emitters and internal visual emitters to communicate with occupants of vehicle 402. By way of example, and not by way of limitation, internal emitters can include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration feedback and/or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. Emitters 408 in this example can also include external emitters. By way of example, and not by way of limitation, external emitters in this example can include lights to signal the direction of movement or other indicators of the vehicle's actions (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) to acoustically communicate with pedestrians or other nearby vehicles, one or more of which include acoustic beam steering technology.

The communication connection 410 may enable communication between the vehicle 402 and one or more other local or remote computing devices. For example, the communication connection 410 may facilitate communication with other local computing devices on the vehicle 402 and/or the drive module 414. The communication connection 410 may also enable the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). The communication connection 410 may also enable the vehicle 402 to communicate with remote teleoperation computing devices or other remote services.

The communication connection 410 may include physical and/or logical interfaces for connecting the vehicle computing device 404 to another computing device or network, such as the network 436. For example, the communication connection 410 may enable Wi-Fi based communications, such as via frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communications protocol that enables each computing device to interface with other computing devices.

In at least one example, the vehicle 402 can include a drive module 412. In one example, the vehicle 402 can include a single drive module 412. In at least one example, the vehicle 402 can have multiple drive modules 412, each of which is located on opposite ends of the vehicle 402 (e.g., front and rear, etc.). In at least one example, the drive module 412 can include one or more sensor systems to detect conditions around the drive module 412 and/or the vehicle 402. By way of example, and not by way of limitation, the sensor systems associated with the drive module 412 can include one or more wheel encoders (e.g., rotational encoders) that sense the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) that measure the orientation and acceleration of the drive module, cameras or other imaging sensors, ultrasonic sensors that acoustically detect objects in the vicinity of the drive module, LiDAR sensors, radar sensors, etc. Some sensors of such wheel encoders may be specific to the drive module 412. In some cases, a sensor system on the drive module 412 may overlap or complement a corresponding system (e.g., sensor system 406) of the vehicle 402.

The drive module 412 may include many of the vehicle systems, including a high voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and/or pneumatic components, a stability control system to distribute braking forces to mitigate loss of friction and maintain control, an HVAC system, lighting (e.g., lighting such as headlights/taillights that illuminate the exterior surroundings of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC/DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive module 412 may include a drive module controller that may receive and preprocess data from the sensor systems and control the operation of the various vehicle systems. In one example, the drive module controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more modules that perform various functions of the drive module 412. Additionally, the drive modules 412 may include one or more communication connections that enable each drive module to communicate with one or more other local or remote computing devices.

In at least one example, the direct connection 414 can provide a physical interface that couples the drive module 412 to the body of the vehicle 402. For example, the directional connection 414 can enable the transfer of energy, fluids, air, data, etc. between the drive module 412 and the vehicle. In one example, the direct connection 414 can further releasably secure the drive module 412 to the body of the vehicle 402.

In at least one example, the localization component 420, the perception component 422, the prediction component 424, the planning component 426, the reflex recognition component 428, the system controller 430, the map 432, and/or the tracker 434 can process the sensor data as described above and send their respective outputs to one or more computing devices 438 via one or more networks 436. In at least one example, the localization component 420, the perception component 422, the prediction component 424, the planning component 426, the reflex recognition component 428, the system controller 430, the map 432, and/or the tracker 434 can send their respective outputs to one or more computing devices 438 at a particular frequency, such as after a predetermined period of time, in near real time, or the like.

In some examples, the vehicle 402 can send sensor data to the computing device 438, for example, via the network 436. In some examples, the vehicle 402 can send raw sensor data to the computing device 438. In other examples, the vehicle 402 can send processed sensor data and/or a representation of the sensor data (e.g., spatial grid data) to the computing device 438. In some examples, the vehicle 402 can send sensor data to the computing device 438 at a particular frequency, such as after a predetermined period of time, in near real-time, etc. In some instances, the vehicle 402 can send sensor data (raw or processed) to the computing device 438 as one or more log files.

The computing device 438 may include a processor 440 and a memory 442 that stores one or more maps 444 and/or reflex recognition components 446.

In one example, map 444 can be similar to map 432. Reflex recognition component 448 may perform substantially the same functions as those described with respect to reflex recognition component 428, in addition to or instead of performing functions in vehicle computing device 404.

The processor 416 of the vehicle 402 and the processor 440 of the computing device 438 can be any suitable processor capable of processing data and executing instructions to perform the operations described herein. By way of example, and not by way of limitation, the processors 416 and 440 can comprise one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts it to other electronic data that may be stored in registers and/or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices can also be considered processors so long as they are configured to implement the encoded instructions.

Memories 418 and 442 are examples of non-transitory computer-readable media. Memories 418 and 442 may store an operating system, as well as one or more software applications, instructions, programs, and/or data that implement the methods described herein and the functions assigned to the various systems. In various implementations, the memory may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile/flash type memory, or any other type of memory capable of storing information. The architecture, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which the components shown in the accompanying figures are merely examples relevant to the description herein.

Note that while FIG. 4 is illustrated as a distributed system, in alternative examples, components of the vehicle 402 can be associated with the computing device 438 and/or components of the computing device 438 can be associated with the vehicle 402. That is, the vehicle 402 can perform one or more of the functions associated with the computing device 438, and the computing device 438 can perform one or more of the functions associated with the vehicle 402. Furthermore, aspects of the reflex recognition components 428, 446, and/or the tracker 434 can also be executed on any of the devices described herein.

FIG. 5 is a flow diagram of an example process 500 for determining whether a radar return is a reflected return, according to an embodiment of the present disclosure. Although described in the context of radar data, the example process 500 may be used in the context of and/or in combination with LiDAR data, sonar data, time-of-flight image data, and/or other types of data.

At operation 502, process 500 may include receiving radar data about the environment. For example, the radar data may include radar returns including position information, speed information, and/or intensity information. In one example, the radar system may include one or more radar sensors and may be a sensor system of an autonomous vehicle, such as vehicle 102 described above. The radar data may include a variety of returns, including static returns corresponding to objects having a position and zero speed, and dynamic returns or radar tracks corresponding to moving objects having a position and non-zero speed.

At operation 504, the process 500 may include determining a first velocity at a first location along a first radial direction based on a first radar return in the radar data. The first radar return may include a velocity having a direction along a first direction between the object and the vehicle, e.g., along a radial direction extending from the vehicle (radar sensor on the vehicle) through the object. In one example, the first radar return may be an object return, e.g., corresponding to a known object in the environment. In one example, the first radar return may be determined to correspond to an object return based on tracking information generated prior to receiving the radar data. However, in other examples, the techniques described herein may work without a priori knowledge of the return at all. In other words, the first return (and further returns, including a second return described below) may not be associated with an object in the environment, and such association may be unnecessary in some examples. The position of the first radar return may be a distance along the first radar direction.

At operation 506, process 500 may include determining a second velocity at a second location along the second radial direction based on a second radar return in the radar data. For example, like the first radar return, the second radar return (and other radar returns) may not be associated with a known object in the environment. The second radar return may be a return from another object in the environment, e.g., a newly detectable object, and/or a return from some intervening object in the environment. In the latter case, the second radar return may identify a phantom "object" at a location along the second radial direction.

In operation 508, the process 500 can determine a projected velocity as a projection of the first velocity on the second radial direction. For example, operation 508 can project one of the reflected waves in the pair of reflected waves, e.g., the first reflected wave, onto a unit vector associated with the other reflected wave in the pair, e.g., the second reflected wave. As can be appreciated, the reflected reflected wave cannot be closer than the object (e.g., direct) reflected wave, so operation 508 can project the velocity associated with the closer of the first and second reflected waves onto a direction associated with the farther of the reflected waves. For example, with reference to the example of FIG. 2, the first reflected wave can be the object reflected wave 122 and the second reflected wave can be the first reflected reflected wave 138. The projected velocity determined by operation 508 may be the radial component of the object reflected wave velocity 160 reflected on a line 210 that bisects and is perpendicular to the line 208 connecting the object reflected wave 122 and the first reflected reflected wave 138. Thus, the projected velocity may be the velocity resulting from reflection of wireless energy first from the object and then from an intervening object located along a second direction.

At operation 510, the process 500 may determine whether the projected velocity corresponds to a second velocity. For example, at operation 510, a radial component of the projected velocity, e.g., a magnitude of the component along the second radial direction, may be compared to a magnitude of the second velocity (measured along the second radial direction by the radar sensor). For example, the comparison may determine whether the magnitude of the radial component is substantially equal to the magnitude of the second velocity. As used herein, the term substantially equal, and similar terms, may refer to two values being equal or within some threshold disparity of each other. For example, the disparity may be an absolute value (e.g., 0.1 meters per second, 0.5 meters per second, 2 meters per second, or the like), a percentage (e.g., 0.5%, 1%, 10%), or some other measurement. In some examples, the disparity may be based on the fidelity of the radar sensor, the distance of the object, the distance associated with the return, the velocity of the object, the velocity associated with the return, and/or other features and/or factors.

If, at operation 510, it is determined that the projected velocity corresponds to a second velocity, then, at operation 512, process 500 may identify the second radar return as (or as likely to be) a reflected radar return. For example, the vehicle computing device may determine that the return is (or is likely to be) a reflected return because the return closely corresponds to a (theoretical) reflection of the first return.

At operation 514, process 500 may optionally receive further sensor data, and at operation 516, process 500 may optionally confirm the presence of an intervening object based on the further sensor data. For example, operations 514, 516 may confirm that the second radar return is a reflected return by confirming the presence of an intervening object (having reflected radio energy) that causes the reflection. As described herein, the further sensor data may be any type of sensor data from one or more sensor modalities disposed on or otherwise associated with the vehicle. In one example, the presence of the intervening object may be confirmed from map data (e.g., when the intervening object is a facility, a topographical feature, or the like). In other implementations, the further sensor data received at operation 514 may be subsequently received data, including, but not limited to, subsequent radar data. As a non-limiting example, the subsequently received data may be used to track the second return over time. As described herein, the reflected echo may be transient, and attempts to track a second echo using prior and/or subsequent additional sensor data may be futile.

At operation 518, process 500 may control the vehicle with the radar clutter removed. For example, control of the vehicle may not rely on the radar clutter because it has been determined that the radar clutter is a reflected clutter and does not represent an actual object in the environment. As described herein, conventional planning systems have controlled the vehicle to react (e.g., by braking, steering, or the like) to phantom "objects" represented by the radar clutter. However, by identifying and removing the reflected clutter, the techniques described herein can provide improved control, e.g., respond only to actual objects in the environment.

In contrast, if in operation 512 it is determined that the projected velocity does not correspond to the second velocity, then in operation 520 process 500 may identify the radar return as a potential additional object in the environment. For example, when the magnitude of the second velocity does not correspond to the radial component of the projected velocity, the second radar return is likely not a reflected return. Instead, the second return may correspond to an actual object in the environment. For example, the second return may be from a newly detected object, such as a pedestrian emerging from a building in the example of FIG. 2.

At operation 522, process 500 may optionally confirm the presence of an additional object. For example, operation 522 may include receiving additional sensor information and determining the presence of an object at a location associated with the second radar return based at least in part on the additional sensor information. The additional sensor information may include one or more of LiDAR data, image data, additional radar data, time-of-flight data, or the like. In one example, operation 522 may be substantially identical to operation 516, but confirms the presence of an object at the second location instead at some intervening point.

At operation 524, the process 500 may include controlling the vehicle based at least in part on the second radar return. For example, the second radar return may be associated with an actual dynamic object, so that the techniques described herein may control the vehicle in relation to the object. In an example, a trajectory of the vehicle may be determined based at least in part on the second radar return. As a non-limiting example, a prediction system may determine a predicted trajectory of the further object, similar to the prediction component 324, and a planning system, such as the planning component 326, may generate a trajectory for the vehicle in relation to the predicted trajectory of the further object. In an implementation, operation 524 may also or alternatively include tracking the further object, such that, for example, subsequent returns from the further object may be treated as object returns. Such object returns may be used to determine reflected returns caused by the further object.

FIG. 6 is a flow diagram of another example process 600 for determining whether a radar return is a reflected return, according to an embodiment of the present disclosure. Process 600 may be used in place of or in addition to process 500 described above. Process 600 is not limited to the environment shown in FIG. 3, although aspects of process 600 may correspond to the techniques described above with reference to FIG. 3. Additionally, although described in the context of radar data, example process 600 may be used in the context of and/or in combination with LiDAR data, sonar data, time-of-flight image data, and/or other types of data.

At operation 602, the process 600 may include receiving radar data about the environment. For example, the radar data may include radar returns including position, speed, and/or intensity information. In one example, the radar system may include one or more radar sensors and may be a sensor system of an autonomous vehicle, such as the vehicles 102, 402 described above. The radar data may include a variety of returns, including static returns corresponding to objects having a position and zero speed, and dynamic returns or radar tracks corresponding to moving objects having a position and non-zero speed.

At operation 604, process 600 may include identifying a first location of a first return of the radar data. The first radar return may identify the depth or range of the return, the location, e.g., the angular location of the return relative to the sensor, and/or the velocity. In some examples, the first radar return may be an object return, e.g., corresponding to a known object in the environment. In some examples, the first radar return may be determined to correspond to an object return based on tracking information generated prior to receiving the radar data. However, in other examples, the techniques described herein may work without any a priori knowledge of the return. In other words, the first return (and further returns, including a second return described below) may not be associated with an object in the environment, and such association may be unnecessary in some examples. The location of the first radar return may be a location along a first radial direction, an x-y coordinate in a coordinate system, or some other location information.

At operation 606, process 600 may include identifying a second location of a second return of the radar data. The second radar return may identify the depth or range of the return, the location, e.g., the angular location of the return relative to the sensor, and/or the velocity. In other implementations, the location of the second radar return may be an x-y coordinate in a coordinate system, or some other location information. The second radar return may be a return from another object in the environment, e.g., a newly detectable object, and/or a return from some intervening object in the environment. In the latter case, the radar return may identify a phantom "object" at a location along a radial direction. Techniques described herein may be used to determine if the return is a reflected return.

In operation 608, the process 600 can determine a location of the reflection point based on the first location and the second location. For example, the techniques described herein can determine the reflection point as a point in space along the second radial direction where the wireless energy reflected from the object associated with the first reflected wave that was reflected by the sensor to generate the second reflected wave. In one example, operation 610 can determine the reflection point using geometry associated with the location of the first reflected wave and the second reflected wave. FIG. 3 shows that the reflection point can be determined as the intersection of a first line extending between the sensor and the second reflected wave and a second line that is perpendicular to and bisects the line extending between the first reflected wave and the second reflected wave. Additionally or alternatively, an assumption that the line segments from the reflection point to the two points are equal can be used to determine where along the line the reflection point is. As can be seen, for any given pair of points, the reflection point will necessarily be on a line between the sensor and the reflected wave that is radially farther away from the sensor, and thus in the example of FIG. 6, the second reflected wave is farther away than the first reflected wave.

At operation 610, process 600 may receive data about objects in the environment. For example, operation 610 may include receiving additional sensor data about the environment. Such additional sensor data may be any type of sensor data from one or more sensor modalities disposed on or otherwise associated with the vehicle. In an example, operation 610 may include receiving map data of the environment. In the examples described herein, the data about the objects may be from any source capable of providing information about the objects, whether static or dynamic, in the environment.

In operation 612, the process 600 determines whether an object is at the location of the reflection point. As explained above, the geometry of the reflected waves allows for the determination of a theoretical reflection point, i.e., a point where the radio energy initially reflected from the object associated with the first reflected wave subsequently reflects to give a reflected wave at the location of the second reflected wave. Such reflections are ghost or phantom reflections in contrast to (direct) reflections from objects in the environment. Thus, while such theoretical points can be determined for any pair of reflected waves, the data about the object received in 610 can be used to determine whether an object is actually present at the theoretical reflection point.

If, in operation 612, it is determined that an object is present at the reflection point, then, in operation 614, the process 600 may identify the second radar return as a reflected radar return. For example, because the location of the second return closely corresponds to the location of the (theoretical) reflection of the first return at the reflection point, and an object is present at the theoretical reflection point, the vehicle computing device may determine that the return is a reflected return. For example, the vehicle computing device may flag the second return as a potential reflected return and/or send information to other components of the vehicle.

At operation 616, the process 600 may control the vehicle to exclude or otherwise take into account the second radar return. For example, the control of the vehicle may not depend on the second radar return because it has been determined that the second radar return is a reflected return and does not represent an actual object in the environment. As described herein, conventional planning systems have controlled the vehicle to react (e.g., by braking, steering, or the like) to the phantom "object" represented by the second radar return. However, by identifying and excluding the reflected return, the techniques described herein can provide improved control, e.g., respond only to actual objects in the environment. In some implementations, the techniques described herein can also track the second return to confirm that it is a reflection point. As described above, the geometry of the environment creates conditions that allow for reflected returns, but the geometry is constantly changing. Therefore, while a reflected clutter may be present in one radar scan, the likelihood that it will be present in a subsequent (or preceding) scan is low, and therefore attempting to track a transient clutter may not be possible.

In contrast, if it is determined in operation 612 that no object is present at the reflection point, then in operation 518, process 600 may identify the second radar return as a further potential object in the environment. For example, when no object is present at the reflection point, the radar return is likely not a reflected return. Instead, the second return may correspond to an actual object in the environment. For example, the return may be a newly detected object, such as a pedestrian emerging from the building in the example of FIG. 3.

At operation 620, process 600 may optionally confirm the presence of an additional object. For example, operation 624 may include receiving additional sensor information and determining the presence of an object at a location associated with the second radar return based at least in part on the additional sensor information. The additional sensor information may include one or more of LiDAR data, image data, additional radar data, time-of-flight data, or the like. In one implementation, operation 620 may be substantially identical to operations 610, 612, but instead of determining the presence of an object at the reflection point, process 600 may determine the presence of an object at a second location, i.e., the second return.

At operation 622, the process 600 may include controlling the vehicle based at least in part on the second reflected wave. For example, the second reflected wave may be associated with an actual dynamic object, so that the techniques described herein may control the vehicle in relation to that object. In an example, the trajectory of the vehicle may be determined based at least in part on the second reflected wave. As a non-limiting example, a prediction system may determine a predicted trajectory of the further object, similar to the prediction component 424, and a planning system, such as the planning component 426, may generate a trajectory of the vehicle in relation to the predicted trajectory of the further object. In an implementation, operation 622 may also or alternatively include tracking the further object, such that, for example, subsequent reflected waves from the further object may be treated as object reflected waves. Such object reflected waves may be used to determine reflected reflected waves caused by the further object.

The operations of processes 500, 600 may be performed sequentially and/or in parallel. As a non-limiting example, the radar data received in operations 502, 504, 602 may include multiple radar returns. In some implementations, operations 506, 508, 510, 512, 608, 610, 612, and others may be performed, for example, in parallel, for each of the radar returns. Thus, all reflected returns from the radar data may be relatively quickly identified and removed from consideration as described herein. Furthermore, while only a single pair of returns is referenced in FIG. 5 and FIG. 6, multiple pairs of returns may be compared using the techniques described herein. As a non-limiting example, the same return may be compared to multiple other returns (which may or may not be known to be associated with objects in the environment). In one example, multiple radar returns for each known object may be received and those returns may be compared to other returns not associated with the object, e.g., using process 400 or process 500. Further, pairs of returns may be processed, e.g., by both process 400 and process 500, to obtain further confidence that the return is a return. In yet another implementation, the multiple radar returns may be filtered, e.g., such that only some subset of the multiple radar returns is processed. For example, radar returns that are relatively closer to the vehicle 102 than the known object may not be reflected returns and therefore may not be searched for by aspects of process 500. Furthermore, radar returns that have a speed below a minimum speed, e.g., non-zero, may be excluded from the search, e.g., because those returns may have very little effect on the vehicle even if they are reflected returns. Additionally, returns having a threshold angle, e.g., an angle (relative to the radar sensor 104/vehicle 102) of greater than or equal to 45 degrees, 60 degrees, 90 degrees, or the like, as well as other filtering techniques and criteria may be used.

5 and 6 illustrate an exemplary process according to an embodiment of the present disclosure. The processes illustrated in FIGS. 5 and 6 may, but need not, be performed as multiple sub-processes, for example, by various components of the vehicle 102, 402. The processes 500, 600 are illustrated as logical flow charts that represent a sequence of operations, each of which may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, cause a computer or autonomous vehicle to perform the described operations. In general, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform a particular function or implement a particular abstract data type. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to perform the process.

(Example clause)
An example autonomous vehicle includes a radar sensor on the autonomous vehicle, one or more processors, and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform operations comprising receiving radar data of an environment from the radar sensor, the radar data comprising a first radar return having a first velocity and a first range along a first radial direction from the radar sensor, and a second radar return having a second velocity and a second range from the radar sensor, projecting the first velocity onto a point corresponding to the second radar return as a projected velocity, determining that the second radar return corresponds to a reflected radar return that reflected off an intervening surface based at least in part on the second velocity and the projected velocity, and controlling the autonomous vehicle within the environment to exclude the second radar return.

B The autonomous vehicle of example A, wherein projecting the first velocity comprises determining a reflection line based at least in part on a first range on a first radial direction and a second range on a second radial direction, reflecting the first velocity from the first location to the second location about the reflection line, and determining the projected velocity as a component of the reflected velocity along the second radial direction.

C. The autonomous vehicle of example A or example B, wherein determining that the second radar return corresponds to the reflected radar return comprises comparing a first magnitude of the projected velocity to a second magnitude of the second velocity, and determining that the first magnitude is substantially equal to the second magnitude.

D. The autonomous vehicle of any one of Examples A to C, wherein the operation further comprises, prior to receiving the radar data, receiving a previous radar return associated with the object, and identifying the first return as an object return associated with the object based at least in part on the previous radar return.

E. The autonomous vehicle of any one of examples A to D, further comprising at least one additional sensor on the vehicle, wherein the operation further comprises receiving additional sensor data from the at least one additional sensor, and verifying the presence of an intervening object based at least in part on the additional sensor data.

F. The autonomous vehicle of any one of Examples A to E, wherein the operation further comprises selecting the first and second reflections from the plurality of candidate radar reflections based at least in part on one or more of a distance associated with the first and second reflections, a position associated with the first and second reflections, or a speed associated with the first and second reflections.

G An exemplary method includes capturing radar data of an environment, including a plurality of radar returns, with a radar sensor on the vehicle; determining a first velocity along a first radial direction extending from the vehicle based at least on a first radar return of the plurality of radar returns; determining a second velocity along a second radial direction extending from the vehicle based at least in part on a second radar return of the plurality of radar returns; determining a projected velocity of the first velocity along the second direction; and determining, based at least in part on a comparison of the projected velocity and the second velocity, that the second radar return corresponds to a reflected radar return reflected from the object and an intervening object between the object and the radar sensor.

H The method of example G, further comprising receiving sensor data from at least one of the radar sensor or the additional sensor, and identifying, based at least in part on the sensor data, the first reflected wave as being associated with an object in the environment.

I The method of Example G or Example H, further comprising tracking an object in the environment based at least in part on the sensor data and prior to capturing the first radar return and the second radar return, and determining that the second return is a reflected return is further based at least in part on the tracking.

J The method of any one of Examples G to I, wherein the projected velocity comprises a velocity projected along the second direction onto a location corresponding to the second reflected wave.

K The method of any one of Examples G through J, wherein determining the projected velocity comprises determining a reflection line based at least in part on a first range associated with the first reflected wave and a second range associated with the second reflected wave, reflecting the first velocity about the reflection line to determine the reflected velocity, and determining the projected velocity as a component of the reflected velocity along a second radial direction.

L The method of any one of Examples G to K, further comprising determining a reflection point as an intersection of the reflection line with a line extending between the radar sensor and the second location, the reflection point being a location associated with an intervening object.

M The method of any one of Examples G to L, further comprising receiving at least one of additional sensor data or map data, and identifying an intervening object at the reflection point based at least in part on the additional sensor data or map data.

N The method of any one of Examples G to M, in which the first reflected wave and the second reflected wave are selected based at least in part on at least one of a distance associated with the first reflected wave and the second reflected wave, a position associated with the first reflected wave and the second reflected wave, or a velocity associated with the first reflected wave and the second reflected wave.

O The method of any one of Examples G to N, in which the first reflection is associated with an object in the environment and the second reflection is determined based at least in part on at least one of a second radar reflection having a second distance greater than a first distance associated with the first radar reflection, or a second speed greater than or equal to a threshold speed.

P One or more exemplary non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including capturing, with a radar sensor, radar data of an environment including a first radar return having a first speed, a first direction, and a first range, and a second radar return having a second speed, a second direction, and a second range, determining a projected velocity of the first speed with respect to the second direction, and determining that the second radar return corresponds to a reflected radar return based at least in part on a comparison of the projected velocity and the second speed.

Q One or more non-transitory computer-readable media of Example P, wherein determining the projected velocity comprises determining a reflection line based at least in part on the first reflected wave and the second reflected wave, reflecting the first velocity from the first location to the second location about the reflection line to determine the reflected velocity, and determining the projected velocity as a component of the reflected velocity along the second direction.

R The method of Example P or Example Q further comprising determining the reflection point as a point along the second direction such that a line from the reflection point bisects a connecting line from the first point to the second point, receiving additional sensor data, determining the presence of a surface at the reflection point based at least in part on the additional sensor data, and verifying that the second radar return corresponds to a reflected radar return.

S. One or more non-transitory computer-readable media of any one of Examples P through R, wherein the operations further comprise receiving sensor data from at least one of the radar sensor or the additional sensor, and determining, based at least in part on the sensor data, that the first reflected echo is associated with an object in the environment.

T one or more non-transitory computer-readable media of any one of Examples P to S, wherein the operation further comprises identifying one or more candidate reflections from the plurality of reflections, the one or more candidate reflections including a second radar reflection, and the identifying is based at least in part on at least one of a range associated with each of the one or more candidate reflections or a speed associated with each of the one or more candidate reflections.

U An exemplary autonomous vehicle includes one or more sensors on the autonomous vehicle including at least a radar sensor, one or more processors, and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform operations including receiving radar data of an environment from the radar sensor, the radar data including a first radar return having an associated first position with a first range along a first radial direction from the radar sensor, and a second radar return having an associated second position with a second range along a second radial direction from the radar sensor, determining a reflection point along the second radial direction based at least in part on the first position and the second position, receiving further sensor data from the one or more sensors, identifying an object in the environment based at least in part on the further sensor data, determining that the second radar return is a reflected return based at least in part on the object located at the reflection point, and excluding the second radar return to control the autonomous vehicle in the environment.

V The autonomous vehicle of example U, wherein determining the reflection point comprises determining a reflection line based at least in part on the first position and the second position, and determining the reflection point as an intersection of the reflection line and a line extending along the second direction.

W The autonomous vehicle of Example U or Example V, wherein the operation further comprises receiving previous sensor data associated with the second object from the radar sensor and prior to receiving the radar data, and identifying the first return wave as an object return wave associated with the second object based at least in part on the previous sensor data.

X The autonomous vehicle of any one of Examples U to W, wherein the operation further comprises selecting a second reflected wave from the plurality of reflected waves based at least in part on the second range being greater than the first range.

Y The autonomous vehicle of any one of Examples U through X, wherein the operations further comprise receiving a further radar return associated with the environment after receiving the radar data from the radar sensor, and identifying the second radar return as a reflected return based at least in part on the further radar return.

Z An exemplary method comprising receiving radar data of an environment from a radar sensor on a vehicle, the radar data including a plurality of radar returns; determining a first location on a first radial direction extending from the radar sensor based at least on a first radar return of the plurality of radar returns; determining a second location on a second radial direction extending from the radar sensor based at least in part on a second radar return of the plurality of radar returns; determining a reflection point along the second radial direction and between the radar sensor and the second location; and determining that the second radar return is a reflected return based at least in part on further data about the environment.

The method of example Z further comprising controlling the vehicle within the environment by excluding the second radar return AA.

BB The method of Example Z or Example AA, wherein determining the reflection point comprises determining a reflection line based at least in part on the first location and the second location.

CC The method of any one of Examples Z to BB, wherein the further data comprises at least one of sensor data or map data, and the sensor data comprises one or more of LiDAR data, further radar data, or image data.

The method of any one of Examples Z to CC further comprising receiving from the DD radar sensor a further radar return associated with the environment, and identifying the second radar return as a reflected return based at least on the further radar return.

EE The method of any one of Examples Z through DD, further comprising receiving a previous radar return associated with a tracked object in the environment from the radar sensor and prior to receiving the radar data, and identifying the first return as an object return associated with the tracked object based at least in part on the previous radar return.

FF The method of any one of Examples Z to EE, in which the first and second reflected waves are selected based at least in part on at least one of a range associated with the first and second reflected waves, a first and second position associated with the first and second reflected waves, or a velocity.

GG The method of any one of Examples Z through FF, in which the first radar reflection is associated with an object in the environment and the second radar reflection is determined based at least in part on at least one of the second radar reflection having a second range that is greater than the first range associated with the first radar reflection, or the second velocity of the second radar reflection being equal to or greater than a threshold velocity.

HH The method of any one of Examples Z to GG, further comprising receiving additional sensor data from an additional sensor on the vehicle and identifying an object associated with the first reflected wave based at least in part on the additional sensor data.

II One or more exemplary non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including receiving radar data of an environment from a radar sensor on a vehicle, the radar data including a plurality of radar returns; determining a first location on a first radial direction extending from the radar sensor based at least in part on a first radar return of the plurality of radar returns; determining a second location on a second radial direction extending from the radar sensor based at least in part on a second radar return of the plurality of radar returns; determining a reflection point along the second radial direction and between the radar sensor and the second location; determining a presence of an object at a location corresponding to the reflection point based at least in part on further data about the environment; and determining that the second radar return is a reflected return based at least in part on the presence of an object at the location.

JJ One or more non-transitory computer-readable media of Example II, wherein determining the reflection point comprises determining a reflection line based at least in part on the first location and the second location.

KK One or more non-transitory computer-readable media of Example II or Example JJ, wherein the further data comprises at least one of sensor data or map data, and the sensor data comprises one or more of LiDAR data or image data.

LL One or more non-transitory computer-readable media of any one of Examples II through KK, wherein the operations further comprise receiving, from the radar sensor, a further radar return associated with the environment, and verifying that the second radar return is a reflected return based at least in part on the further radar return.

One or more non-transitory computer-readable media of any one of Examples II through LL, wherein the MM operation further comprises receiving, from the radar sensor and prior to receiving the radar data, a previous radar return associated with a tracked object in the environment, and identifying the first return as an object return associated with the tracked object based at least in part on the previous radar return.

NN One or more non-transitory computer readable media of any one of Examples II through MM, in which the first reflected wave and the second reflected wave are selected based at least in part on at least one of a range associated with the first reflected wave and the second reflected wave, a first position and a second position associated with the first reflected wave and the second reflected wave, or a velocity.

While the example clauses described above are described with respect to one particular implementation, it should be understood that in the context of this specification, the contents of the example clauses may also be implemented via methods, devices, systems, computer-readable media, and/or other implementations.

While the example clauses described above are described with respect to one particular implementation, it should be understood that in the context of this specification, the contents of the example clauses may also be implemented via methods, devices, systems, computer-readable media, and/or other implementations.

(Conclusion)
While one or more examples of the techniques described herein have been described, various modifications, additions, permutations, and equivalents of those examples are included within the scope of the techniques described herein.

In describing the examples, reference is made to the accompanying drawings, which form a part of the description, showing, by way of illustration, specific examples of the claimed subject matter. It is understood that other examples can be used and that modifications or variations, such as structural changes, can be made. Such examples, modifications, or variations do not necessarily depart from the intended scope of the claimed subject matter. While the steps described herein may be presented in a certain order, in some cases the order can be changed such that some inputs are provided at different times or in a different order without changing the functionality of the systems and methods described. The procedures disclosed can also be performed in a different order. Furthermore, the various calculations herein need not be performed in the order disclosed, and other examples using alternative orders of calculations can be readily implemented. In addition to being reordered, the calculations can also be decomposed into sub-calculations with the same results.

Claims (13)

1. An autonomous vehicle, comprising:
a radar sensor of the autonomous vehicle;
one or more processors;
When executed by the one or more processors, causes the autonomous vehicle to:
receiving radar data of an environment from the radar sensor, the radar data comprising:
a first radar return including a first velocity along a first radial direction from the radar sensor and a first position at a first range along the first radial direction;
a second radar return having a second velocity and a second range along a second radial direction from the radar sensor;
determining, based at least in part on at least one of the first and second positions or the first and second velocities, that the second radar return corresponds to a reflected radar return reflected off an intermediate surface;
and a memory storing processor-executable instructions for performing operations to control the autonomous vehicle in the environment excluding the second radar return,
Determining that the second radar return corresponds to the reflected radar return includes:
projecting the first velocity to a point corresponding to the second radar return as a projected velocity;
determining that a first magnitude of the projected velocity is equal to a second magnitude of the second velocity;
Including, autonomous vehicles. Projecting the first velocity includes:
determining a reflected line based at least in part on the first range in the first radial direction and the second range in the second radial direction;
reflecting the first velocity about the reflection line from the first location to the second location;
and determining the projected velocity as a component of the reflected velocity along the second radial direction. Determining that the second radar return corresponds to the reflected radar return comprises:
determining a reflection point along the second radial direction based at least in part on the first location and the second location;
receiving further sensor data from one or more further sensors;
identifying objects in the environment based at least in part on the further sensor data; and
and determining that the object is located at the reflection point . Determining the reflection point comprises:
determining a reflected line based at least in part on the first location and the second location;
and determining the reflection point as an intersection of the reflection line and a line extending along the second radial direction . The operation includes:
receiving a previous radar return associated with an object prior to receiving the radar data;
5. The autonomous vehicle of claim 1 , further comprising: identifying the first radar return as an object return associated with the object based at least in part on the previous radar return. and further comprising at least one additional sensor in the autonomous vehicle, the operation comprising:
receiving further sensor data from the at least one further sensor;
6. The autonomous vehicle of claim 1, further comprising : verifying a presence of the intermediate surface based at least in part on the further sensor data. Capturing radar data of an environment with a radar sensor of the vehicle, the radar data including a plurality of radar returns;
determining at least one of a first velocity along a first radial direction extending from the vehicle or a first position at a first range along the first radial direction based at least on a first radar return of the plurality of radar returns;
determining at least one of a second velocity along a second radial direction extending from the vehicle or a second position at a second range along the second radial direction based at least in part on a second radar return of the plurality of radar returns;
determining, based at least in part on at least one of the first and second positions or the first and second velocities, that the second radar return corresponds to a radar return reflected from an object and an intermediate object between the object and the radar sensor ;
projecting the first velocity to a point corresponding to the second radar return as a projected velocity;
determining that a first magnitude of the projected velocity is equal to a second magnitude of the second velocity;
A method for providing the above. The projecting of the first velocity includes:
determining a reflected line based at least in part on the first range in the first radial direction and the second range in the second radial direction;
reflecting the first velocity about the reflection line from the first location to the second location;
and determining the projected velocity as a component of the reflected velocity along the second radial direction. 9. The method of claim 8, further comprising determining a reflection point as an intersection of the reflection line with a line extending between the radar sensor and the second location, the reflection point being a location associated with the intermediate object. determining a reflection point along the second radial direction based at least in part on the first location and the second location;
receiving further sensor data from one or more further sensors;
and identifying a further object in the environment based at least in part on the further sensor data.
the determination that the second radar return is a reflected return is based, at least in part, on the further object being located at the reflection point.
10. The method according to any one of claims 7 to 9 . The step of determining the reflection points comprises:
determining a reflection line based at least in part on the first location and the second location;
determining the reflection point as an intersection of the reflection line and a line extending along the second radial direction;
and determining the reflection point as an intersection of the reflection line with a line extending along the second radial direction . 12. The method of claim 7, further comprising selecting the first radar return and the second radar return based at least in part on at least one of the first range and the second range, the first position and the second position, or the first velocity and the second velocity . One or more non-transitory computer readable media storing instructions that, when executed, cause one or more processors to perform the method of any one of claims 7 to 12 .

Images