Classifications

• • G—PHYSICS
  • G08—SIGNALLING
  • G08G—TRAFFIC CONTROL SYSTEMS
  • G08G1/00—Traffic control systems for road vehicles
  • G08G1/14—Traffic control systems for road vehicles indicating individual free spaces in parking areas
  • G08G1/141—Traffic control systems for road vehicles indicating individual free spaces in parking areas with means giving the indication of available parking spaces
  • G08G1/143—Traffic control systems for road vehicles indicating individual free spaces in parking areas with means giving the indication of available parking spaces inside the vehicles
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
  • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
  • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
  • G01C21/34—Route searching; Route guidance
  • G01C21/3407—Route searching; Route guidance specially adapted for specific applications
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
  • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
  • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
  • G01C21/34—Route searching; Route guidance
  • G01C21/36—Input/output arrangements for on-board computers
  • G01C21/3679—Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities
  • G01C21/3685—Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities the POI's being parking facilities

Definitions

• This description relates to identifying stopping places for an autonomous vehicle.
• a typical activity of an autonomous vehicle (AV) 10 is to safely and reliably drive through a road environment 12 to a goal position 14, while avoiding vehicles, pedestrians, cyclists and other obstacles 16 and obeying the rules of the road.
• AV autonomous vehicle
• the autonomous driving capability of an AV typically is supported by an array of technology 18, 20 including hardware, software, and stored and real time data that we together sometimes refer to as an AV system 22.
• Some or all of the technology is onboard the AV and some of the technology may be at a server, for example, in the cloud.
• Most AV systems include some or all of the following basic components:
• Sensors 24 able to measure or infer or both properties of the AV's state and condition, such as the vehicle's position, linear and angular velocity and acceleration, and heading (i.e., orientation of the leading end of the AV).
• sensors include but are not limited to, e.g., GPS, inertial measurement units that measure both vehicle linear accelerations and angular rates, individual wheel speed sensors and derived estimates of individual wheel slip ratios, individual wheel brake pressure or braking torque sensors, engine torque or individual wheel torque sensors, and steering wheel angle and angular rate sensors.
• Sensors 26 able to measure properties of the vehicle's surroundings.
• Such sensors include but are not limited to, e.g., LIDAR, RADAR, monocular or stereo video cameras in the visible light, infrared, or thermal spectra, ultrasonic sensors, time-of-flight (TOF) depth sensors, as well as temperature and rain sensors.
• Devices 28 able to communicate the measured or inferred or both properties of other vehicles' states and conditions, such as other vehicles' positions, linear and angular velocities, and accelerations, and headings.
• These devices include Vehicle-to- Vehicle (V2) and Vehicle-to- Infrastructure (V2I) communication devices, and devices for wireless communications over point-to-point or ad-hoc networks or both.
• the devices can operate across the electromagnetic spectrum (including radio and optical communications) or other media (e.g., acoustic
• Data sources 30 providing historical, real-time, or predictive information or combinations of them about the local environment, including traffic congestion updates and weather conditions. Such data may be stored on a memory storage unit 32 on the vehicle or transmitted to the vehicle by wireless communication from a remotely located database 34.
• Data sources 36 providing digital road map data drawn from GIS databases, potentially including high-precision maps of the roadway geometric properties, maps describing road network connectivity properties, maps describing roadway physical properties (such as the number of vehicular and cyclist travel lanes, lane width, lane traffic direction, lane marker type and location), and maps describing the spatial locations of road features such as crosswalks, traffic signs of various types (e.g., stop, yield), and traffic signals of various types (e.g., red- yellow-green indicators, flashing yellow or red indicators, right or left turn arrows).
• Such data may be stored on a memory storage unit on the AV or transmitted to the AV by wireless communication from a remotely located database.
• Data sources 38 providing historical information about driving properties (e.g., typical speed and acceleration profiles) of vehicles that have previously traveled along the local road section at a similar time of day. Such data may be stored on a memory storage unit on the AV or transmitted to the AV by wireless communication from a remotely located database. 7.
• a computer system 40 located on the AV that is capable of executing algorithms (e.g., processes 42) for the on-line (that is, real-time on board) generation of control actions based on both real-time sensor data and prior information, allowing an AV to execute its autonomous driving capability.
• a display device 44 that is connected to the computer system to provide information and alerts of various types to occupants of the AV.
• a wireless communication device 46 to transmit data from a remotely located database 34 to the AV and to transmit vehicle sensor data or data related to driver performance to a remotely located database 34.
• Functional devices and features 48 of the AV that are instrumented to receive and act on commands for driving (e.g., steering, acceleration, deceleration, gear selection) and for auxiliary functions (e.g., turn indicator activation) from the computer system.
• commands for driving e.g., steering, acceleration, deceleration, gear selection
• auxiliary functions e.g., turn indicator activation
• a vehicle is caused to drive autonomously through a road network toward a defined goal position.
• Current information is analyzed about potential stopping places in the vicinity of the goal position, to make a choice of a currently selected stopping place that is acceptable and feasible.
• the vehicle is caused to drive autonomously toward the currently selected stopping place.
• the analyzing of current information includes analyzing a combination of static information that is known prior to the vehicle beginning to drive autonomously through the road network and information that is obtained during the autonomous driving. The analyzing of current
• information about potential stopping places includes continuously analyzing information from sensors on the vehicle or information from one or more other sources or both.
• the analyzing of current information about potential stopping places includes applying a predefined strategy for choosing a currently selected stopping place.
• the strategy includes stopping at the first feasible stopping place in the vicinity of the goal position.
• the strategy includes stopping at the most desirable among the currently feasible stopping places in the vicinity of the goal position.
• the strategy includes, if the vehicle has not stopped at a currently selected stopping place within a specified period of time, stopping at the first available stopping place in the vicinity of the goal position.
• the strategy includes trading off between choosing a best possible stopping place and a cost incurred in continuing to analyze current information to make a choice.
• the activities are repeated using a relaxed threshold for acceptability, a relaxed threshold for feasibility, or both. If the vehicle has not stopped at a currently selected stopping place that is acceptable and feasible, the activities are repeated based on a redefined goal position. If the vehicle has not stopped at a currently selected stopping place, the activities are repeated based on an expanded vicinity of the goal position. If the vehicle has not stopped at a currently selected stopping place, the activities are repeated during an extended period of time beyond the specified amount of time. If the vehicle has not stopped at a currently selected stopping place, the vehicle is enabled to be controlled from a remote location. If the vehicle has not stopped at a currently selected stopping place, the stop is aborted.
• the analyzing of current information includes analyzing map data about the road network.
• the analyzing of current information includes analyzing information about features in the vicinity of the goal position as perceived by sensors. The sensors are on the vehicle.
• the analyzing of current information includes analyzing distances of respective stopping places from the goal position.
• the analyzing of current information includes analyzing information about respective positions of the vehicle at the potential stopping places.
• the analyzing of current information includes analyzing information about whether the vehicle can legally stop at the potential stopping places.
• the analyzing of current information includes analyzing relative desirability of potential stopping places.
• the analyzing relative desirability of potential stopping places includes applying a statistical model to predict an expected feasibility state of a stopping place based on historical statistics of level of demand for parking and current traffic volumes.
• the analyzing of current information includes identifying a set of potential stopping places that are within a proximity region in the vicinity of the goal position.
• the analyzing of current information includes removing potential stopping places from the set that are not feasible stopping places for the vehicle. If no potential stopping places remain after the removing, the proximity region is enlarged. There may be no stopping place that is currently acceptable.
• the additional potential stopping places lie within a larger vicinity of the goal position.
• the larger vicinity is determined based on information provided by a passenger of the vehicle.
• the larger vicinity is determined automatically based on pre-specified rules.
• the size of the larger vicinity is less than a predetermined upper limit.
• and autonomous vehicle includes steering, acceleration, and deceleration devices that respond to signals from an autonomous driving control system to drive the vehicle autonomously on a specified route through a road network defined by map data and toward a specified goal position. Sensors on the vehicle perceive characteristics of potential stopping places in the vicinity of the goal position.
• a communication feature sends the currently perceived characteristics to the autonomous driving control system and receives from the autonomous driving control system current information indicative of commands for the steering, acceleration, and deceleration devices to cause the vehicle to drive to and stop at a currently selected stopping place.
• the autonomous driving control system includes elements onboard the vehicle.
• the autonomous driving control system includes elements remote from the vehicle.
• the sensors on the vehicle include video capture, LIDAR, or RADAR devices.
• the current information received from the autonomous driving control system includes a continuously updated choice of a selected stopping place.
• a planning process is associated with an autonomous vehicle.
• the planning process has inputs including static map data and dynamic data from sensors on the autonomous vehicle and outputs including a route to be driven through a road network to reach a goal position, a continually updated choice of a currently selected stopping place in the vicinity of the goal position, and a trajectory to be executed through a road network to reach the currently selected stopping place.
• a communication element communicates information about the updated choice of a currently selected stopping place to a device of a passenger, receives from the device of the passenger information about the goal position, and delivers the information to the planning process as an input.
• Implementations may include one or a combination of two or more of the following features.
• the currently selected stopping place is updated based on non-feasibility of a currently selected stopping place.
• the planning process updates the selection of a stopping place based on the information from the device of the passenger about the stopping place.
• the communication element communicates to the device of the passenger that the planning process has selected a stopping place.
• the information received from the device of the passenger includes an indication that a stopping place farther from the goal position would be acceptable.
• the information received from the device of the passenger includes an indication of a maximum acceptable distance.
• the information received from the device of the passenger includes an indication of a new goal position to be considered by the planning process.
• the information received from the device of the passenger includes an indication that further time spent searching for an acceptable stopping place would be acceptable to the passenger.
• stored data is maintained indicative of potential stopping places that are currently feasible stopping places for a vehicle within a region.
• the potential stopping places are identified as part of static map data for the region.
• Current signals are received from sensors or one or more other sources current signals representing perceptions of actual conditions at one or more of the potential stopping places.
• the stored data is updated based on changes in the perceptions of actual conditions.
• the updated stored data is exposed to a process that selects a stopping place for the vehicle from among the currently feasible stopping places.
• Implementations may include one or a combination of two or more of the following features.
• the potential stopping places are initialized as all of the potential stopping places identified as part of the static map data for the region.
• the potential stopping places are discretized as a finite number of points within the region corresponding to potential stopping places.
• a potential stopping place is defined as a shape containing one of the points, the shape corresponding to a footprint of the vehicle.
• An orientation is attributed to the shape, the orientation corresponding to a direction of traffic flow.
• the potential stopping places are initialized as potential stopping places expected to be feasible based on prior signals from sensors representing perceptions of actual conditions at one or more of the potential stopping places.
• the sensors include sensors located on the vehicle.
• the sensors include sensors located other than on the vehicle.
• the current signals received from sensors are received through V2V or V2I communication.
• the one or more other sources include crowd-sourced data sources.
• the vehicle is part of a fleet of vehicles managed from a central server and the method includes the server distributing information received from sensors at one of the vehicles to other vehicles of the fleet.
• a vehicle is caused to drive autonomously through a road network toward a defined goal position, and, if no potential stopping place that is in the vicinity of the goal position and is acceptable and feasible can be identified, a human operator not onboard the vehicle is enabled to drive the vehicle to a stopping place.
• Implementations may include one or a combination of two or more of the following features, information is provided for presenting to the human operator a view associated with a state of the vehicle.
• the human operator is enabled to provide inputs to the steering, throttle, brake, or other actuators of the vehicle.
• the human operator is enabled to directly control a trajectory planning process for the vehicle by manually selecting a goal position for the vehicle or influencing the planned trajectory to the goal position.
• a passenger in the vehicle is informed or consent by a passenger in the vehicle is requested for the human operator to drive the vehicle to the stopping place.
• a vehicle is caused to drive autonomously through a road network toward a defined goal position at which a stopped activity is to occur.
• a set of acceptable stopping places is identified in the vicinity of the goal position.
• the identifying of the set includes: identifying a proximity region in the vicinity of the goal position, identifying a goal region within the proximity region, discretizing the goal region to identify acceptable stopping places given characteristics of the vehicle and of the stopped activity analyzing current information about potential stopping places in the goal region in the vicinity of the goal position to make a choice of a currently selected stopping place that is acceptable and feasible. Current information is analyzed about the potential stopping places, to make a choice of a currently selected stopping place that is acceptable and feasible the vehicle is caused to drive
• Implementations may include one or a combination of two or more of the following features.
• the analyzing of current information includes analyzing a combination of static information that is known prior to the vehicle beginning to drive autonomously through the road network and information that is obtained during the autonomous driving.
• the analyzing of current information about potential stopping places includes continuously analyzing information from sensors on the vehicle or information from one or more other sources or both. If the vehicle has not stopped at a currently selected stopping place that is acceptable and feasible within a specified amount of time and within specified thresholds for acceptability and feasibility, activities are repeated using a relaxed threshold for acceptability, a relaxed threshold for feasibility, or both.
• the analyzing of current information includes analyzing map data about the road network.
• the analyzing of current information includes analyzing information about features in the vicinity of the goal position as perceived by sensors.
• the analyzing of current information includes analyzing distance of respective stopping places from the goal position.
• the analyzing of current information includes analyzing information about respective positions of the vehicle at the potential stopping places.
• the analyzing of current information includes analyzing information about whether the vehicle can legally stop at the potential stopping places.
• the analyzing of current information includes analyzing relative desirability of potential stopping places.
• the analyzing of current information includes identifying a set of potential stopping places that are within a proximity region in the vicinity of the goal position.
• the analyzing of current information includes removing potential stopping places from the set that are not feasible stopping places for the vehicle.
• Figure 1 is a block diagram.
• Figures 2, 10, and 13 are schematic views of maps.
• Figure 3 is a screenshot.
• Figure 4 is a schematic diagram.
• Figures 5 through 9 are schematic views of maps.
• Figures 11, 12, 14, and 15 are flow diagrams. The use of the following terms in this description are intended broadly and meant to include, for example, what is recited after each of the terms.
• Annotated map data conventional road network map data used, for example, for autonomous driving systems that has been augmented with data associated with potential stopping places.
• Conventional road network map data can include some or all of the data mentioned in item 5. under the Background heading above.
• AV Autonomous vehicle
• Autonomous driving capability an ability to safely and reliably drive through a road environment to a goal position while avoiding vehicles, pedestrians, cyclists, and other obstacles and obeying the rules of the road.
• Coordinates geographic longitude and latitude.
• Acceptable satisfies one or more criteria for evaluating the appropriateness of a stopping place as a place where, for example, the pick up or drop-off or other stopped activity can be done safely or legally or conveniently or expeditiously, among other criteria.
• Stop come to a halt for a limited period of time with the intention of completing a stopped activity.
• Stopped activity an action that may occur at a stopping place (other than, for example, a stop made in the course of driving such as at a stop sign or a traffic light), such as picking up or dropping off a passenger or parcel, or other action.
• AV system a set of elements or components or processes located on an autonomous vehicle or at other locations, or a combination of them, and that together do the things that must be done in order to enable an autonomous vehicle to operate.
• Passenger a human being who is to be picked up or dropped off at a stopping place by an autonomous vehicle and (for convenience and simplicity in the prose) a human being who wants to have a parcel picked up or dropped off by an autonomous vehicle.
• Stopping place an area that the vehicle occupies (identified by a defined shape, typically a rectangle, at a defined location in the world) and a direction in which a vehicle is facing when stopped at the stopping place.
• Acceptable stopping place a stopping place that is acceptable because it is safe, legal, and convenient for the passenger.
• Potential stopping place a stopping place that is under consideration by an AV system.
• Feasible stopping place a stopping place that is possible for the AV to reach and stop at.
• selected stopping place a stopping place that is currently selected by an AV system.
• Actual stopping place a stopping place where an AV actually stops.
• Availability layer a layer of annotated road data that identifies potential stopping places included in the road data.
• Goal position a position that is an intended destination, is identified by coordinates on a road network, and has been specified by a passenger or an element of an AV system.
• Proximity region a region defined around or in the vicinity of the goal position and entirely within a fixed configurable distance of a point within the region (e.g., its center). The proximity region need not be round because the distance metric employed in calculation of the region may be defined in a non-Euclidean metric space.
• Goal region a region defined around or in the vicinity of the goal position such that any stopping place within the goal region is an acceptable stopping place for the AV.
• Sub-region a portion of a region.
• an AV system can determine and continuously update a currently selected stopping place 100 for an AV at or in the vicinity of a goal position 102 to which the AV is driving.
• the AV will stop and engage in or enable a stopped activity, including, for example, picking up or dropping off a passenger 104 or a package 106 or both.
• stop broadly to include, for example, stop, park, or stand.
• the AV's goal position is often specified directly as geographic coordinates (latitude and longitude) or indirectly by an address or some other form of identifier that can be translated into geographic coordinates.
• the goal position is specified by a passenger who wants to be picked up or dropped off by the AV roughly at that position, or by a user who wants to load a parcel onto the AV at that position for delivery to another location, or unload a parcel.
• a passenger who wants to be picked up or dropped off by the AV roughly at that position, or by a user who wants to load a parcel onto the AV at that position for delivery to another location, or unload a parcel.
• the word passenger we use the word passenger to refer not only to a human rider but also to a user who is to load or unload a parcel even though that user may not be a rider in the AV.
• a passenger 110 can specify a goal position by typing out an address 112 or by dropping a pin 114 on a map 116 using a touch-based user interface 118 of a mobile app running on a user device 120.
• the coordinates that are entered directly or that result from translation of such an address or other identifier may correspond to a goal position that does not lie on a drivable road 122 or other drivable feature of the road network, such as a goal position within a building, waterway, park, or other non-drivable feature.
• the coordinates may correspond to a location on the road that is not an acceptable stopping place (e.g., because it is not a safe or legal stopping place), such as a stopping place in the middle of the road or at the side of a busy highway.
• acceptable broadly to refer to satisfying one or more criteria for evaluating the appropriateness of a stopping place as a place where, for example, the pick up or drop-off or other stopped activity can be done safely or legally or conveniently or expeditiously, among other criteria.
• the coordinates of the goal position may correspond to an acceptable stopping place 130 on a road or other drivable feature where, for example, it is normally safe and legal for the AV to stop (e.g., a taxi stand or a reserved parking space). Yet for a variety of reasons such as the temporary presence of a parked vehicle or other obstacle or construction works 132, it may be temporarily impossible for the AV to stop there. In other words, such an acceptable stopping place is not a feasible stopping place.
• the coordinates of the goal position may correspond to an acceptable stopping place where a road or driveway exists, but the AV may not have map information sufficient to enable it to determine that the goal position represents such an acceptable stopping place. This circumstance may occur on private roads or grounds, such as private residences, shopping malls, corporate campuses, or other private sites. In other words, such an acceptable stopping place is not a feasible stopping place. 5.
• the coordinates of the goal position may correspond to an acceptable stopping place that would require passing through a feature that the AV cannot navigate.
• the acceptable stopping place may lie beyond the entrance of a parking structure that requires human interaction (e.g., to retrieve a parking ticket), beyond a guard post or checkpoint, or beyond a road region that the AV has otherwise identified as impassable (e.g., due to the presence of road construction). In such circumstances, the otherwise acceptable stopping place is not a feasible stopping place.
• the AV needs to find a stopping place on the road that is not only acceptable but is also a feasible stopping place.
• a stopping place on the road that is not only acceptable but is also a feasible stopping place.
• circumstances may sometimes require that the AV stop in the vicinity of a goal position that is not a passenger-specified goal position.
• the AV may be automatically re-directed to the closest medical center.
• a centrally coordinated service where a central optimization algorithm asks an idle AV to reposition itself (for example, in expectation of higher passenger or freight demand around the new location).
• the techniques that we describe here are applicable to such other goal positions as well.
• goal position broadly to include, for example, any position on the road network to which the AV is driving for the purpose of stopping.
• a goal position is a single specific location defined by geographic coordinates
• a goal region 140 of a configurable size and geometry can be defined around or in the vicinity of the goal position coordinates.
• a goal region is the region within which it would be acceptable for the AV to stop to engage in stopped activities, for a goal position located at the specified coordinates. If the AV were to stop at any stopping place within the goal region, it would be considered to have stopped acceptably near to the goal position, that is, at an acceptable stopping place.
• All potential stopping places that are acceptable (and therefore within the goal region) and feasible may not be equally desirable. For example, some of those feasible stopping places may be closer to the goal position than others, making them more desirable.
• the techniques and systems that we describe here enable the AV to select and then drive to a selected stopping place within the goal region that is acceptable, feasible, and desirable.
• the process shown in figure 4 illustrates an example of how a selected stopping place may be determined.
• a wide variety of other sequences and components of the process could be used.
• the geographic coordinates of the goal position 200 of an AV are specified directly or inferred by the AV system 202 in one or more of the following ways and in other ways or combinations of them: a.
• a passenger indicates a point on a map or an address 206 as the goal position of the AV. This could be done on a mobile app, a kiosk, a notebook, a tablet, or a workstation 207, to name a few examples.
• the map or address is communicated wirelessly to and received by a communication element 208 of the AV system 202.
• a communication element 208 of the AV system 202 is a communication element 208 of the AV system 202.
• the AV system itself sets a goal position based on one or more processes, for example, an emergency response process 210 that routes a vehicle to the nearest hospital or a vehicle rebalancing process 212 that routes the vehicle to a different goal position in a city, etc.
• a goal region 214 is defined by the AV system 202.
• the goal region contains potential stopping places (based on available annotated map data) where it is acceptable for the AV to stop to perform a stopped activity. As noted before, not all of these potential stopping places may be feasible or equally desirable. Also, the goal region may be updated for a variety of reasons, for example, as more information becomes available.
• the AV system first creates and stores a proximity region 214.
• the proximity region comprises both acceptable and unacceptable stopping places.
• the goal region is defined as the subset of the proximity region where all stopping places are acceptable. However, not all of these acceptable stopping places in the goal region are feasible or equally desirable. So finally, a stopping place is selected from the goal region that is both feasible and desirable, and the AV navigates to the currently selected stopping place.
• the proximity region is defined to be within, for example, a fixed, configurable distance 218 from the goal position.
• the center of this proximity region could be the goal position itself, or a point 222 on the drivable area of the map that is near (e.g., "nearest") to the goal position, or something similar.
• This fixed configurable distance could be calculated by the AV system in a number of ways, including but not restricted to one or a combination of:
• the goal region is then determined by excluding from the proximity region sub-regions where stopping is not allowed, and retaining the sub-regions where stopping is allowed, given the nature of the stopped activity, expected duration of stopping, nature of the AV, time of the day, etc. This can be done, for example, by determining the intersection of the areas in the annotated map data where stopping is allowed (given the nature of the stopped activity, expected duration of stopping, nature of the AV, time of the day, etc.) with the proximity region.
• Such geospatial intersections may be performed using available commercial or open-source software, for example ArcGIS, MongoDB.
• sub-regions include one or more of the following: a. Certain areas such as loading zones might be used for cargo (we sometimes use the word cargo interchangeably with parcel) pickups and drop-offs, but not for passengers. Similarly, certain driveways might be usable for passengers but not for cargo. Such restrictions could be part of the annotated map data. Therefore, depending on the purpose of the stopped activity (which could be provided by the passenger or the AV system or another source), certain sub-regions can be excluded from the proximity region. b. Some sub-regions might be excluded because the allowed stopping time in the stopping places in that sub-region is not sufficient to carry out the stopped activity. The estimated duration of the stopped activity can be influenced by a number of factors including one or more of the following.
• a pickup is generally slower than a drop-off as the passenger might need first to locate the AV and confirm his identity to the AV.
• Cargo stops might be slower than for passengers because of time needed to load or unload.
• Multiple passengers getting in or out of the AV might be slower than a single passenger.
• Certain sub-regions of the goal region might have restrictions (captured in the annotated map data) on how long a vehicle can stop there and therefore might not be suitable for some stopped activities.
• Independent configurable time parameters can be defined that relate to the expected time required for a pickup, drop-off, loading of cargo, unloading of cargo, and loading and unloading of one or multiple passengers or for other stopped activities.
• Certain sub-regions can then be excluded from the proximity region if the allowed stopping time in that sub-region is less than the expected stopping time of the stopping activity.
• Some vehicle types e.g., trucks or buses
• the vehicle type is known to the AV system.
• Certain sub-regions can then be removed from the proximity region if they correspond to zones where the AV is not allowed to stop.
• Some large vehicles 240 e.g., trucks or buses
• an AV footprint area (normally a rectangle) 242 is defined by the AV system corresponding to the footprint of the AV with a margin added in each dimension to account for overhangs and errors and provide a safety margin. Any sub-region of the proximity region that is unable to fully contain this AV footprint area is then excluded from the proximity region. e. Even though some part of the proximity region, say a parking space at a particular stopping place, is able to accommodate the AV configuration area, the driving approaches to reach that space may be too small for the AV footprint area. In other words, the stopping place is not a feasible stopping place because no feasible path exists for the AV to reach that stopping place. This situation is accommodated as part of the trajectory planning process described later.
• a proximity region in the vicinity of the road network can be annotated to exclude sub-regions that relate to the particular stopped activity, the AV, or the time of day.
• goal region 250 An example of a goal region 250 is shown in a checkerboard pattern in figure 6. The exclusion of sub-regions from the proximity region yields the goal region 250.
• the goal region or the proximity region can be defined as a mathematical set having potentially an infinite number of potential stopping places 252, in some implementations this can be discretized into a set having a finite number of potential stopping places using a number of well-known discretization strategies (e.g., random sampling, uniform sampling). These sampling strategies would yield a finite number of points within the proximity region.
• a stopping place could be constructed around each of these sampled points, for example, by drawing a rectangle corresponding to the AV's footprint around the sampled point. The sampled point could be the centroid of the rectangle. The size of the rectangle would have to be sufficient to accommodate the footprint of the AV including space for overhangs or a safety buffer on all sides. The orientation of the rectangle would be determined by the direction of the traffic flow at that point.
• the vehicle would have to stop in the direction of the traffic flow.
• the rectangle would be oriented accordingly and would be characterized both by its size, boundaries and its orientation. If the rectangle thus specified falls entirely within the proximity region, the stopping place that it represents would be considered to be a part of the proximity region.
• each potential stopping place in the goal region is associated with a direction in which it is legal for the leading end of the AV to point when stopped.
• This direction is inferred from the annotated map data which specifies legal driving directions for all parts of the roads and other drivable features, including traffic lanes.
• Figure 7 illustrates a few sample stopping places 251 (rectangles with arrows pointing in the direction that the leading end of the AV must face).
• the direction of a stopping place is inferred from the direction of travel 253 in the lanes which are shown with dashed arrows.
• a goal region might have an infinite number of valid stopping places and the stopping places shown in figure 7 are merely a sample.
• the system may expand the size of the proximity region and re-calculate the goal region. This expansion may be done automatically by the AV system, or by presenting the option to the passenger through the user interface, or by a combination of both.
• the goal region 430 may be empty (i.e. it contains no acceptable stopping places). The passenger could be prompted (through a
• the user interface could allow the passenger to specify the maximum walking distance that is acceptable to the passenger.
• the user interface could allow the passenger to specify the proximity region more directly, for instance by allowing the passenger to draw the region or by providing a mechanism for the passenger to relocate the boundaries of the region, for example using a boundary expansion tool 450 (which is an example of a touch-based drag and drop tool).
• the expanded proximity region 460 and the expanded goal region 470 may also be shown to the passenger.
• the expansion may be subject to some upper limit on the size of the proximity region. It is possible that despite expansion up to the upper limit, the goal region may still be empty. This could happen, for example, if the goal position is in the middle of a large field or a military installation. In these cases, the passenger will be informed through the user interface that no acceptable stopping places may be found in the vicinity of the goal position, and the passenger may be requested to select a different goal position.
• Figure 15 shows a flowchart of the activities involved in an example of the process used if the goal region is empty.
• stopping place is equally desirable for stopping.
• the desirability of a stopping place depends on, for example, quantifiable factors such as, but not limited to, the following:
• This distance 256 (figure 4) is computed by the AV system using one or a combination of distance metrics, such as the
• this covered distance 258 can be computed using one or a combination of distance metrics, such as the Euclidean distance, Manhattan distance, or another metric. Generally, a stopping place that has a shorter covered distance is more desirable. In some scenarios a covered walking path or data about the coverage of walking paths might not exist.
• Type of road Generally, it is desirable for the AV to stop on a road that has fewer lanes or has a lower speed limit or both.
• the lane configuration and speed limit information may be part of the annotated map data.
• Expected or actual traffic Generally, it is desirable for the AV to pick up or drop off a passenger or parcel on a road that is less trafficked as determined, for example, by AV system analysis of historical traffic data or data 270 collected by sensors mounted on the AV or both.
• Designated stopping area Generally, it is desirable for the AV to stop at a pre-defined stopping zone such as a taxi stand, a hotel pick-up and drop-off zone, a loading zone or other pre-defined stopping zone, compared to stopping in the travel lane of a road.
• a pre-defined stopping zone such as a taxi stand, a hotel pick-up and drop-off zone, a loading zone or other pre-defined stopping zone, compared to stopping in the travel lane of a road.
• the AV system creates a generalized cost or utility function 274 for stopping places that normalizes all factors (using calibrated weights or scaling factors) to create a single cost or utility value 276 associated with each stopping place. These costs or utilities, being numbers, can then be directly compared.
• the cost function can be continuous (e.g., the Euclidean distance from the stopping place to the goal position) or binary (e.g., a certain non-zero cost if a sightline between the stopping place and destination point does not exist, and a cost of zero if a sightline does exist).
• the cost can also depend on the type of stopped activity, e.g., whether it is a pickup or drop-off of a passenger, multiple passengers, or a parcel.
• the AV system converts each factor to a 0-1 range and applies fuzzy logic rules to compare stopping places.
• the AV system uses a prioritized comparison, in which factors that are defined to be more important are compared before factors that are less important. This kind of prioritized comparison can also impose minimum and maximum values on each factor to ensure that the selected stopping place meets the maximum and minimum requirements for each factor. 4.
• the AV system creates ranks such that a stopping place with a higher rank is more desirable than a stopping place with a lower rank.
• the goal region might be subdivided into sub-regions each with its own rank, and all stopping places within a sub-region could share the same rank, i.e., desirability.
• the AV system can determine and store the goal region 250 containing a set of stopping places all within the goal region, where stopping is acceptable given the nature of the stopped activity and the AV.
• a desirability index or utility or cost or rank i.e., a measure of desirability
• Figure 11 shows a flowchart of the activities involved in an example of the process of defining a goal region.
• the AV system also maintains what we term an availability layer 282 that can be thought of as an overlay on the map's potential stopping places. This availability layer of the annotated map data identifies to the AV system, for each potential stopping place, whether the potential stopping place is a feasible stopping place.
• the availability layer can be initialized by assuming that all potential stopping places are feasible stopping places.
• the availability layer is continually updated in real time as, for example, the AV perceives new information from its sensors. This information can come, for example, from a variety of sensors such as LIDAR, Radar, ultrasonic, video camera, IR, etc., which allow the AV to determine the shape and position of objects in its environment. For example, LIDAR data allows an AV to find other vehicles in its environment.
• the areas occupied by those vehicles can then be removed from the availability layer as these areas are not available to the AV for stopping.
• objects such as traffic cones or road signs (denoting construction work) can be detected using a combination of LIDAR and video cameras.
• an object can be detected but cannot be classified, for example, a fallen tree or construction debris.
• the AV system may note that the related area is not available and therefore remove it from the availability layer.
• an object may be detected but does not represent an obstacle to stopping (for example, a moving pedestrian).
• the corresponding stopping places can be retained in the availability layer.
• Figure 8 shows an example of what an AV 300 that is approaching its goal position 302 perceives and how the availability layer 282 is updated as a result of that perception.
• the checkerboard pattern 306 denotes the part of the goal region that includes only stopping places that are feasible.
• the availability layer is updated frequently, using both the AV's own perception data and external information.
• the frequent updating is important because the AV can stop only at a stopping place 308 that it is currently perceiving as being feasible.
• a stopping place that was assumed to be feasible (because of previous perception or information from another vehicle, etc.) might not actually be available (e.g., feasible) when the AV approaches that stopping place and sees for "itself .
• a stopping place that was previously thought to be unavailable (and therefore infeasible) might actually be available (and therefore feasible) when the AV perceives it directly.
• the availability layer in the annotated map data could also store, for each potential stopping place:
• the time of the most recent update of the stopping place status 310 is a measure of the current accuracy of the information.
• a stopping place that was reported being infeasible two minutes ago is more likely to remain unfeasible than a stopping place that was reported being infeasible twenty minutes ago.
• the likelihood that a stopping place that is marked as infeasible in the availability layer might become feasible by the time the AV reaches that stopping place (and vice versa) depends on several factors, including among others: freshness of the information (time elapsed since last update); historical statistics on the level of demand for parking relative to supply in that area at that time of the day; the reason for the infeasibility of that stopping place; and the current traffic volumes around that stopping place (derived from the AV's perception system, potentially supplemented by information from other AVs or sensors).
• the AV system could use a statistical model 314 that predicts the expected feasibility state of a stopping place (or a similar metric), given some or all of the data points, along with a confidence bound for that estimate. Such a metric could contribute to the calculation of the desirability value of a potential stopping place.
• a stopping place that is more likely to be available is more desirable than an equivalent stopping place that is less likely to be available.
• the availability layer can be updated using information received from other AVs (either directly or through a central cloud server). Therefore, as part of an interconnected fleet of AVs or manually driven vehicles that are equipped with V2V (vehicle-to-vehicle) communication capabilities, the AV might have foreknowledge of which stopping places are available without the AV actually having seen them.
• the availability layer can also be updated using information from sensors that are fixed (e.g., sensors inside parking garages or sensors monitoring city parking spaces), from crowd-sourced data (e.g., apps such as Waze on which people report construction work, etc.) and from a variety of other sources.
• the AV system executes a trajectory planning process 326 as part of its autonomous capability, which attempts to identify a trajectory from the AV's current position to a specified destination on the drivable area of a map.
• the result of the trajectory planning process is a continuously updated selected stopping place in the goal region, and a feasible trajectory to reach the currently selected stopping place, if one exists.
• the trajectory planning process routes the vehicle to the currently selected stopping place, and not the goal position specified by the user or the system, as that may not represent an acceptable and feasible stopping place.
• trajectory planning process updates its choice of selected stopping place. This may happen, for example, if the approaches to the currently selected stopping place are blocked or cannot accommodate the AV. This trajectory planning process continues as long as the AV has not stopped at the currently selected stopping place.
• Figure 12 shows a flowchart of the activities involved in an example of the process of trajectory planning.
• the trajectory planning process is executed simultaneously and asynchronously with the AV's perception process 328.
• the currently selected stopping places may become unfeasible, for example, because some other vehicle has now occupied one of the stopping places.
• the trajectory planning process may be forced to update its selected stopping place and the corresponding trajectory to that selected stopping place, multiple times.
• the stopping place in the goal region that is selected by the trajectory planning process depends on: (1) feasibility of the acceptable stopping places within the goal region, as determined in realtime by the availability layer via the perception process, (2) the relative desirability of the acceptable stopping places within the goal region, as computed by static or real-time data or both, and (3) the optimization objective of the algorithm that determines the trade-off between coming to a stop sooner versus spending more time to find a more desirable stopping place.
• Some examples of strategies are:
• the AV system may ask the user to choose a stopping place from among a set of stopping places.
• a touch-based user interface 500 can show the passenger a choice of, for example, three acceptable and feasible stopping places (A, B and C) 520 that are within the proximity region defined around the goal position 510. The user may select from one of the three stopping places by touching the appropriate stopping place. In some instances, the user may be able to select the desired stopping place using a voice command that references the stopping place name, for example, "Stop at A".
• the goal region may contain an infinite number of potential stopping places or a finite number which is still too large to present to the passenger. Therefore, the AV system may choose a limited (likely pre-specified) number of stopping places to present to the passenger. These may be stopping places that are relatively desirable and relatively more likely to be available (e.g., the status of the update was updated relatively recently) while being sufficiently different from each other (e.g., non-overlapping).
• the user input may be optional, in that the AV system may automatically choose a stopping place if the user does not make a choice within a specified amount of time.
• the selected stopping place may be communicated to the passenger, for example, on a map-based interface on the passenger's mobile device or on a display located in the vehicle. Further, the passenger may be given the choice of changing the system-selected stopping place to one of the passenger's own choosing through an interface, such as the one described for figure 13.
• the AV system can revisit stopping places that had been perceived as unfeasible in the hope that they might now be feasible.
• the trajectory planning process can communicate with a passenger to keep her informed of the AV's progress.
• a passenger can be informed that the AV has found an acceptable stopping place or informed when the AV has stopped at a stopping place, or both. If the AV is unable to stop at an acceptable, feasible stopping place within a specified amount of time, the AV system may adopt strategies to deal with that situation.
• Figure 14 shows a flowchart of the activities involved in an example of the process of expanding the analysis if the AV cannot come to a stop, including the following: 1. Expand the goal region around the goal position with the understanding that this might require the passenger to walk a greater distance. This may be done automatically by the AV system.
• the AV system may have stored a specified maximum distance from the goal position that is acceptable for the goal region.
• the initial proximity region (and goal regions) might include stopping places that are significantly closer to the goal position than this specified maximum distance to enable the AV system to find as close a stopping place as possible.
• the proximity region may incrementally be expanded to include more stopping places that are farther away from the goal position, but still within the specified maximum distance from the goal position .
• the passenger may provide information or choices that control the increasing of the size of the proximity region (and correspondingly the goal region) in response to requests posed to the passenger through the user interface (such as a smartphone or a tablet, located in the car or belonging to the user).
• One of the pieces of information that may be specified by the user is the maximum distance (or other distance measure) from the goal position that is acceptable to the user. If the system is unable to find a stopping place within this distance, the user may be prompted to increase the specified distance, subject to some specified limit.
• a combination of processes running on the AV system and input from the passenger can be used to control the expansion of the regions.
• the AV may switch to a tele-operation or remote operation mode in which the control of the operation of the AV switches, partially or fully, from the AV system to a human operator typically located at a central operations control center.
• Tele-operation systems typically stream live data from the AV to a remote location using wireless transmitters located on the AV.
• This data may include, but is not limited to, some combination of: (1) raw data from the sensors onboard the AV (for example, a live video stream from the on-board cameras), (2) processed data from the computers on board the AV (for example, data about detected and classified objects, or a rendering of the world around the AV that has been created by fusing data from multiple sensors), (3) data from other systems (for example, outputs from the trajectory planning process), (4) data about the actuators on the AV (such as the throttle, brake, steering), (5) data about the current position, speed, acceleration and orientation of the car from the localization system, and (6) data from the AV's health monitoring system (for example, sensor health, battery status, etc.).
• raw data from the sensors onboard the AV for example, a live video stream from the on-board cameras
• processed data from the computers on board the AV for example, data about detected and classified objects, or a rendering of the world around the AV that has been created by fusing data from multiple sensors
• data from other systems for example,
• This data is typically viewed by a tele-operator or other remote operator who is located at a central operations control center and who can then decide how to drive and otherwise control the AV.
• the tele-operator may have the ability to control the AV by directly providing inputs to the steering, throttle, brake and other actuators (for example, in the manner in which driver training simulators function).
• the tele-operator may have the ability to directly control the trajectory planning process by manually selecting a goal position for the vehicle, or influencing the trajectory to the goal position (by specifying the entire trajectory, or providing waypoints, or by some other method). The transition from autonomous to tele-operation mode should be handled with care.
• Such a transition would normally take place when the AV is not moving, although the AV may not be stopped at an acceptable stopping place. If there is a passenger in the AV, the passenger may be informed about or asked to approve the remote operation. This passenger communication may take place through a user interface in a smartphone belonging to the passenger, or a smartphone or tablet located in the AV, or some other such device.
• Providing the passenger the option of switching the AV from an autonomous mode to a partially or fully manual mode if there is a passenger in the AV who is legally authorized and willing to drive it seated in the driver' s seat and the AV has a manual mode) so that the passenger may locate an acceptable, feasible stopping place.
• the transition from autonomous to manual mode should be handled with care. A transition would normally take place when the AV is not moving, although the AV may not be stopped at an acceptable stopping place.
• the passenger may be informed about the option, and her approval sought, through a user interface in a smartphone belonging to the passenger, or a smartphone or tablet located in the AV, or some other such device.
• the passenger may be required to ensure that the stopping place chosen manually is one from which the AV can resume autonomous operation after the passenger has exited the AV.
• a tele-operator may take control of the AV after the passenger has exited the AV, and bring the AV to a position at which autonomous mode may be engaged.
• the AV system may inform the passenger that a stopping place cannot be found and that the pickup request has been canceled.

Landscapes

• Engineering & Computer Science (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Automation & Control Theory (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Traffic Control Systems (AREA)
• Control Of Driving Devices And Active Controlling Of Vehicle (AREA)

Applications Claiming Priority (6)

Publications (2)

Family

ID=62018722

Family Applications (1)

Country Status (3)

Cited By (2)

Families Citing this family (5)

Family Cites Families (15)

• 2017
  • 2017-10-04 EP EP17863085.1A patent/EP3529562A4/de active Pending
  • 2017-10-04 CN CN201780079090.XA patent/CN110140028A/zh active Pending
  • 2017-10-04 WO PCT/US2017/055127 patent/WO2018075242A1/en unknown

Also Published As

Similar Documents

Legal Events