US20180299274A1 - Real-time updates to maps for autonomous navigation - Google Patents
Real-time updates to maps for autonomous navigation Download PDFInfo
- Publication number
- US20180299274A1 US20180299274A1 US15/488,945 US201715488945A US2018299274A1 US 20180299274 A1 US20180299274 A1 US 20180299274A1 US 201715488945 A US201715488945 A US 201715488945A US 2018299274 A1 US2018299274 A1 US 2018299274A1
- Authority
- US
- United States
- Prior art keywords
- map
- vehicle
- data
- side unit
- road side
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3885—Transmission of map data to client devices; Reception of map data by client devices
- G01C21/3893—Transmission of map data from distributed sources, e.g. from roadside stations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
- G01C21/30—Map- or contour-matching
- G01C21/32—Structuring or formatting of map data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3635—Guidance using 3D or perspective road maps
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3885—Transmission of map data to client devices; Reception of map data by client devices
- G01C21/3896—Transmission of map data from central databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G06F17/30241—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
- H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
Definitions
- the present disclosure relates to vehicle navigation.
- Autonomous vehicles use various computing systems to aid in the transport of a person or people, items, or cargo from one location to another. Some autonomous vehicles may require initial input or continuous input from an operator. Other systems, including, e.g., autopilot systems, may be used only when the system has been engaged, which permits the operator to switch from a manual mode (where the operator exercises a high degree of control over the movement of the vehicle) to an autonomous mode (where the vehicle essentially drives itself) to modes that lie somewhere in between.
- a manual mode where the operator exercises a high degree of control over the movement of the vehicle
- autonomous mode where the vehicle essentially drives itself
- These autonomous vehicles may maneuver themselves between locations using highly detailed maps and/or models in conjunction with sensors for detecting the vehicle's surroundings. If the detailed maps are incorrect, it may be particularly difficult or impossible for the vehicle to safely and efficiently navigate without input from the operator.
- FIG. 1 depicts a scene and network components to maintain updated maps and related information for autonomous vehicles in accordance with an example embodiment.
- FIG. 2 is depicts features of an autonomous vehicle in accordance with an example embodiment.
- FIG. 3 depicts features of a road-side unit in accordance with an example embodiment.
- FIG. 4 is a flow chart depicting a series of operations that may be performed by map update logic operated by a road-side unit in accordance with an example embodiment.
- FIG. 5 is another flow chart depicting a series of operations that may be performed by map update logic in a road-side unit in accordance with an example embodiment.
- the techniques include determining that a vehicle has come within a predetermined range of a road side unit, establishing a communication link with the vehicle, receiving, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, selecting a map, stored by the road side unit, for the vehicle, sending a query to a neighbor road side unit seeking data to augment the map, in response to the query, receiving the data to augment the map from the neighbor road side unit, updating the map based on the data to augment the map to obtain an updated map, and sending at least aspects of the updated map to the vehicle.
- a device that includes an interface unit configured to enable network communications, a memory, and one or more processors coupled to the interface unit and the memory, and configured to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle.
- FIG. 1 depicts a scene 100 and network components that are used to maintain updated maps and related information for autonomous vehicles in accordance with an example embodiment.
- scene 100 includes a road intersection 110 .
- Several vehicles 105 a (V 1 ), 105 b (V 2 ), 105 c (V 3 ) are depicted in the roadway.
- a plurality of road-side units (RSUs) 300 a, 300 b, 300 c are deployed in the vicinity of the intersection 110 and along (e.g., adjacent, above) the roadways.
- Each of the RSUs is also in communication with a central or cloud server 150 and one or more external sensors or information providers 160 .
- Cloud server 150 may be configured to maintain overall control over the RSUs, and map generation and distribution. That is, cloud server 150 may operate as both an administrator of the RSUs and as a processor to generate maps, or at least provide data to the RSUs to generate maps locally.
- scene 100 may include a host of other elements such as lane markers (white, yellow lines, etc.), curbs, guardrails, highway dividers, buildings, trees, parked cars, pedestrians, road work sites, stop signs, traffic signals, among other possible features. Such features can all be part of the detailed maps that are generated and supplied to the vehicles.
- vehicles 105 a, 105 b, 105 c operate by, among other things, consulting a stored detailed (3D) map to compute where to drive/navigate. As such, it is important that the detailed map is as accurate and up to date as possible.
- elements of a scene such as scene 100 , can change relatively quickly compared to a baseline map that might be stored within a given vehicle. A given change might be transient such as a lane closure, or an accident. Other changes may be more permanent including a new traffic sign or a new building that has been constructed. A vehicle that is presented with a scene that does not match its currently stored map may become confused and either stop in a failsafe mode, or drive/navigate in a manner that is unsafe.
- the several RSUs 200 a, 200 b, 200 c are used, in accordance with the instant embodiments, to maintain updated versions of the maps and to provide to relevant vehicles the updated versions of the maps or a set of deviation data that enables a given vehicle to properly update its stored map to a most up to date version.
- the RSUs can communicate with one another, the vehicles, cloud server 150 and other external sensors/information provider 160 to better ascertain if/when updates to a given map may be warranted.
- the RSUs can also be used to control what data may be uploaded to cloud server 150 . That is, each RSU may be used to throttle the amount of data that flows toward cloud server 150 in an effort to avoid using too much bandwidth and overwhelming cloud server 150 with unnecessary, outdated or duplicative data.
- FIG. 2 depicts features of an autonomous vehicle 105 in accordance with an example embodiment.
- Autonomous vehicle 105 may be any type of vehicle including, but not limited to, a car, truck, drone, motorcycle, bicycle, bus, boat, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, piece of farm equipment, piece of construction equipment, tram, golf cart, train, and trolley.
- the vehicle may have one or more computers, such as a computer 210 containing a processor 220 , memory 230 and other components typically present in general purpose computers.
- Processor 220 may be any conventional processor, such as commercially available central processing units (CPUs) and/or graphics processing units (GPUs). Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor.
- FIG. 2 functionally illustrates the processor, memory, and other elements of computer 210 as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computer, or memory may actually comprise multiple processors, computers, or memories that may or may not be stored within the same physical housing.
- the processor 220 may be located remotely from the vehicle 105 and communicate with the vehicle 105 wirelessly. In other aspects, some of the processes described herein are executed on a processor disposed within the vehicle 105 and others by a remote processor, including taking the steps to execute a given maneuver.
- the memory 230 stores information accessible by the processor 220 , including autonomous vehicle logic (or instructions) 232 and data 234 that may be executed or otherwise used by the processor 220 .
- the memory 230 may be of any type capable of storing information accessible by the processor, including a computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, ROM, RAM, DVD or other optical disks, as well as other write-capable and read-only memories.
- Systems and methods may include different combinations of the foregoing, whereby different portions of the instructions and data are stored on different types of media.
- the autonomous vehicle logic 232 may be any set of instructions to be executed directly (such as machine code) or indirectly (such as scripts) by the processor.
- the instructions may be stored as computer code on the computer-readable medium.
- the terms “instructions” and “programs” may be used interchangeably herein.
- the data 134 may be retrieved, stored or modified by processor 220 in accordance with the autonomous vehicle logic 232 .
- the data may include environmental data that was obtained at a previous point in time and is expected to persist regardless of the vehicle's presence in the environment.
- data 234 may include a map 236 , e.g., a highly detailed 3D map that represents the shape and elevation of roadways, lane lines, intersections, crosswalks, speed limits, traffic signals, buildings, signs, real time traffic information, or other such features and information. These features may be persistent, and may be updated, as described in more detail below, when the vehicle 105 approaches the location of a feature in the map 236 , or when the vehicle communicates with one or more RSUs.
- the map 236 may also include explicit speed limit information associated with various roadway segments.
- the speed limit data may be entered manually or scanned from previously taken images of a speed limit sign using, for example, optical-character recognition.
- the map 236 may also include two-dimensional street-level imagery, such as highly detailed image data depicting the surroundings of a vehicle from the vehicle's point-of-view. And, as noted, the map 236 may also include 3D terrain maps incorporating one or more of the objects listed above.
- the data may include, for example, environmental data that is transient, such as the accumulated snow since the last snow removal truck came by, which may have been 5 mins ago. Note that some of this transient information is beneficial to capture and share at the local RSU, but may have less value to send and add to the map in the cloud.
- Data 234 may also include traffic pattern model information 238 , e.g., a highly detailed model indicating the distribution of typical or expected speeds, trajectories, locations, accelerations/decelerations (changes in speed), or other such characteristics of vehicles or other moving objects on the locations of the map 236 .
- This data may be generated, for example, by observing how vehicles, pedestrians, bicycles, etc. move at different locations in the map 236 . That is, data for the traffic pattern model information 238 might be generated based on long term observation and learning of traffic patterns.
- the traffic pattern model information 238 may indicate information pertinent to the entire road.
- the traffic pattern model information 238 may indicate a range of speeds for vehicles travelling along a road, specific to each particular lane or independent of the lanes.
- the traffic pattern model information 238 may indicate a range of trajectories for vehicles travelling in particular lanes.
- the vehicle may also be equipped with one or more sensors or detection components 244 for detecting objects external to the vehicle such as other vehicles, obstacles in the roadway, traffic signals, signs, trees, buildings, weather, etc.
- the detection components or sensors 244 may include lasers, sonar, radar, LIDAR, cameras or any other detection devices which record data and which may be processed by the computer 210 .
- Computer 210 may be capable of communicating with various components of the vehicle 105 .
- the computer 210 may be in communication with the vehicle's central processor 260 and may send and receive information from the various systems of vehicle 105 , for example the braking 280 , acceleration 282 , signaling 284 , navigation 286 , direction/speed detection device 252 and geographic position component 250 systems in order to control the movement, speed, etc. of vehicle 105 .
- computer 210 when engaged, computer 210 may control some or all of these functions of vehicle 105 and thus be fully or partially autonomous.
- vehicle 105 includes a network interface 239 that may be used to communicate, e.g., wirelessly, with a given RSU 300 . This wireless communication provides a rich set of data to the RSU 300 that can be leveraged to augment and update the map 236 , as will be discussed below.
- FIG. 3 depicts features of an RSU 300 in accordance with an example embodiment.
- a road-side unit 300 includes a computer 310 , which may be similar to computer 210 , and a processor 320 , which may be similar to processor 220 .
- a memory 330 may be configured to store map update logic 332 , i.e., computer logic or code encoded in a computer readable medium similar to that described for autonomous vehicle logic 232 , and data 334 for multiple vehicles 105 that may be within a predetermined range of the RSU 300 .
- Map update logic 332 i.e., computer logic or code encoded in a computer readable medium similar to that described for autonomous vehicle logic 232
- Data 334 may include sensor data 336 received wirelessly from respective vehicles via network interface 339 and may further include, for example, data about a type of the vehicle 337 .
- the vehicle may be a truck or a car, a bicycle, or a drone.
- the type of vehicle can be used to determine what sort of map or map update is to be provided to the vehicle, or what type of information or data might be missing from the vehicle's already stored map 236 such that the RSU 300 will query the vehicle, another RSU, or cloud server 150 to obtain additional/selected information that may be more useful for that particular type of vehicle.
- each RSU 300 is configured to store a base map, and is further configured to update that base map using map update logic 332 .
- cloud server 150 might deliver a base map to RSU 300 at a point in time. That same base map may also be supplied to any one of the vehicles 105 .
- a given vehicle assume vehicle 105 b for purposes of discussion, travels by RSU 300 b and supplies sensor data to RSU 300 b, enabling RSU 300 b to augment or update its base map.
- Updates to the base map might include parked cars, accidents, new road signs, the fact that ice has been detected on the road, or that there is a pot hole in a particular location, among other elements that might further elaborate on, or otherwise update a base map.
- RSU 300 b might also obtain updates from cloud server 150 and external sensors/information provider 160 (e.g., weather forecasts, upcoming construction notices, etc.) to still further update its base map.
- the vehicle 100 d communicates with RSU 300 b to verify that the vehicle's currently stored base map matches the base map which is stored in the RSU. This comparison could be performed, for example, by the vehicle sending, to RSU 300 b, a hash value of the base map stored in the vehicle 105 d. RSU 300 b can then confirm that the base maps match. RSU 300 b can then send, e.g., a map deviation data set to the vehicle 105 d based on the information RSU 300 b previously received from the earlier passing vehicle 105 b.
- RSU 300 b might send to the vehicle 105 d a collection of information delineating the differences between the base map stored in the vehicle 105 d and the updated map that has been generated by the RSU over recent time. It is noteworthy that because a deviation data set typically has less overall data than a full map, the transfer time needed for the vehicle to obtain the map deviation data set from RSU 300 b is less than the time needed to send a full map.
- RSU 300 b determines that vehicle 105 d has a different base map from the currently-stored based map in RSU 300 b, then RSU 300 b can send the updated map, or its currently-stored base map, as a new base map to vehicle 105 d.
- the map deviation data set transmitted to vehicle 105 d can be a function of (1) location of vehicle, (2) current direction (e.g., speed) and type of vehicle, and (3) intended or anticipated next steps in driving trajectory.
- a location of vehicle e.g., a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that is a vehicle that can be a function of (1) location of vehicle, (2) current direction (e.g., speed) and type of vehicle, and (3) intended or anticipated next steps in driving trajectory.
- an overlay of dynamic information such as traffic can be delivered.
- Such context is particularly relevant and scalable at the “edge,” i.e., at the RSU network level, as opposed to the cloud server 150 level.
- the vehicle may provide such information enabling the RSU 300 to provide the most appropriate information back to the vehicle.
- the RSU can communicate and query a neighbor RSU to obtain further map data to update its own map and then deliver the updated map or map deviation data set to the vehicle.
- FIG. 4 is a flow chart depicting a series of operations that may be performed by map update logic 332 operated by a road-side unit 300 in accordance with an example embodiment.
- the RSU determines that a vehicle has come within a predetermined range.
- the RSU establishes a communication link with the vehicle.
- the RSU receives, from the vehicle, data sufficient to identify a type of the vehicle (or where it is travelling, or any other number of characteristics about the vehicle) This step might also include obtaining a type/version of the map that that vehicle has stored to establish the base map.
- the RSU selects a map, stored by the road side unit, for the vehicle.
- Selecting a map may comprise, e.g., selecting aspects of the base map that are most relevant to the type of vehicle or where the vehicle is programmed to travel.
- the RSU sends a query to a neighbor RSU seeking data to augment the map (or type of map) that has been selected.
- the RSU receives the data to augment the map from the neighbor RSU.
- the RSU updates the selected map (or type of map) based on the data to augment the map to obtain an updated map.
- the RSU sends at least a portion of the updated map to the vehicle. That is, the RSU sends a map deviation data set to the vehicle or sends a complete updated map to the vehicle.
- RSUs communicate with one another it is possible to minimize the amount of information exchanged with a cloud server 150 (which might otherwise be responsible for collecting all data from all RSU and coordinating which information to provide to which vehicles), thus minimizing the amount of bandwidth consumed in the network.
- a cloud server 150 which might otherwise be responsible for collecting all data from all RSU and coordinating which information to provide to which vehicles
- the scope of the updated map or map deviation data set sent from the RSU 300 to the vehicle 105 might depend on vehicle type, e.g., a light bicycle versus an 18-wheeler truck.
- vehicle type e.g., a light bicycle versus an 18-wheeler truck.
- the RSU 300 can scale the map in size, information content and perspective, and create a version that is tailored to the target vehicle' needs and capabilities.
- the RSU 300 can instead process this data or telemetry and determine common aspects and characteristics and thus send a reduced amount of data to the cloud.
- the RSU can be configured to identify duplicate information and only transmit non-duplicative information to the cloud server 150 .
- the RSU can be configured to identify complementary information and send only the complementary information to the cloud server 150 .
- the RSU can use the sensor data to compose a map or model and only send the resulting map or model to the cloud server 150 .
- the RSU can be configured to use the sensor information to refine the map or model and only send the refinement to the cloud server 150 .
- the RSU can be configured to identify higher priority information (e.g., detecting an erratic car driving in the wrong direction), and give bandwidth priority to such information.
- higher priority information e.g., detecting an erratic car driving in the wrong direction
- the RSU can build and continuously refine a map or model, estimating gaps, and then requesting the types of data that would fill the gaps.
- a key element of the above is feedback between the RSU and either the vehicle and/or other RSUs to guide what information is sent.
- RSU may have a detailed map or model of the environment from the south side of a scene looking north
- another RSU may have a detailed map or model of the environment from the north side of a scene looking south.
- a given RSU can query a neighbor RSU having the data or information that is being sought to fill a gap in the map or model stored in the given RSU.
- the resulting updated map or model can then be shared with (1) vehicles, (2) still other neighboring RSUs, and (3) the cloud server 150 .
- the cloud server 150 is not relied upon for initial or even substantial processing. Processing is instead performed at the edge, i.e., at the RSU level. This not only mitigates bandwidth usage, it can also help avoid latency with having to rely on communication with the cloud server 150 .
- FIG. 5 is another flow chart depicting a series of operations that may be performed by map update logic 332 in a road-side unit 300 in accordance with an example embodiment. These operations are employed to throttle or control the amount or quality or type of data or information that is sent to cloud server 150 . That is, the described operations are performed by an RSU to determine what aspects or elements of information received from multiple vehicles should be sent to the cloud server 150 , as opposed to sending all received information or data to the cloud server 150 .
- the RSU receives data or telemetry from one or more vehicles.
- the RSU may receive information from a laser, camera, radar unit, etc.
- the RSU processes the data or telemetry to detect one or more objects and one or more characteristics (e.g., location, size, color, movement, etc.) of those one or more objects.
- the RSU compares the one or more characteristics of the detected object to its stored map. For example, a characteristic for the location or size of a stationary object may be compared to the locations of objects in the map.
- the RSU determines a deviation value for the compared one or more characteristics.
- Deviation values can be based on percent deviation, minimum or maximum deviation, etc.
- the RSU determines whether the deviation values are within threshold values for the relevant characteristics. These thresholds may be determined by an administrator and may be different for different kinds of objects. If the deviation value is within a threshold, more data is collected from vehicles for processing. If at 510 the deviation values are not within the relevant threshold deviation values, the RSU, i.e., map update logic 332 , updates its map.
- the RSU sends the updated map or a map deviation data set to the cloud server 150 . It should be understood by those skilled in the art, that the same updated map or map deviation data set can also be shared with neighbor RSUs and vehicles.
- Telemetry need not be received from all vehicles for map updating. Telemetry could be limited to certain “trusted vehicles” such as public transportation vehicles (e.g., busses, light trains), police cars, firetrucks, etc. Alternatively, the RSUs can be configured to statistically weigh telemetry received from vehicles, and that such weighting might not begin until telemetry from a threshold number of vehicles has been received. Higher weights may be given to selected vehicles whose past data has consistently proven to be accurate. Further, less transient changes can be supplied by, e.g., cloud server 150 , thereby alleviating the need to provide frequent updates to every vehicle. That is, the cloud server 150 can be responsible for distributing new base maps when significant non-transient changes are made to the scene or environment.
- trusted vehicles such as public transportation vehicles (e.g., busses, light trains), police cars, firetrucks, etc.
- the RSUs can be configured to statistically weigh telemetry received from vehicles, and that such weighting might not begin until telemetry from a threshold number of vehicles has been received
- autonomous vehicles use high definition 3D maps to augment their perception of the environment captured through their own sensors to establish a highly accurate position within the environment.
- the detailed maps suffer from enormous data sizes and the challenge of keeping them up to date, especially when relying on a cloud or central server.
- Embodiments described herein push much of the map processing to the edge of the network, i.e. to local RSUs, which are configured to update maps locally, communicate those maps or map deviation data sets to vehicles, and communicate selected data to the cloud.
- the RSUs can also communicate with another to enhance the map updating process.
- a method in one form, includes determining that a vehicle has come within a predetermined range of a road side unit, establishing a communication link with the vehicle, receiving, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, selecting a map, stored by the road side unit, for the vehicle, sending a query to a neighbor road side unit seeking data to augment the map, in response to the query, receiving the data to augment the map from the neighbor road side unit, updating the map based on the data to augment the map to obtain an updated map, and sending at least aspects of the updated map to the vehicle
- sending at least aspects of the updated map comprises sending differences between a map stored in the vehicle and the updated map.
- the method may further include receiving telemetry from a plurality of vehicles within the predetermined range, and updating the map based on the telemetry.
- the method may still further include determining a deviation value for an aspect of the telemetry compared to an aspect of the map, and not updating the map when the deviation value is within a predetermined threshold value.
- the method may further include determining a deviation value for an aspect of the telemetry compared to an aspect of the map, and further updating the map when the deviation value is beyond a predetermined threshold value.
- the method may still further include sending an indication of the aspect of the telemetry to a central server.
- updating the map may be based on data received from a central server. Updating the map may also be based on data from an external information source other than a central server.
- the method may also include sending a query to the vehicle, the query enabling the vehicle to request selected information as part of the updated map.
- a device in another form, includes: an interface unit configured to enable network communications, a memory, and one or more processors coupled to the interface unit and the memory, and configured to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle.
- the one or more processors are further configured to send at least aspects of the updated map by sending differences between a map stored in the vehicle and the updated map.
- the one or more processors are further configured to receive telemetry from a plurality of vehicles within the predetermined range, and the one or more processors are further configured to update the map based on the telemetry.
- the one or more processors may be further configured to determine a deviation value for an aspect of the telemetry compared to an aspect of the map, and not update the map when the deviation value is within a predetermined threshold value.
- the one or more processors may also be configured to determine a deviation value for an aspect of the telemetry compared to an aspect of the map, and further update the map when the deviation value is beyond a predetermined threshold value.
- the one or more processors may still be further configured to send an indication of the aspect of the telemetry to a central server.
- a non-transitory tangible computer readable storage media encoded with instructions that, when executed by at least one processor, is configured to cause the processor to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle.
- the instructions may further case the processor to send at least aspects of the updated map by sending differences between a map stored in the vehicle and the updated map, and/or to receive telemetry from a plurality of vehicles within the predetermined range.
Abstract
Description
- The present disclosure relates to vehicle navigation.
- Autonomous vehicles use various computing systems to aid in the transport of a person or people, items, or cargo from one location to another. Some autonomous vehicles may require initial input or continuous input from an operator. Other systems, including, e.g., autopilot systems, may be used only when the system has been engaged, which permits the operator to switch from a manual mode (where the operator exercises a high degree of control over the movement of the vehicle) to an autonomous mode (where the vehicle essentially drives itself) to modes that lie somewhere in between.
- These autonomous vehicles may maneuver themselves between locations using highly detailed maps and/or models in conjunction with sensors for detecting the vehicle's surroundings. If the detailed maps are incorrect, it may be particularly difficult or impossible for the vehicle to safely and efficiently navigate without input from the operator.
-
FIG. 1 depicts a scene and network components to maintain updated maps and related information for autonomous vehicles in accordance with an example embodiment. -
FIG. 2 is depicts features of an autonomous vehicle in accordance with an example embodiment. -
FIG. 3 depicts features of a road-side unit in accordance with an example embodiment. -
FIG. 4 is a flow chart depicting a series of operations that may be performed by map update logic operated by a road-side unit in accordance with an example embodiment. -
FIG. 5 is another flow chart depicting a series of operations that may be performed by map update logic in a road-side unit in accordance with an example embodiment. - Presented herein are techniques for updating detailed maps used to navigate an autonomous vehicle. The techniques include determining that a vehicle has come within a predetermined range of a road side unit, establishing a communication link with the vehicle, receiving, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, selecting a map, stored by the road side unit, for the vehicle, sending a query to a neighbor road side unit seeking data to augment the map, in response to the query, receiving the data to augment the map from the neighbor road side unit, updating the map based on the data to augment the map to obtain an updated map, and sending at least aspects of the updated map to the vehicle.
- Also presented herein is a device that includes an interface unit configured to enable network communications, a memory, and one or more processors coupled to the interface unit and the memory, and configured to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle.
-
FIG. 1 depicts ascene 100 and network components that are used to maintain updated maps and related information for autonomous vehicles in accordance with an example embodiment. Specifically, in the example embodiment,scene 100 includes aroad intersection 110.Several vehicles 105 a (V1), 105 b (V2), 105 c (V3) are depicted in the roadway. A plurality of road-side units (RSUs) 300 a, 300 b, 300 c are deployed in the vicinity of theintersection 110 and along (e.g., adjacent, above) the roadways. Each of the RSUs is also in communication with a central orcloud server 150 and one or more external sensors orinformation providers 160.Cloud server 150 may be configured to maintain overall control over the RSUs, and map generation and distribution. That is,cloud server 150 may operate as both an administrator of the RSUs and as a processor to generate maps, or at least provide data to the RSUs to generate maps locally. - Although not shown in
FIG. 1 , those skilled in the art will appreciate thatscene 100 may include a host of other elements such as lane markers (white, yellow lines, etc.), curbs, guardrails, highway dividers, buildings, trees, parked cars, pedestrians, road work sites, stop signs, traffic signals, among other possible features. Such features can all be part of the detailed maps that are generated and supplied to the vehicles. - As will be described more fully below,
vehicles scene 100, can change relatively quickly compared to a baseline map that might be stored within a given vehicle. A given change might be transient such as a lane closure, or an accident. Other changes may be more permanent including a new traffic sign or a new building that has been constructed. A vehicle that is presented with a scene that does not match its currently stored map may become confused and either stop in a failsafe mode, or drive/navigate in a manner that is unsafe. - Thus, the several RSUs 200 a, 200 b, 200 c are used, in accordance with the instant embodiments, to maintain updated versions of the maps and to provide to relevant vehicles the updated versions of the maps or a set of deviation data that enables a given vehicle to properly update its stored map to a most up to date version. As will also be discussed further below, the RSUs can communicate with one another, the vehicles,
cloud server 150 and other external sensors/information provider 160 to better ascertain if/when updates to a given map may be warranted. The RSUs can also be used to control what data may be uploaded tocloud server 150. That is, each RSU may be used to throttle the amount of data that flows towardcloud server 150 in an effort to avoid using too much bandwidth andoverwhelming cloud server 150 with unnecessary, outdated or duplicative data. -
FIG. 2 depicts features of anautonomous vehicle 105 in accordance with an example embodiment.Autonomous vehicle 105 may be any type of vehicle including, but not limited to, a car, truck, drone, motorcycle, bicycle, bus, boat, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, piece of farm equipment, piece of construction equipment, tram, golf cart, train, and trolley. The vehicle may have one or more computers, such as acomputer 210 containing aprocessor 220,memory 230 and other components typically present in general purpose computers. -
Processor 220 may be any conventional processor, such as commercially available central processing units (CPUs) and/or graphics processing units (GPUs). Alternatively, the processor may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. AlthoughFIG. 2 functionally illustrates the processor, memory, and other elements ofcomputer 210 as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computer, or memory may actually comprise multiple processors, computers, or memories that may or may not be stored within the same physical housing. - In various aspects described herein, the
processor 220 may be located remotely from thevehicle 105 and communicate with thevehicle 105 wirelessly. In other aspects, some of the processes described herein are executed on a processor disposed within thevehicle 105 and others by a remote processor, including taking the steps to execute a given maneuver. - The
memory 230 stores information accessible by theprocessor 220, including autonomous vehicle logic (or instructions) 232 anddata 234 that may be executed or otherwise used by theprocessor 220. Thememory 230 may be of any type capable of storing information accessible by the processor, including a computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, ROM, RAM, DVD or other optical disks, as well as other write-capable and read-only memories. Systems and methods may include different combinations of the foregoing, whereby different portions of the instructions and data are stored on different types of media. - The
autonomous vehicle logic 232 may be any set of instructions to be executed directly (such as machine code) or indirectly (such as scripts) by the processor. For example, the instructions may be stored as computer code on the computer-readable medium. In that regard, the terms “instructions” and “programs” may be used interchangeably herein. - The data 134 may be retrieved, stored or modified by
processor 220 in accordance with theautonomous vehicle logic 232. - The data may include environmental data that was obtained at a previous point in time and is expected to persist regardless of the vehicle's presence in the environment. For example,
data 234 may include amap 236, e.g., a highly detailed 3D map that represents the shape and elevation of roadways, lane lines, intersections, crosswalks, speed limits, traffic signals, buildings, signs, real time traffic information, or other such features and information. These features may be persistent, and may be updated, as described in more detail below, when thevehicle 105 approaches the location of a feature in themap 236, or when the vehicle communicates with one or more RSUs. Themap 236 may also include explicit speed limit information associated with various roadway segments. The speed limit data may be entered manually or scanned from previously taken images of a speed limit sign using, for example, optical-character recognition. Themap 236 may also include two-dimensional street-level imagery, such as highly detailed image data depicting the surroundings of a vehicle from the vehicle's point-of-view. And, as noted, themap 236 may also include 3D terrain maps incorporating one or more of the objects listed above. The data may include, for example, environmental data that is transient, such as the accumulated snow since the last snow removal truck came by, which may have been 5 mins ago. Note that some of this transient information is beneficial to capture and share at the local RSU, but may have less value to send and add to the map in the cloud. -
Data 234 may also include trafficpattern model information 238, e.g., a highly detailed model indicating the distribution of typical or expected speeds, trajectories, locations, accelerations/decelerations (changes in speed), or other such characteristics of vehicles or other moving objects on the locations of themap 236. This data may be generated, for example, by observing how vehicles, pedestrians, bicycles, etc. move at different locations in themap 236. That is, data for the trafficpattern model information 238 might be generated based on long term observation and learning of traffic patterns. - In some aspects, the traffic
pattern model information 238 may indicate information pertinent to the entire road. For example, the trafficpattern model information 238 may indicate a range of speeds for vehicles travelling along a road, specific to each particular lane or independent of the lanes. The trafficpattern model information 238 may indicate a range of trajectories for vehicles travelling in particular lanes. - The vehicle may also be equipped with one or more sensors or
detection components 244 for detecting objects external to the vehicle such as other vehicles, obstacles in the roadway, traffic signals, signs, trees, buildings, weather, etc. The detection components orsensors 244 may include lasers, sonar, radar, LIDAR, cameras or any other detection devices which record data and which may be processed by thecomputer 210. -
Computer 210 may be capable of communicating with various components of thevehicle 105. For example, thecomputer 210 may be in communication with the vehicle'scentral processor 260 and may send and receive information from the various systems ofvehicle 105, for example thebraking 280,acceleration 282, signaling 284,navigation 286, direction/speed detection device 252 andgeographic position component 250 systems in order to control the movement, speed, etc. ofvehicle 105. In addition, when engaged,computer 210 may control some or all of these functions ofvehicle 105 and thus be fully or partially autonomous. Finally,vehicle 105 includes anetwork interface 239 that may be used to communicate, e.g., wirelessly, with a givenRSU 300. This wireless communication provides a rich set of data to theRSU 300 that can be leveraged to augment and update themap 236, as will be discussed below. -
FIG. 3 depicts features of anRSU 300 in accordance with an example embodiment. A road-side unit 300 includes acomputer 310, which may be similar tocomputer 210, and aprocessor 320, which may be similar toprocessor 220. Amemory 330 may be configured to storemap update logic 332, i.e., computer logic or code encoded in a computer readable medium similar to that described forautonomous vehicle logic 232, anddata 334 formultiple vehicles 105 that may be within a predetermined range of theRSU 300.Data 334 may includesensor data 336 received wirelessly from respective vehicles vianetwork interface 339 and may further include, for example, data about a type of thevehicle 337. For example, the vehicle may be a truck or a car, a bicycle, or a drone. In accordance with embodiments described herein, the type of vehicle can be used to determine what sort of map or map update is to be provided to the vehicle, or what type of information or data might be missing from the vehicle's already storedmap 236 such that theRSU 300 will query the vehicle, another RSU, orcloud server 150 to obtain additional/selected information that may be more useful for that particular type of vehicle. - In operation, each
RSU 300 is configured to store a base map, and is further configured to update that base map usingmap update logic 332. For example,cloud server 150 might deliver a base map toRSU 300 at a point in time. That same base map may also be supplied to any one of thevehicles 105. Thereafter, a given vehicle, assumevehicle 105 b for purposes of discussion, travels byRSU 300 b and supplies sensor data toRSU 300 b, enablingRSU 300 b to augment or update its base map. Updates to the base map might include parked cars, accidents, new road signs, the fact that ice has been detected on the road, or that there is a pot hole in a particular location, among other elements that might further elaborate on, or otherwise update a base map.RSU 300 b might also obtain updates fromcloud server 150 and external sensors/information provider 160 (e.g., weather forecasts, upcoming construction notices, etc.) to still further update its base map. - When yet another vehicle, e.g., 105 d, enters a zone covered by
RSU 300 b, the vehicle 100 d communicates withRSU 300 b to verify that the vehicle's currently stored base map matches the base map which is stored in the RSU. This comparison could be performed, for example, by the vehicle sending, toRSU 300 b, a hash value of the base map stored in thevehicle 105 d.RSU 300 b can then confirm that the base maps match.RSU 300 b can then send, e.g., a map deviation data set to thevehicle 105 d based on theinformation RSU 300 b previously received from the earlier passingvehicle 105 b. That is,RSU 300 b might send to thevehicle 105 d a collection of information delineating the differences between the base map stored in thevehicle 105 d and the updated map that has been generated by the RSU over recent time. It is noteworthy that because a deviation data set typically has less overall data than a full map, the transfer time needed for the vehicle to obtain the map deviation data set fromRSU 300 b is less than the time needed to send a full map. - If
RSU 300 b determines thatvehicle 105 d has a different base map from the currently-stored based map inRSU 300 b, thenRSU 300 b can send the updated map, or its currently-stored base map, as a new base map tovehicle 105 d. - The map deviation data set transmitted to
vehicle 105 d can be a function of (1) location of vehicle, (2) current direction (e.g., speed) and type of vehicle, and (3) intended or anticipated next steps in driving trajectory. Depending on the context (such as location, direction, speed, intended trajectory, etc.), an overlay of dynamic information such as traffic can be delivered. Such context is particularly relevant and scalable at the “edge,” i.e., at the RSU network level, as opposed to thecloud server 150 level. - For example, if a vehicle is driving northbound on a road, the information that is most valuable to it will probably be different from information related to driving southbound. Similarly, when a vehicle approaches an intersection, the most valuable information depends on (a) where it is coming from and (b) where it is going, e.g., driving northbound (coming from the south) and planning to turn right at the intersection. Therefore, when a vehicle communicates with an
RSU 300, the vehicle may provide such information enabling theRSU 300 to provide the most appropriate information back to the vehicle. - Thus, in the instant embodiments, once a given
RSU 300 learns where a vehicle might be going next, or determines that information might be missing from its own stored map, the RSU can communicate and query a neighbor RSU to obtain further map data to update its own map and then deliver the updated map or map deviation data set to the vehicle. -
FIG. 4 is a flow chart depicting a series of operations that may be performed bymap update logic 332 operated by a road-side unit 300 in accordance with an example embodiment. At 410 the RSU determines that a vehicle has come within a predetermined range. At 412, the RSU establishes a communication link with the vehicle. At 414, the RSU receives, from the vehicle, data sufficient to identify a type of the vehicle (or where it is travelling, or any other number of characteristics about the vehicle) This step might also include obtaining a type/version of the map that that vehicle has stored to establish the base map. At 416, and based on the vehicle type (or other characteristic), the RSU selects a map, stored by the road side unit, for the vehicle. Selecting a map may comprise, e.g., selecting aspects of the base map that are most relevant to the type of vehicle or where the vehicle is programmed to travel. At 418, the RSU sends a query to a neighbor RSU seeking data to augment the map (or type of map) that has been selected. At 420, the RSU receives the data to augment the map from the neighbor RSU. At 422 the RSU updates the selected map (or type of map) based on the data to augment the map to obtain an updated map. At 424 the RSU sends at least a portion of the updated map to the vehicle. That is, the RSU sends a map deviation data set to the vehicle or sends a complete updated map to the vehicle. - By having RSUs communicate with one another it is possible to minimize the amount of information exchanged with a cloud server 150 (which might otherwise be responsible for collecting all data from all RSU and coordinating which information to provide to which vehicles), thus minimizing the amount of bandwidth consumed in the network.
- In connection with selecting an appropriate map, the scope of the updated map or map deviation data set sent from the
RSU 300 to thevehicle 105 might depend on vehicle type, e.g., a light bicycle versus an 18-wheeler truck. At the network edge (the RSU level), theRSU 300 can scale the map in size, information content and perspective, and create a version that is tailored to the target vehicle' needs and capabilities. - In accordance with a further embodiment, there can be a communication exchange between the
RSU 300 and thevehicle 105 during which theRSU 300 queries the vehicle to determine what data would be most useful to transmit, and what data might be unnecessary to transmit. That is, the vehicle might request specific information. For example, a passenger might want to know if there is traffic several blocks away. The RSU can then communicate with the relevant neighbor RSU to obtain the requested data. - Adaptive Edge Computing
- Conventional approaches to ensuring that vehicles have the most up to date maps are cloud-based where sensor data collected by the vehicle is sent to the cloud, e.g.,
cloud server 150. In such configurations, the RSUs might operate merely as pass through devices or conduits for information between a given vehicle and the cloud. However, it can take a significant amount of bandwidth and related cost to send large volumes of data to the cloud. The cost may come in the form of scaling that involves increased CPU and memory resources in the cloud. Such scaling issues might also result in increased communication latency between a vehicle and the cloud, proving detrimental to the vehicle's decision making process. Adaptive processing at the edge (RSU level) addresses this problem. For example, consider an RSU that experiences a continuous stream of vehicles that travel by. More specifically, assume that 20 vehicles pass a given RSU over the course of one minute. These 20 vehicles will likely send the identical or very nearly identical sensor data or telemetry. In accordance with the instant embodiments, instead of transmitting each of the 20 vehicles' sensor data to the cloud, theRSU 300 can instead process this data or telemetry and determine common aspects and characteristics and thus send a reduced amount of data to the cloud. - For example, the RSU can be configured to identify duplicate information and only transmit non-duplicative information to the
cloud server 150. - The RSU can be configured to identify complementary information and send only the complementary information to the
cloud server 150. - The RSU can use the sensor data to compose a map or model and only send the resulting map or model to the
cloud server 150. - The RSU can be configured to use the sensor information to refine the map or model and only send the refinement to the
cloud server 150. - The RSU can be configured to identify higher priority information (e.g., detecting an erratic car driving in the wrong direction), and give bandwidth priority to such information.
- Thus, the RSU can build and continuously refine a map or model, estimating gaps, and then requesting the types of data that would fill the gaps. A key element of the above is feedback between the RSU and either the vehicle and/or other RSUs to guide what information is sent.
- In addition, and in connection with the prior discussion, there can be interactions between multiple, usually neighboring RSU's. For example, one RSU may have a detailed map or model of the environment from the south side of a scene looking north, while another RSU (perhaps a more northern RSU) may have a detailed map or model of the environment from the north side of a scene looking south. Because the RSUs know their local neighbors and associated coverage areas, a given RSU can query a neighbor RSU having the data or information that is being sought to fill a gap in the map or model stored in the given RSU. The resulting updated map or model can then be shared with (1) vehicles, (2) still other neighboring RSUs, and (3) the
cloud server 150. Notably, thecloud server 150 is not relied upon for initial or even substantial processing. Processing is instead performed at the edge, i.e., at the RSU level. This not only mitigates bandwidth usage, it can also help avoid latency with having to rely on communication with thecloud server 150. -
FIG. 5 is another flow chart depicting a series of operations that may be performed bymap update logic 332 in a road-side unit 300 in accordance with an example embodiment. These operations are employed to throttle or control the amount or quality or type of data or information that is sent tocloud server 150. That is, the described operations are performed by an RSU to determine what aspects or elements of information received from multiple vehicles should be sent to thecloud server 150, as opposed to sending all received information or data to thecloud server 150. - At 502 the RSU receives data or telemetry from one or more vehicles. For example, as described above, the RSU may receive information from a laser, camera, radar unit, etc. At 504, the RSU processes the data or telemetry to detect one or more objects and one or more characteristics (e.g., location, size, color, movement, etc.) of those one or more objects. At 506, for each detected object, the RSU compares the one or more characteristics of the detected object to its stored map. For example, a characteristic for the location or size of a stationary object may be compared to the locations of objects in the map. At 508, the RSU then determines a deviation value for the compared one or more characteristics. Deviation values can be based on percent deviation, minimum or maximum deviation, etc. At 510, the RSU determines whether the deviation values are within threshold values for the relevant characteristics. These thresholds may be determined by an administrator and may be different for different kinds of objects. If the deviation value is within a threshold, more data is collected from vehicles for processing. If at 510 the deviation values are not within the relevant threshold deviation values, the RSU, i.e.,
map update logic 332, updates its map. At 514, the RSU sends the updated map or a map deviation data set to thecloud server 150. It should be understood by those skilled in the art, that the same updated map or map deviation data set can also be shared with neighbor RSUs and vehicles. - It should also be noted that telemetry need not be received from all vehicles for map updating. Telemetry could be limited to certain “trusted vehicles” such as public transportation vehicles (e.g., busses, light trains), police cars, firetrucks, etc. Alternatively, the RSUs can be configured to statistically weigh telemetry received from vehicles, and that such weighting might not begin until telemetry from a threshold number of vehicles has been received. Higher weights may be given to selected vehicles whose past data has consistently proven to be accurate. Further, less transient changes can be supplied by, e.g.,
cloud server 150, thereby alleviating the need to provide frequent updates to every vehicle. That is, thecloud server 150 can be responsible for distributing new base maps when significant non-transient changes are made to the scene or environment. - In sum, autonomous vehicles use high definition 3D maps to augment their perception of the environment captured through their own sensors to establish a highly accurate position within the environment. However, the detailed maps suffer from enormous data sizes and the challenge of keeping them up to date, especially when relying on a cloud or central server. Embodiments described herein push much of the map processing to the edge of the network, i.e. to local RSUs, which are configured to update maps locally, communicate those maps or map deviation data sets to vehicles, and communicate selected data to the cloud. The RSUs can also communicate with another to enhance the map updating process.
- In summary, in one form, a method is provided. The method includes determining that a vehicle has come within a predetermined range of a road side unit, establishing a communication link with the vehicle, receiving, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, selecting a map, stored by the road side unit, for the vehicle, sending a query to a neighbor road side unit seeking data to augment the map, in response to the query, receiving the data to augment the map from the neighbor road side unit, updating the map based on the data to augment the map to obtain an updated map, and sending at least aspects of the updated map to the vehicle
- In an embodiment, sending at least aspects of the updated map comprises sending differences between a map stored in the vehicle and the updated map.
- The method may further include receiving telemetry from a plurality of vehicles within the predetermined range, and updating the map based on the telemetry. The method may still further include determining a deviation value for an aspect of the telemetry compared to an aspect of the map, and not updating the map when the deviation value is within a predetermined threshold value. The method may further include determining a deviation value for an aspect of the telemetry compared to an aspect of the map, and further updating the map when the deviation value is beyond a predetermined threshold value. The method may still further include sending an indication of the aspect of the telemetry to a central server.
- In accordance with an embodiment, updating the map may be based on data received from a central server. Updating the map may also be based on data from an external information source other than a central server.
- The method may also include sending a query to the vehicle, the query enabling the vehicle to request selected information as part of the updated map.
- In another form, a device is provided that includes: an interface unit configured to enable network communications, a memory, and one or more processors coupled to the interface unit and the memory, and configured to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle.
- The one or more processors are further configured to send at least aspects of the updated map by sending differences between a map stored in the vehicle and the updated map.
- The one or more processors are further configured to receive telemetry from a plurality of vehicles within the predetermined range, and the one or more processors are further configured to update the map based on the telemetry. The one or more processors may be further configured to determine a deviation value for an aspect of the telemetry compared to an aspect of the map, and not update the map when the deviation value is within a predetermined threshold value. The one or more processors may also be configured to determine a deviation value for an aspect of the telemetry compared to an aspect of the map, and further update the map when the deviation value is beyond a predetermined threshold value. The one or more processors may still be further configured to send an indication of the aspect of the telemetry to a central server.
- In still another embodiment, a non-transitory tangible computer readable storage media encoded with instructions is provided that, when executed by at least one processor, is configured to cause the processor to: determine that a vehicle has come within a predetermined range of a road side unit, establish a communication link with the vehicle, receive, from the vehicle, data sufficient to identify a vehicle type of the vehicle, based on the vehicle type, select a map, stored by the road side unit, for the vehicle, send a query to a neighbor road side unit seeking data to augment the map, in response to the query, receive the data to augment the map from the neighbor road side unit, update the map based on the data to augment the map to obtain an updated map, and send at least aspects of the updated map to the vehicle. The instructions may further case the processor to send at least aspects of the updated map by sending differences between a map stored in the vehicle and the updated map, and/or to receive telemetry from a plurality of vehicles within the predetermined range.
- The above description is intended by way of example only. Various modifications and structural changes may be made therein without departing from the scope of the concepts described herein and within the scope and range of equivalents of the claims.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/488,945 US10907974B2 (en) | 2017-04-17 | 2017-04-17 | Real-time updates to maps for autonomous navigation |
US17/130,978 US11747169B2 (en) | 2017-04-17 | 2020-12-22 | Real-time updates to maps for autonomous navigation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/488,945 US10907974B2 (en) | 2017-04-17 | 2017-04-17 | Real-time updates to maps for autonomous navigation |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/130,978 Continuation US11747169B2 (en) | 2017-04-17 | 2020-12-22 | Real-time updates to maps for autonomous navigation |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180299274A1 true US20180299274A1 (en) | 2018-10-18 |
US10907974B2 US10907974B2 (en) | 2021-02-02 |
Family
ID=63791807
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/488,945 Active 2038-06-17 US10907974B2 (en) | 2017-04-17 | 2017-04-17 | Real-time updates to maps for autonomous navigation |
US17/130,978 Active 2038-06-19 US11747169B2 (en) | 2017-04-17 | 2020-12-22 | Real-time updates to maps for autonomous navigation |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/130,978 Active 2038-06-19 US11747169B2 (en) | 2017-04-17 | 2020-12-22 | Real-time updates to maps for autonomous navigation |
Country Status (1)
Country | Link |
---|---|
US (2) | US10907974B2 (en) |
Cited By (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180348003A1 (en) * | 2017-06-02 | 2018-12-06 | Apple Inc. | Providing Light Navigation Guidance |
US20190132709A1 (en) * | 2018-12-27 | 2019-05-02 | Ralf Graefe | Sensor network enhancement mechanisms |
US10380886B2 (en) * | 2017-05-17 | 2019-08-13 | Cavh Llc | Connected automated vehicle highway systems and methods |
CN110136564A (en) * | 2019-06-12 | 2019-08-16 | 四川长虹电器股份有限公司 | A kind of method of parking lot mark display information in real-time update map |
US20200050205A1 (en) * | 2018-08-07 | 2020-02-13 | Cnh Industrial America Llc | System and method for updating a mapped area |
CN110972085A (en) * | 2019-11-27 | 2020-04-07 | 北京梧桐车联科技有限责任公司 | Information interaction method, device, storage medium, equipment and system |
US10647317B2 (en) * | 2017-06-02 | 2020-05-12 | Honda Motor Co., Ltd. | Automatic traveling control system and server device |
CN111309832A (en) * | 2019-12-31 | 2020-06-19 | 厦门雅迅网络股份有限公司 | Electronic horizon construction method and system |
US10692365B2 (en) | 2017-06-20 | 2020-06-23 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
CN111339111A (en) * | 2020-02-26 | 2020-06-26 | 北京邮电大学 | High-precision map data updating method and system |
EP3720097A1 (en) * | 2019-04-03 | 2020-10-07 | Hyundai Motor Company | Method and apparatus for operating autonomous shuttle using edge computing |
CN111757288A (en) * | 2019-03-27 | 2020-10-09 | 阿里巴巴集团控股有限公司 | Perception base station in road traffic environment and message sending method and device thereof |
US10867512B2 (en) | 2018-02-06 | 2020-12-15 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
CN112097779A (en) * | 2020-09-15 | 2020-12-18 | 黑龙江省交投千方科技有限公司 | Data service system based on roadside high-precision map |
US20210005085A1 (en) * | 2019-07-03 | 2021-01-07 | Cavh Llc | Localized artificial intelligence for intelligent road infrastructure |
US20210101612A1 (en) * | 2019-10-08 | 2021-04-08 | Qualcomm Incorporated | Edge System for Providing Local Dynamic Map Data |
CN112804661A (en) * | 2021-03-18 | 2021-05-14 | 湖北亿咖通科技有限公司 | Map data transmission method, system, edge server and storage medium |
US11017674B1 (en) * | 2018-10-15 | 2021-05-25 | Waymo Llc | Managing and tracking scouting tasks using autonomous vehicles |
US20210231459A1 (en) * | 2020-01-29 | 2021-07-29 | Toyota Jidosha Kabushiki Kaisha | Apparatus and method for collecting data for map generation |
US20210364632A1 (en) * | 2020-05-24 | 2021-11-25 | Havenshine Technologies, Inc. | Methods and Systems for Map Creation and Calibration of Localization Equipment in an Outdoor Environment |
CN113891281A (en) * | 2021-09-28 | 2022-01-04 | 安徽江淮汽车集团股份有限公司 | Dynamic adjustment method for road side unit map |
CN114440859A (en) * | 2022-01-26 | 2022-05-06 | 广州小鹏自动驾驶科技有限公司 | Map updating method and device, computer equipment and storage medium |
US11330410B2 (en) * | 2018-04-03 | 2022-05-10 | Corning Research & Development Corporation | Pathside communication relay (PCR) for collecting and distributing pathside data among client devices |
US11359930B2 (en) * | 2017-12-29 | 2022-06-14 | Zte Corporation | Map construction and navigation method, and device and system |
US11373122B2 (en) | 2018-07-10 | 2022-06-28 | Cavh Llc | Fixed-route service system for CAVH systems |
US11395208B2 (en) * | 2019-12-30 | 2022-07-19 | Subaru Corporation | Mobility information provision system for mobile bodies, server, and vehicle |
US11423062B2 (en) | 2019-09-26 | 2022-08-23 | Here Global B.V. | Apparatus and methods for generating update data for a map database |
EP3943352A4 (en) * | 2019-06-11 | 2022-11-02 | Apollo Intelligent Driving Technology (Beijing) Co., Ltd. | Driving control method, apparatus, device, medium, and system |
CN115294764A (en) * | 2022-07-28 | 2022-11-04 | 阿波罗智联(北京)科技有限公司 | Pedestrian crossing area determination method, device and equipment and automatic driving vehicle |
US11495126B2 (en) | 2018-05-09 | 2022-11-08 | Cavh Llc | Systems and methods for driving intelligence allocation between vehicles and highways |
US11567928B2 (en) | 2019-09-26 | 2023-01-31 | Here Global B.V. | Apparatus and methods for updating a map database |
US11735041B2 (en) | 2018-07-10 | 2023-08-22 | Cavh Llc | Route-specific services for connected automated vehicle highway systems |
US11735035B2 (en) | 2017-05-17 | 2023-08-22 | Cavh Llc | Autonomous vehicle and cloud control (AVCC) system with roadside unit (RSU) network |
US20230266146A1 (en) * | 2022-02-23 | 2023-08-24 | Plusai, Inc. | Methods and apparatus for navigating an autonomous vehicle based on a map updated in regions |
US11747169B2 (en) | 2017-04-17 | 2023-09-05 | Cisco Technology, Inc. | Real-time updates to maps for autonomous navigation |
US11842642B2 (en) | 2018-06-20 | 2023-12-12 | Cavh Llc | Connected automated vehicle highway systems and methods related to heavy vehicles |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070152804A1 (en) * | 1997-10-22 | 2007-07-05 | Intelligent Technologies International, Inc. | Accident Avoidance Systems and Methods |
US20100063729A1 (en) * | 2006-12-15 | 2010-03-11 | Kabushiki Kaisha Kenwood | Mobile body position information transmitting device for navigation system, and mobile body position information transmission method and program for navigation system |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4446252B2 (en) | 2005-03-16 | 2010-04-07 | 株式会社デンソー | Vehicle navigation apparatus and map data update system including the same |
US9518830B1 (en) | 2011-12-28 | 2016-12-13 | Intelligent Technologies International, Inc. | Vehicular navigation system updating based on object presence |
US8718861B1 (en) | 2012-04-11 | 2014-05-06 | Google Inc. | Determining when to drive autonomously |
US8527199B1 (en) | 2012-05-17 | 2013-09-03 | Google Inc. | Automatic collection of quality control statistics for maps used in autonomous driving |
US9435654B2 (en) | 2013-06-01 | 2016-09-06 | Savari, Inc. | System and method for creating, storing, and updating local dynamic MAP database with safety attribute |
US10907974B2 (en) | 2017-04-17 | 2021-02-02 | Cisco Technology, Inc. | Real-time updates to maps for autonomous navigation |
-
2017
- 2017-04-17 US US15/488,945 patent/US10907974B2/en active Active
-
2020
- 2020-12-22 US US17/130,978 patent/US11747169B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070152804A1 (en) * | 1997-10-22 | 2007-07-05 | Intelligent Technologies International, Inc. | Accident Avoidance Systems and Methods |
US20100063729A1 (en) * | 2006-12-15 | 2010-03-11 | Kabushiki Kaisha Kenwood | Mobile body position information transmitting device for navigation system, and mobile body position information transmission method and program for navigation system |
Cited By (55)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11747169B2 (en) | 2017-04-17 | 2023-09-05 | Cisco Technology, Inc. | Real-time updates to maps for autonomous navigation |
US11955002B2 (en) | 2017-05-17 | 2024-04-09 | Cavh Llc | Autonomous vehicle control system with roadside unit (RSU) network's global sensing |
US10380886B2 (en) * | 2017-05-17 | 2019-08-13 | Cavh Llc | Connected automated vehicle highway systems and methods |
US11935402B2 (en) | 2017-05-17 | 2024-03-19 | Cavh Llc | Autonomous vehicle and center control system |
US20240029555A1 (en) * | 2017-05-17 | 2024-01-25 | Cavh Llc | Autonomous vehicle intelligent system (avis) |
US11482102B2 (en) | 2017-05-17 | 2022-10-25 | Cavh Llc | Connected automated vehicle highway systems and methods |
US20220366783A1 (en) * | 2017-05-17 | 2022-11-17 | Cavh Llc | Autonomous vehicle control system with traffic control center/traffic control unit (tcc/tcu) and roadside unit (rsu) network |
US11735035B2 (en) | 2017-05-17 | 2023-08-22 | Cavh Llc | Autonomous vehicle and cloud control (AVCC) system with roadside unit (RSU) network |
US11879746B2 (en) | 2017-06-02 | 2024-01-23 | Apple Inc. | Providing light navigation guidance |
US10907984B2 (en) | 2017-06-02 | 2021-02-02 | Apple Inc. | Presenting suggested routes based on local route ranking |
US11118929B2 (en) * | 2017-06-02 | 2021-09-14 | Apple Inc. | Providing light navigation guidance |
US11231291B2 (en) | 2017-06-02 | 2022-01-25 | Apple Inc. | Presenting non-recommended routes |
US20180348003A1 (en) * | 2017-06-02 | 2018-12-06 | Apple Inc. | Providing Light Navigation Guidance |
US11650068B2 (en) | 2017-06-02 | 2023-05-16 | Apple Inc. | Presenting suggested routes based on local route ranking |
US10647317B2 (en) * | 2017-06-02 | 2020-05-12 | Honda Motor Co., Ltd. | Automatic traveling control system and server device |
US11430328B2 (en) | 2017-06-20 | 2022-08-30 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
US11881101B2 (en) | 2017-06-20 | 2024-01-23 | Cavh Llc | Intelligent road side unit (RSU) network for automated driving |
US10692365B2 (en) | 2017-06-20 | 2020-06-23 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
US11359930B2 (en) * | 2017-12-29 | 2022-06-14 | Zte Corporation | Map construction and navigation method, and device and system |
US10867512B2 (en) | 2018-02-06 | 2020-12-15 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
US11854391B2 (en) | 2018-02-06 | 2023-12-26 | Cavh Llc | Intelligent road infrastructure system (IRIS): systems and methods |
US11330410B2 (en) * | 2018-04-03 | 2022-05-10 | Corning Research & Development Corporation | Pathside communication relay (PCR) for collecting and distributing pathside data among client devices |
US11495126B2 (en) | 2018-05-09 | 2022-11-08 | Cavh Llc | Systems and methods for driving intelligence allocation between vehicles and highways |
US11842642B2 (en) | 2018-06-20 | 2023-12-12 | Cavh Llc | Connected automated vehicle highway systems and methods related to heavy vehicles |
US11373122B2 (en) | 2018-07-10 | 2022-06-28 | Cavh Llc | Fixed-route service system for CAVH systems |
US11735041B2 (en) | 2018-07-10 | 2023-08-22 | Cavh Llc | Route-specific services for connected automated vehicle highway systems |
US20200050205A1 (en) * | 2018-08-07 | 2020-02-13 | Cnh Industrial America Llc | System and method for updating a mapped area |
US11804136B1 (en) | 2018-10-15 | 2023-10-31 | Waymo Llc | Managing and tracking scouting tasks using autonomous vehicles |
US11017674B1 (en) * | 2018-10-15 | 2021-05-25 | Waymo Llc | Managing and tracking scouting tasks using autonomous vehicles |
US20220182793A1 (en) * | 2018-12-27 | 2022-06-09 | Intel Corporation | Sensor network enhancement mechanisms |
US20190132709A1 (en) * | 2018-12-27 | 2019-05-02 | Ralf Graefe | Sensor network enhancement mechanisms |
US11153721B2 (en) * | 2018-12-27 | 2021-10-19 | Intel Corporation | Sensor network enhancement mechanisms |
CN111757288A (en) * | 2019-03-27 | 2020-10-09 | 阿里巴巴集团控股有限公司 | Perception base station in road traffic environment and message sending method and device thereof |
US11409293B2 (en) | 2019-04-03 | 2022-08-09 | Hyundai Motor Company | Method and apparatus for operating autonomous shuttle using edge computing |
EP3720097A1 (en) * | 2019-04-03 | 2020-10-07 | Hyundai Motor Company | Method and apparatus for operating autonomous shuttle using edge computing |
EP3943352A4 (en) * | 2019-06-11 | 2022-11-02 | Apollo Intelligent Driving Technology (Beijing) Co., Ltd. | Driving control method, apparatus, device, medium, and system |
CN110136564A (en) * | 2019-06-12 | 2019-08-16 | 四川长虹电器股份有限公司 | A kind of method of parking lot mark display information in real-time update map |
US20210005085A1 (en) * | 2019-07-03 | 2021-01-07 | Cavh Llc | Localized artificial intelligence for intelligent road infrastructure |
US11423062B2 (en) | 2019-09-26 | 2022-08-23 | Here Global B.V. | Apparatus and methods for generating update data for a map database |
US11567928B2 (en) | 2019-09-26 | 2023-01-31 | Here Global B.V. | Apparatus and methods for updating a map database |
US20210101612A1 (en) * | 2019-10-08 | 2021-04-08 | Qualcomm Incorporated | Edge System for Providing Local Dynamic Map Data |
CN114502444A (en) * | 2019-10-08 | 2022-05-13 | 高通股份有限公司 | Edge system for providing local dynamic map data |
CN110972085A (en) * | 2019-11-27 | 2020-04-07 | 北京梧桐车联科技有限责任公司 | Information interaction method, device, storage medium, equipment and system |
US11395208B2 (en) * | 2019-12-30 | 2022-07-19 | Subaru Corporation | Mobility information provision system for mobile bodies, server, and vehicle |
CN111309832A (en) * | 2019-12-31 | 2020-06-19 | 厦门雅迅网络股份有限公司 | Electronic horizon construction method and system |
WO2021135147A1 (en) * | 2019-12-31 | 2021-07-08 | 厦门雅迅网络股份有限公司 | Electronic horizon creation method and system |
US20210231459A1 (en) * | 2020-01-29 | 2021-07-29 | Toyota Jidosha Kabushiki Kaisha | Apparatus and method for collecting data for map generation |
CN111339111A (en) * | 2020-02-26 | 2020-06-26 | 北京邮电大学 | High-precision map data updating method and system |
US20210364632A1 (en) * | 2020-05-24 | 2021-11-25 | Havenshine Technologies, Inc. | Methods and Systems for Map Creation and Calibration of Localization Equipment in an Outdoor Environment |
CN112097779A (en) * | 2020-09-15 | 2020-12-18 | 黑龙江省交投千方科技有限公司 | Data service system based on roadside high-precision map |
CN112804661A (en) * | 2021-03-18 | 2021-05-14 | 湖北亿咖通科技有限公司 | Map data transmission method, system, edge server and storage medium |
CN113891281A (en) * | 2021-09-28 | 2022-01-04 | 安徽江淮汽车集团股份有限公司 | Dynamic adjustment method for road side unit map |
CN114440859A (en) * | 2022-01-26 | 2022-05-06 | 广州小鹏自动驾驶科技有限公司 | Map updating method and device, computer equipment and storage medium |
US20230266146A1 (en) * | 2022-02-23 | 2023-08-24 | Plusai, Inc. | Methods and apparatus for navigating an autonomous vehicle based on a map updated in regions |
CN115294764A (en) * | 2022-07-28 | 2022-11-04 | 阿波罗智联(北京)科技有限公司 | Pedestrian crossing area determination method, device and equipment and automatic driving vehicle |
Also Published As
Publication number | Publication date |
---|---|
US20210108927A1 (en) | 2021-04-15 |
US10907974B2 (en) | 2021-02-02 |
US11747169B2 (en) | 2023-09-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11747169B2 (en) | Real-time updates to maps for autonomous navigation | |
US11934962B2 (en) | Object association for autonomous vehicles | |
US11842639B2 (en) | Power and thermal management systems and methods for autonomous vehicles | |
US11537133B2 (en) | Dynamic routing for autonomous vehicles | |
US11354820B2 (en) | Image based localization system | |
US10489686B2 (en) | Object detection for an autonomous vehicle | |
US10006779B2 (en) | Transmission necessity determination apparatus and route planning system | |
US10829116B2 (en) | Affecting functions of a vehicle based on function-related information about its environment | |
US11619940B2 (en) | Operating an autonomous vehicle according to road user reaction modeling with occlusions | |
US20200133272A1 (en) | Automatic generation of dimensionally reduced maps and spatiotemporal localization for navigation of a vehicle | |
WO2017010209A1 (en) | Peripheral environment recognition device and computer program product | |
CN110799804A (en) | Map generation system and method | |
US20170359561A1 (en) | Disparity mapping for an autonomous vehicle | |
GB2614379A (en) | Systems and methods for vehicle navigation | |
US20200149896A1 (en) | System to derive an autonomous vehicle enabling drivable map | |
EP3479182A1 (en) | Affecting functions of a vehicle based on function-related information about its environment | |
WO2023129656A1 (en) | Calculating vehicle speed for a road curve | |
US20220242440A1 (en) | Methods and system for generating a lane-level map for an area of interest for navigation of an autonomous vehicle | |
US20210241612A1 (en) | Systems and methods for zone tagging | |
US11958503B2 (en) | Techniques for navigating an autonomous vehicle based on perceived risk | |
US20230154024A1 (en) | Producing a depth map from a monocular two-dimensional image | |
DE102022103060A1 (en) | AUTOMATICALLY DETECTING TRAFFIC SIGNALS USING SENSOR DATA |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CISCO TECHNOLOGY, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MOGHE, ASHOK K.;APOSTOLOPOULOS, JOHN G.;POUPKO, AVRAHAM A.;SIGNING DATES FROM 20170412 TO 20170414;REEL/FRAME:042029/0727 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |