KR20220034843A - Systems and methods for real-time control of autonomous devices - Google Patents

Abstract

An autonomous vehicle having sensors, preferably in which capabilities are varied, preferably located, and preferably not affected by environmental conditions. For example, receive a map comprising substantially discrete surface features along with data from sensors, generate an occupancy grid based on the map and data, and autonomously based on the type of surface the autonomous vehicle navigates to. A system running on an autonomous vehicle capable of changing the configuration of the vehicle. The device can safely navigate surfaces and surface features, including traversing discrete surfaces and other obstacles.

Description

Systems and methods for real-time control of autonomous devices

CROSS-REFERENCE TO RELATED APPLICATIONS

This utility patent application is filed in U.S. Patent Application Serial No. 16/800,497, entitled “System and Method for Surface Feature Detection and Traversal,” filed on February 25, 2020, which is incorporated herein by reference in its entirety, and is incorporated herein by reference in its entirety. It is a continuation application in part under Accession No. AA164). This patent application is filed in U.S. Provisional Patent Application Serial No. 62/872,396 (Attorney Docket No. AA028), entitled “Apparatus for Long and Short Range Sensors on an Autonomous Delivery Device,” filed July 10, 2019, March 2020; U.S. Provisional Patent Application No. 62/990,485 (Attorney Docket No. AA037), entitled “System and Method for Managing an Occupancy Grid,” filed on July 17, and “System and Method for Real,” filed July 10, 2019 -Time Control of the Configuration of an Autonomous Device."

This application is entitled to U.S. Patent Application No. 16/035,205 (Attorney Docket No. X80), entitled “MOBILITY DEVICE,” filed on July 13, 2018, and entitled “MOBILITY DEVICE,” filed on October 18, 2017. U.S. Patent Application Serial No. 15/787,613 (Attorney Docket No. W10); U.S. Patent Application Serial No. 15/982,737 (Attorney Docket No. X55), entitled “SYSTEM AND METHOD FOR SECURE REMOTE CONTROL OF A MEDICAL DEVICE,” filed on July 17, and “MOBILITY DEVICE IMPROVEMENTS,” filed July 15, 2017. U.S. Provisional Application No. 62/532,993 (Attorney Docket No. U30), entitled U.S. Provisional Application No. 62/559,263, entitled "MOBILITY DEVICE SEAT", filed September 15, 2017 (Attorney Docket No. V85), and 2017 to U.S. Provisional Application No. 62/581,670 (Attorney Docket No. W07), entitled “MOBILITY DEVICE SEAT,” filed on November 4, 2011, which is incorporated herein by reference in its entirety.

The present teachings generally relate to AVs, and more particularly, autonomous route planning, global occupancy grid management, on-vehicle sensors, surface feature detection and traversal ( surface feature detection and traversal, and real-time vehicle configuration changes.

Navigation of AVs and semi-autonomous vehicles (AV) typically relies on long-range sensors including, but not limited to, LIDAR, cameras, stereo cameras, and radar. Long range sensors can sense from 4 to 100 meters from the AV. In contrast, object avoidance and/or surface detection typically relies on short-range sensors including, but not limited to, stereo-cameras, short-range radar, and ultrasonic sensors. These short-range sensors typically observe an area or volume up to about 5 meters around the AV. Sensors may enable, for example, to orient an AV within its environment and navigate streets, sidewalks, obstacles, and open spaces to reach a desired destination. Sensors may also enable visualization of people, signs, traffic lights, obstacles, and surface features.

Surface feature traversal can be difficult because surface features such as, but not limited to, substantially discontinuous surface features (SDSFs) may be found among heterogeneous topologies. , and its topology may be unique to a particular geography. However, for example, but not limited to, slopes, edges, curves, steps, and curve-like geometries (herein, non-limiting In a way, SDSFs (referred to as SDSFs or simply surface features) may include some typical properties that may aid their identification. Surface/road conditions and surface types can be recognized and classified, for example, by fusing multisensory data, which can be complex and expensive. Surface features and conditions can be used to control the physical reconstruction of the AV in real time.

Sensors may be used to enable the creation of an occupancy grid that may represent the world for route planning purposes for AV. Path planning requires a grid that identifies space as free, occupied, or unknown. However, the probability that a space will be occupied can improve decision making about the space. A Logodds representation of probabilities can be used to increase accuracy at numerical boundaries of probabilities of 0 and 1. The probability that the cell is occupied may depend at least on the new sensor information, the old sensor information, and the previous occupancy information.

What is needed is a system that combines collected and real-time sensor data with changes in the vehicle's physical configuration to achieve variable terrain traversal. Favorable sensor placement is needed to achieve physical configuration changes, variable terrain traversal, and object avoidance. What is needed is the ability to locate SDSFs based on a multi-part model that is associated with several criteria for SDSF identification. It is necessary to determine candidate surface feature traversals based on criteria such as a candidate traversing approach angle, a candidate traversing driving surface on both sides of the candidate surface feature, and real-time determination of candidate traversing path obstacles. What is needed is a system and method for incorporating drivable surface and device mode information into an occupancy grid determination.

The AV of the present teachings may autonomously navigate to a desired location. In some configurations, the AV is a device controller including sensors, a perception subsystem, an autonomous subsystem, and a driver subsystem, a power base, four powered wheels, two caster wheels, and a cargo container. In some configurations, the perceptual and autonomous subsystems can receive and process sensor information (perception) and map information (perception and autonomy), and can provide direction to the driver subsystem. The map information may include surface classifications and associated device mode. Movement of the AV controlled by the driver subsystem and enabled by the power base may be sensed by the sensor subsystem to provide a feedback loop. In some configurations, SDSFs may be accurately identified from point cloud data, eg, stored in a map according to the process described herein. Portions of the map associated with the location of the AV may be provided to the AV during navigation. The perceptual subsystem may maintain an occupancy grid that may notify the AV about the probability that the path to be traversed is currently occupied. In some configurations, AV may operate in multiple distinct modes. Mods may enable traversing complex terrain, among other advantages. The combination of map (e.g., surface classification), sensor data (detecting features surrounding the AV), occupancy grid (probability that an upcoming path point will be occupied), and mode (ready or not to traverse difficult terrain) is It can be used to identify the direction, configuration, and velocity of the AV.

With respect to preparing the map, in some configurations, the method of the present teachings is for generating a map for navigating to at least one SDSF encountered by the AV, wherein the AV travels a path over the surface, and includes at least one SDSF, the path includes a start point and an end point, and the method includes, but is not limited to, accessing point cloud data representing a surface, filtering the point cloud data, a filtered point forming the cloud data into processable portions, and merging the processable portions into at least one concave polygon. The method may include locating and labeling the at least one SDSF in the at least one concave polygon. Positioning and labeling may form labeled point cloud data. The method may include generating graphing polygons based at least on the at least one concave polygon, and selecting a path from a starting point to an ending point based at least on the graphing polygons. When navigating, the AV may traverse at least one SDSF along the path.

Filtering the point cloud data includes conditionally removing from the point cloud data points representing transient objects and points representing outliers, and replacing the removed points with a preselected height. may optionally be included. Forming the processing portions may optionally include segmenting the point cloud data into processable portions, and removing points of a preselected height from the processable portions. Merging processable parts includes reducing the size of processable parts by analyzing outliers, voxels, and normals, growing regions from reduced size processable parts, growth Determining initial drivable surfaces from the segmented regions, segmenting and meshing the initial drivable surfaces, positioning polygons within the segmented and meshed initial drivable surfaces, and polygons may optionally include setting the drivable surfaces based at least on locating and labeling the at least one SDSF feature includes sorting the point cloud data of the drivable surfaces according to an SDSF filter, the SDSF filter comprising at least three categories of points, and the categories of points , in combination, locating the at least one SDSF point based at least on whether the at least one first preselected criterion is met. The method may optionally include generating the at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points, in combination, satisfy the at least one second preselected criterion. Generating the graphing polygons may also optionally include generating the at least one polygon from the at least one drivable surface. At least one polygon may include edges. Generating the graphing polygons includes smoothing edges, forming a driving margin based on the smoothed edges, adding at least one SDSF trajectory to the at least one drivable surface, and at least one third and removing edges from the at least one drivable surface according to a preselected criterion. Smoothing the edges may optionally include trimming the edges outward. Forming the driving margin of the smoothed edges may optionally include trimming the outer edges inward.

In some configurations, the system of the present teachings is for generating a map for navigating to the at least one SDSF the AV encounters, the AV moving a path over the surface, the surface comprising the at least one SDSF, The path includes a start point and an end point, and the system is capable of processing from, but not limited to, a first processor to access point cloud data representative of a surface, a first filter to filter the point cloud data, and to process from the filtered point cloud data. a second processor forming the portions, a third processor merging the processable portions into the at least one concave polygon, a fourth processor locating and labeling the at least one SDSF in the at least one concave polygon, wherein the positioning and labeling is labeling forming point cloud data, a fifth processor that generates graphing polygons, and a path selector that selects a path from a starting point to an ending point based at least on the graphing polygons. The AV may traverse at least one SDSF along the path.

The first filter may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. You can optionally include enable codes. A segmenter may optionally include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions, and removing points of a preselected height from the processable portions. can A third processor includes, but is not limited to, reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced size processable portions, grown regions determining initial drivable surfaces from, segmenting and meshing the initial drivable surfaces, positioning polygons within the segmented and meshed initial drivable surfaces, and at least based on the polygons the drivable surface It may optionally include executable code that may include configuring them. A fourth processor includes, but is not limited to, sorting the point cloud data of the drivable surfaces according to the SDSF filter, the SDSF filter comprising at least three categories of points, and the categories of points, in combination, at least may optionally include executable code that may include locating the at least one SDSF point based at least on whether the one first preselected criterion is met. The system may include, but is not limited to, generating at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points, in combination, satisfy at least one second preselected criterion. You can optionally include code.

Generating the graphing polygons includes, but is not limited to, generating at least one polygon from at least one drivable surface, wherein the at least one polygon includes edges, smoothing edges, smoothed edges forming a driving margin based on , adding at least one SDSF trajectory to the at least one drivable surface, and removing edges from the at least one drivable surface according to at least one third preselected criterion. It can optionally include executable code that it can include. Smoothing the edges may optionally include executable code, which may include, but is not limited to, trimming the edges outward. Forming the driving margin of the smoothed edges may optionally include executable code, which may include, but is not limited to, trimming the outer edges inward.

In some configurations, the method of the present teachings is for generating a map for navigating to the at least one SDSF the AV encounters, the AV moving a path over the surface, the surface comprising the at least one SDSF, A route includes a start point and an end point, and the method may include, but is not limited to, accessing a route topology. The route topology may include at least one graphing polygon that may include filtered point cloud data. The point cloud data may include labeled features and a drivable margin. The method includes transforming the point cloud data into a global coordinate system, determining boundaries of at least one SDSF, generating SDSF buffers of a preselected size around the boundaries, at least one SDSF traversing criterion determining which of the at least one SDSF can be traversed based at least on selecting a path from the starting point to the destination point based at least on the edge/weight graph.

The at least one SDSF traversal criterion is a minimum ingress distance between the at least one SDSF and the AV, comprising a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a drivable surface. ) and a minimum egress distance, and a minimum egress distance between the at least one SDSF and the AV, which can accommodate approximately 90° access to the at least one SDSF by the AV.

In some configurations, the system of the present teachings is for generating a map for navigating to the at least one SDSF the AV encounters, the AV moving a path over the surface, the surface comprising the at least one SDSF, The path includes a start point and an end point, and the system may include, but is not limited to, a sixth processor to access the route topology. The route topology may include at least one graphing polygon that may include filtered point cloud data. The point cloud data may include labeled features and a drivable margin. The system may include a seventh processor that transforms the point cloud data into a global coordinate system, and an eighth processor that determines boundaries of the at least one SDSF. The eighth processor may create SDSF buffers of a preselected size around the boundaries. The system is configured to determine which of the at least one SDSF can be traversed based at least on the ninth processor based at least on the at least one SDSF traversal criterion, at least based on the at least one SDSF traversal criterion, the transformed point cloud data, and a route topology. a tenth processor to generate an edge/weight graph, and a base controller to select a path from a starting point to a destination point based at least on the edge/weight graph.

In some configurations, the method of the present teachings is for generating a map for navigating to the at least one SDSF the AV encounters, the AV moving a path over the surface, the surface comprising the at least one SDSF, The path includes a start point and an end point, and the method may include, but is not limited to, accessing point cloud data representing the surface. The method may include filtering the point cloud data, forming the filtered point cloud data into processable portions, and merging the processable portions into at least one concave polygon. The method may include locating and labeling the at least one SDSF in the at least one concave polygon. Positioning and labeling may form labeled point cloud data. The method may include generating the graphing polygons based at least on the at least one concave polygon. The graphing polygons may form a route topology, and the point cloud data may include labeled features and a drivable margin. The method comprises transforming the point cloud data into a global coordinate system, determining boundaries of at least one SDSF, generating SDSF buffers of a preselected size around the boundaries, at least based on at least one SDSF traversal criterion. determining which of the one SDSF can be traversed, generating an edge/weight graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and the root topology, and in the edge/weight graph selecting a route from the starting point to the destination point based on at least.

Filtering the point cloud data may include conditionally removing from the point cloud data points representing transient objects and points representing outliers, and optionally replacing the removed points having a preselected height. Forming the processing portions may optionally include segmenting the point cloud data into processable portions, and removing points of a preselected height from the processable portions. Merging processable parts includes reducing the size of processable parts by analyzing outliers, voxels, and normals, growing regions from reduced sized processable parts, and initial decomposition from grown regions. determining the diverable surfaces, segmenting and meshing the initial drivable surfaces, positioning polygons within the segmented and meshed initial drivable surfaces, and setting the drivable surfaces based at least on the polygons may optionally include Locating and labeling the at least one SDSF feature comprises aligning the point cloud data of the drivable surfaces according to an SDSF filter, the SDSF filter comprising at least three categories of points, and the categories of points, in combination , locating the at least one SDSF point based at least on whether the at least one first preselected criterion is met. The method may optionally include generating the at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points, in combination, satisfy the at least one second preselected criterion. Generating the graphing polygons may also optionally include generating the at least one polygon from the at least one drivable surface. At least one polygon may include edges. Generating the graphing polygons includes smoothing edges, forming a driving margin based on the smoothed edges, adding at least one SDSF trajectory to the at least one drivable surface, and at least one third and removing edges from the at least one drivable surface according to a preselected criterion. may optionally include outwardly trimming the smoothing edges of the edges. Forming the driving margin of the smoothed edges may optionally include trimming the outer edges inward. The at least one SDSF traversal criterion includes a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF, a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV, including a drivable surface, and AV may optionally include a minimum entry distance between the at least one SDSF and the AV, which may accommodate approximately 90° access to the at least one SDSF by

In some configurations, the system of the present teachings is for generating a map for navigating to the at least one SDSF the AV encounters, the AV moving a path over the surface, the surface comprising the at least one SDSF, The path includes a start point and an end point, and the system is capable of processing from, but not limited to, a point cloud accessor to access point cloud data representing a surface, a first filter to filter the point cloud data, and to process from the filtered point cloud data. a segmenter forming the parts, a third processor merging the processable parts into the at least one concave polygon, a fourth processor locating and labeling the at least one SDSF in the at least one concave polygon, wherein the positioning and labeling is labeled forming the point cloud data—can include a fifth processor that generates the graphing polygons. The route topology may include at least one graphing polygon that may include filtered point cloud data. The point cloud data may include labeled features and a drivable margin. The system may include a seventh processor that transforms the point cloud data into a global coordinate system, and an eighth processor that determines boundaries of the at least one SDSF. The eighth processor may create SDSF buffers of a pre-selected size around the boundaries. The system is configured to determine which of the at least one SDSF can be traversed based at least on the ninth processor based at least on the at least one SDSF traversal criterion, at least based on the at least one SDSF traversal criterion, the transformed point cloud data, and a route topology. a tenth processor to generate an edge/weight graph, and a base controller to select a path from a starting point to a destination point based at least on the edge/weight graph.

The first filter may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. You can optionally include enable codes. A segmenter may optionally include executable code that may include, but is not limited to, segmenting the point cloud data into processable portions, and removing points of a preselected height from the processable portions. can A third processor includes, but is not limited to, reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced size processable portions, grown regions determining initial drivable surfaces from, segmenting and meshing the initial drivable surfaces, positioning polygons within the segmented and meshed initial drivable surfaces, and at least based on the polygons the drivable surface It may optionally include executable code that may include configuring them. A fourth processor includes, but is not limited to, sorting the point cloud data of the drivable surfaces according to the SDSF filter, the SDSF filter comprising at least three categories of points, and the categories of points, in combination, at least may optionally include executable code that may include locating the at least one SDSF point based at least on whether the one first preselected criterion is met. The system may include, but is not limited to, generating at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points, in combination, satisfy at least one second preselected criterion. You can optionally include code.

Generating the graphing polygons includes, but is not limited to, generating at least one polygon from at least one drivable surface, wherein the at least one polygon includes edges, smoothing edges, smoothed edges forming a driving margin based on , adding at least one SDSF trajectory to the at least one drivable surface, and removing edges from the at least one drivable surface according to at least one third preselected criterion. It can optionally include executable code that it can include. Smoothing the edges may optionally include executable code, which may include, but is not limited to, trimming the edges outward. Forming the driving margin of the smoothed edges may optionally include executable code, which may include, but is not limited to, trimming the outer edges inward.

In some configurations, the SDSF can be identified by its dimensions. For example, the curve may include, but is not limited to, a width of about 0.6-0.7 m. In some configurations, point cloud data may be processed to locate SDSFs, and these data may be used to prepare a route for the AV from a starting point to a destination. In some configurations, the path may be included in a map and provided to the perceptual subsystem. When the AV is moving its path, in some configurations, SDSF traversal may be accommodated via sensor-based positioning of the AV, enabled in part by the perceptual subsystem. The perceptual subsystem may execute on at least one processor within the AV.

An AV may include, but is not limited to, a power base comprising two powered front-wheels, two powered back-wheels, an energy storage, and at least one processor. can The power base may be configured to move at a commanded speed. The AV may include a cargo platform mechanically attached to the power base, comprising a plurality of short-range sensors. The AV may in some configurations be mounted on top of a cargo platform and include a cargo container having a volume for receiving one or more objects to be shipped. The AV may include, but is not limited to, a long-range sensor suite mounted on top of the cargo container in some configurations, which may include a LIDAR and one or more cameras. The AV may include a controller capable of receiving data from the long range sensor suite and the short range sensor suite.

The short range sensor suite may optionally detect at least one characteristic of the drivable surface and may optionally include stereo cameras, an IR projector, two image sensors, an RGB sensor, and radar sensors. The short-range sensor suite can optionally feed RGB-D data to the controller. The controller may optionally determine the geometry of the road surface based on RGB-D data received from the short-range sensor suite. The short-range sensor suite may optionally detect objects within 4 meters of the AV, and the long-range sensor suite may optionally detect objects more than 4 meters away from the AV.

The perception subsystem may use the data collected by the sensors to populate the occupancy grid. The occupancy grid of the present teachings may be constructed as a 3D grid of points surrounding an AV, with the AV occupying a central point. In some configurations, the occupancy grid can stretch 10 m to the left, right, rear, and front of the AV. The grid can roughly include the height of the AV, and can virtually move with the AV as it moves, representing obstacles surrounding the AV. A grid can be transformed into two dimensions by reducing its vertical axis, for example, but not limited to being divided into polygons approximately 5 cm x 5 cm in size. Obstacles appearing in 3D space around the AV can be reduced to a 2D shape. If the 2D shape overlaps any segment of one of the polygons, the polygon can be given a value of 100, indicating that space is occupied. Any polygons that are not filled can be given a value of 0, and can be referred to as free space in which AV can move.

As the AV navigates, one may encounter a situation in which a change in the configuration of the AV may be required. In some configurations, the method of the present teachings is for real-time control of a configuration in some configurations, and in some configurations is operatively coupled with at least one of a chassis, at least four wheels, at least four wheels. A method comprising, but not limited to, a first side of the chassis and an opposite second side of the chassis operatively coupled to at least one of the at least four wheels, the method comprising, but not limited to, receiving environmental data, at least in the environmental data determining a surface type based on, determining a mode based at least on the surface type and the first configuration, determining a second configuration based at least on the mode and the surface type, moving instructions based at least on the second configuration and controlling the configuration of the device using move commands to change the device from the first configuration to the second configuration.

The method may optionally include populating the occupancy grid based at least on the surface type and mode. The environment data may optionally include RGB-D image data and a topology of the road surface. The configuration may optionally include clustered two pairs of at least four wheels. A first pair of the two pairs may be located on the first side, and a second pair of the two pairs may be located on the second side. The first pair may include a first front wheel and a first rear wheel, and the second pair may include a second front wheel and a second rear wheel. Control of the configuration may optionally include regulated powering of the first pair and the second pair based at least on the environmental data. The control of the configuration includes the two wheels with the clustered first pair and the clustered second pair rotated to lift the first front wheel and the second front wheel from driving the pair of at least four wheels and retracted casters. It may optionally include switching to actuating. The pair of casters may be operatively coupled with the chassis. The device may rest on a pair of first rear wheels, second rear wheels and casters. Control of the configuration can optionally include rotating a pair of clusters operatively coupled with the two powered wheels on the first side and the two powered wheels on the second side based at least on the environmental data.

A system of the present teachings for real-time control of the configuration of an AV may include, but is not limited to, a device processor and a power base processor. The AV may include a chassis, at least four wheels, a first side of the chassis, and an opposing second side of the chassis. The device processor receives real-time environmental data surrounding the AV, determines a surface type based at least on the environmental data, determines a mode based at least on the surface type and a first configuration, and determines a second mode based on at least the mode and the surface type. 2 configuration can be determined. The power base processor enables the AV to move based at least on the second configuration, and enables the AV to change from the first configuration to the second configuration. The device processor may optionally include populating the occupancy grid based at least on the surface type and mode.

During navigation, the AV may encounter SDSFs that may require steering the AV for successful traversal. In some configurations, a method of the present teachings for navigating an AV along a path line in a movement area towards a target point over at least one SDSF, wherein the AV has a leading edge and a trailing edge. including, but is not limited to, receiving SDSF information and obstacle information for a movement area, detecting at least one candidate SDSF from the SDSF information, and at least one candidate based on at least one selection criterion. selecting the SDSF line from the SDSF line. The method includes determining at least one traversable portion of the selected SDSF line based on at least one location of at least one obstacle found in obstacle information in the vicinity of the selected SDSF line, heading an AV, the AV moving at a first speed towards the at least one traversable portion by turning to move along a line perpendicular to the traversable portion, and continuing heading of the AV based on the relationship between the heading and the vertical line. This may include corrections. The method may include driving the AV at a second speed by adjusting a first speed of the AV based at least on the heading and a distance between the AV and the traversable portion. If the SDSF associated with the at least one traversable portion is raised relative to the surface of the route of travel, the method traverses the SDSF by raising the leading edge relative to the trailing edge, and a third increased speed per degree of elevation ), and driving the AV at the fourth speed until the AV clears the SDSF.

Detecting the at least one candidate SDSF from the SDSF information comprises (a) drawing a closed polygon comprising the location of the AV and the location of the target point, (b) drawing a path line between the target point and the location of the AV; (c) selecting two SDSF points from the SDSF information, wherein the SDSF points are located within a polygon, and (d) drawing an SDSF line between the two points. Detecting the at least one candidate SDSF may include (e) selecting the SDSF points if there are fewer than a first preselected number of points within a first preselected distance of the SDSF line, and drawing a line between them. and repeat steps (c)-(e) if there are fewer than the second preselected number of trials in having fewer than the first preselected number of points around the SDSF line. may include doing Detecting the at least one candidate SDSF comprises (f) fitting a curve to SDSF points that are within a first preselected distance of the SDSF line if there is a first preselected number of points or more points. ), (g) if the first number of SDSF points within the first preselected distance of the curve exceeds the second number of SDSF points within the first preselected distance of the SDSF line, and at which the curve intersects the path line. and identifying the curve as an SDSF line if there are gaps greater than a second preselected distance between SDSF points on the curve. Detecting the at least one candidate SDSF comprises (h) the number of points within the first preselected distance of the curve does not exceed the number of points within the first preselected distance of the SDSF line, or the curve does not intersect the path line. or if there are gaps exceeding a second preselected distance between SDSF points on the curve, and if the SDSF line does not remain stable, and if steps (f)-(h) are repeating steps (f)-(h) if no more than the selected number of attempts have been attempted.

The closed polygon may optionally include a preselected width, and the preselected width may optionally include a width dimension of the AV. Selecting SDSF points may optionally include random selection. The at least one selection criterion may optionally include that a first number of SDSF points within a first preselected distance of the curve exceeds a second number of SDSF points within a first preselected distance of the SDSF line, wherein the curve comprises a path There are no gaps that intersect the line and exceed the second preselected distance between SDSF points on the curve.

Determining the at least one traversable portion of the selected SDSF may optionally include selecting a plurality of obstacle points from the obstacle information. Each of the plurality of obstacle points may include a probability that the obstacle point is associated with at least one obstacle. Determining the at least one traversable portion has a probability greater than a preselected percentage, if any of the plurality of obstacle points lies between the SDSF line and the target point, and if any of the plurality of obstacle points are from the SDSF line. projecting the plurality of obstacle points onto the SDSF line if less than the third preselected distance, forming at least one projection. Determining the at least one traversable portion comprises connecting at least two of the at least one projections to each other, locating end points of the connected at least two projections along an SDSF line, traversing the connected at least two projections. may optionally include marking as a non-traversable section, and marking an SDSF line outside the non-traversable section as the at least one traversable section.

Traversing the at least one traversable portion of the SDSF includes heading the AV, operating at a first speed towards the traversable portion, rotating the AV to move along a line perpendicular to the traversable portion, heading and continuing to correct the heading of the AV based on the relationship between the vertical lines, and driving the AV at a second speed by adjusting the first speed of the AV based at least on the heading and the distance between the AV and the traversable portion; may optionally be included. traversing the at least one traversable portion of the SDSF includes, when the SDSF is raised relative to the surface of the travel route, traversing the SDSF by raising the leading edge relative to the trailing edge and driving the AV at a third increased speed per altitude. , and optionally driving the AV at the fourth speed until the AV clears the SDSF.

Traversing at least one traversable portion of the SDSF may alternatively include: (a) if the heading error is less than a third preselected amount for a line perpendicular to the SDSF line, ignore the updated SDSF information and preselected rate (b) driving the AV forward when the elevation of the front portion of the AV relative to the rear portion of the AV is between the sixth preselected amount and the fifth preselected amount, and increasing the speed of the AV at a speed, (c) driving the AV forward at a seventh preselected speed if the front portion is raised less than a sixth preselected amount for the rear portion, and (d) rearward repeating steps (a)-(d) if the portion is less than or equal to a fifth preselected distance from the SDSF line.

In some configurations, the wheels of SDSF and AV may be automatically aligned to avoid system instability. Auto-alignment may be implemented by, for example, but not limited to, continuously testing and correcting the AV's heading as it approaches the SDSF. Another aspect of the SDSF traversing feature of the present teachings is to automatically verify that there is sufficient free space around the SDSF before the SDSF traversing feature attempts to traverse. Another aspect of the SDSF traversing feature of the present teachings is that it is possible to traverse SDSFs of various geometries. Geometries may include, for example, but not limited to square and outlined SDSFs. The orientation of the AV with respect to the SDSF can determine at what speed and direction the AV travels. The SDSF traversal feature can adjust the speed of the AV near SDSFs. As the AV climbs the SDSF, the speed may be increased to assist the AV traversing the SDSF.

1. An autonomous delivery vehicle comprising: a power base comprising two motorized front wheels, two motorized rear wheels and an energy store, the power base comprising at a commanded speed and configured to move in a commanded direction—; a cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to a power base; a cargo container having a volume for accommodating at least one object, the cargo container mounted on top of a cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suit mounted on top of a cargo container; and a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, wherein the controller determines a commanded speed and a commanded direction based at least on the data, the controller determining a commanded speed and a commanded direction to the power base to complete the transport; An autonomous delivery vehicle comprising: providing commanded direction. 2. The autonomous delivery vehicle of clause 1, wherein data from the plurality of short-range sensors comprises at least one characteristic of the surface on which the power base moves. 3. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors comprises at least one stereo camera. 4. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors comprises at least one IR projector, at least one image sensor, and at least one RGB sensor. 5. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors comprises at least one radar sensor. 6. The autonomous delivery vehicle of clause 1, wherein the data from the plurality of short-range sensors comprises RGB-D data. 7. The autonomous delivery vehicle of clause 1, wherein the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. 8. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors detect objects within 4 meters of the AV, and the long-range sensor suite detects objects that are more than 4 meters away from the autonomous delivery vehicle. 9. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors comprises a cooling circuit. 10. The autonomous delivery vehicle of clause 1, wherein the plurality of short-range sensors comprises an ultrasonic sensor. 11. The autonomous delivery vehicle of clause 2, wherein the controller comprises executable code, the executable code comprising accessing a map formed by a map processor, the map processor accessing point cloud data from the long range sensor suite a first processor, wherein the point cloud data represents a surface; a filter to filter the point cloud data; a second processor forming processable portions from the filtered point cloud data; a third processor for merging the processable portions into the at least one polygon; a fourth processor for locating and labeling at least one substantially discontinuous surface feature (SDSF) in the at least one polygon, wherein the locating and labeling, if present, forms labeled point cloud data; a fifth processor for generating graphing polygons from the labeled point cloud data; and a sixth processor that selects a path from a start point to an end point based at least on the graphing polygons, wherein the AV traverses at least one SDSF along the path. 12. The autonomous delivery vehicle as in clause 11, wherein the filter comprises a seventh processor executing code, wherein the code conditionally removes from the point cloud data points representing transient objects and points representing outliers. ; and replacing the removed points having a preselected height. 13. The autonomous delivery vehicle as in clause 11, wherein the second processor comprises executable code, the executable code comprising: segmenting the point cloud data into processable portions; and removing points of a preselected height from the processable portions. 14. The autonomous delivery vehicle as in clause 11, wherein the third processor comprises executable code, the executable code to reduce the size of the processable portions by analyzing outliers, voxels, and normals; growing regions from reduced size processable portions; determining initial drivable surfaces from the grown regions; segmenting and meshing the initial drivable surfaces; positioning polygons within the segmented and meshed initial drivable surfaces; and setting the at least one drivable surface based at least on the polygons. 15. The autonomous delivery vehicle as in clause 14, wherein the fourth processor comprises executable code, the executable code to sort the point cloud data of the initial drivable surfaces according to the SDSF filter, the SDSF filter comprising comprising at least three categories; and locating the at least one SDSF point based at least on whether the at least three categories of points, in combination, satisfy the at least one first preselected criterion. 16. The autonomous delivery vehicle as in clause 15, wherein the fourth processor comprises executable code, wherein the plurality of at least one SDSF points, in combination, satisfy at least one second preselected criterion. and generating at least one SDSF trajectory based at least on 17. The autonomous delivery vehicle as in clause 14, wherein generating the graphing polygons comprises an eighth processor comprising executable code, wherein the executable code generates the at least one polygon from the at least one drivable surface. to do, wherein at least one polygon includes outer edges; smoothing the outer edges; forming a driving margin based on the smoothed outer edges; adding at least one SDSF trajectory to the at least one drivable surface; and removing the inner edges from the at least one drivable surface according to the at least one third preselected criterion. 18. The autonomous delivery vehicle as in clause 17, wherein smoothing the outer edges comprises a ninth processor comprising executable code, wherein the executable code comprises trimming the outer edges outward to form the outer edges. Including, autonomous delivery vehicles. 19. The autonomous delivery vehicle as in clause 18, wherein forming a driving margin of the smoothed outer edges comprises a tenth processor comprising executable code, wherein the executable code comprises trimming the outer edges inward. Including, autonomous delivery vehicles. 20. The autonomous delivery vehicle as in clause 1, wherein the controller comprises: a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by the autonomous delivery vehicle (AV), wherein the AV is on the surface. moving a path, wherein the surface comprises at least one SDSF, and the path comprises a start point and an end point, wherein the subsystem comprises: a first processor accessing the root topology - the root topology is a filtered point cloud at least one graphing polygon comprising data, the filtered point cloud data comprising labeled features, and the point cloud data comprising a drivable margin; a second processor for converting the point cloud data into a global coordinate system; a third processor that determines boundaries of the at least one SDSF, wherein the third processor creates SDSF buffers of a preselected size around the boundaries; a fourth processor that determines which of the at least one SDSF can be traversed based at least on the at least one SDSF traversal criterion; a fifth processor for generating an edge/weight graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and a base controller that selects a path from a start point to an end point based at least on the edge/weight graph. 21. The autonomous delivery vehicle as in clause 20, wherein the at least one SDSF traversal criterion comprises: a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF; a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV comprising a drivable surface, and a minimum entry distance between the at least one SDSF and the AV that accommodates approximately 90° access to the at least one SDSF by the AV Autonomous delivery vehicles, including streets.

22. A method for managing a global occupancy grid for an autonomous device, the global occupancy grid comprising globally occupied grid cells, wherein the global occupancy grid cells are associated with an occupancy probability, the method comprising: receiving data; generating a local occupancy grid based at least on the sensor data, the local occupancy grid having locally occupied grid cells; accessing historical data associated with the second area when the autonomous device has moved from the first area to the second area; generating a static grid based at least on the historical data; moving the global occupancy grid to maintain the autonomous device in a central position of the global occupancy grid; updating the moved global occupancy grid based on the static grid; if at least one of the globally occupied grid cells matches the location of the autonomous device, then marking at least one of the globally occupied grid cells as unoccupied; calculating, for each of the locally occupied grid cells, the location of the locally occupied grid cell on the global occupied grid; accessing a first occupancy probability from a global occupancy grid cell at the location; accessing a second occupancy probability from a locally occupied grid cell at the location; and calculating a new occupancy probability at the location on the global occupancy grid based at least on the first occupancy probability and the second occupancy probability. 23. The method as in clause 22, further comprising range-checking the new occupancy probability. 24. The method as in clause 23, wherein the range-checking comprises: if the new occupancy probability < 0, setting the new occupancy probability to zero; and setting the new occupancy probability to 1 if the new occupancy probability>1. 25. The method as in clause 22, further comprising setting the global occupancy grid cell to a new occupancy probability. 26. The method as in clause 23, further comprising setting the global occupancy grid cell to a range-checked new occupancy probability.

27. A method for creating and managing occupancy grids, comprising: converting, by a local occupancy grid generating node, sensor measurements into a frame of reference associated with a device; creating a time-stamped measurement occupancy grid; publishing the time-stamped measurement occupancy grid as a local occupancy grid; creating a plurality of local occupancy grids; generating a static occupancy grid based on the surface properties in the repository, the surface properties being associated with a location of the device; moving the global occupancy grid associated with the location of the device to maintain the device and the local occupancy grid approximately centered relative to the global occupancy grid; adding information from the static occupancy grid to the global occupancy grid; marking an area in the global occupancy grid currently occupied by the device as unoccupied; determining, for each of the at least one cell in each local occupancy grid, a location of the at least one cell in the global occupancy grid; accessing a first value at the location; determining a second value at the location based on a relationship between the cell value and the first value in at least one cell in the local occupancy grid; comparing the second value to a preselected range of probability; and setting the global occupancy grid to a new value if the probability value is within the preselected probability range. 28. A method as in clause 27, further comprising posting a global occupancy grid. 29. A method as in clause 27, wherein the surface properties include surface type and surface discontinuities. 30. A method as in clause 27, wherein the relationship comprises a summation. 31. A system for creating and managing occupancy grids, comprising a plurality of local grid generating nodes that create at least one local occupancy grid, wherein at least one local occupancy grid is associated with a location of a device, the at least one local occupancy grid contains at least one cell; a global occupancy grid manager accessing at least one local occupancy grid, the global occupancy grid manager generating a static occupancy grid based on surface properties in the repository, the surface properties associated with a location of the device; move the global occupancy grid associated with the location of the device to maintain the device and the at least one local occupancy grid approximately centered with respect to the global occupancy grid; add information from the static occupancy grid to the at least one global occupancy grid; mark an area in the global occupancy grid currently occupied by the device as unoccupied; determine, for each of the at least one cell in each local occupancy grid, a location of the at least one cell in the global occupancy grid; access a first value at the location; determine a second value at the location based on a relationship between the cell value and the first value in at least one cell in the local occupancy grid; compare the second value to a preselected range of probability; and setting the global occupancy grid to a new value if the probability value is within a preselected probability range.

32. A method for updating a global occupancy grid, comprising: when an autonomous device has moved to a new location, updating the global occupancy grid with information from the static grid associated with the new location; analyzing the surfaces at the new location; if the surfaces are drivable, updating the surfaces and updating the global occupancy grid with the updated surfaces; and updating the global occupancy grid with values from the store of static values, the static values associated with the new location. 33. The method as in clause 32, wherein updating the surfaces comprises: accessing a local occupancy grid associated with the new location; accessing, for each cell in the local occupancy grid, a local occupancy grid surface classification confidence value and a local occupancy grid surface classification; If the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, the global surface classification confidence value in the global occupancy grid is added to the local occupancy grid surface classification confidence value to form a sum, and the cell as the sum updating the global occupancy grid in ; If the local occupancy grid surface classification is not the same as the global surface classification in the global occupancy grid in the cell, subtract the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid to form a difference; updating the global occupancy grid with if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification. 34. The method as in clause 32, wherein updating the global occupancy grid with values from the store of static values comprises, for each cell in the local occupancy grid, from the local occupancy grid, the local occupancy of the cell. accessing the occupancy grid probability value, log odds value; updating the log odds value in the global occupancy grid with the local occupancy grid log odds value in the cell; If the preselected certainty that the cell is not occupied is met, and if the autonomous device is moving within lane barriers, and if the locally occupied grid surface classification indicates a drivable surface, the cell is reducing the log odds occupied in the occupancy grid; increasing the log odds in the local occupancy grid if the autonomous device expects to encounter relatively uniform surfaces, and if the local occupancy grid surface classification indicates a relatively non-uniform surface; and reducing log odds in the local occupancy grid if the autonomous device expects to encounter relatively uniform surfaces, and if the local occupancy grid surface classification indicates a relatively uniform surface.

35. A method for real-time control of a configuration of a device, the device comprising: a chassis, at least four wheels, a first side of the chassis operatively coupled with at least one of the at least four wheels, and at least one of the at least four wheels; an opposing second side of the chassis operatively coupled, the method comprising: generating a map based at least on prior surface features and an occupancy grid, wherein the map is generated non-real-time, the map comprising: comprising a location, wherein the at least one location is associated with the at least one surface feature, the at least one surface feature is associated with the at least one surface classification and the at least one mode; determining current surface features as the device moves; updating the occupancy grid with current surface features in real time; determining, from the occupancy grid and map, a path a device can travel to traverse the at least one surface feature.

36. A method for real-time control of a configuration of a device, the device comprising: a chassis, at least four wheels, a first side of the chassis operatively coupled with at least one of the at least four wheels, and at least one of the at least four wheels; an opposing second side of the chassis operatively coupled, the method comprising: receiving environmental data; determining a surface type based at least on the environmental data; determining a mode based at least on the surface type and the first configuration; determining a second configuration based at least on the mode and the surface type; determining movement instructions based at least on the second configuration; and controlling the configuration of the device using the move commands to change the device from the first configuration to the second configuration. 37. A method as in clause 36, wherein the environmental data comprises RGB-D image data. 38. A method as in clause 36, further comprising: filling the occupancy grid based at least on the surface type and mode; and determining movement instructions based at least on the occupancy grid. 39. A method as in clause 38, wherein the occupancy grid comprises information based at least on data from the at least one image sensor. 40. A method as in clause 36, wherein the environmental data comprises a topology of the road surface. 41. The method as in clause 36, wherein the configuration comprises clustered two pairs of at least four wheels, a first pair of the two pairs positioned on a first side, a second pair of the two pairs is located on the second side, the first pair comprising a first front wheel and a first rear wheel, and the second pair comprising a second front wheel and a second rear wheel. 42. The method as in clause 41, wherein controlling the configuration comprises supplying conditioned power to the first pair and the second pair based at least on the environmental data. 43. The method as in clause 41, wherein controlling the configuration comprises: from driving at least four wheels and a pair of retracted casters, the pair of casters operatively coupled to the chassis, the first front converting the clustered first pair and the clustered second pair to driving the two wheels rotated to lift the wheel and the second front wheel, the device comprising the first rear wheel, the second rear wheel, and the pair of casters placed on the bed. 44. The method as in clause 41, wherein the controlling the configuration is operable with the first two powered wheels on the first side and the second two powered wheels on the second side based at least on the environmental data. rotating a pair of tightly coupled clusters. 45. The method as in clause 36, wherein the device further comprises a cargo container, wherein the cargo container is mounted on the chassis, wherein the chassis controls the height of the cargo container. 46. A method as in clause 45, wherein the height of the cargo container is based at least on environmental data.

47. A system for real-time control of a configuration of a device, wherein the device comprises a chassis, at least four wheels, a first side of the chassis, and an opposite second side of the chassis, the system comprising: real-time environmental data surrounding the device a device processor to receive, the device processor to determine a surface type based at least on the environmental data, the device processor to determine a mode based at least on the surface type and the first configuration, and the device processor to determine a mode based at least on the mode and the surface type. determine the second configuration; and a power base processor that determines move instructions based at least on the second configuration, wherein the power base processor controls the configuration of the device by using the move instructions to change the device from the first configuration to the second configuration. system. 48. The system as in clause 47, wherein the environmental data comprises RGB-D image data. 49. The system as in clause 47, wherein the device processor comprises populating the occupancy grid based at least on the surface type and mode. 50. The system as in clause 49, wherein the power base processor comprises determining movement instructions based at least on the occupancy grid. 51. The system as in clause 49, wherein the occupancy grid comprises information based at least on data from at least one image sensor. 52. The system as in clause 47, wherein the environmental data comprises a topology of the road surface. 53. The system as in clause 47, wherein the configuration comprises clustered two pairs of at least four wheels, a first pair of the two pairs positioned on a first side, a second pair of the two pairs is located on the second side, the first pair having a first front wheel and a first rear wheel, and the second pair having a second front wheel and a second rear wheel. 54. The system as in clause 53, wherein controlling the configuration comprises supplying conditioned power to the first pair and the second pair based at least on the environmental data. 55. The system as in clause 53, wherein control of the configuration comprises, from driving at least four wheels and a pair of retracted casters, the pair of casters operatively coupled to the chassis, the first front wheel and converting the first pair of clustered and the second pair of clustered to driving the two wheels rotated to lift the second front wheel, the device being on the first rear wheel, the second rear wheel, and the pair of casters A system comprising—placed. 56. A method for maintaining a global occupancy grid comprising: locating a first location of an autonomous device; updating the global occupancy grid with at least one occupancy probability value associated with the first location when the autonomous device moves to a second location associated with the global occupancy grid and the local occupancy grid; updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid; updating the global occupancy grid with surface confidences associated with the at least one drivable surface; updating the global occupancy grid with the log odds of at least one occupancy probability value using a first Bayesian function; and adjusting the log odds based at least on the characteristics associated with the second location. and updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid when the autonomous device is maintained in the first location and the global occupancy grid and the local occupancy grid are co-located; updating the global occupancy grid with surface confidences associated with the at least one drivable surface; updating the global occupancy grid with log odds of at least one occupancy probability value using a second Bayesian function; and adjusting the log odds based at least on characteristics associated with the second location. 57. A method as in clause 35, wherein generating the map comprises: accessing point cloud data representing the surface; filtering the point cloud data; forming the filtered point cloud data into processable portions; merging the processable portions into at least one concave polygon; locating and labeling the at least one SDSF in the at least one concave polygon, wherein the locating and labeling forms labeled point cloud data; generating graphing polygons based at least on the at least one concave polygon; and selecting a path from a start point to an end point based at least on the graphing polygons, wherein the AV traverses the at least one SDSF along the path. 58. The method as in clause 57, wherein filtering the point cloud data comprises: conditionally removing points representing transient objects and points representing outliers from the point cloud data; and replacing the removed points having a preselected height. 59. A method as in clause 57, wherein forming the processable portions comprises: segmenting the point cloud data into the processable portions; and removing points of the preselected height from the processable portions. 60. The method as in clause 57, wherein merging the processable portions comprises: reducing the size of the processable portions by analyzing outliers, voxels, and normals; growing regions from the reduced size processable portions; determining initial drivable surfaces from the grown regions; segmenting and meshing the initial drivable surfaces; positioning polygons within the segmented and meshed initial drivable surfaces; and setting the at least one drivable surface based at least on the polygons. 61. The method as in clause 60, wherein locating and labeling the at least one SDSF comprises: sorting the point cloud data of the initial drivable surfaces according to an SDSF filter, wherein the SDSF filter comprises at least three categories of points. including -; and locating the at least one SDSF point based at least on whether the at least three categories of points, in combination, satisfy the at least one first preselected criterion. 62. The method as in clause 61, further comprising generating at least one SDSF trajectory based at least on whether the plurality of at least one SDSF points, in combination, satisfy at least one second preselected criterion. How to. 63. The method as in clause 62, wherein generating the graphing polygons comprises: generating at least one polygon from the at least one drivable surface, wherein the at least one polygon includes outer edges; smoothing the outer edges; forming a driving margin based on the smoothed outer edges; adding at least one SDSF trajectory to the at least one drivable surface; and removing the inner edges from the at least one drivable surface according to the at least one third preselected criterion. 64. A method as in clause 63, wherein smoothing the outer edges comprises trimming the outer edges outward to form the outer edges. 65. The method as in clause 63, wherein forming a driving margin of the smoothed outer edges comprises trimming the outer edges inward.

66. An autonomous delivery vehicle, comprising: a power base comprising two powered front wheels, two powered rear wheels and an energy store, the power base configured to move at a commanded speed; a cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to a power base; a cargo container having a volume for receiving one or more objects to be shipped, the cargo container mounted on top of a cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suit mounted on top of a cargo container; and a controller that receives data from the long-range sensor suite and the plurality of short-range sensors. 67. The autonomous delivery vehicle of clause 66, wherein the plurality of short-range sensors detect at least one characteristic of the drivable surface. 68. The autonomous delivery vehicle of clause 66, wherein the plurality of short-range sensors are stereo cameras. 69. The autonomous delivery vehicle of clause 66, wherein the plurality of short-range sensors comprises an IR projector, two image sensors and an RGB sensor. 70. The autonomous delivery vehicle of clause 66, wherein the plurality of short-range sensors are radar sensors. 71. The autonomous delivery vehicle of clause 66, wherein the short-range sensors supply RGB-D data to the controller. 72. The autonomous delivery vehicle of clause 66, wherein the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. 73. The autonomous delivery vehicle of clause 66, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects that are more than 4 meters away from the autonomous delivery vehicle. .

74. An autonomous delivery vehicle, comprising: a power base comprising at least two powered rear wheels, caster front wheels and an energy storage, wherein the power base is configured to move at a commanded speed; a cargo platform comprising a plurality of short-range sensors, the cargo platform being mechanically attached to a power base; a cargo container having a volume for receiving one or more objects to be shipped, the cargo container mounted on top of a cargo platform; a long-range sensor suite comprising a LIDAR and one or more cameras, the long-range sensor suit mounted on top of a cargo container; and a controller that receives data from the long-range sensor suite and the plurality of short-range sensors. 75. The autonomous delivery vehicle of clause 74, wherein the plurality of short-range sensors detect at least one characteristic of the drivable surface. 76. The autonomous delivery vehicle of clause 74, wherein the plurality of short-range sensors are stereo cameras. 77. The autonomous delivery vehicle of clause 74, wherein the plurality of short-range sensors comprises an IR projector, two image sensors and an RGB sensor. 78. The autonomous delivery vehicle of clause 74, wherein the plurality of short-range sensors are radar sensors. 79. The autonomous delivery vehicle of clause 74, wherein the short-range sensors supply RGB-D data to the controller. 80. The autonomous delivery vehicle of clause 74, wherein the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. 81. The autonomous delivery vehicle of clause 74, wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle, and the long-range sensor suite detects objects that are more than 4 meters away from the autonomous delivery vehicle. . 82. The autonomous delivery vehicle of clause 74, further comprising a second set of powered wheels capable of engaging the ground while the caster wheels are lifted from the ground.

83. An autonomous delivery vehicle, comprising: a power base comprising at least two powered rear wheels, caster front wheels and an energy storage, wherein the power base is configured to move at a commanded speed; a cargo platform mechanically attached to the power base; and a short-range camera assembly mounted to the cargo platform to detect at least one characteristic of the drivable surface, the short-range camera assembly comprising: a camera; first light; and a first liquid cooling heat sink, wherein the first liquid cooling heat sink cools the first light and camera. 84. The autonomous delivery vehicle according to clause 83, wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid cooling heat sink. 85. The autonomous delivery vehicle according to clause 83, wherein the first light and camera are recessed in a cover having openings that deflect illumination from the first light away from the camera. 86. The autonomous delivery vehicle according to clause 83, wherein the lights are angled downwards by at least 15° and are recessed at least 4 mm in the cover to minimize lighting that distracts pedestrians. 87. The autonomous delivery vehicle according to clause 83, wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that are diffused to illuminate the field of view of the camera. vehicle. 88. The autonomous delivery vehicle according to clause 87, wherein the lights are angled approximately 50° apart and the lenses produce a 60° beam. 89. The autonomous delivery vehicle according to clause 83, wherein the short-range camera assembly comprises an ultrasonic sensor mounted above the camera. 90. The autonomous delivery vehicle according to clause 83, wherein the short-range camera assembly is mounted in a central location on the front of the cargo platform. 91. The autonomous delivery vehicle according to clause 83, further comprising at least one corner camera assembly mounted on at least one corner of the front of the cargo platform, the at least one corner camera assembly comprising: an ultrasonic sensor; corner camera; a second light; and a second liquid cooled heat sink, wherein the second liquid cooled heat sink cools the second light and corner camera. 92. A method as in clause 22, wherein the historical data comprises surface data. 93. A method as in clause 22, wherein the historical data comprises discontinuous data.

The present teachings will be more readily understood with reference to the following description taken in conjunction with the accompanying drawings.
1IA is a schematic block diagram of the main components of the system of the present teachings.
1IB is a schematic block diagram of the major components of a map processor of the present teachings.
1IC is a schematic block diagram of the major components of a perceptual processor of the present teachings.
1ID is a schematic block diagram of the major components of an autonomous processor of the present teachings.
1A is a schematic block diagram of a system of the present teachings for preparing a travel path for AV.
1B is a pictorial diagram of an exemplary configuration of a device incorporating the system of the present teachings.
1C is a side view of an automated delivery vehicle showing the fields of view of some long-range and short-range sensors.
1D is a schematic block diagram of a map processor of the present teachings.
1E is a pictorial diagram of a first portion of the flow of a map processor of the present teachings.
1F is an image of a segmented point cloud of the present teachings.
1G is a pictorial diagram of a second portion of a map processor of the present teachings.
1H is an image of a drivable surface detection result of the present teachings.
1I is a pictorial diagram of the flow of the SDSF finder of the present teachings.
1J is a pictorial diagram of SDSF categories of the present teachings.
1K is an image of SDSFs identified by the system of the present teachings.
11 and 1M are pictorial diagrams of polygon processing in the present teachings.
1N is an image of polygons and SDSFs identified by the system of the present teachings.
2A is an isometric view of an autonomous vehicle of the present teachings.
2B is a plan view of a cargo container showing fields of view of a selected one of the long-range sensors;
2C-2F are views of a long range sensor assembly.
2G is a top view of a cargo container showing fields of view of a selected one of the short-range sensors.
2H is an isometric view of a cargo platform of the present teachings.
2I-2L are isometric views of a short-range sensor.
2M-2N are isometric views of an autonomous vehicle of the present teachings.
2O-2P are isometric views of an autonomous vehicle of the present teachings with skin panels removed.
2Q is an isometric view of the autonomous vehicle of the present teachings with a portion of the top panel removed.
2R-2V are diagrams of long-range sensors on an autonomous vehicle of the present teachings.
2W to 2Z are diagrams of an ultrasonic sensor.
2AA-2BB are views of a central short-range camera assembly.
2cc-2dd are views of a corner short range camera assembly.
2EE-2HH are various views of a central short-range camera assembly.
3A is a schematic block diagram of a system of one configuration of the present teachings.
3B is a schematic block diagram of a system of another configuration of the present teachings.
3C is a schematic block diagram of a system of the present teachings capable of initially creating a global occupancy grid.
3D is a pictorial representation of a static grid of the present teachings.
3E and 3F are pictorial representations of the creation of an occupancy grid of the present teachings.
3G is a pictorial representation of a previous occupancy grid of the present teachings.
3H is a pictorial representation of an update of the global occupancy grid of the present teachings.
3I is a flow diagram of a method of the present teachings for publishing a global occupancy grid.
3J is a flow diagram of a method of the present teachings for updating a global occupancy grid.
3K-3M are flow diagrams of another method of the present teachings for updating a global occupancy grid.
4A is a perspective view of a device of the present teachings in various modes.
4B is a schematic block diagram of a system of the present teachings.
4C is a schematic block diagram of driving surface processor components of the present teachings.
4D is a schematic block/figure flow diagram of the process of the present teachings.
4E and 4F are perspective and side views, respectively, of a configuration of a device of the present teachings in a standard mode.
4G and 4H are perspective and side views, respectively, of a configuration of a device of the present teachings in a four-wheel mode.
4I and 4J are perspective and side views, respectively, of a configuration of a device of the present teachings in an elevated four-wheel mode;
4K is a flow diagram of a method of the present teachings.
5A is a schematic block diagram of a device controller of the present teachings.
5B is a schematic block diagram of an SDSF processor of the present teachings.
5C is an image of SDSF accesses identified by the system of the present teachings.
5D is an image of a route topology generated by the system of the present teachings.
5E is a schematic block diagram of modes of the present teachings.
5F-5J are flow diagrams of a method of the present teachings for traversing SDSFs.
5K is a schematic block diagram of a system of the present teachings for traversing SDSFs.
Figures 5L-5N are pictorial representations of the method of Figures 5F-5H.
Fig. 5o is a pictorial representation of converting an image into a polygon.

The system and method of the present teachings can use on-board sensors and previously developed maps to develop an occupancy grid, and include reconstructing the AV based on the surface type and the previous AV across surface features. You can use these aids to navigate to.

Referring now to FIG. 1IA , the AV system 100 may include a structure in which sensors 10701 may be mounted, within which a device controller 10111 may execute. The structure may include a power base 10112 that may direct movement of the wheels that are part of the structure and may enable movement of the AV. The device controller 10111 may execute on at least one processor located on the AV, and may receive data from the sensors 10701 , which may be located on the AV, but not limited thereto. Device controller 10111 may provide speed, direction, and configuration information to base controller 10114 , which may provide movement commands to power base 10112 . The device controller 10111 may receive map information from the map processor 10104 that may prepare a map of an area surrounding the AV. The device controller 10111 may include, but is not limited to, a sensor processor 10703 that may receive and process input from sensors 10701 , including on-AV sensors. In some configurations, the device controller 10111 can include a perceptual processor 2143 , an autonomous processor 2145 , and a driver processor 2127 . Perceptual processor 2143 may, for example, but not be limited to, locate static and dynamic obstacles, determine traffic light conditions, create an occupancy grid, and classify surfaces. The autonomous processor 2145 determines, for example, but not limited to, the maximum speed of the AV, and the AV is navigating to, for example, on a road, on a sidewalk, at an intersection, and/or under remote control. You can determine the type of situation. The driver processor 2127 may, for example, but not be limited to, generate instructions according to the direction of the autonomous processor 2145 and send them to the base controller 10114 .

Referring now to FIG. 1IB , map processor 10104 may generate a map of surface features and may provide the map via device controller 10111 to perceptual processor 2143 , which may update the occupancy grid. The map processor 10104 includes, among many other aspects, a feature extractor 10801, a point cloud organizer 10803, a transient processor 10805, a segmenter 10807, a polygon generator 10809; SDSF line generator 10811 , and combiner 10813 . The feature extractor 10801 may include a first processor that accesses point cloud data representing the surface. The point cloud organizer 10803 may include a second processor that forms processable portions from the filtered point cloud data. The transient processor 10805 may include a first filter to filter the point cloud data. Segmenter 10807 includes executable code that may include, but is not limited to, segmenting the point cloud data into processable portions, and removing points of a preselected height from the processable portions. can do. The first filter may include, but is not limited to, conditionally removing points representing transient objects and points representing outliers from the point cloud data, and replacing the removed points with a preselected height. You can optionally include enable codes. The polygon generator 10809 may include a third processor that merges the processable portions into at least one concave polygon. A third processor includes, but is not limited to, reducing the size of the processable portions by analyzing outliers, voxels, and normals, growing regions from the reduced size processable portions, grown regions determining initial drivable surfaces from, segmenting and meshing the initial drivable surfaces, locating polygons within the segmented and meshed initial drivable surfaces, and drivable based at least on the polygons It may optionally include executable code that may include setting up surfaces. The SDSF line generator 10811 may include a fourth processor for locating and labeling the at least one SDSF in the at least one concave polygon, the locating and labeling forming labeled point cloud data. A fourth processor includes, but is not limited to, sorting the point cloud data of the drivable surfaces according to the SDSF filter, the SDSF filter comprising at least three categories of points, and the categories of points, in combination, at least may optionally include executable code that may include locating the at least one SDSF point based at least on whether the one first preselected criterion is met. The combiner 10813 may include a fifth processor that generates graphing polygons. Generating the graphing polygons includes, but is not limited to, generating at least one polygon from at least one drivable surface, wherein the at least one polygon includes edges, smoothing edges, smoothed edges forming a driving margin based on , adding at least one SDSF trajectory to the at least one drivable surface, and removing edges from the at least one drivable surface according to at least one third preselected criterion. It can optionally include executable code that it can include. Smoothing the edges may optionally include executable code, which may include, but is not limited to, trimming the edges outward. Forming the driving margin of the smoothed edges may optionally include executable code, which may include, but is not limited to, trimming the outer edges inward.

Referring now to FIG. 1IC , among other things, when the AV delivers goods, it receives on-board sensors, powered wheels, sensor and map data and uses these data to power the AV to traverse various types of surfaces. Maps may be provided to the AV, which may include processors for configuring. On-board sensors can populate the occupancy grid and provide data that can be used to detect dynamic obstacles. Occupancy grids can also be populated by maps. The device controller 10111 may include a perceptual processor 2143 that may receive and process sensor data and map data, and update the occupancy grid with these data.

Referring now to FIG. 1ID , the device controller 10111 can include a configuration processor 41023 that can automatically determine the configuration of the AV based at least on the mode of the AV and the encountered surface features. Autonomic processor 2145 determines what kind of surface needs to be traversed and what configurations the AV needs to traverse, based at least on the map (planned route to be followed), information from configuration processor 41023, and the mode of the AV. may include a control processor 40325 that may determine whether it needs to assume in order to traverse the . The autonomous processor 2145 may supply instructions to the motor drive processor 40326 to implement the instructions.

Referring now to FIG. 1A , map processor 10104 is a navigation device in environments where a device, eg, but not limited to an AV or semi-autonomous device, may include features such as SDSFs. Gating may be enabled. Features in the map, along with on-board sensors, may enable the AV to move on various surfaces. In particular, SDSFs can be accurately identified and labeled so that the AV can automatically maintain the performance of the AV during entry and exit of the SDSF, and AV speed, configuration, and direction can be controlled for safe SDSF traversal.

With continued reference to FIG. 1A , in some configurations, system 100 for managing traversal of SDSFs includes an AV 10101 , a core cloud infrastructure 10103 , AV services 10105 , a device controller 10111 , sensor(s) 10701 , and a power base 10112 . AV 10101 may provide transport and escort services from source to destination along a dynamically determined route, for example, but not limited to, as modified by incoming sensor information. can AV 10101 may include, but is not limited to, devices having autonomous modes, devices capable of fully autonomous operation, devices capable of at least partially remotely operated devices, and combinations of these features. may include devices. Transportation device services 10105 may provide drivable surface information including features to device controller 10111 . The device controller 10111 may modify the drivable surface information according to at least, but not limited to, incoming sensor information and feature traversing requirements, and based on the modified drivable surface information, the AV 10101 ) can be selected. Device controller 10111 may present commands to power base 10112 that may instruct power base 10112 to provide speed, direction, and configuration commands to wheel motors and cluster motors, the commands being Causes the AV 10101 to follow the selected path, raising and lowering its cargo accordingly. Transport device services 10105 may access route related information from core cloud infrastructure 10103 , which may include, but is not limited to, storage and content distribution facilities. In some configurations, core cloud infrastructure 10103 may include commercial products such as, but not limited to, for example, but not limited to AMAZON WEB SERVICES®, GOOGLE CLOUD ^™ , and ORACLE CLOUD®.

Referring now to FIG. 1B , an exemplary AV that may include a device controller 10111 ( FIG. 1A ) capable of receiving information from a map processor 10104 ( FIG. 1A ) of the present teachings is, for example, limited to Power Base, including but not limited to, U.S. Patent Application Serial No. 16/035,205, filed July 13, 2018, for the name Mobility Device, or 8, 2001 for Control System and Method. It may include a power base assembly, such as the power base fully described in US Pat. No. 6,571,892, filed Jan. 15, all of which are incorporated herein by reference in their entirety. An exemplary power base assembly is not intended to limit the present teachings, but is instead described herein to clarify features of any power base assembly that may be useful in implementing the techniques of the present teachings. An exemplary power base assembly can optionally include a power base 10112 , a wheel cluster assembly 11100 , and a payload carrier height assembly 10068 . The exemplary power base assembly can selectively provide electrical and mechanical power to drive the wheels 11203 and the clusters 11100 that can raise and lower the wheels 11203 . The power base 10112 may control rotation of the cluster assembly 11100 and lift of the payload carrier height assembly 10068 to support the substantially discontinuous surface traversing of the present teachings. Other such devices may be used to accommodate SDSF detection and traversal of the present teachings.

Referring again to FIG. 1A , in some configurations, sensors within the exemplary power base may detect the orientation and rate of change of orientation of AV 10101 , motors may enable servo operation, and controllers may include internal It can assimilate information from sensors and motors. Appropriate motor commands may be calculated to achieve vehicle performance and implement path following commands. Left and right wheel motors may drive the wheels on either side of AV 10101 . In some configurations, the front and rear wheels can be coupled to drive together, such that the two left wheels can drive together and the two right wheels can drive together. In some configurations, rotation may be achieved by driving the left and right motors at different rates, and the cluster motor may rotate the wheelbase in a forward/rear direction. This may allow the AV 10101 to remain level while the front wheels are higher or lower than the rear wheels. This feature can be useful, for example, but not limited to, when climbing up and down SDSFs. The payload carrier 10173 may be automatically raised and lowered based at least on the underlying terrain.

With continued reference to FIG. 1A , in some configurations, the point cloud data may include route information for an area to which the AV 10101 will move. Possibly the point cloud data collected by a mapping device similar to or identical to AV 10101 may be time tagged. A path along which the mapping device moves may be referred to as a mapped trajectory. The point cloud data processing described herein may occur when the mapping device traverses a mapped trajectory, or after point cloud data collection is complete. After the point cloud data is collected, they may undergo initial filtering and point cloud data processing, which may include point reduction, point cloud segmentation, and feature detection, as described herein. In some configurations, core cloud infrastructure 10103 may provide long-term or short-term storage for the collected point cloud data, and may provide data to AV services 10105 . AV services 10105 may select from among possible point cloud datasets to find a dataset that covers an area surrounding a desired starting point for AV 10101 and a desired destination for AV 10101 . AV services 10105 may include, but are not limited to, a map processor 10104 that may reduce the size of the point cloud data and determine features represented in the point cloud data. In some configurations, map processor 10104 can determine the location of SDSFs from point cloud data. In some configurations, polygons may be created from the point cloud data as a technique for segmenting the point cloud data and ultimately establishing a drivable surface. In some configurations, SDSF finding and drivable surface determination may proceed in parallel. In some configurations, SDSF finding and drivable surface determination may proceed sequentially.

Referring now to FIG. 1C , in some configurations, the AV may be configured to deliver cargo and/or perform other functions involving autonomously navigating to a desired location. In some applications, the AV may be guided remotely. In some configurations, AV 201000 includes a cargo container that can be opened remotely, in response to user inputs, automatically or manually, to allow users to place or remove packages and other items. The cargo container 20110 is mounted on the cargo platform 20160, which is mechanically connected to the power base 20170. The power base 20170 includes four power wheels 20174 and two caster wheels 20176 . The power base provides speed and direction control for moving the cargo container 201110 along the ground and over obstacles including curves and other discontinuous surface features.

With continued reference to FIG. 1C , the cargo platform 20160 is connected to the power base 20170 via two U-frames 20162 . Each U-frame 20162 is rigidly attached to the structure of the cargo platform 20160 such that a rotatable joint 20164 is formed with the end of each arm 20172 on the power base 20170. It contains two holes for The power base controls the rotational position of the arms, thus controlling the height and posture of the cargo container 201110.

With continued reference to FIG. 1C , in some configurations, AV 201000 includes one or more processors that receive data, navigate a route, and select a direction and speed of power base 20170 .

Referring now to FIG. 1D , in some configurations, a map processor 10104 of the present teachings may place SDSFs on a map. Map processor 10104 includes, but is not limited to, feature extractor 10801 , point cloud organizer 10803 , transient processor 10805 , segmenter 10807 , polygon generator 10809 , SDSF line generator 10811 . , and a data combiner 10813 .

With continued reference to FIG. 1D , feature extractor 10801 ( FIG. 1IB ) may include, but is not limited to, gaze filtering 10121 and mapped trajectory 10133 of point cloud data 10131 . Gaze filtering removes hidden points from the direct gaze of sensors to collect point cloud data and form a mapped trajectory. The point cloud organizer 10803 ( FIG. 1IB ) may organize 10151 the reduced point cloud data 10132 according to preselected criteria possibly associated with a particular feature. In some configurations, the transient processor 10805 ( FIG. 1IB ) removes 10153 transient points from the mapped trajectory 10133 and the point cloud data organized by any number of methods, including the method described herein. )can do. Transient points can complicate the process, especially when certain features are stationary. The segmenter 10807 ( FIG. 1IB ) may partition the processed point cloud data 10135 into processable chunks. In some configurations, the processed point cloud data 10135 may be segmented 10155 into sections having a preselected minimum number of points, such as, but not limited to, about 100,000 points. In some configurations, the further point reduction may be based on preselected criteria that may relate to the features to be extracted. For example, if points above a certain height are not critical to the location of a feature, those points may be deleted from the point cloud data. In some configurations, the height of at least one of the sensors collecting point cloud data can be considered an origin, the points above the origin being, for example, when only points of interest are associated with surface features. may be removed from the point cloud data. After the filtered point cloud data 10135 has been segmented and formed segments 10137 , the remaining points can be divided into drivable surface sections, and surface features can be positioned. In some configurations, the polygon generator 10809 ( FIG. 1IB ) generates drivable surfaces by, for example, but not limited to, generating 10161 polygons 10139 as described herein. position can be determined. In some configurations, the SDSF line generator 10811 ( FIG. 1IB ) is a surface feature by generating 10163 SDSF lines 10141 , such as, but not limited to, as described herein. can be positioned. In some configurations, combiner 10813 ( FIG. 1IB ) generates an actual path that AV 10101 ( FIG. 1A ) can travel on by combining polygons 10139 and SDSFs 10141 by combining 10165 . data sets that can be further processed for

Referring now primarily to FIG. 1E , from the point cloud data 10131 ( FIG. 1D ), removing objects transient to the mapped trajectory 10133 , such as exemplary time-stamped points 10751 . 10153 (FIG. 1D) is a ray from time-stamped points on mapped trajectory 10133 to each time-stamped point in point cloud data 10131 (FIG. 1D) with substantially the same time stamp. 10753) can be included. If ray 10753 intersects a point between the time-stamped point on mapped trajectory 10133 and the end point of ray 10753, e.g., point D 10755, then point of intersection D 10755 is It can be assumed that the point cloud data was entered during different sweeps of the camera. An intersection point, e.g., intersection point D 10755, may be assumed to be part of a transient object, and may be removed from the reduced point cloud data 10132 (FIG. 1D) as not representing a fixed feature such as SDSF. there is. The result is, for example, but not limited to, processed point cloud data 10135 ( FIG. 1D ) free of transient objects. Points that have been removed as portions of transient objects, but are also substantially at ground level, may be returned 10754 as processed point cloud data 10135 ( FIG. 1D ). Transient objects cannot include certain features, such as, but not limited to, SDSFs 10141 ( FIG. 1D ), and thus point when SDSFs 10141 ( FIG. 1D ) are the detected features. Cloud data 10131 ( FIG. 1D ) can be removed without disturbing the integrity.

With continued reference to FIG. 1E , the segmentation 10155 ( FIG. 1D ) processed point cloud data 10135 ( FIG. 1D ) is divided into sections 10757 having a preselected size and shape, e.g., limited to Although not, it is possible to create rectangles 10154 ( FIG. 1F ) having a minimum preselected side length and containing about 100,000 points. From each section 10757, points not required for a particular task, such as, but not limited to, points that are above a pre-selected level may be removed 10157 to reduce the dataset size ( Fig. 1d). In some configurations, the preselected level may be the height of the AV 10101 ( FIG. 1A ). Removing these points can result in more efficient processing of the dataset.

Referring back primarily to FIG. 1D , the map processor 10104 generates at least one dataset that may be used to generate direction, speed and configuration commands for controlling the AV 10101 ( FIG. 1A ) to the device controller 10111 . can be supplied to The at least one dataset may include points that can be connected to other points in the dataset, wherein each of the lines connecting the points in the dataset traverses the drivable surface. To determine these route points, the segmented point cloud data 10137 can be divided into polygons 10139 , and the vertices of polygons 10139 can possibly be route points. Polygons 10139 may include features such as, for example, SDSF 10141 .

With continued reference to FIG. 1D , in some configurations generating the processed point cloud data 10135 may include filtering the voxels. To reduce the number of points subject to future processing, in some configurations, the centroid of each voxel in the dataset may be used to approximate the points within the voxel, and all points except the centroid will be removed from the point cloud data. can In some configurations, the center of the voxel can be used to approximate points within the voxel. Other methods of reducing the size of the filtered segments 10251 ( FIG. 1G ) may be used, such as, but not limited to, taking random point subsamples, and thus the filtered segments 10251 ) (Fig. 1g) a fixed number of points randomly selected uniformly can be removed.

With further reference to FIG. 1D , in some configurations generating the processed point cloud data 10135 may include calculating normals from a dataset in which outliers have been removed and downsized via voxel filtering. there is. The normals for each point in the filtered dataset can be used for a variety of processes, including curve reconstruction algorithms. In some configurations, estimating and filtering normals in a dataset may include obtaining an underlying surface from the dataset using surface meshing techniques, and calculating normals from the surface mesh. In some configurations, estimating normals may include, but is not limited to, determining a normal to a fitting plane obtained by applying a total least squares method to the k neighbors closest to a point, such as, but not limited to, a dataset may involve using approximations to directly infer surface normals from In some configurations, the value of k may be selected based at least on empirical data. Filtering the normals may include removing any normals that are greater than about 45° from normal to the x-y plane. In some configurations, a filter may be used to align normals in the same direction. If a portion of the dataset represents a planar surface, redundant information contained in adjacent normals may be filtered out by performing random subsampling, or by filtering a point in one of the related sets of points. In some configurations, selecting the point may include recursively decomposing the dataset into boxes until each box contains at most k points. A single normal can be computed from the k points in each box.

및

과 같은 알고리즘들에 의해 달성될 수 있다.With continued reference to FIG. 1D , in some configurations, generating the processed point cloud data 10135 comprises growing regions within the dataset by clustering points that are geometrically compatible with the surface represented by the dataset. and fine-tuning the surface as the region grows to obtain the best approximation of the largest number of points. Region growth can merge points in terms of smoothness constraints. In some configurations, the smoothness constraint may be determined empirically, for example, or may be based on a desired surface smoothness. In some configurations, the smoothness constraint can include a range from about 10π/180 to about 20π/180. The output of region growth is a set of point clusters, each point cluster being a set of points, each of which is considered to be part of the same smooth surface. In some configurations, region growth can be based on comparison of angles between normals. Region growth may include, but is not limited to, region growth segmentation.

and

This can be achieved by algorithms such as

Referring now to FIG. 1G , segmented point cloud data 10137 ( FIG. 1D ) may be used to create polygons 10759 , eg, 5m×5m polygons 10161 ( FIG. 1D ). Point sub-clusters may be transformed into polygons 10759 using, for example, meshing. Meshing includes, for example, but not limited to marching cubes, marching tetrahedrons, surface nets, greedy meshing, and dual contouring. ) can be achieved by standard methods such as In some configurations, polygons 10759 can be created by projecting a point's local neighborhood along the point's normal and connecting the unconnected points. The resulting polygons 10759 are based at least on the size of the neighborhood, the maximum acceptable distance to the point to be considered, the maximum edge length for the polygon, the minimum and maximum angles of the polygons, and the maximum deviation the normals can take from each other. can do. In some configurations, polygons 10759 may be filtered according to whether polygons 10759 are too small for AV 10101 ( FIG. 1A ) to pass through. In some configurations, a circle may be dragged around each of polygons 10759 by means of which the size of AV 10101 ( FIG. 1A ) is known. If the circle is substantially within polygon 10759 , polygon 10759 and thus the resulting drivable surface can receive AV 10101 ( FIG. 1A ). In some configurations, the area of polygon 10759 can be compared to the footprint of AV 10101 ( FIG. 1A ). The polygons are assumed to be irregular, so that the first step for determining the area of the polygons 10759 can separate the polygon 10759 into regular polygons 10759A by known methods. For each regular polygon 10759A, standard area equations may be used to determine its size. The regions of each regular polygon 10759A can be added together to find the region of polygon 10759 , which can be compared to the footprint of AV 10101 ( FIG. 1A ). The filtered polygons may include a subset of polygons that satisfy size criteria. The filtered polygons can be used to establish the final drivable surface.

에서 이용가능한 것들과 같은 통계적 분석 기술들과 같은 종래의 수단에 의해 이상치를 제거함으로써 처리될 수 있다. 필터링은, 제한적인 것은 아니지만, 포인트 클라우드 라이브러리,

에서 이용가능한 것과 같은 복셀화된 그리드 접근법(voxelized grid approach)을 포함하는 종래의 수단에 의해 세그먼트들(10137)(도 1d)를 다운사이징하는 것을 포함할 수 있다. 오목 다각형들(10263)은, 예를 들어, 제한적인 것은 아니지만, A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets, Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012에 제시된 프로세스에 의해 생성될 수 있다.With continued reference to FIG. 1G , in some configurations, polygon 10759 can be, for example, but not limited to, a Point Cloud Library;

can be dealt with by removing outliers by conventional means, such as statistical analysis techniques such as those available in Filtering includes, but is not limited to, point cloud libraries,

downsizing the segments 10137 ( FIG. 1D ) by conventional means including a voxelized grid approach such as that available in . Concave polygons 10263 are described, for example, but not limited to, A New Concave Hull Algorithm and Concaveness Measure for n-dimensional Datasets, Park et al., Journal of Information Science and Engineering 28, pp. 587-600, 2012.

Referring now primarily to FIG. 1H , in some configurations, processed point cloud data 10135 ( FIG. 1D ) may be used to determine an initial drivable surface 10265 . Area growth can create point clusters that can include points that are part of a drivable surface. In some configurations, a reference plane can be fitted to each of the point clusters to determine the initial drivable surface. In some configurations, point clusters may be filtered according to a relationship between the orientation of the point clusters and a reference plane. For example, if the angle between the point cluster plane and the reference plane is, for example, but not limited to, less than about 30°, the point cluster may, preliminarily, be considered to be part of the initial drivable surface. . In some configurations, point clusters may be filtered based on, for example, but not limited to, a size constraint. In some configurations, point clusters with a point size greater than about 20% of all points in point cloud data 10131 ( FIG. 1D ) may be considered too large, and point clusters in point cloud data 10131 ( FIG. 1D ) may be considered too large. Point clusters smaller in size than about 0.1% of all points may be considered too small. The initial drivable surface may include a filtered out of point clusters. In some configurations, point clusters may be partitioned to continue further processing by any of several known methods. In some configurations, density based spatial clustering of applications with noise (DBSCAN) may be used to partition point clusters, whereas in some configurations, k-means clustering is It can be used to partition clusters. DBSCAN can group together points that are closely packed together, and mark points that are substantially isolated or in low-density regions as outliers. In order to be considered closely packed, a point must be within a preselected distance from the candidate point. In some configurations, the scaling factor for the preselected distance may be determined empirically or dynamically. In some configurations, the scaling factor may be in the range of about 0.1 to 1.0.

Referring primarily to FIG. 1I , generating SDSF lines 10163 ( FIG. 1D ) includes locating the SDSFs by further filtering of concave polygons 10263 on the drivable surface 10265 ( FIG. 1H ). can do. In some configurations, the points from the point cloud data that make up the polygons are categorized as upper donut point 10351 ( FIG. 1J ), lower donut point 10353 ( FIG. 1J ), or cylindrical point 10355 ( FIG. 1J ). can be pissed off The upper donut points 10351 ( FIG. 1J ) may fall within the shape of the SDSF model 10352 furthest from the ground. The lower donut points 10353 ( FIG. 1J ) may fall within the shape of the SDSF model 10352 closest to the ground or at ground level. Cylindrical points 10355 ( FIG. 1J ) may fall within the shape between upper donut points 10351 ( FIG. 1J ) and lower donut points 10353 ( FIG. 1J ). The combination of categories can form donut 10371 . To determine whether the donuts 10371 form the SDSF, certain criteria are tested. For example, in each donut 10371 , there should be a minimum number of points that are top donut points 10351 ( FIG. 1J ) and a minimum number of points that are bottom donut points 10353 ( FIG. 1J ). In some configurations, the minimums may be chosen empirically and may fall within the range of about 5-20. Each donut 10371 may be divided into multiple portions, eg, two hemispheres. Another criterion for determining whether the points in donut 10371 represent SDSF is whether most of the points are within opposite hemispheres of portions of donut 10371 . The cylindrical points 10355 ( FIG. 1J ) may occur in the first cylindrical region 10357 ( FIG. 1J ) or the second cylindrical region 10359 ( FIG. 1J ). Another criterion for SDSF selection is that there must be a minimum number of points in both cylindrical regions 10357/10359 (FIG. 1J). In some configurations, the minimum number of points may be selected empirically and may fall within the range of 3-20. Another criterion for SDSF selection is that donut 10371 falls into three categories of points: upper donut point 10351 (FIG. 1J), lower donut point 10353 (FIG. 1J), and cylindrical point 10355 ( 1j) should include at least two of them.

With continued reference primarily to FIG. 1I , in some configurations polygons may be processed in parallel. Each category worker 10362 may search its assigned polygon for SDSF points 10789 ( FIG. 1N ) and SDSF points 10789 to categories 10763 ( FIG. 1G ). (FIG. 1n) can be assigned. When the polygons are processed, the resulting point categories 10763 ( FIG. 1G ) can be combined 10363 to form combined categories 10366 , which can be shortened 10365 to form shortened combined categories. (10368) may be formed. Shortening the SDSF points 10789 ( FIG. 1N ) may include filtering the SDSF points 10789 ( FIG. 1N ) for their distances from the ground. The shortened combined categories 10368 can be averaged by searching the area around each SDSF point 10766 (FIG. 1G) and generating average points 10765 (FIG. 1G), possibly averaging Processed in parallel by workers 10373 , the points in the category form an averaged set of donuts 10375 . In some configurations, the radius around each SDSF point 10766 ( FIG. 1G ) may be determined empirically. In some configurations, the radius around each SDSF point 10766 ( FIG. 1G ) may include a range from 0.1 m to 1.0 m. The change in height between one point and another on the SDSF trajectory 10377 ( FIG. 1G ) may be calculated for the SDSF at the average point 10765 ( FIG. 1G ). Connecting the averaged donuts 10375 together can create the SDSF trajectory 10377 ( FIGS. 1G and 1K ). When generating the SDSF trajectory 10377 ( FIGS. 1G and 1K ), if there are two next candidate points within the search radius of the starting point, the next point is a possible between the previous line segments, the starting point and the candidate destination point. may be selected based at least on forming a straight line, wherein the candidate next point represents the smallest change in SDSF height between the previous points and the candidate next point. In some configurations, the SDSF height may be defined as the difference between the height of the upper donut 10351 ( FIG. 1J ) and the lower donut 10353 ( FIG. 1J ).

에서 발견되는 것들과 같은 알려진 방법들을 이용하여 달성될 수 있다. 병합된 다각형(10771)은 에지들을 평활화하고 확장된 다각형(10772)을 형성하도록 확장될 수 있다. 확장된 다각형(10772)은 드라이빙 마진을 제공하도록 수축될 수 있어, SDSF 궤적들(10377)(도 1m)이 추가될 수 있는 수축된 다각형(10774)을 형성한다. 안쪽으로의 트리밍(수축)은 AV(10101)(도 1a)의 크기에 적어도 기초하여 미리 선택된 양만큼 드라이빙가능 표면의 크기를 감소시킴으로써 AV(10101)(도 1a)가 이동할 에지들 근처에 공간이 있는 것을 보장할 수 있다. 다각형 확장 및 수축은, 예를 들어, 제한적인 것은 아니지만, ARCGIS® 클립 명령 (http://desktop.arcgis.com/en/arcmap/10.3/manage-data/editing-existing-features/clipping-a-polygon-feature.htm)과 같은 상업적으로 이용가능한 기술에 의해 달성될 수 있다.Referring now primarily to FIG. 1L , combining concave polygons and SDSF lines 10165 ( FIG. 1D ) is a dataset comprising polygons 10139 ( FIG. 1D ) and SDSFs 10141 ( FIG. 1D ). , and the dataset can be manipulated to create graphing polygons with SDSF data. Manipulating the concave polygons 10263 may include, but is not limited to, merging the concave polygons 10263 to form a merged polygon 10771 . Merging the concave polygons 10263 may include, but is not limited to,

can be accomplished using known methods such as those found in The merged polygon 10771 may be expanded to smooth the edges and form an expanded polygon 10772 . The expanded polygon 10772 can be contracted to provide driving margin, forming a contracted polygon 10774 to which SDSF trajectories 10377 (FIG. 1M) can be added. The inward trimming (retraction) reduces the size of the drivable surface by a pre-selected amount based at least on the size of the AV 10101 ( FIG. 1A ) so that there is space near the edges on which the AV 10101 ( FIG. 1A ) will move. can guarantee that there is Polygon expansion and contraction can be performed by, for example, but not limited to, the ARCGIS® clip command (http://desktop.arcgis.com/en/arcmap/10.3/manage-data/editing-existing-features/clipping-a- polygon-feature.htm) can be achieved by commercially available technologies.

에서 발견되는 것들을 포함할 수 있다. SDSF 궤적(10377)은 다각형 정점들(10781)에 접속될 수 있는 SDSF 포인트들(10789)(도 1n)를 포함할 수 있다. 정점들(10781)은 AV(10101)(도 1a)에 대한 가능한 이동 경로들을 형성하기 위해 서로 접속될 수 있는 가능한 경로 포인트들인 것으로 간주될 수 있다. 데이터세트에서, SDSF 포인트들(10789)(도 1n)은 그와 같이 라벨링될 수 있다. 분할이 진행됨에 따라, 예를 들어, 제한적인 것은 아니지만, 에지들(10777 및 10779)과 같은 중복 에지들이 도입되는 것이 가능하다. 에지들(10777 또는 10779) 중 하나를 제거하는 것은 추가 분석의 복잡성을 감소시킬 수 있고 다각형 메시를 유지할 수 있다. 일부 구성들에서, Hertel-Mehlhorn 다각형 분할 알고리즘은 에지를 제거하고, 피처들로서 라벨링된 에지들을 스킵하는데 이용될 수 있다. 라벨링된 피처들을 포함한 다각형들(10778)의 세트는 가능한 경로 포인트들의 수를 감소시키기 위해 추가의 단순화를 겪을 수 있고, 가능한 경로 포인트들은 점유 그리드를 채우는데 이용될 수 있는 주석달린 포인트 데이터(10379)(도 5b)의 형태로 디바이스 제어기(10111)(도 1a)에 제공될 수 있다.Referring now primarily to FIG. 1M , the collapsed polygon 10774 may be divided into polygons 10778 , each of which may be traversed without encountering a non-drivable surface. The constricted polygon 10774 is optimized by, for example, but not limited to, z-order curve hashing and expanded to handle holes, twisted polygons, degenerates, and self-intersections. It can be divided by conventional means such as ear slicing. Commercially available ear slicing implementations include, but are not limited to,

may include those found in SDSF trajectory 10377 may include SDSF points 10789 ( FIG. 1N ), which may be connected to polygonal vertices 10781 . Vertices 10781 may be considered to be possible path points that may be connected to each other to form possible travel paths for AV 10101 ( FIG. 1A ). In the dataset, SDSF points 10789 ( FIG. 1N ) may be labeled as such. As the segmentation progresses, it is possible for overlapping edges to be introduced, such as, for example, but not limited to, edges 10777 and 10779 . Removing one of the edges 10777 or 10779 may reduce the complexity of further analysis and preserve the polygonal mesh. In some configurations, the Hertel-Mehlhorn polygon segmentation algorithm can be used to remove an edge and skip edges labeled as features. The set of polygons 10778 containing labeled features may undergo further simplification to reduce the number of possible path points, the possible path points being annotated point data 10379 that may be used to populate the occupancy grid. It may be provided to the device controller 10111 (FIG. 1A) in the form of (FIG. 5B).

Referring now to FIGS. 2A and 2B , the sensor data collected by the AV may also be used to populate the occupancy grid. The processors in the AV are from sensors in the long-range sensor assembly 20400 mounted on top of the cargo container 20110 and from the short-range sensors 20510 , 20520 , 20530 , 20540 located in the cargo platform 20160 and other It can receive data from sensors. The processors may also receive data from an optional short-range sensor 20505 mounted near the top of the front of the cargo container 201110 . The processors may also receive data from one or more antennas 20122A, 20122B ( FIG. 1C ) including cellular, WiFi, and/or GPS. In one example, AV 201000 includes GPS antenna 20122A ( FIG. 1C ) positioned on top of long range sensor assembly 20400 and/or antenna 20122B positioned on top of cargo container 201110 ( FIG. 1C ). ) has Processors may be located anywhere within AV 201000 . In some examples, one or more processors may be located in the long range sensor assembly 20400 . Additional processors may be located in the cargo platform 20160 . In other examples, the processors may be located in the cargo container 201110 and/or as part of the power base 20170 .

With continued reference to FIGS. 2A and 2B , the long range sensor assembly 20400 is mounted on top of the cargo container to provide an improved view of the environment surrounding the AV. In one example, the long range sensor assembly 20400 is more than 1.2 m feet above the moving surface or ground. In other examples where the cargo container is taller or the power base configuration elevates the cargo platform 20160 , the long range sensor assembly 20400 may be 1.8 m above the ground the AV is moving upwards. The long range sensor assembly 20400 provides information about the environment around the AV from a minimum distance to a maximum range. The minimum distance may be defined by the relative positions of the long-range sensors 20400 and the cargo container 201110 . The minimum distance may be further defined by the field of view (FOV) of the sensors. The maximum distance may be defined by the processors and/or by the range of the long-range sensors in the long-range sensor assembly 20400 . In one example, the range of long-range sensors is limited to 20 meters. In one example, the Velodyne Puck LIDAR has a range of up to 100 m. The long range sensor assembly 20400 can provide data about objects in all directions. The sensor assembly may provide information about structures, surfaces, and obstacles over a 360° angle around the AV 201000 .

With continued reference to FIG. 2A , three long range cameras viewing through windows 20434 , 20436 and 20438 can provide horizontal FOVs 20410 , 20412 , 20414 together providing a 360° FOV. The horizontal FOV may be defined by the selected camera and the location of the cameras within the long range camera assembly 20400 . In describing the fields of view, a zero angle is a ray located in a vertical plane that is through the center of the AV 2010 and is perpendicular to the front of the AV. The zero angle ray passes in front of the AV. The forward long range camera looking through window 20434 has a 96° FOV 20410 of 311° to 47°. The left long-range camera looking through window 20436 has a FOV 20412 between 47° and 180°. The right long-range camera looking through window 20438 has a FOV 20414 of 180° to 311°. The long range sensor assembly 20400 may include an industrial camera positioned to view through a window 20432 that provides more detailed information about objects and surfaces in front of the AV 2020 than the long range cameras. An industrial camera located behind the window 20432 may have a FOV 20416 defined by the selected camera and the location of the cameras within the long range camera assembly 20400 . In one example, the industrial camera behind window 20432 has a FOV of 23° to 337°.

Referring now to FIG. 2B , LIDAR 20420 provides a 360° horizontal FOV around AV 201000 . The vertical FOV may be limited by the LIDAR instrument. In one example, a 40° vertical FOV 20418 mounted 1.2 m to 1.8 m above the ground sets the minimum distance of the sensor from 3.3 m to 5 m from the AV 2020.

Referring now to FIGS. 2C and 2D , a long range sensor assembly 20400 is shown with a cover 20430 . The cover 20430 includes windows 20434 , 20432 , 20436 through which long-range cameras and industrial cameras view the environment around the AV 2020 . The cover 20430 for the long range sensor assembly 20400 is sealed from the weather by an O-ring between the cover 20430 and the top of the cargo container 201110 .

Referring now to FIGS. 2E and 2F , cover 20430 has been removed to show examples of cameras and processors. The LIDAR sensor 20420 provides data regarding the distance or range to surfaces around the AV. These data may be provided to a processor 20470 located in the long range sensor assembly 20400 . The LIDAR is mounted on structure 20405 over cover 20430 and long range cameras 20440A - 20440C. The LIDAR sensor 20420 is an example of a ranging sensor based on reflected laser pulsed light. Other ranging sensors may also be used, such as radar using reflected radio waves. In one example, the LIDAR sensor 20420 is a Puck sensor by VELODYNE LIDAR® of San Jose, CA. Three long range cameras 20440A, 20440B, 20440C provide digital images of objects, surfaces and structures around AV 201000 . Three long range cameras 20440A, 20440B, 20440C are arranged around structure 20405 relative to cover 20430 to provide three horizontal FOVs covering a full 360° around the AV. Long range cameras 20440A, 20440B, 20440C are on a raised ring structure 20405 mounted to cargo container 201110 . Long range cameras 20440A, 20440B, 20440C receive images through windows 20434 , 20436 , 20438 mounted to cover 20430 . Long range cameras may include a camera and lens on a printed circuit board (PCB).

Referring now to FIG. 2F , one example of a long range camera 20440A may include a digital camera 20444 having a fisheye lens 20442 mounted to the front of the digital camera 20444 . The fisheye lens 20442 can extend the FOV of the camera to a much wider angle. In one example, the fisheye lens expands the field of view to 180°. In one example, digital camera 20444 is similar to e-cam52A_56540_MOD by E-con Systems of San Jose, CA. In one example, fisheye lens 20442 is similar to model DSL227 by Sunex of Carlsbad, CA.

With continued reference to FIG. 2F , the long range sensor assembly 20400 may also include an industrial camera 20450 that receives visual data through a window 20432 in the cover 20430 . The industrial camera 20450 provides additional data to the processor 20470 regarding objects, surfaces and structures in front of the AV. The camera may be similar to the Kowa industrial camera part number LM6HC. Industrial camera 20450 and long-range cameras 20440A-20440C are located 1.2 m to 1.8 m above the surface on which AV 2010 is moving.

With continued reference to FIG. 2F , mounting the long range sensor assembly 20400 on top of a freight container provides at least two advantages. Fields of view for long-range sensors, including long-range cameras 20440A-20440C, industrial camera 20450, and LIDAR 20420, when the sensor is further mounted above the ground, provide a view of nearby objects such as people, cars, low walls, etc. less frequently blocked by Additionally, pedestrian paths are configured to provide visual cues including signage, fence height, etc. for human perception, with typical eye levels in the range of 1.2m to 1.8m. Mounting the long range sensor assembly 20400 on top of the cargo container places the long range cameras 20440A-20440C, 20450 at the same level as the signal and above the visual cues directed to the pedestrian. The long range sensors are mounted on a structure 20405 that provides a substantial and rigid mount that resists deflections caused by movement of the AV 201000 .

Referring back to FIGS. 2E and 2F , the long range sensor assembly may include an inertial measurement unit (IMU) and one or more processors that receive data from the long range sensors and output the processed data to other processors for navigation. . An IMU 20460 with a vertical reference (VRU) is mounted to the structure 20405 . An IMU/VRU 20460 may be located directly below the LIDAR 20420 to provide location data on the LIDAR 20420 . The position and orientation from the IMU/VRU 20460 may be combined with data from other long-range sensors. In one example, the IMU/VRU 20460 is a model MTi 20 supplied by Xsens Technologies of the Netherlands. The one or more processors can include at least a processor 20465 that receives data from the industrial camera 20450 . In addition, the processor 20470 may receive data from at least one of the following: LIDAR 20420, long-range cameras 20440A-20440C, industrial camera 20450, and IMU/VRU 20460. . The processor 20470 may be cooled by a liquid cooling heat exchanger 20475 that is connected to a circulating coolant system.

Referring now to FIG. 2G , AV 201000 may include multiple short-range sensors that detect drivable surfaces and obstacles within a predetermined distance from the AV. Short-range sensors 20510 , 20520 , 20530 , 20540 , 20550 , and 20560 are located on the perimeter of container platform 20160 . These sensors are located below the cargo container 201110 ( FIG. 2B ) and are closer to the ground than the long range sensor assembly 20400 ( FIG. 2C ). The short-range sensors 20510 , 20520 , 20530 , 20540 , 20550 , and 20560 are used to provide FOVs that capture surfaces and objects that cannot be seen by the sensors in the long-range sensor assembly 20400 ( FIG. 2C ). angle downwards. The view of a sensor positioned closer to the ground and angled downward is less likely to be obstructed by nearby objects and pedestrians than sensors mounted further away from the ground. In one example, short-range sensors provide information about ground surfaces and objects up to 4 m from AV 201000 .

Referring again to FIG. 2B , vertical FOVs for two of the short range sensors are shown in a side view of AV 201000 . The vertical FOV 20542 of the aft facing sensor 20540 is centered about the center line 20544 . The center line 20544 is angled below the top surface of the cargo platform 20160 . In one example, the sensor 20540 has a vertical FOV of 42° and a centerline angled 22° to 28° (20546) below the plane 20547 defined by the top plate of the cargo platform 20160 . In one example, short range sensors 20510 and 20540 are approximately 0.55 m to 0.71 m above ground level. The resulting vertical FOVs 20512 and 20542 cover the ground from 0.4 m to 4.2 m from the AV. The short range sensors 20510 , 20520 , 20530 , 20550 ( FIG. 2G ), 20560 ( FIG. 2G ) mounted on the cargo base 20160 have similar vertical fields of view and center line angle to the top of the cargo platform. Short range sensors mounted on cargo platform 20160 can see the ground from 0.4 to 4.7 meters from the outer edge of AV 201000 .

With continued reference to FIG. 2B , the short range sensor 20505 may be mounted on a front surface near the top of the cargo container 201110 . In one example, the sensor 20505 may provide additional views of the ground in front of the AV to the view provided by the short range sensor 20510 . In another example, the sensor 20505 may provide a view of the ground in front of the AV instead of the view provided by the short range sensor 20510 . In one example, the short range sensor 20505 may have a vertical FOV 20507 of 42° and the angle of the center line to the top of the cargo platform 20160 is 39°. The resulting view of the ground extends from 0.7 m to 3.75 m from the AV.

Referring back to FIG. 2G , the horizontal FOVs of short range sensors 20510 , 20520 , 20530 , 20540 , 20550 , 20560 cover all directions around AV 201000 . The horizontal FOVs 20522 and 20532 of adjacent sensors, such as 20520 and 20530, overlap at a distance from the AV 201000. In one example, the horizontal FOVs 20522 , 20532 and 20562 , 20552 of adjacent sensors 20520 , 20530 and 20560 , 20550 overlap at 0.5 to 2 meters from the AV. The short-range sensors are distributed around the perimeter of the cargo base 20160, have horizontal fields of view, and are positioned at specific angles to provide nearly complete visual coverage of the ground surrounding the AV. In one example, the short range sensors have a horizontal FOV of 69°. The forward sensor 20510 faces forward at a zero angle to AV and has a FOV 20512 . In one example, the two front corner sensors 20520 , 20560 are angled such that the center lines are at an angle 20564 of 65°. In one example, the rear side sensors 20530 , 20550 are angled such that the center lines of 20530 and 20560 are at an angle 20534 of 110°. In some configurations, other numbers of sensors with different horizontal FOVs are possible that are mounted around the perimeter of the cargo base 20160 to provide a nearly complete view of the ground around the AV 201000 .

Referring now to FIG. 2H , the short-range sensors 20510 , 20520 , 20530 , 20540 , 20550 , and 20560 are located on the periphery of the cargo base 20160 . Short-range cameras are mounted on protuberances that set the angle and position of the short-range sensors. In another configuration, the sensors are mounted on the interior of the cargo base and receive visual data through windows aligned with the outer skin of the cargo base 20160 .

Referring now to FIGS. 2I and 2J , short range sensors 20600 are mounted to skin element 20516 of cargo base 20160 and may include a liquid cooling system. The skin element 20516 includes a formed protrusion 20514 that holds the short-range sensor assembly 20600 at a predetermined position and perpendicular to the top of the cargo base 20160 and at an angle relative to the front of the cargo base 20160. include In some configurations, the short-range sensor 20510 is angled downwardly with respect to the cargo platform 20160 by 28°, and the short-range sensors 20520 and 20560 are angled downwardly by 18° and forward by 25°; The short-range sensors 20530 and 20550 are angled 34° downward and 20° rearward, and the short-range sensor 20540 is angled downwardly with respect to the cargo platform 20160 by 28°. The skin element 20516 includes a cavity 20517 for receiving the camera assembly 20600 . Skin element 20516 may also include a plurality of elements 20518 for receiving mechanical fasteners including, but not limited to, rivets, screws, and buttons. Alternatively, the camera assembly may be adhesively mounted or held in place with clips that fasten to skin element 20516 . A gasket 20519 may provide a seal against the front of the camera 20610 .

Referring now to FIGS. 2K and 2L , a short-range sensor assembly 20600 includes a short-range sensor 20610 mounted on a bracket 20622 attached to a water-cooled plate 20626 . To better visualize the heat dissipation element sensor block 20616 and the electronic block 20620 of the short-range sensor 20610, an outer case 20612, a transparent cover 20614, and a heat sink 20618 in FIGS. has been partially removed. The short range sensor assembly 20600 may include one or more thermoelectric coolers (TECs) 20630 between the bracket 20622 and the liquid cooling plate 20626 . The liquid cooling plate 20626 is cooled by a coolant pumped through 20628 thermally connected to the plate 20626 . TECs are electrical power supply elements having a first and a second side. The electrical power supply TEC cools the first side while rejecting the thermal energy removed from the first side plus power at the second side. In the short range sensor assembly 20600 , the TECs 20630 cool the bracket 20622 and transfer thermal cooling energy plus electrical energy to the water cooled plate 20626 . Alternatively, the TEC 20630 may be used to actively control the temperature of the camera 20600 by varying the magnitude and polarity of the voltage supplied to the TEC 20630 .

Operating the TEC 20630 in cool mode causes the short range sensor 20610 to operate at a temperature below the coolant temperature. Bracket 20622 is thermally connected to short-range sensor 20610 in two places to maximize cooling of sensor block 20616 and electronics block 20620 . Bracket 20622 includes tabs 20624 that are thermally attached to heat sink 20618 via screws 20625 . The heat sink 20618 is thermally connected to the sensor block 20616 . The bracket is thus thermally connected to the sensor block 20616 via a heat sink 20618 , screws 20625 , and tabs 20624 . A bracket 20622 is also mechanically attached to the electronics block 20620 to provide direct cooling of the electronics block 20620 . Bracket 20622 may include a plurality of mechanical attachments including, but not limited to, screws and rivets that engage elements 20518 of FIG. 2J . The short-range sensor 20610 may include a camera, a stereo camera, an ultrasonic sensor, a short-range radar, and one or more sensors including, but not limited to, infrared projectors and CMOS sensors. One exemplary short-range sensor is similar to the real-sense depth camera D435 by Intel of Santa Clara, CA, which includes an IR projector, two imager chips and an RGB camera.

Referring now to FIGS. 2M-2O , another embodiment of an AV 201000 is shown. The AV 20100A includes a cargo container 201110 and a power base 20170 mounted on a cargo platform 20160 . AV 201000A includes a plurality of long-range and short-range sensors. The main long range sensors are mounted to a sensor pylon 20400A on top of the cargo container 201110 . The sensor pylon may include a LIDAR 20420 targeting divergent directions to provide a wide field of view and a plurality of long-range cameras (not shown). In some configurations, the LIDAR 20420 provides point cloud data that may enable, for example, but not limited to, the population of an occupancy grid, a landmark, and a landmark, as described elsewhere herein. may be used to provide information for identifying navigable spaces, locating the AV 201000 within its environment, and/or determining a navigable space. In some configurations, a long range camera from Leopard Imaging Inc. may be used to identify landmarks, locate the AV 201000 within its environment, and/or determine a navigable space.

With continued reference to FIGS. 2M-2O , short-range sensors are primarily mounted on cargo platform 20160 and provide information about obstacles near AV 201000A. In some embodiments, short-range sensors supply data about obstacles and surfaces within 4 m of AV 201000A. In some configurations, short-range sensors provide information up to 10 m from AV 201000A. A plurality of at least partially forward facing cameras are mounted to the cargo platform 20160 . In some configurations, the plurality of cameras may include three cameras.

Referring now to FIG. 2O , the top cover 20830 is partially cut out to reveal the sub-loop 20810 . The sub-loop 20810 provides a single piece to which a plurality of antennas 20820 can be mounted. In the example, ten antennas 20820 are mounted in sub-loop 20810 . Also, for example, 4 cellular communication channels each have 2 antennas, and there are 2 WiFi antennas. The antennas are wired as primary and secondary antennas for cellular transmission and reception. Auxiliary antennas may improve cellular functionality in several ways including, but not limited to, reducing interference and achieving 4G LTE connectivity. The sub-roof 20810 as well as the top cover 20830 are non-metallic. The sub-loop 20810 is a plastic surface within 10 mm to 20 mm of the top cover 20830 , which is not structural and allows the antennas to connect to the processors before the top cover 20830 is attached. Antenna connections are often high impedance and susceptible to dust, grease and mishandling. Mounting and connecting the antenna to the sub-loop 20810 allows the top cover to be installed and removed without touching the antenna connections. Maintenance and repair operations may include removing the top cover without removing the sub-loop or disconnecting the antenna. Assembly of the antennas spaced apart from installing the top cover 20830 facilitates testing/repair. The top cover 20830 is weatherproof and prevents water and grit from entering the cargo container 201110 . Mounting the antennas on the sub-loop minimizes the number of openings on the top cover 20830.

Referring now to FIGS. 2P, 2Q, and 2R, another example of a long range sensor assembly (LRSA) 20400A mounted on top of a freight container (not shown) is shown. The LRSA may include a LIDAR and a plurality of long-range cameras mounted at different locations on the LRSA structure 20950 to provide a panoramic view of the environment of the AV 201000A. A LIDAR 20420 may be mounted on top of the LRSA structure 20950 to provide an uninterrupted view. LIDAR may include VELODYNE LIDAR. A plurality of long range cameras 20910A - 20910D are mounted on the LRSA structure 20950 on the next level below the LIDAR 20420 . In the example, four cameras are mounted around the structure, one every 90°, providing four views of the environment around the AV 201000 . In some examples, the four views will overlap. In some examples, each camera is aligned with or orthogonal to the direction of movement. In the example, one camera is aligned with each of the principle faces of AV 201000A: front, rear, left and right. In the example, the long range cameras are model LI-AR01 44-MIPI-M12 from Leopard Imaging Inc. Long-range cameras may have a MIPI CSI-2 interface that provides high-speed data transfer to the processor. Long range cameras may have a horizontal field of view between 50° and 70° and a vertical field of view between 30° and 40°.

Referring now to FIGS. 2S and 2T , a long range processor 20940 is located on the LRSA structure 20950 below the long range cameras 20910A - 20910D and the LIDAR 20420 . Long range processor 20940 receives data from long range cameras and LIDAR. The long-range processor communicates with one or more processors elsewhere in the AV 201000A. Long-range processor 20940 provides data derived from long-range cameras and LIDAR to one or more processors located elsewhere on AV 201000A as described elsewhere herein. Long range processor 20940 may be liquid cooled by cooler 20930 . The cooler 20930 may be mounted to long-range cameras and structures below the LIDAR. The cooler 20930 may provide a mounting location for the long range processor 20940 . Cooler 20930 is described in U.S. Patent Application Serial No. 16/883,668 (Attorney Docket No. AA280), entitled Apparatus for Electronic Cooling on an Autonomous Device, filed May 26, 2020, which application is It is incorporated herein by reference in its entirety. The cooler has a liquid supply conduit and a return conduit that provides cooling liquid to the cooler 20930 .

Referring again to FIGS. 2M and 2N , short-range camera assemblies 20740A-20740C are mounted to the front of container platform 20160 , and are mounted on a moving surface and obstacles, steps, curves, and other substantially discontinuous surface features. Formed to collect information about the SDSFs. Camera assemblies 20740A-20740C include one or more LED lights for illuminating moving surfaces, objects on the ground, and SDSFs.

Referring now to FIGS. 2U-2X , camera assemblies 20740A and 20740B include lights 20732 that illuminate the ground and objects to provide improved image data from cameras 20732 . Camera assembly 20740A is a mirror of 20740C, and the descriptions of 20740A apply implicitly to 20740C. The camera 20732 may include a single vision camera, a stereo camera, and/or an infrared projector and CMOS sensor. One example of a camera is the Real-Sense Depth D435 camera by Intel of Santa Clara, CA, which includes an IR projector, two imager chips with lenses and an RGB camera. The LED lights 20734 can be used at night or in low light conditions or always used to improve image data. One theory of operation is that lights create contrast by illuminating projection surfaces and creating shadows in depressions. The LED lights may be white LEDs in an example. In the example, the LED lights 20734 are Xlamp XHP50 from Cree Inc. In another example, LED lights may emit in infrared to provide illumination to camera 20372 without distracting or annoying nearby pedestrians or drivers.

With continued reference to FIGS. 2U-2X , the placement and angle of lights 20374 and the shape of covers 20736A, 20736B prevent camera 20732 from seeing lights 20374 . The angle and placement of lights 20374 and covers 20736A, 20736B prevent the lights from interfering with drivers or bothering pedestrians. The camera 20732 is positioned on the light 20734 such that the sensor in the camera 20732 is obscured by the light 20734 and thus prevents detecting lower light signals from the ground and objects on the front and side of the AV 20100A. It is advantageous not to be exposed. The camera 20732 and/or lights 20734 may be cooled with a liquid flowing into and out of the camera assemblies through ports 20736 .

Referring now to FIGS. 2W and 2X , short-range camera assembly 20740A includes an ultrasonic or sonar short-range sensor 20730A. The second short-range camera assembly 20740C also includes an ultrasonic short-range sensor 20730B ( FIG. 2N ).

Referring now to FIG. 2Y , the ultrasonic sensor 20730A is mounted above the camera 20732 . In the example, the center line of the ultrasonic sensor 20730A is parallel to the base of the cargo container 201110, which often means that the sensor 20730A is horizontal. The sensor 20730A is not facing forward and is angled at 45°. Cover 20376A provides a horn 20746 for directing ultrasound waves coming from and received by ultrasound sensor 20730A.

With continued reference to FIG. 2Y , a cross-section of camera 20732 , light 20734 in camera assembly 20740A, shows angles and openings in cover 20736 . The cameras in short range camera assemblies are angled downward to better image the ground in front and side of the AV. The central camera assembly 20740B is oriented straight forward in a horizontal plane. Corner camera assemblies 20740A, 20740C are angled at a 25° angle to their respective sides relative to the front in a straight line in a horizontal plane. Camera 20732 is angled 20° downward relative to the top of the cargo platform in a vertical plane. As the AV normally holds the cargo platform level, the camera is thus at an angle of 20° below the level. Similarly, central camera assembly 20740B ( FIG. 2M ) is angled by 28° below horizontal. In an example, cameras in camera assemblies may be angled downward by 25° to 35°. In another example, the cameras in camera assemblies may be angled downward by 15° to 45°. LED lights 20734 are similarly angled downward to illuminate the ground imaged by camera 20732 and to minimize distraction to pedestrians. In one example, the LED light center line 20742 is parallel within 5° of the camera center line 20738 . Cover 20736A protects camera 20732 and pedestrians from the bright light of LEDs 20734 in both camera assemblies 20740A - 20740C. The cover that isolates the light emitted by the LEDs also provides a flared opening 20737 to maximize the field of view of the camera 20732 . The lights are recessed at least 4 mm from the opening in the cover. The light opening is defined by an upper wall 20739 and a lower wall 20744 . Top wall 20739 is approximately parallel (± 5°) with center line 2074 . The lower wall 20744 is flared approximately 18° from the center line 20742 to maximize illumination of the ground and proximate objects.

Referring now to FIGS. 2Z-2A , in one configuration, a light 20734 includes two LEDs 20734A, each under a square lens 20734B to produce a beam of light. The LEDs/lenses are positioned at an angle relative to the camera 20372 to illuminate the camera's field of view with minimal excess of light outside the FOV of the camera 20732 . The two LEDs/lenses are mounted together on a single PCB 20752 at an angle 20762 defined between the two lights. In another configuration, the two LEDs/lenses are individually mounted on the heat sink 20626A on separate PCBs at an angle to each other. In the example, the lights are Xlamp XHP50s from Cree, Inc. and the lens is a 60° lens HB-SQ-W from LEDil. The lights are at an angle of approximately 50° to each other, so that the angle 20762 between the fronts of the lenses is 130°. The lights are positioned approximately 18 mm (± 5 mm) 20764 behind the front of the camera 20732 and approximately 30 mm (20766) below the center line of the camera 20732 .

Referring now to FIGS. 2AA and 2BB , camera 20732 is cooled by a thermoelectric cooler (TEC) 20630 , which is cooled along with light 20734 by liquid coolant flowing through cooling block 20626A. do. The camera is attached to the bracket 20622 via screws 20625 that fit into the sensor block portion of the camera, the back of the bracket 20622 being bolted to the camera's electronics block. The bracket 20622 is cooled by two TEzCs to maintain the performance of the IR imaging chips (CMOS chips) in the camera 20732 . The TECs reject heat from the bracket 20622 and the power they draw to the cooling block 20626A.

Referring now to FIG. 2BB , coolant is directed through a U-shaped path created by a central fin 20626D. The coolant flows directly behind the LED/lens/PCB of the light 20734 . Fins 20626B, 20626C improve heat transfer from light 20734 to coolant. The coolant flows upwardly past the hot side of the TEC 20630 . The fluid path is created by a plate 20737 (FIG. 2X) attached to the back of the cooling block 20626A.

Referring now to FIG. 3A , sensor data and map data may be used to update the occupancy grid. Systems and methods of the present teachings can manage a global occupancy grid for devices that are autonomously navigating with respect to a grid map. A grid map may include routes or routes that a device may follow from a starting point to a destination. The global occupancy grid may include free space indicators that may indicate where it is safe for a device to navigate. Possible routes and free space indications can be combined on the global occupancy grid to establish an optimal route the device can travel to safely reach its destination.

With continued reference to FIG. 3A , as the device moves, a global occupancy grid that will be used to determine an undisturbed navigation route may be accessed based on the device's location, and the global occupancy grid may be updated as the device moves. can Updates are made at least to current values associated with the global occupancy grid at the device's location, a static occupancy grid, which may include historical information about the neighborhood the device is navigating to, and data collected by sensors as the device moves. can be based The sensors may be placed on the device as described herein, and they may be placed elsewhere.

With still further reference to FIG. 3A , the global occupancy grid may include cells, and the cells may be associated with occupied probability values. Each cell of the global occupancy grid is located at and around the location of the moving surface as determined by whether obstacles have been identified at the cell's location, as determined from previously collected data and as determined by the data collected when the device is navigating. It may be associated with information such as characteristics and discontinuities, and previous occupancy data associated with its location. Data captured as a device navigates may be stored in a local occupancy grid whose center is the device. When updating the global occupancy grid, the static previously collected data is combined with the local occupancy grid data and the global occupancy data determined from the previous update to create a new global occupancy grid in which the space occupied by the device is marked as unoccupied. can In some configurations, the Bayesian method may be used to update the global occupancy grid. The method includes, for each cell in the local occupancy grid, calculating the position of the cell on the global occupancy grid, accessing a value at that position from the current global occupancy grid, the value at the position from the static occupancy grid. accessing , accessing a value at a location from the local occupancy grid, and adding a new value at a location on the global occupancy grid to the current value from the global occupancy grid, the value from the static occupancy grid, and the local occupancy grid. calculating as a function of the value of In some configurations, the relationship used to compute the new value may include the sum of the static value and the current value minus the locally occupied grid value. In some configurations, the new value may be bounded by preselected values, for example based on a computational constraint.

With continued reference to FIG. 3A , the system 30100 of the present teachings may manage a global occupancy grid. The global occupancy grid can start with initial data and can be updated as devices move. Creating the initial global occupancy grid may include a first process, and updating the global occupancy grid may include a second process. System 30100 may include, but is not limited to, a global occupancy server 30121 that may receive information from a variety of sources and update a global occupancy grid 30505 based at least on the information. The information may be supplied by, for example, but not limited to, sensors located on and/or elsewhere on the device, static information, and navigation information. In some configurations, sensors may include, for example, cameras and radar capable of detecting surface characteristics and obstacles. For example, sensors may be advantageously positioned on the device to provide sufficient ambient coverage to enable safe movement by the device. In some configurations, LIDAR 30103 may provide LIDAR point cloud (PC) data 30201 that may enable populating a local occupancy grid with LIDAR free space information 30213 . In some configurations, a conventional ground detection inverse sensor model (ISM) 30113 may process the LIDAR PC data 30201 to generate LIDAR free space information 30213 .

With continued reference to FIG. 3A , in some configurations, RGB-D cameras 30101 can provide RGB-D PC data 30202 and RGB camera data 30203 . RGB-D PC data 30202 may populate the local occupancy grid with depth free space information 30209 , and RGB-D camera data 30203 may populate the local occupancy grid with surface data 30211 . In some configurations, RGB-D PC data 30202 may be processed by, for example, but not limited to, conventional stereo free space ISM 30109, RGB-D camera data 30203 may include: For example, but not by way of limitation, it may be fed to a conventional surface detection neural network 30111 . In some configurations, RGB MIPI cameras 30105 can provide RGB data 30205 and combine with LIDAR PC data 30201 to create a local occupancy grid with LIDAR/MIPI free space information 30215 . there is. In some configurations, RGB data 30205 can be fed to a conventional free space neural network 30115 , the output of which is fed to a conventional 2D-3D registration 30117 , along with LIDAR PC data 30201 . Before, it may be subjected to a preselected mask 30221 that can identify which portions of the RGB data 30205 are most important for accuracy. 2D-3D registration 30117 can project an image from RGB data 30205 onto LIDAR PC data 30201 . In some configurations, 2D-3D registration 30117 is not required. Any combination of sensors and methods for processing sensor data may be used to collect data to update the global occupancy grid. Any number of free space estimation procedures may be used and combined to enable determination and verification of occupied probabilities within the global occupancy grid.

With continued reference to FIG. 3A , in some configurations, historical data may be provided, for example, by a store 30107 of previously collected and processed data having information associated with a navigation area. In some configurations, storage 30107 may contain, for example, but not limited to, route information, such as, for example, polygons 30207 . In some configurations, these data are to be fed to a conventional polygon parser 30119 that can provide edges 30303 , discontinuities 30503 , and surfaces 30241 to a global occupancy grid server 30121 . can The global occupancy grid server 30121 may fuse the local occupancy grid data collected by the sensors with the processed storage data to determine the global occupancy grid 30505 . A grid map 30601 (FIG. 3D) may be generated from global occupancy data.

Referring now to FIG. 3B , in some configurations, the sensors can include a sonar 30141 that can provide a locally occupied grid with sonar free space 30225 to a globally occupied grid server 30121 . The depth data 30209 may be processed by a conventional free space ISM 30143 . The locally occupied grid with sonar free space 30225 is a locally occupied grid with surfaces and discontinuities 30223 , a locally occupied grid with LIDAR free space 30213 , and a locally occupied grid with LIDAR/MIPI free space 30215 . Grid, a locally occupied grid with stereo free space 30209, and edges 30303 (FIG. 3F), discontinuities 30503 (FIG. 3F), navigation points 30501 (FIG. 3F), surface reliability (FIG. 3F) 30513 ( FIG. 3F ), and surfaces 30241 ( FIG. 3F ) may be fused to form a global occupancy grid 30505 .

3C-3F, to initialize the global occupancy grid, the global occupancy grid initialization 30200 is generated by the global occupancy grid server 30121, the global occupancy grid 30505, and the static grid 30249. may include The global occupancy grid 30505 can be created by fusing data from the local occupancy grids 30118 with edges 30303, discontinuities 30503, and surfaces 30241 located in the region of interest. Static grid 30249 ( FIG. 3D ) is configured to include data such as, but not limited to, surface data 30241 , discontinuity data 30503 , edges 30303 and polygons 30207 , for example. can be created The initial global occupancy grid 30505 adds occupancy probability data from the static grid 30249 ( FIG. 3E ) to the occupancy data derived from the collected data from the sensors 30107A, and It can be calculated by subtracting the occupancy data from 30505A (FIG. 3F). Local occupancy grids 30118 include, but are not limited to, local occupancy grid data resulting from stereo free space estimation 30209 (FIG. 3B) via ISM, surface/discontinuity detection results 30223 (FIG. 3B). Local occupancy grid data resulting from LIDAR free space estimation 30213 ( FIG. 3B ) via ISM, and local occupancy resulting from LIDAR/MIPI free space estimation 30215 ( FIG. 3B ) following in some configurations. grid data, 2D-3D registration 30117 (FIG. 3B), but is not limited thereto. In some configurations, local occupancy grids 30118 may include local occupancy grid data resulting from sonar free space estimation 30225 via ISM. In some configurations, the various local occupancy grids with free space estimate may be fused into local occupancy grids 30118 according to preselected known processes. The global occupancy grid 30505 may generate a grid map 30601 ( FIG. 3E ) that may include occupancy and surface data in the vicinity of the device. In some configurations, grid map 30601 ( FIG. 3E ) and static grid 30249 ( FIG. 3D ) may be published using, for example, robotic operating system (ROS) subscribe/publish features, but are limited to this. doesn't happen

Referring now to FIGS. 3G and 3H , to update the occupancy grid as the device moves, occupancy grid update 30300 updates the local occupancy grid with respect to the measured data as the device moves, and converts these data to a static grid ( 30249)). The static grid 30249 is accessed when the device moves out of range of the operating occupancy grid. The device may be located at the occupancy grid 30245A at the first location 30513A at the first time. When the device moves to the second location 30513B, the device is placed in an occupancy grid 30245B comprising a set of values derived from the new location and possibly values in the occupancy grid 30245A. The data from the static grid 30249 and the surface data from the initial global occupancy grid 30505 (FIG. 3C) that positionally coincide with the cells in the occupancy grid 30245B at the second time include the measured surface data and the occupied surface data. Along with probabilities, it can be used to update each grid cell according to a preselected relationship. In some configurations, the relationship can include summing the static data with the measured data. The resulting occupancy grid 30245C at the third time and third location 30513C may be made available to the mobility manager 30123 to inform navigation of the device.

Referring now to FIG. 3I , a method 30450 for creating and managing an occupancy grid includes, but is not limited to, converting, by a local occupancy grid creation node 30122, sensor measurements into a frame of reference associated with a device ( 30451 ), generating a time-stamped measurement occupancy grid 30453 , and publishing 30455 the time-stamped measurement occupancy grid as a local occupancy grid 30234 ( FIG. 3G ). The system associated with method 30450 may include multiple local grid generating nodes 30122 , eg, one for each sensor, such that multiple local occupancy grids 30234 ( FIG. 3G ) will occur. make it possible Sensors may include, but are not limited to, RGB-D cameras 30325 ( FIG. 3G ), LIDAR/MIPI 30231 ( FIG. 3G ), and LIDAR 30233 ( FIG. 3G ). A system associated with method 30450 may include a global occupancy grid server 30121 that may receive local occupancy grid(s) and process them in accordance with method 30450 . In particular, the method 30450 includes loading surfaces 30242 , such as but not limited to accessing surface discontinuities such as curves 30504 , and, for example, but not limited to, steps 30504 . , generating 30248 the static occupancy grid 30249 from any properties available in the reservoir 30107, which may include a surface and surface discontinuities. The method 30450 can include receiving a posted local occupancy grid ( 30456 ) and moving the global occupancy grid to maintain the device in the center of the map ( 30457 ). The method 30450 includes setting 30459 new areas on the map with old information from a static previous occupancy grid 30249, and marking 30461 an area currently occupied by the device as unoccupied. may include Method 30450 may execute loop 30463 for each cell in each local occupancy grid. Loop 30463 includes calculating the position of the cell on the global occupancy grid, accessing a previous value at the position on the global occupancy grid, and based on the relationship between the previous value and the value in the cell in the local occupancy grid. calculating a new value at the cell location, but is not limited thereto. This relationship may include, but is not limited to, summing values. The loop 30463 may include comparing the new value to the preselected acceptable probability range, and setting the global occupancy grid to the new value. The comparison may include setting the probability to a minimum or maximum acceptable probability if the probability is lower or higher than a minimum or maximum acceptable probability. Method 30450 can include posting 30467 a global occupancy grid.

Referring now to FIG. 3J , an alternative method 30150 for creating a global occupancy grid includes, but is not limited to, if the device has moved 30151 , the occupied map area associated with the old map area (which was before the device was moved) is accessing the probability values 30153 and updating the global occupancy grid on the new map area (which was after the device was moved) with values from the old map area, the drivable surface associated with the cell of the global occupancy grid in the new map area accessing 30155 , updating cells in the updated global occupancy grid with drivable surfaces, and proceeding to step 30159 . If the device has not moved at 30151, and if the global occupancy grid is co-located with the local occupancy grid, then the method 30150 includes the global occupancy possibly updated with surface confidences associated with drivable surfaces from the at least one local occupancy grid. Updating the grid 30159, eg, but not limited to, updating the updated global occupancy grid with the log odds of occupied probability values from at least one local occupancy grid using a Bayesian function 30161 ), and adjusting 30163 the log odds based at least on the characteristics associated with the location. If at 30157 the global occupancy grid is not co-located with the local occupancy grid, the method 30150 may include returning to step 30151 . Characteristics may include, but are not limited to, setting the location of the device to unoccupied.

Referring now to FIG. 3K , in another configuration, a method 30250 for creating a global occupancy grid includes, but is not limited to, if the device has moved at 30251, the global occupancy with information from the static grid associated with the device's new location. updating the grid (30253). Method 30250 can include analyzing 30257 surfaces at the new location. If the surfaces are drivable at 30259, the method 30250 updates the surfaces on the global occupancy grid 30261, and updates 30263 the global occupancy grid with values from the store of static values associated with the new location on the map. ) may be included.

Referring now to FIG. 3L , updating surfaces 30261 may include, but is not limited to, accessing 30351 a local occupancy grid (LOG) for a particular sensor. If there are more cells in the local occupancy grid to process at 30353 , the method 30261 may include accessing the surface classification and surface classification confidence value from the local occupancy grid ( 30355 ). If at 30357 the surface classification in the cell in the local occupancy grid is the same as the surface classification in the global occupancy grid at the location of the cell, then the method 30261 calculates the new global occupancy grid (GOG) surface reliability to the old global occupancy grid surface. setting 30461 to the sum of the reliability and the locally occupied grid surface reliability. If at 30357 the surface classification in the cell in the local occupancy grid is not the same as the surface classification in the global occupancy grid at the location of the cell, then the method 30261 compares the new global occupancy grid surface confidence with the old global occupancy grid surface confidence. setting 30359 to the difference between the locally occupied grid surface reliability. If at 30463 the new global occupancy grid surface confidence is less than zero, the method 30261 may include setting 30469 the new global occupancy grid surface classification to a value of the local occupancy grid surface classification.

Referring now to FIG. 3M , the step 30263 of updating the global occupancy grid with values from the store of static values includes, but is not limited to, if there are more cells in the local occupancy grid to process at 30361, the method ( 30263 may include accessing 30363 log odds from the local occupancy grid and updating 30365 log odds in the global occupancy grid with values from the local occupancy grid at the location. If the maximum certainty that the cell is empty is met at 30367, and if the device is moving within predetermined lane barriers at 30369, and if the surface is drivable at 30371, then the method 30263 determines the probability that the cell is occupied. updating 30373 and returning to continue processing more cells. If the maximum certainty that the cell is empty is not reached at 30367, or if the device is not moving in a predetermined lane at 30369, or if at 30371 the surface is not drivable in the mode in which the device is currently moving, the method ( 30263) may include returning to consider more cells without updating the log odds. If the device is in standard mode, i.e., a mode in which the device can navigate relatively uniform surfaces, and the surface classification indicates that the surface is not relatively uniform, the method 30263 updates the log odds By increasing the probability of this being occupied, the path of the device can be adjusted. If the device is in standard mode and the surface classification indicates that the surface is relatively uniform, the method 30263 can adjust the path of the device by reducing the probability that the cell is occupied by updating 30373 the log odds. . If the device is moving in a four-wheel mode, that is, if the device is able to navigate non-uniform terrain, then adjustments to the probability that the cell is occupied may not be necessary.

Referring now to FIG. 4A , the AV may move in a particular mode that may be associated with a device configuration, eg, the configuration shown in device 42114A and the configuration shown in device 42114B. A system of the present teachings for real-time control of a configuration of a device, based on at least one environmental factor and a context of the device, includes, but is not limited to, a chassis operatively coupled with sensors, means of movement, sensors and means of movement; The moving means is driven by motors and a power source—it may include a device processor that receives data from sensors, and a power base processor that controls the moving means. In some configurations, the device processor may receive environmental data, determine environmental factors, determine configuration changes according to the environmental factors and a situation of the device, and provide the configuration changes to the power base processor. The power base processor may issue commands to the vehicle to move the device around and physically reconfigure the device as required by the road surface type.

With continued reference to FIG. 4A , sensors collecting environmental data include, for example, cameras, LIDAR, radar, thermometers, pressure sensors, and weather condition sensors, some of which are described herein. may include, but are not limited to. From this data classification, the device processor can determine environmental factors in which the device configuration may change. In some configurations, environmental factors may include, for example, but not limited to surface factors such as surface type, surface features, and surface conditions. The device processor may determine in real time how to change the configuration to accommodate the traversal of the detected surface type, based on environmental factors and the current situation of the device.

With continued reference to FIG. 4A , in some configurations, a configuration change of device 42114A/B/C (collectively referred to as device 42114 ) may include, for example, a change in configuration of the means of transportation. can Other configuration changes are contemplated, such as user information displays and sensor controls that may depend on the current mode and surface type. In some configurations, as described herein, the means of movement includes at least four drive wheels 442101 , two on each side of chassis 42112 , and at least two operatively coupled chassis 42112 . It may include caster wheels 42103 . In some configurations, drive wheels 442101 can be operatively coupled into pairs 42105 , where each pair 42105 includes a first drive wheel 42101A and a second drive of four drive wheels 442101 . may include a wheel 42101B, with pairs 42105 positioned on opposite sides of the chassis 42112, respectively. The operable coupling may include a wheel cluster assembly 42110 . In some configurations, the power base processor 41016 ( FIG. 4B ) can control rotation of the cluster assembly 42110 . Left and right wheel motors 41017 ( FIG. 4B ) may drive wheels 442101 on either side of chassis 42112 . Rotation may be accomplished by driving the left and right wheel motors 41017 ( FIG. 4B ) at different rates. Cluster motors 41019 ( FIG. 4B ) may rotate the wheelbase in a forward/rear direction. Rotation of the wheelbase may allow the load to rotate independently from the drive wheels 442101 , while, for example, when encountering discontinuous surface features, the front drive wheels 442101A and the rear drive wheels 442101B Higher or lower may be allowed. The cluster assembly 42110 may operate each pair of two wheels 42105 independently, thereby providing forward, backward, and rotational movement of the device 42114 upon command. The cluster assembly 42110 may provide structural support for the pairs 42105 . The cluster assembly 42110 may provide mechanical power to rotate the wheel drive assemblies together, such as, but not limited to, cluster assembly rotation, such as, but not limited to, climbing discontinuous surface features, various surface types, and non-uniform terrain. Allows functions that depend on Additional details of the operation of clustered wheels are found in U.S. Patent Application Serial No. 16,035,205, Attorney Docket No. X80, entitled Mobility Device, filed July 13, 2018, which is incorporated herein by reference in its entirety. can be

With continued reference to FIG. 4A , the configuration of device 42114 may be associated with, but not associated with, mode 41033 ( FIG. 4B ) of device 42114 . Device 42114 may operate in several modes 41033 ( FIG. 4B ). In standard mode 10100 - 1 ( FIG. 5E ), device 42114B can operate on two of drive wheels 442101B and two of caster wheels 42103 . Standard mode 10100 - 1 ( FIG. 5E ) may provide rotational performance and mobility on relatively solid level surfaces, for example, but not limited to indoor environments, sidewalks, and pavement. In enhanced mode 10100 - 2 ( FIG. 5E ), or four-wheel mode, device 42114A/C can command four of drive wheels 442101A/B and actively stabilize via onboard sensors and may raise and/or redirect the chassis 42112 , casters 42103 , and cargo. The four-wheel mode 10100-2 (FIG. 5E) can provide mobility in a variety of environments, allowing the device 42114A/C to move up steep slopes and over smooth, uneven terrain. In the four-wheel mode 10100 - 2 ( FIG. 5E ), all four of the drive wheels 442101A/B can be deployed and the caster wheel 42103 can be retracted. Rotation of cluster 42110 may allow operation on non-uniform terrain, and drive wheels 442101A/B may drive over and over discontinuous surface features. This capability may provide the device 42114A/C with mobility in a wide variety of outdoor environments. Device 42114B is robust and stable but can operate on wet outdoor surfaces. Frost heaves and other natural phenomena can degrade outdoor surfaces, creating cracks and loose material. In 4-wheel mode 10100 - 2 ( FIG. 5E ), device 42114A/C can operate on these degraded surfaces. Modes 41033 (FIG. 4B) are described in detail in US Pat. No. 6,571,892 ('892), entitled Control System and Method, issued Jun. 3, 2003, which is incorporated herein by reference in its entirety. there is.

Referring now to FIG. 4B , system 41000 processes inputs from sensors 41031 and generates commands to wheel motors 41017 to drive wheels 442101 ( FIG. 4A ) and clusters By generating commands to the cluster motors 41019 to drive 42110 ( FIG. 4A ), it can drive the device 42114 ( FIG. 4A ). System 41000 may include, but is not limited to, a device processor 41014 and a power base processor 41016 . The device processor 41014 may receive and process the environmental data 41022 from the sensors 41031 , and provide the configuration information 40125 to the power base processor 41016 . In some configurations, the device processor 41014 can include a sensor processor 41021 that can receive and process environmental data 41022 from the sensors 41031 . Sensors 41031 may include, but are not limited to, cameras, as described herein. From this data, for example, information about the driving surface being traversed by device 42114 ( FIG. 4A ) can be accumulated and processed. In some configurations, the driving surface information may be processed in real time. The device processor 41014 can include, for example, a configuration processor 41023 that can determine the surface type 40121 from the environmental data 41022 being traversed by the device 42114 ( FIG. 4A ). The configuration processor 41023 may, for example, generate a driving surface classification layer, a driving surface trust layer, and an occupancy layer from the environmental data 41022, e.g., a driving surface processor 41029 (FIG. 4C). may include This data may be used by the power base processor 41016 to generate move instructions 40127 and motor instructions 40128, as described herein by the global occupancy grid processor 41025, It can be used to update the occupancy grid that can be used for route planning. Configuration 40125 may be based, at least in part, on surface type 40121 . The surface type 40121 and the mode 41033 may be used to at least partially determine the occupancy grid information 41022 , the occupancy grid information 41022 may include a probability that a cell within the occupancy grid will be occupied. The occupancy grid may enable, at least in part, the determination of a path that device 42114 ( FIG. 4A ) may take.

With continued reference to FIG. 4B , the power base processor 41016 may receive the configuration information 40125 from the device processor 41014 and process the configuration information 40125 along with other information, for example, path information. . The power base processor 41016 includes a control processor 40325 that can generate move instructions 40127 based at least on the configuration information 40125 and provide the move instructions 40127 to the motor drive processor 40326 . may include The motor drive processor 40326 can generate motor instructions 40128 that can induce and move the device 42114 ( FIG. 4A ). Specifically, motor drive processor 40326 can generate motor instructions 40128 that can drive wheel motors 41017 , and motor commands 40128 that can drive cluster motors 41019 . can create

Referring now to FIG. 4C , real-time surface detection of the present teachings may include, but is not limited to, a configuration processor 41023 , which may include a driving surface processor 41029 . The driving surface processor 41029 can determine characteristics of the driving surface that the device 42114 ( FIG. 4A ) is navigating to. The characteristics may be used to determine a future configuration of device 42114 ( FIG. 4A ). The driving surface processor 41029 may include, but is not limited to, a neural network processor 40207 , data transformations 40215 , 40219 , 40239 , a layer processor 40241 , and an occupancy grid processor 40242 . Together these components can generate information that can direct a change in configuration of device 42114 ( FIG. 4A ) and can inform route planning for movement of device 42114 ( FIG. 4A ) occupancy grid 40244 . ) (Fig. 4c) can be modified.

Referring now to FIGS. 4C and 4D , the neural network processor 40207 may indicate, for each point of data collected by the sensors 41031 ( FIG. 4B ), the type of surface the point is likely to represent. The environment data 41022 (FIG. 4B) may be applied to the trained neural network. Environment data 41022 ( FIG. 4B ) may be received as, but not limited to, camera images 40202 , cameras may be associated with camera characteristics 40204 . The camera images 40202 may include 2D grids of points 40201 having an X-resolution 40205 ( FIG. 4D ) and a Y-resolution 40204 ( FIG. 4D ). In some configurations, camera images 40202 may include RGB-D images, X-resolution 40205 ( FIG. 4D ) may include 40,640 pixels, and Y-resolution 40204 ( FIG. 4D ). ) may include 40,480 pixels. In some configurations, camera images 40202 may be converted into formatted images according to the requirements of the selected neural network. In some configurations, data can be normalized, scaled, and transformed from 2D to 1D, which can improve the processing efficiency of the neural network. A neural network can be trained in a number of ways, including but not limited to training with RGB-D camera images. In some configurations, the trained neural network represented by the neural network file 40209 ( FIG. 4D ) is available to the neural network processor 40207 via a direct connection to the processor executing the trained neural network or via, for example, a communication channel. can be done In some configurations, the neural network processor 40207 may use the trained neural network file 40209 ( FIG. 4D ) to identify surface types 40121 ( FIG. 4C ) in the environmental data 41022 ( FIG. 4B ). . In some configurations, surface types 40121 ( FIG. 4C ) may include, but are not limited to, non-drivable, hard-drivable, soft-drivable, and curve. In some configurations, surface types 40121 ( FIG. 4C ) are non-drivable/background, asphalt, concrete, brick, packed dust, wood flank, gravel/pebbles, grass, mulch, sand, curve, may include, but are not limited to, solid metal, metal grates, tactile pavement, snow/ice, and train tracks. The result of the neural network processing may include a surface classification grid 40303 of points 40213 with X-resolution 40205 and Y-resolution 40203 , and a center 40211 . Each point 40213 in the surface classification grid 40303 may be associated with a likelihood of being a particular one of the surface types 40121 ( FIG. 4C ).

With continued reference to FIGS. 4C and 4D , the driving surface processor 41029 ( FIG. 4C ) in 3D real-world coordinates as seen by the camera from the 2D surface classification grid 40303 ( FIG. 4D ) in the 2D camera frame. 2D to 3D transform 40215 that can be deprojected into 3D image cube 40307 (FIG. 4D) of De-projection may recover 3D properties of 2D data, and may convert a 2D image from an RGB-D camera to a 3D camera frame 40305 (FIG. 4C). Points 40233 ( FIG. 4D ) in cube 40307 ( FIG. 4D ) may each be associated with a specific one of surface types 40121 ( FIG. 4C ), and depth coordinates as well as possibly X/Y coordinates. . The dimensions of the point cube 40307 ( FIG. 4D ) depend, for example, on camera characteristics 40204 ( FIG. 4C ), such as, for example, focal length x, focal length y, and projection center 40225 . may be limited, but is not limited thereto. For example, camera characteristics 40204 may include a maximum range that the camera can reliably project. There may also be features of device 42114 ( FIG. 4A ) that may interfere with image 40202 . For example, the casters 42103 ( FIG. 4A ) may interfere with the view of the camera(s) 40227 ( FIG. 4D ). These factors may limit the number of points in point cube 40307 ( FIG. 4D ). In some configurations, the camera(s) 40227 ( FIG. 4D ) cannot reliably project beyond about 6 meters, which may represent an upper limit of the range of the camera 40227 ( FIG. 4D ), and point cubes You can limit the number of points at 40307 ( FIG. 4D ). In some configurations, features of device 42114 ( FIG. 4A ) may act as a minimum limit of the range of camera 40227 ( FIG. 4D ). For example, the presence of casters 42103 ( FIG. 4A ) may imply, in some configurations, a minimum limit that may be set to approximately one meter. In some configurations, the points in the point cubes 40307 ( FIG. 4D ) can be limited to points that are at least 1 meter from the camera 40227 ( FIG. 4D ) and 6 meters or less from the camera 40227 ( FIG. 4D ). there is.

With continued reference to FIGS. 4C and 4D , the driving surface processor 41029 ( FIG. 4C ) generates a 3D cube of points with coordinates associated with the device 42114 ( FIG. 4A ), ie, the baselink frame 40309 ( FIG. 4C ). ) may include a baselink conversion 40219 that can be converted to. Baselink transformation 40219 converts 3D data points 40223 (FIG. 4D) in cube 40307 (FIG. 4D) to cube 40308 ( points 40233 ( FIG. 4D ) in FIG. 4D ). Driving surface processor 41029 (FIG. 4C) calculates points 40233 (FIG. 4D) in cube 40308 (FIG. 4D) to points 40237 (FIG. 4D) in cube 40311 (FIG. 4D). can include an OG prep 40239 that can be projected onto the occupancy grid 40244 (FIG. 4D) as The layer processor 40241 may flatten the points 40237 ( FIG. 4D ) into various layers 40312 ( FIG. 4C ), depending on the data represented by the points 40237 ( FIG. 4D ). In some configurations, the layer processor 40241 may apply a scalar value to the layers 40312 ( FIG. 4C ). In some configurations, the layers 40312 ( FIG. 4C ) include the probability of the occupying layer 40243 , the surface classification layer 40245 ( FIG. 4D ) as determined by the neural network processor 40207 , and the surface type trust layer. 40247 ( FIG. 4D ). In some configurations, the surface type trust layer 40247 ( FIG. 4D ) calculates the class scores from the neural network processor 40207 and converts the class scores as log(class score)/Sigma log (each class) to the output classes. It can be determined by transforming it into scores that can be determined by normalizing to a probability distribution. In some configurations, one or more layers may be replaced or augmented by a layer that provides a probability of a non-drivable surface.

With continued reference to FIGS. 4C and 4D , in some configurations, the probability value in the occupying layer 40243 may be expressed as a log odds (log odds -> ln(p/(1-p)) value. In configurations, the probability value in the occupation layer 40243 may be based at least on the combination of the mode 41033 ( FIG. 4B ) and the surface type 40121 ( FIG. 4B ) In some configurations, the probability value in the occupation layer 40243 ) are, for example, but not limited to: (1) when the surface type 40121 ( FIG. 4A ) is hard and drivable, and the device 42114 ( FIG. 4A ) preselected set. when in the modes 41033 ( FIG. 4B ) of, or (2) when the surface type 40121 ( FIG. 4A ) is soft and drivable, and when the device 42114 ( FIG. 4A ) is, for example, standard mode, or (3) when the surface type 40121 (FIG. 4A) is soft and drivable, and when the device 42114 (FIG. 4A) is in a four-wheel mode, for example. or (4) when the surface type 40121 (FIG. 4A) is discontinuous, and the device 42114 (FIG. 4A) is in a specific preselected mode, e.g., standard mode mode, or (5) when the surface type 40121 (FIG. 4A) is discontinuous, and when the device 42114 (FIG. 4A) is in a particular preselected mode, such as a 4-wheel mode, or (6) ) surface type 40121 (FIG. 4A) is not drivable, and when device 42114 (FIG. 4A) is in a preselected set of modes 41033 (FIG. 4B). In some configurations, the probability values may include, but are not limited to, those set forth in Table I. In some configurations, the neural network-prediction probability may be tuned, if necessary, and is listed in Table I. You can substitute probabilities.

Table I.

Referring back to FIG. 4C , the occupancy grid processor 40242 may affect probability values of the occupancy grid 40244 , such as, but not limited to, surface type 40121 and occupancy grid information 41022 . parameters may be provided to the global occupancy grid processor 41025 in real time. For example, but not by way of limitation, configuration information 40125 ( FIG. 4B ) such as mode 41033 and surface type 40121 may be provided to power base processor 41016 ( FIG. 4B ). The power base processor 41016 ( FIG. 4B ) provides motor instructions 40128 ( FIG. 4B ) that may set the configuration of the device 42114 ( FIG. 4A ) based at least on the configuration information 40125 ( FIG. 4B ). can be decided

Referring now to FIGS. 4E and 4F , device 42100A may be configured in accordance with the present teachings to operate in a standard mode. In standard mode, casters 42103 and second drive wheel 42101B may rest on the ground as device 42100A navigates its path. The first drive wheel 42101A may be raised by a preselected amount 42102 (FIG. 4F) and clear the driving surface. Device 42100A can successfully navigate on relatively solid level surfaces. When driving in standard mode, the occupancy grid 40244 (FIG. 4C) may reflect the surface type restrictions (see Table I) and thus may enable selection of compatible modes, or may vary depending on the surface type and current mode. based configuration changes may be possible.

Referring now to FIGS. 4G-4J , device 42100B/C may be configured to operate in a four-wheel mode in accordance with the present teachings. In one configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B may rest on the ground as the device 42100A navigates its path. Casters 42103 may be retracted and clear the driving surface by a preselected amount 42104 ( FIG. 4H ). In another configuration in the four-wheel mode, the first drive wheel 42101A and the second drive wheel 42101B may substantially rest on the ground as the device 42100A navigates its path. The casters 42103 may be retracted and the chassis 42111 may, for example, be rotated to accommodate a discontinuous surface (thus moving the casters 42103 further away from the ground). In this configuration, the casters 42103 may clear the driving surface by a preselected amount 42108 ( FIG. 1J ). Devices 42100A/C can successfully navigate on a variety of surfaces, including soft surfaces and discontinuous surfaces. In another configuration in the four-wheel mode, the second drive wheel 42101B is connected to the device 42100A while the first drive wheel 42101A can be raised as the device 42100C ( FIG. 1J ) navigates its path. ) can be placed on the ground as The casters 42103 may be retracted and the chassis 42111 may, for example, be rotated to accommodate a discontinuous surface (thus moving the casters 42103 further away from the ground). When driving in 4-wheel mode, the occupancy grid 40244 (FIG. 4C) may reflect the surface type (see Table I) and thus may enable selection of a compatible mode, or the surface type and current mode It is possible to enable configuration changes based on

Referring now to FIG. 4K , a method 40150 for real-time control of a device configuration of a device such as an AV, eg, but not limited to, moving a path based on at least one environmental factor and device configuration, comprises: Although not limited thereto, the steps of receiving sensor data ( 40151 ), determining a surface type based at least on the sensor data ( 40153 ), and determining a current mode based at least on the surface type and current device configuration ( 40155) may be included. The method 40150 includes determining a next device configuration based at least on a current mode and a surface type ( 40157 ), determining move commands based at least on the next device configuration ( 40159 ), and at least based on the move commands. It may include changing the current device configuration to the next device configuration ( 40161 ).

Referring now primarily to FIG. 5A , annotated point data 10379 ( FIG. 5B ) may be provided to the device controller 10111 to respond to objects appearing in the path of the AV. Annotated point data 10379 ( FIG. 5B ), which may be the basis for route information that may be used to instruct AV 10101 ( FIG. 1A ) to move a route, includes, but is not limited to, navigable edges , eg, but not limited to, a mapped trajectory, such as mapped trajectory 10413/10415 ( FIG. 5D ), and labeling such as, but not limited to, SDSFs 10377 ( FIG. 5C ). may include features. The mapped trajectory 10413/10415 ( FIG. 5C ) may include a graph of initial weights assigned to edges of the root space and portions of the root space. A graph of edges may include characteristics such as, but not limited to, directionality and capacitance, and edges may be categorized according to these characteristics. The mapped trajectory 10413/10415 ( FIG. 5C ) may include a cost modifier associated with surfaces of the root space and a driving mode associated with edges. Driving modes may include, but are not limited to, path following and SDSF climbing. Other modes may include, for example, but not limited to, modes of operation such as autonomous, mapping, and waiting for intervention. Ultimately, a path may be selected based at least on the lower cost modifiers. Topologies that are relatively far from the mapped trajectories 10413/10415 (FIG. 5C) may have higher cost modifiers and may be of less interest when forming paths. As the AV 10101 ( FIG. 1A ) operates, the initial weights can be adjusted, possibly resulting in a modification of the path. The adjusted weights may be used to adjust the edge/weight graph 10381 ( FIG. 5B ) and may be based at least on the current driving mode, current surface and edge category.

With continued reference to FIG. 5A , the device controller 10111 may include a feature processor capable of performing certain tasks related to incorporating the eccentricities of any features into a path. In some configurations, the feature processor may include, but is not limited to, the SDSF processor 10118 . In some configurations, the device controller 10111 may include, but is not limited to, an SDSF processor 10118 , a sensor processor 10703 , a mode controller 10122 , and a base controller 10114 each described herein. there is. SDSF processor 10118 , sensor processor 10703 , and mode controller 10122 may provide inputs to base controller 10114 .

With continued reference to FIG. 5A , the base controller 10114 is configured such that, based at least on the inputs provided by the mode controller 10122 , the SDSF processor 10118 , and the sensor processor 10703 , the power base 10112 is edge/weight Information available for driving the AV 10101 ( FIG. 1A ) on the path determined by the base controller 10114 based at least on the graph 10381 ( FIG. 5B ) may be determined. In some configurations, the base controller 10114 can ensure that the AV 10101 ( FIG. 1A ) can follow a predetermined route from a starting point to a destination, based at least on external and/or internal conditions. A predetermined path can be modified. In some configurations, external conditions may include, but are not limited to, stop lights, SDSFs, and obstacles in or near the path driven by AV 10101 ( FIG. 1A ). In some configurations, internal conditions may include, but are not limited to, mode transitions that reflect the response AV 10101 ( FIG. 1A ) makes to external conditions. The device controller 10111 may determine commands to transmit to the power base 10112 based at least on external and internal conditions. The commands may include, but are not limited to, speed and direction commands that may instruct AV 10101 ( FIG. 1A ) to move at a commanded speed in a commanded direction. Other instructions may include, for example, groups of instructions that enable a feature response, such as, for example, SDSF climbing. The base controller 10114 may route by conventional methods including, but not limited to, for example, Interior Point Optimizer (IPOPT) large-scale nonlinear optimization (https://projects.coin-or.org/Ipopt). It is possible to determine the desired speed between waypoints of The base controller 10114 may determine the desired path based at least on conventional techniques, such as, but not limited to, techniques based on, for example, Dykstra's algorithm, the A* search algorithm, or the Breadth-first search algorithm. there is. The base controller 10114 may form a box around the mapped trajectory 10413/10415 ( FIG. 5C ) to establish an area in which obstacle detection may be performed. The height of the payload carrier, when adjustable, may be adjusted based, at least in part, on the directed velocity.

With continued reference to FIG. 5A , the base controller 10114 may convert speed and direction determinations into motor commands. For example, but not limited to, when encountering an SDSF such as a curve or slope, the base controller 10114 raises the payload carrier 10173 (FIG. 1A), in SDSF climbing mode, You can instruct the power base 10112 to align the AV 10101 ( FIG. 1A ) with the SDSF at an angle of approximately 90° and reduce the speed to a relatively low level. When AV 10101 (FIG. 1A) is climbing a substantially discontinuous surface, base controller 10114 causes the speed to increase as increased torque is required to move AV 10101 (FIG. 1A) up an incline. It may direct the power base 10112 to transition into the climbing phase. When the AV 10101 ( FIG. 1A ) encounters a relatively level surface, the base controller 10114 may reduce its velocity so that it remains on top of any flat portion of the SDSF. In the case of a reduction ramp associated with a flat portion, the AV 10101 (FIG. 1A) begins to descend a substantially discontinuous surface, and when both wheels are on the reduction ramp, the base controller 10114 causes the speed to increase. can allow For example, but not by way of limitation, when faced with an SDSF such as a slope, a slope may be identified and treated as a structure. For example, the features of the structure may include a preselected size of the ramp. The ramp may include an inclination of approximately 30°, and optionally but not limited to on either side of a plateau. The device controller 10111 (FIG. 5A) can differentiate between the obstacle and the slope by comparing the angle of the perceived feature to the expected slope ramp angle, where the angle is to be received from the sensor processor 10703 (FIG. 5A). can

Referring now primarily to FIG. 5B , SDSF processor 10118 performs traversal by AV 10101 ( FIG. 1A ), from blocks of drivable surfaces formed by a mesh of polygons represented in annotated point data 10379 , by AV 10101 ( FIG. 1A ). can locate navigable edges that can be used to create a path for Within SDSF buffer 10407 (FIG. 5C), which may form an area of a preselected size around SDSF line 10377 (FIG. 5C), navigable edges may be erased in preparation for special processing for a given SDSF traversal. can be (see FIG. 5d). Closed line segments, such as segment 10409 ( FIG. 5C ) may be drawn to bisect SDSF buffer 10407 ( FIG. 5C ) between previously determined pairs of SDSF points 10789 ( FIG. 1N ). In some configurations, for a closed line segment to be considered a candidate for SDSF traversal, segment ends 10411 ( FIG. 5C ) may belong to an undisturbed portion of the drivable surface, and AV 10101 ( FIG. 5C ) There may be sufficient space for 1a) to move between adjacent SDSF points 10789 ( FIG. 1N ) along line segments, and the area between SDSF points 10789 ( FIG. 1N ) may be a drivable surface. . Segment ends 10411 ( FIG. 5C ) may be connected to the underlying topology to form vertices and drivable edges. For example, line segments 10461 , 10463 , 10465 , and 10467 ( FIG. 5C ) that meet the traversal criteria are shown as part of the topology in FIG. 5D . In contrast, line segment 10409 ( FIG. 5C ) does not meet at least the criteria because segment end 10411 ( FIG. 5C ) does not belong on a drivable surface. Overlapping SDSF buffers 10506 ( FIG. 5C ) may indicate an SDSF discontinuity that may be weighted for SDSF traversal of SDSFs within overlapping SDSF buffers 10506 ( FIG. 5C ). SDSF line 10377 (FIG. 5C) may be smoothed, the positions of SDSF points 10789 (FIG. 1N) may be adjusted such that they are separated by a preselected distance, and the preselected distance may be AV 10101 (FIG. 1A). is based at least on the footprint of

With continued reference to FIG. 5B , the SDSF processor 10118 can transform the annotated point data 10379 into an edge/weight graph 10381 that includes topology modifications for SDSF traversal. The SDSF processor 10118 may include a seventh processor 10601 , an eighth processor 10702 , a ninth processor 10603 , and a tenth processor 10605 . The seventh processor 10601 may transform the coordinates of the points in the annotated point data 10379 into a global coordinate system to achieve compatibility with GPS coordinates and generate a GPS-compatible dataset 10602 . . The seventh processor 10601 may generate the GPS-compatible dataset 10602 using, for example, conventional processes such as, but not limited to, an affine matrix transform and a PostGIS transform. The World Geodetic System (WGS) can be used as a standard coordinate system because it considers the curvature of the earth. The map can be stored in a Universal Transverse Mercator (UTM) coordinate system and switched to WGS when it is necessary to find where specific addresses are located.

Referring now primarily to FIG. 5C , an eighth processor 10702 ( FIG. 5B ) smoothes the SDSFs and determines the boundary of the SDSF 10377 , creates buffers 10407 around the SDSF boundary, and modifies the cost of the surface. The ruler can be increased further away from the SDSF boundary. The mapped trajectories 10413/10415 may be the special case suboptimal with the lowest cost modifier. The lower cost modifiers 10406 may generally be located near the SDSF boundary, while the higher cost modifiers 10408 may generally be located relatively further away from the SDSF boundary. The eighth processor 10702 may provide the point cloud data with costs 10704 ( FIG. 5B ) to the ninth processor 10603 ( FIG. 5B ).

With continued reference primarily to FIG. 5C , the ninth processor 10603 ( FIG. 5B ) calculates approximately 90° approaches 10604 ( FIG. 5B ) to the AV 10101 ( FIG. 1A ), labeling them as traversable. SDSFs 10377 that have met the criteria for Criteria may include SDSF width and SDSF smoothness. Line segments, such as line segment 10409, whose length may be the minimum entry distance that AV 10101 ( FIG. 1A ) may require to access SDSF 10377 , and the minimum entry distance that may be required to exit SDSF 10377 . It can be created to represent a minimum exit distance. Segment endpoints, such as endpoint 10411, may be integrated with a basic routing topology. The criteria used to determine whether SDSF access is possible may eliminate some accessibility possibilities. SDSF buffers, such as SDSF buffer 10407 , may be used to compute valid accesses and compute root topology edge generation.

Referring again primarily to FIG. 5B , a tenth processor 10605 generates an edge/weight graph 10381 from the graph of topology, edges and weights, developed herein, that may be used to calculate paths through the map. can create The topology may include cost modifiers and driving modes, and edges may include directionality and capacity. Weights may be adjusted at runtime based on information from any number of sources. The tenth processor 10605 may provide the base controller 10114 with at least one sequence of ordered points, along with a recommended driving mode at the specific points, to enable route generation. Each point in each sequence of points represents a labeling and location of a possible path point on the treated drivable surface. In some configurations, the labeling may indicate that the point represents some of a feature that may be encountered along a path, such as, but not limited to, an SDSF. In some configurations, the feature may be further labeled with a suggested treatment based on the type of feature. For example, in some configurations, if a path point is labeled as SDSF, the additional labeling may include a mode. The mode is by AV 10101 (FIG. 1A), e.g., AV 10101 (FIG. 1A) to enable AV 10101 (FIG. 1A) to traverse SDSF 10377 (FIG. 5C). may be interpreted as suggested driving commands for AV 10101 (FIG. 1A), such as switching to SDSF climbing mode 100-31 (FIG. 5E).

Referring now to FIG. 5E , in some configurations, the mode controller 10122 can provide directions to the base controller 10114 ( FIG. 5A ) for effecting a mode transition. The mode controller 10122 may establish the mode in which the AV 10101 (FIG. 1A) is moving. For example, the mode controller 10122 may change the mode indication when the SDSF is identified along a movement path, e.g., changing between the path following mode 10100-32 and the SDSF climbing mode 10100-31. may be provided to the base controller 10114 . In some configurations, the annotated point data 10379 ( FIG. 5B ) may include mode identifiers at various points along the route, for example, when the mode is changed to accommodate the route. For example, if SDSF 10377 (FIG. 5C) has been labeled in annotated point data 10379 (FIG. 5B), device controller 10111 can determine the mode identifier(s) associated with the root point(s) and , possibly adjusting the commands to the power base 10112 (FIG. 5A) based on the desired mode. In addition to SDSF climbing mode 10100-31 and path following mode 10100-32, in some configurations, AV 10101 (FIG. -wheel) mode 10100 - 2 may be included, but operation modes not limited thereto may be supported. The height of the payload carrier 10173 ( FIG. 1A ) can be adjusted to provide the necessary clearance through obstacles and along slopes.

Referring now to FIG. 5F , a method 11150 for navigating an AV towards a target point across at least one SDSF includes, but is not limited to, SDSF information related to the SDSF, the location of the target point, and the location of the AV. and receiving 11151 . The SDSF information may include, but is not limited to, a set of points each classified as SDSF points, and an associated probability for each point that the point is an SDSF point. The method 11150 may include the steps of drawing 11153 a closed polygon including the location of the AV, the location of the target point, and drawing a path line between the target point and the location of the AV. The closed polygon may include a preselected width. Table I includes possible ranges for the preselected variables discussed herein. Method 11150 may include selecting 11155 two of the SDSF points located within the polygon, and drawing 11157 an SDSF line between the two points. In some configurations, the selection of SDSF points may be made randomly or in any other manner. If there are fewer points than the first preselected number of points within the first preselected distance of the SDSF line at 11159 , and at 11161 select SDSF points, draw a line between them, and If there are a second preselected number of attempts in having a smaller number of points than the first preselected number of points, the method 11150 can include returning to step 11155 . If there are a second preselected number of attempts at selecting SDSF points at 11161, drawing a line between them, and having fewer than the first preselected number of points around the SDSF line, the method ( 11150 may include a notification 11163 that no SDSF lines were detected.

Referring now primarily to FIG. 5G , if there is a first preselected number of points or more at 11159 ( FIG. 5F ), the method 11150 fits the curve to points within a first preselected distance of the SDSF line. It may include a step 11165 of If the number of points within the first preselected distance of the curve at 11167 exceeds the number of points within the first preselected distance of the SDSF line, and at 11171 if the curve intersects the path line, and at 11173 at 11173 If there are no gaps between points on the curve that exceed the selected distance, the method 11150 can include identifying the curve as an SDSF line 11175 . if the number of points within the first preselected distance of the curve at 11167 does not exceed the number of points within the first preselected distance of the SDSF line, or if the curve does not intersect the path line at 11171, or at 11173 If there are gaps between points on the curve that exceed 2 preselected distances, and at 11177 the SDSF line does not remain stable, and at 11169 the curve fitting is not attempted more than a second preselected number of trials. If not, the method 11150 may include returning to step 11165 . A stable SDSF line is the result of subsequent iterations yielding equal or fewer points.

Referring now primarily to FIG. 5H , method 11150 if the curve fitting attempted a second preselected number of trials at 11169 ( FIG. 5G ), or if the SDSF line remains stable or degrades at 11177 ( FIG. 5G ). may include receiving 11179 occupancy grid information. The occupancy grid can provide the probability that obstacles are present at specific points. Occupancy grid information may augment SDSF and path information found in polygons surrounding AV paths and SDSF(s) when the occupancy grid contains captured and/or computed data over a common geographic area having polygons. Method 11150 may include selecting 11181 a point and its associated probability from a common geographic area. if the probability that an obstacle is present at the selected point is greater than the preselected percentage at 11183, and if the obstacle lies between the AV and the target point at 11185, and at 11186 the obstacle is a third preselected from the SDSF line between the SDSF line and the target point If less than the distance, the method 11150 may include projecting the obstacle to the SDSF line 11187 . If at 11183 the location is less than or equal to a preselected percentage probability that includes the obstacle, or at 11185 the obstacle does not lie between the AV and the target point, or at 11186 the obstacle is a third pre-selected from the SDSF line between the SDSF and the target point. If the distance is greater than or equal to the selected distance, and if there are more obstacles to process at 11189 , the method 11150 may include resuming processing at 11179 .

Referring now primarily to FIG. 5I , when there are no more obstacles to process at 11189 ( FIG. 5H ), the method 11150 connects the projections and finds the endpoints of the connected projections along the SDSF line 11191 . may include The method 11150 can include marking 11193 the portion of the SDSF line between the projection end points as non-traversal. Method 11150 can include marking 11195 a portion of the SDSF line that is outside of the non-traversable section as traversable. The method 11150 may include converting the AV into a fifth preselected amount perpendicular to the traversable section of the SDSF line 11197 . If at 11199 the heading error for the line perpendicular to the traversable section of the SDSF line is greater than the first preselected amount, the method 11150 may include slowing the AV by a ninth preselected amount 11251 . The method 11150 may include driving the AV forward towards the SDSF line and slowing it down 11253 by a second preselected amount per meter distance between the AV and the traversable SDSF line. If the distance of the AV from the traversable SDSF line at 11255 is less than a fourth preselected distance, and if the heading error is greater than or equal to a third preselected amount for a line perpendicular to the SDSF line at 11257, the method 11150 determines the AV and delaying 11252 by a ninth preselected amount.

Referring now primarily to FIG. 5J , if the heading error at 11257 ( FIG. 5I ) is less than a third preselected amount for a line perpendicular to the SDSF line, the method 11150 ignores the updated SDSF information and proceeds at the preselected rate. It may include driving the AV ( 11260 ). If at 11259 the elevation of the front portion of the AV relative to the rear portion of the AV is between the sixth preselected amount and the fifth preselected amount, then the method 11150 drives the AV forward and an eighth preselected amount per degree of lift and increasing the speed of the AV to 11261 . If at 11263 the elevation of the forward relative to the rear of the AV is less than the sixth preselected amount, the method 11150 may include driving the AV forwardly at the seventh preselected speed 11265 . If at 11267 the back of the AV is greater than the fifth preselected distance from the SDSF line, the method 11150 may include informing 11269 that the AV has completed traversing the SDSF. If at 11267 the back of the AV is less than or equal to the fifth preselected distance from the SDSF line, the method 11150 may include returning to step 11260 .

Referring now to FIG. 5K , a system 51100 for navigating an AV towards a target point across at least one SDSF may include a path line processor 11103 , an SDSF detector 11109 and an SDSF controller 11127 . can, but is not limited to. System 51100 may be operatively coupled with a surface processor 11601 capable of processing sensor information, which may include, but is not limited to, images of the surroundings of AV 10101 (FIG. 5L), for example. there is. The surface processor 11601 may provide real-time surface feature updates including an indication of SDSFs. In some configurations, cameras can provide RGB-D data whose points can be classified according to surface type. In some configurations, system 51100 can process points classified as SDSFs and their associated probabilities. System 51100 may be operatively coupled with a system controller 11602 that may manage aspects of operation of AV 10101 ( FIG. 5L ). The system controller 11602 may maintain an occupancy grid 11138 that may contain information from available sources regarding the navigable area near the AV 10101 ( FIG. 5L ). Occupancy grid 11138 may include probabilities that obstacles are present. This information can be used in conjunction with the SDSF information to determine if the SDSF 10377 (FIG. 5N) can be traversed by the AV 10101 (FIG. 5L) without encountering an obstacle 11681 (FIG. 5M). there is. The system controller 11602 may determine a rate limit 11148 that the AV 10101 ( FIG. 5N ) must not exceed based on the environment and other information. The speed limit 11148 can be used as a guide to the speeds set by the system 51100 or can be overridden. System 51100 may be operatively coupled with a base controller 10114 that may send driving commands 11144 generated by SDSF controller 11127 to driving components of AV 10101 ( FIG. 5L ). . Base controller 10114 may provide information to SDSF controller 11127 regarding the orientation of AV 10101 ( FIG. 5L ) during SDSF traversal.

With continued reference to FIG. 5K , the path line processor 11103 may continue to receive in real time surface classification points 10789 , which may include, but are not limited to, points classified as SDSFs. The path line processor 11103 may, for example, but not be limited to, the position of the target point 11139 and the AV position as indicated by the center 11202 ( FIG. 51 ) of the AV 10101 ( FIG. 5L ). (11141) may be received. The system 51100 has a polygon processor 11105 that draws a polygon 11147 that includes an AV position 11141 , a position of a target point 11139 , and a path 11214 between the target point 11139 and the AV position 11141 . may include. Polygon 11147 may include a preselected width. In some configurations, the preselected width may include approximately the width of AV 10101 ( FIG. 5L ). SDSF points 10789 that fall within polygon 11147 may be identified.

With continued reference to FIG. 5K , SDSF detector 11109 may receive surface classification points 10789 , path 11214 , polygon 11147 , and target point 11139 , available within incoming data. According to the criteria presented herein, the most suitable SDSF line 10377 may be determined. The SDSF detector 11109 may include, but is not limited to, a point processor 11111 and an SDSF line processor 11113 . The point processor 11111 may include selecting two of the SDSF points 10789 located within the polygon 11147 and drawing the SDSF 10377 line between the two points. If there are fewer than a first preselected number of points within a first preselected distance of SDSF line 10377, select SDSF points 10789, draw a line between the two points, , if there are fewer attempts than the second preselected number of attempts in having fewer than the first preselected number of points around the SDSF line, then the point processor 11111 is As mentioned, this may involve looping again through a select-picture-testing loop. If there are a second preselected number of attempts in selecting SDSF points, drawing a line between them, and having fewer than the first preselected number of points around the SDSF line, point processor 11111 ) may include indicating that no SDSF lines were detected.

With continued reference to FIG. 5K , SDSF line processor 11113 generates curves 11609 - 11611 ( FIG. 5L ) of first preselected line 10377 of SDSF line 10377 when there are more than a first preselected number of points 10789 . fitting to points 10789 that are within a distance. If the number of points 10789 within the first preselected distance of the curve 11609-11611 (FIG. 51) exceeds the number of points 10789 within the first preselected distance of the SDSF line 10377, and a gap between points 10789 on curves 11609-11611 ( FIG. 5L ) when curves 11609-11611 ( FIG. 5L ) intersect path line 11214 , and exceeding a second preselected distance. In the absence of , SDSF line processor 11113 may (eg) identify curves 11609 - 11611 ( FIG. 5L ) as SDSF line 10377 . the number of points 10789 that are within the preselected distance of the curves 11609 - 11611 ( FIG. 5L ) does not exceed the number of points 10789 that are within the first preselected distance of the SDSF line 10377 , or The gap between the points 10789 on the curve 11609-11611 ( FIG. 5L ) where the curve 11609-11611 ( FIG. 5L ) does not intersect the path line 11214 , or exceeds a second preselected distance. are present, and if the SDSF line 10377 does not remain stable, and if the curve fitting has not been attempted more than a second preselected number of attempts, the SDSF line processor 11113 executes the curve fitting loop. You can run it again.

With continued reference to FIG. 5K , SDSF controller 11127 may receive SDSF line 10377 , occupancy grid 11138 , AV orientation changes 11142 and rate limit 11148 , and SDSF 10377 ( FIG. SDSF instructions 11144 to drive AV 10101 (FIG. 5L) to precisely traverse 5n). The SDSF controller 11127 may include, but is not limited to, an obstacle processor 11115 , an SDSF approach 11131 , and an SDSF traversal 11133 . Obstacle processor 11115 may receive SDSF line 10377 , target point 11139 and occupancy grid 11138 , any of the obstacles identified in occupancy grid 11138 , any of which are SDSF 10377 . It can be determined whether the AV 10101 (FIG. 5N) can be obstructed when traversing (FIG. 5N). The obstacle processor 11115 may include, but is not limited to, an obstacle selector 11117 , an obstacle tester 11119 , and a traversing locator 11121 . The obstacle selector 11117 may include, but is not limited to, receiving an occupancy grid 11138 as described herein. The obstacle selector 11117 may include selecting the occupancy grid points and their associated probabilities from a geographic area common to both the occupancy grid 11138 and the polygon 11147 . Obstacle tester 11119 determines if the probability that an obstacle exists at the selected grid point is higher than a preselected percentage, and if the obstacle is between AV 10101 ( FIG. 5L ) and target point 11139 , and if the obstacle is an SDSF line less than a third preselected distance from the SDSF line 10377 between the 10377 and the target point 11139 , wherein the obstacle tester 11119 projects the obstacle onto the SDSF line 10377 , the SDSF line forming projections 11621 that intersect with 10377 . If the location is less than or equal to a preselected percentage probability that contains the obstacle, or if the obstacle is not between the AV 10101 ( FIG. 5L ) and the target point 11139 , or if the obstacle is between the SDSF line 10377 and the target point If at a distance greater than or equal to a third preselected distance from SDSF line 10377 between may include

With continued reference to FIG. 5K , the traversal locator 11121 is located along the SDSF line 10377 with the connecting projection points of the connected projections 11621 ( FIG. 5M ) and the positioning end points 11622/11623 ( FIG. 5M ). ) may be included. The traversing locator 11121 may include a marking portion 11624 ( FIG. 5M ) of the SDSF line 10377 between the projection end points 11622/11623 ( FIG. 5M ) as non-traversable. The traversing locator 11121 may include marking portions 11626 ( FIG. 5M ) of the SDSF line 10377 that are external to the non-traversable portion 11624 ( FIG. 5M ) as being traversable.

With continued reference to FIG. 5K , SDSF approach 11131 is to divert AV 10101 ( FIG. 5N ) into a fifth preselected amount perpendicular to traversable portion 11626 ( FIG. 5N ) of SDSF line 10377 . sending SDSF commands 11144 . If the heading error for the vertical line 11627 ( FIG. 5N ) perpendicular to the traversable section 11626 ( FIG. 5N ) of the SDSF line 10377 is greater than the first preselected amount, then the SDSF access 11131 is a ninth sending SDSF commands 11144 to slow AV 10101 (FIG. 5N) by a preselected amount. In some configurations, the ninth preselected amount may range from very slow to completely stationary. SDSF approach 11131 sends SDSF commands 11144 to drive AV 10101 (FIG. 5N) forward towards SDSF line 10377, a second preselected amount of AV ( sending SDSF commands 11144 to slow 10101 ( FIG. 5N ). If the distance between AV 10101 ( FIG. 5N ) and traversable SDSF line 11626 ( FIG. 5N ) is less than a fourth preselected distance, and the heading error is a third for a line perpendicular to SDSF line 10377 ) If greater than or equal to the preselected amount, SDSF access 11131 may include sending SDSF commands 11144 to slow AV 10101 ( FIG. 5N ) by a ninth preselected amount.

With continued reference to FIG. 5K , if the heading error is less than a third preselected amount for a line perpendicular to SDSF line 10377 , SDSF traversal 11133 ignores the updated SDSF information and AV 10101 at the preselected rate. ) ( FIG. 5N ), sending SDSF commands 11144 . AV orientation changes 11142 result in the rise of leading edge 11701 (FIG. 5N) of AV 10101 (FIG. 5N) relative to trailing edge 11703 (FIG. 5N) of AV 10101 (FIG. 5N). If it indicates that it is between the 6 preselected amount and the fifth preselected amount, SDSF traversal 11133 sends SDSF commands 11144 to drive AV 10101 (FIG. 5N) forward, per degree of ascent. sending SDSF commands 11144 to increase the speed of AV 10101 (FIG. 5N) at a selected rate. AV orientation changes 11142 indicate that the rise of leading edge 11701 (FIG. 5N) relative to trailing edge 11703 (FIG. 5N) of AV 10101 (FIG. 5N) is less than a sixth preselected amount. In that case, SDSF traversal 11133 may include sending SDSF commands 11144 to drive AV 10101 ( FIG. 5N ) forward at a seventh preselected speed. If AV position 11141 indicates that trailing edge 11703 (FIG. 5N) is greater than the fifth preselected distance from SDSF line 10377, SDSF traversal 11133 indicates that AV 10101 (FIG. 5N) 10377), indicating that traversing has been completed. If the AV position 11141 indicates that the trailing edge 11703 (FIG. 5N) is less than or equal to the fifth preselected distance from the SDSF line 10377, then the SDSF traversal 11133 begins ignoring the updated SDSF information. This may involve re-executing the loop.

Some exemplary ranges for preselected values described herein may include, but are not limited to, those listed in Table II.

Table II.

Referring now to FIG. 5O , to support real-time data collection, in some configurations, a system of the present teachings may receive data, such as, but not limited to, RGD-D camera image data, for example. It is possible to create positions in three-dimensional space of various surface types in a city. The system can rotate the images 12155 and transform them from the camera coordinate system 12157 to the UTM coordinate system 12159 . The system may generate polygon files from the transformed images, which may represent three-dimensional locations associated with the surface type 12161 . Method 12150 for locating features 12151 from camera images 12155 received by AV 10101 - AV 10101 having pose 12163 - includes, but is not limited to, AV receiving camera images 12155 by 10101 . Each of the camera images 12155 may include an image timestamp 12171 , and each of the images 12155 may include image color pixels 12167 and image depth pixels 12169 . The method 12150 includes receiving a pose 12163 of the AV 10101 , a pose 12163 having a pose timestamp 12171 , and an image timestamp 12165 closest to the pose timestamp 12171 . determining the selected image 12173 by identifying the image from the camera images 12155 . The method 12150 includes separating image color pixels 12167 from image depth pixels 12169 in a selected image 12173 , and adding the image color pixels 12167 to a first machine learning model 12177 and determining image surface classifications 12161 for the selected image 12173 by providing the image depth pixels 12169 to the second machine learning model 12179 . The method 12150 may include determining peripheral points 12181 of features in the camera image 12173, wherein the features may include feature pixels 12151 within the perimeter, and the feature pixels ( 12151 ) each have the same surface classification 12161 , and each of the surrounding points 12181 has a set of coordinates 12157 . Method 12150 may include transforming each of the sets of coordinates 12157 to UTM coordinates 12159 .

Compositions of the present teachings relate to computer systems for accomplishing the methods discussed in the description herein, and computer readable media containing programs for performing these methods. The raw data and results may be stored, printed, displayed, transmitted to another computer, and/or transmitted elsewhere for future retrieval and processing. Communication links may be wired or wireless, using, for example, cellular communication systems, military communication systems, and satellite communication systems. Portions of the system may run on a computer with a variable number of CPUs. Other alternative computer platforms may be used.

The present configuration also relates to software for accomplishing the methods discussed herein, and computer-readable media storing the software for achieving these methods. The various modules described herein may be implemented on the same CPU, or may be implemented on different computers. As per the statement, the present composition has been described in language that is rather specific for its structural and methodological features. However, it is to be understood that the present arrangement is not limited to the specific features shown and described, since the means disclosed herein include preferred forms of carrying out the present arrangement.

The methods may be implemented, in whole or in part, electronically. Signals indicative of actions taken by elements of the system and other disclosed configurations may travel through the at least one live communication network. The control and data information may be executed and stored electronically on at least one computer readable medium. The system may be implemented to run on at least one computer node in at least one live communication network. Common forms of at least one computer readable medium include, for example, but not limited to, a floppy disk, a flexible disk, a hard disk, magnetic tape, or any other magnetic medium, compact disk read only memory, or any other optical medium. media, punched cards, paper tape, or any other physical medium having patterns of holes, random access memory, programmable read only memory, and erasable programmable read only memory (EPROM), flash EPROM, or any may include other memory chips or cartridges, or any other computer readable medium. Also, at least one computer-readable medium may include, but is not limited to, Graphic Interchange Format (GIF), Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Scalable Vector Graphics (SVG), and Tagged Image (TIFF). File Format), if necessary, in accordance with the appropriate licenses, and may include the graphs in any form.

Although the present teachings have been described above in terms of specific configurations, it should be understood that they are not limited to these disclosed configurations. Many modifications and other constructions will occur to those skilled in the art to which this application pertains, and are intended and covered by both the present disclosure and the appended claims. The scope of the present teachings is determined by appropriate interpretation and construction of the appended claims and their legal equivalents, as understood by those skilled in the art to rely on the disclosure in this specification and the appended drawings. It is intended to be

Claims (93)

An autonomous delivery vehicle comprising:
a power base comprising two powered front wheels, two powered rear wheels and an energy store, wherein the power base is configured to move at a commanded speed and in a commanded direction to effect transport of at least one object;
a cargo platform comprising a plurality of short-range sensors, the cargo platform mechanically attached to the power base;
a cargo container having a volume for receiving the at least one object, the cargo container mounted on top of the cargo platform;
a long-range sensor chute comprising a LIDAR and one or more cameras, the long-range sensor chute mounted on top of the cargo container; and
a controller for receiving data from the long-range sensor suite and the plurality of short-range sensors, wherein the controller determines the commanded speed and the commanded direction based at least on the data, and the controller is configured to: providing the commanded speed and the commanded direction to the power base;
Autonomous delivery vehicle. According to claim 1,
and the data from the plurality of short-range sensors include at least one characteristic of a surface on which the power base moves. According to claim 1,
wherein the plurality of short-range sensors include at least one stereo camera. According to claim 1,
wherein the plurality of short-range sensors include at least one IR projector, at least one image sensor, and at least one RGB sensor. According to claim 1,
wherein the plurality of short-range sensors include at least one radar sensor. According to claim 1,
and the data from the plurality of short-range sensors comprises RGB-D data. According to claim 1,
and the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. According to claim 1,
wherein the plurality of short-range sensors detect objects within 4 meters of the AV and the long-range sensor suite detects objects more than 4 meters away from the autonomous delivery vehicle. According to claim 1,
wherein the plurality of short-range sensors include a cooling circuit. According to claim 1,
wherein the plurality of short-range sensors include an ultrasonic sensor. 3. The method of claim 2,
the controller comprises executable code;
the executable code comprising accessing a map formed by a map processor;
The map processor,
a first processor accessing point cloud data from the long range sensor suite, the point cloud data representing the surface;
a filter for filtering the point cloud data;
a second processor forming processable portions from the filtered point cloud data;
a third processor for merging the processable portions into at least one polygon;
a fourth processor for locating and labeling at least one substantially discontinuous surface feature (SDSF) in the at least one polygon, wherein the locating and labeling, if present, forms labeled point cloud data;
a fifth processor for generating graphing polygons from the labeled point cloud data; and
and a sixth processor selecting a path from a starting point to an ending point based at least on the graphing polygons, wherein the AV traverses the at least one SDSF along the path. 12. The method of claim 11,
the filter comprises a seventh processor executing code;
The code is
conditionally removing from the point cloud data points indicative of transient objects and points indicative of outliers; and
and replacing the removed points with a preselected height. 12. The method of claim 11,
wherein the second processor comprises the executable code;
This executable code is
segmenting the point cloud data into the processable portions; and
and removing points of a preselected height from the processable portions. 12. The method of claim 11,
wherein the third processor comprises the executable code;
This executable code is
reducing the size of the processable portions by analyzing outliers, voxels, and normals;
growing regions from the reduced size processable portions;
determining initial drivable surfaces from the grown regions;
segmenting and meshing the initial drivable surfaces;
positioning polygons within the segmented and meshed initial drivable surfaces; and
and setting at least one drivable surface based at least on the polygons. 15. The method of claim 14,
the fourth processor comprises the executable code;
This executable code is
sorting the point cloud data of the initial drivable surfaces according to an SDSF filter, wherein the SDSF filter includes at least three categories of points; and
and locating at least one SDSF point based at least on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion. 16. The method of claim 15,
the fourth processor comprises the executable code;
This executable code is
and generating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF point, in combination, satisfies at least one second preselected criterion. 15. The method of claim 14,
generating the graphing polygons comprises an eighth processor comprising the executable code;
This executable code is
generating at least one polygon from the at least one drivable surface, the at least one polygon comprising outer edges;
smoothing the outer edges;
forming a driving margin based on the smoothed outer edges;
adding the at least one SDSF trajectory to the at least one drivable surface; and
and removing inner edges from the at least one drivable surface according to at least one third preselected criterion. 18. The method of claim 17,
smoothing the outer edges comprises a ninth processor comprising the executable code;
This executable code is
and trimming the outer edges outward to form outer edges. 19. The method of claim 18,
forming a driving margin of the smoothed outer edges comprises a tenth processor comprising the executable code;
This executable code is
and trimming the outer edges inward. According to claim 1,
The controller is
a subsystem for navigating at least one substantially discontinuous surface feature (SDSF) encountered by the autonomous delivery vehicle (AV), wherein the AV travels a path over a surface, the surface passing through the at least one SDSF wherein the path includes a start point and an end point;
The subsystem is
A first processor accessing a route topology, the route topology comprising at least one graphing polygon comprising filtered point cloud data, the filtered point cloud data comprising labeled features, wherein the point cloud data comprises: including drivable margin-;
a second processor for converting the point cloud data into a global coordinate system;
a third processor that determines boundaries of the at least one SDSF, wherein the third processor creates SDSF buffers of a preselected size around the boundaries;
a fourth processor to determine which of the at least one SDSF may be traversed based at least on at least one SDSF traversal criterion;
a fifth processor configured to generate an edge/weight graph based at least on the at least one SDSF traversal criterion, the transformed point cloud data, and the route topology; and
and a base controller to select a path from the start point to the end point based at least on the edge/weight graph. 21. The method of claim 20,
The at least one SDSF traversal criterion comprises:
a preselected width of the at least one SDSF and a preselected smoothness of the at least one SDSF;
a minimum entry distance and a minimum exit distance between the at least one SDSF and the AV, including a drivable surface, and
and the minimum entry distance between the AV and the at least one SDSF that accommodates approximately 90° access to the at least one SDSF by the AV. A method for managing a global occupancy grid for autonomous devices, comprising:
The global occupancy grid includes global occupancy grid cells, wherein the global occupancy grid cells are associated with an occupancy probability, the method comprising:
receiving sensor data from sensors associated with the autonomous device;
generating a local occupancy grid based at least on the sensor data, the locally occupied grid having locally occupied grid cells;
When the autonomous device moves from the first area to the second area,
accessing historical data associated with the second area;
generating a static grid based at least on the historical data;
moving the global occupancy grid to maintain the autonomous device in a central position of the global occupancy grid;
updating the moved global occupancy grid based on the static grid;
if at least one of the globally occupied grid cells matches the location of the autonomous device, then marking at least one of the globally occupied grid cells as unoccupied;
For each of the locally occupied grid cells,
calculating a position of the locally occupied grid cell on the global occupied grid;
accessing a first occupancy probability from the global occupancy grid cell at the location;
accessing a second occupancy probability from the locally occupied grid cell at the location; and
calculating a new occupancy probability at the location on the global occupancy grid based at least on the first occupancy probability and the second occupancy probability;
method. 23. The method of claim 22,
range-checking the new occupancy probability. 24. The method of claim 23,
The range-check is,
setting the new occupation probability to 0 if the new occupation probability <0; and
setting the new occupancy probability to 1 if the new occupancy probability>1. 23. The method of claim 22,
and setting the global occupancy grid cell to the new occupancy probability. 24. The method of claim 23,
and setting the global occupancy grid cell to the range-checked new occupancy probability. A method for creating and managing occupancy grids, comprising:
converting, by the local occupancy grid generating node, the sensor measurements into a frame of reference associated with the device;
generating a time-stamped measurement occupancy grid;
publishing the time-stamped measurement occupancy grid as a local occupancy grid;
creating a plurality of local occupancy grids;
generating a static occupancy grid based on surface properties in a repository, wherein the surface properties are associated with a location of the device;
moving a global occupancy grid associated with the location of the device to maintain the device and the local occupancy grid approximately centered relative to the global occupancy grid;
adding information from the static occupancy grid to the global occupancy grid;
marking an area in the global occupancy grid currently occupied by the device as unoccupied;
for each of the at least one cell in each local occupancy grid,
determining a location of the at least one cell in the global occupancy grid;
accessing a first value at the location;
determining a second value at the location based on a relationship between the first value and a cell value in the at least one cell in the local occupancy grid;
comparing the second value to a preselected range of probability; and
setting the global occupancy grid to the new value if a probability value is within the preselected probability range;
method. 28. The method of claim 27,
and posting the global occupancy grid. 28. The method of claim 27,
wherein the surface properties include a surface type and surface discontinuities. 28. The method of claim 27,
wherein the relationship comprises a summation. A system for creating and managing occupancy grids, comprising:
a plurality of local grid generating nodes generating at least one local occupancy grid, wherein the at least one local occupancy grid is associated with a location of a device, the at least one local occupancy grid comprising at least one cell;
a global occupancy grid manager accessing the at least one local occupancy grid;
The global occupancy grid manager,
create a static occupancy grid based on surface properties in the repository, wherein the surface properties are associated with the location of the device;
move a global occupancy grid associated with the location of the device to maintain the device and at least one local occupancy grid approximately centered relative to the global occupancy grid;
add information from the static occupancy grid to at least one global occupancy grid;
mark an area in the global occupancy grid currently occupied by the device as unoccupied;
for each of said at least one cell in each local occupancy grid,
determine a location of the at least one cell in the global occupancy grid;
access a first value at the location;
determine a second value at the location based on a relationship between the first value and a cell value in the at least one cell in the local occupancy grid;
compare the second value to a preselected range of probability;
setting the global occupancy grid to the new value if a probability value is within the preselected probability range;
system. A method for updating a global occupancy grid, comprising:
when an autonomous device has moved to a new location, updating the global occupancy grid with information from a static grid associated with the new location;
analyzing the surfaces at the new location;
if the surfaces are drivable, updating the surfaces and updating the global occupancy grid with the updated surfaces; and
updating the global occupancy grid with values from a repository of static values, wherein the static values are associated with the new location.
method. 33. The method of claim 32,
Updating the surfaces comprises:
accessing a local occupancy grid associated with the new location;
For each cell in the local occupancy grid,
accessing the locally occupied grid surface classification confidence value and the locally occupied grid surface classification;
If the local occupancy grid surface classification is the same as the global surface classification in the global occupancy grid in the cell, a global surface classification confidence value in the global occupancy grid is added to the locally occupied grid surface classification confidence value to form a sum and updating the global occupancy grid in the cell with the sum;
If the local occupancy grid surface classification is not the same as the global surface classification in the global occupancy grid in the cell, subtract the local occupancy grid surface classification confidence value from the global surface classification confidence value in the global occupancy grid forming a difference and updating the global occupancy grid with the difference;
if the difference is less than zero, updating the global occupancy grid with the local occupancy grid surface classification. 33. The method of claim 32,
Updating the global occupancy grid with values from the repository of static values comprises:
For each cell in the local occupancy grid,
accessing, from the local occupancy grid, a local occupancy grid probability value, a log odds value, in which the cell is occupied;
updating the log odds value in the global occupancy grid with the local occupancy grid log odds value in the cell;
If the preselected certainty that the cell is not occupied is met, and if the autonomous device is moving within lane barriers, and if a locally occupied grid surface classification indicates a drivable surface, the cell is reducing the log odds occupied in an occupancy grid;
increasing the log odds in the locally occupied grid if the autonomous device expects to encounter relatively uniform surfaces, and if the locally occupied grid surface classification indicates a relatively non-uniform surface; and
reducing the log odds in the locally occupied grid if the autonomous device expects to encounter relatively uniform surfaces, and if the locally occupied grid surface classification indicates a relatively uniform surface. . A method for real-time control of a configuration of a device, comprising:
The device includes a chassis, at least four wheels, a first side of the chassis operatively coupled with at least one of the at least four wheels, and an opposite side of the chassis operatively coupled with at least one of the at least four wheels. A second side comprising:
generating a map based at least on previous surface features and an occupancy grid, wherein the map is generated in non-real time, the map includes at least one location, the at least one location includes at least one surface feature and associated, wherein the at least one surface feature is associated with at least one surface classification and at least one mode;
determining current surface features as the device moves;
updating the occupancy grid with the current surface features in real time;
determining, from the occupancy grid and the map, a path the device can travel to traverse the at least one surface feature;
method. A method for real-time control of a configuration of a device, comprising:
The device includes a chassis, at least four wheels, a first side of the chassis operatively coupled with at least one of the at least four wheels, and an opposite side of the chassis operatively coupled with at least one of the at least four wheels. A second side comprising:
receiving environmental data;
determining a surface type based at least on the environmental data;
determining a mode based at least on the surface type and the first configuration;
determining a second configuration based at least on the mode and the surface type;
determining movement instructions based at least on the second configuration; and
controlling the configuration of the device using the move commands to change the device from the first configuration to the second configuration;
method. 37. The method of claim 36,
wherein the environmental data comprises RGB-D image data. 37. The method of claim 36,
populating the occupancy grid based at least on the surface type and the mode; and
and determining the movement instructions based at least on the occupancy grid. 39. The method of claim 38,
wherein the occupancy grid includes information based at least on data from at least one image sensor. 37. The method of claim 36,
wherein the environmental data includes a topology of the road surface. 37. The method of claim 36,
The configuration includes clustered two pairs of the at least four wheels, a first one of the two pairs positioned on the first side, a second one of the two pairs on the second side wherein the first pair comprises a first front wheel and a first rear wheel, and the second pair comprises a second front wheel and a second rear wheel. 42. The method of claim 41,
The step of controlling the configuration comprises:
and supplying adjusted power to the first pair and the second pair based at least on the environmental data. 42. The method of claim 41,
The step of controlling the configuration comprises:
the clustered front wheel to lift the first front wheel and the second front wheel from driving the at least four wheels and a pair of retracted casters, the pair of casters being operatively coupled to the chassis. converting the first pair and the clustered second pair to driving two wheels rotated, the device resting on the pair of the first rear wheel, the second rear wheel, and the casters; , method. 42. The method of claim 41,
The step of controlling the configuration comprises:
rotating a pair of clusters operatively coupled with first two powered wheels on the first side and second two powered wheels on the second side based at least on the environmental data; . 37. The method of claim 36,
The device further comprises a cargo container, wherein the cargo container is mounted on the chassis, the chassis controlling a height of the cargo container. 46. The method of claim 45,
and the height of the freight container is based at least on the environmental data. A system for real-time control of the configuration of a device, comprising:
The device comprises a chassis, at least four wheels, a first side of the chassis, and an opposite second side of the chassis, the system comprising:
a device processor receiving real-time environmental data surrounding the device, wherein the device processor determines a surface type based at least on the environmental data, the device processor determines a mode based at least on the surface type and a first configuration; , the device processor determines a second configuration based at least on the mode and the surface type; and
a power base processor that determines move instructions based at least on the second configuration, wherein the power base processor configures the configuration of the device by using the move instructions to change the device from the first configuration to the second configuration. control, including
system. 48. The method of claim 47,
wherein the environment data comprises RGB-D image data. 48. The method of claim 47,
and the device processor populates the occupancy grid based at least on the surface type and the mode. 50. The method of claim 49,
and the power base processor determines the movement instructions based at least on the occupancy grid. 50. The method of claim 49,
wherein the occupancy grid includes information based at least on data from at least one image sensor. 48. The method of claim 47,
wherein the environmental data includes a topology of the road surface. 48. The method of claim 47,
The configuration includes clustered two pairs of the at least four wheels, a first one of the two pairs positioned on the first side, a second one of the two pairs on the second side wherein the first pair has a first front wheel and a first rear wheel and the second pair has a second front wheel and a second rear wheel. 54. The method of claim 53,
The control of the configuration is,
and supplying adjusted power to the first pair and the second pair based at least on the environmental data. 54. The method of claim 53,
Control of the configuration may include controlling the first front wheel and the second front wheel from driving the at least four wheels and a pair of retracted casters, the pair of casters operatively coupled to the chassis. converting the clustered first pair and the clustered second pair to driving two wheels rotated for lifting, wherein the device is disposed on the first rear wheel, the second rear wheel, and the pair of casters. A system comprising — placed on. A method for maintaining a global occupancy grid comprising:
locating a first location of the autonomous device;
when the autonomous device moves to a second location associated with the global occupancy grid and the local occupancy grid;
updating the global occupancy grid with at least one occupancy probability value associated with the first location;
updating the global occupancy grid with at least one drivable surface associated with the local occupancy grid;
updating the global occupancy grid with surface confidences associated with the at least one drivable surface;
updating the global occupancy grid with the log odds of the at least one occupancy probability value using a first Bayesian function; and
adjusting the log odds based at least on characteristics associated with the second location; and
when the autonomous device is maintained in the first position and the global occupancy grid and the local occupancy grid are co-located;
updating the global occupancy grid with the at least one drivable surface associated with the local occupancy grid;
updating the global occupancy grid with the surface confidence levels associated with the at least one drivable surface;
updating the global occupancy grid with the log odds of the at least one occupancy probability value using a second Bayesian function; and
adjusting the log odds based at least on characteristics associated with the second location;
method. 36. The method of claim 35,
Creating the map includes:
accessing point cloud data representing the surface;
filtering the point cloud data;
forming the filtered point cloud data into processable portions;
merging the processable portions into at least one concave polygon;
locating and labeling the at least one SDSF in the at least one concave polygon, wherein the locating and labeling forms labeled point cloud data;
generating graphing polygons based at least on the at least one concave polygon; and
selecting the path from a start point to an end point based at least on the graphing polygons, wherein the AV traverses the at least one SDSF along the path. 58. The method of claim 57,
The step of filtering the point cloud data,
conditionally removing points representing transient objects and points representing outliers from the point cloud data; and
replacing the removed points with a preselected height. 58. The method of claim 57,
Forming the processable portions comprises:
segmenting the point cloud data into the processable portions; and
and removing points of a preselected height from the processable portions. 58. The method of claim 57,
The step of merging the processable parts comprises:
reducing the size of the processable portions by analyzing outliers, voxels, and normals;
growing regions from the reduced size processable portions;
determining initial drivable surfaces from the grown regions;
segmenting and meshing the initially drivable surfaces;
positioning polygons within the segmented and meshed initial drivable surfaces; and
and setting at least one drivable surface based at least on the polygons. 61. The method of claim 60,
Positioning and labeling the at least one SDSF comprises:
sorting the point cloud data of the initial drivable surfaces according to an SDSF filter, the SDSF filter comprising at least three categories of points; and
locating at least one SDSF point based at least on whether the at least three categories of points, in combination, satisfy at least one first preselected criterion. 62. The method of claim 61,
generating at least one SDSF trajectory based at least on whether a plurality of the at least one SDSF points, in combination, satisfy at least one second preselected criterion. 63. The method of claim 62,
The step of generating the graphing polygons,
generating at least one polygon from the at least one drivable surface, the at least one polygon comprising outer edges;
smoothing the outer edges;
forming a driving margin based on the smoothed outer edges;
adding the at least one SDSF trajectory to the at least one drivable surface; and
and removing inner edges from the at least one drivable surface according to at least one third preselected criterion. 64. The method of claim 63,
The smoothing of the outer edges comprises:
and trimming the outer edges outward to form outer edges. 64. The method of claim 63,
The step of forming a driving margin of the smoothed outer edges comprises:
and trimming the outer edges inward. An autonomous delivery vehicle comprising:
a power base comprising two powered front wheels, two powered rear wheels and an energy storage, wherein the power base is configured to move at a commanded speed;
a cargo platform comprising a plurality of short-range sensors, the cargo platform mechanically attached to the power base;
a cargo container having a volume for receiving one or more objects to be shipped, the cargo container mounted on top of the cargo platform;
a long-range sensor chute comprising a LIDAR and one or more cameras, the long-range sensor chute mounted on top of the cargo container; and
a controller receiving data from the long range sensor suite and the plurality of short range sensors;
Autonomous delivery vehicle. 67. The method of claim 66,
wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 67. The method of claim 66,
wherein the plurality of short-range sensors are stereo cameras. 67. The method of claim 66,
wherein the plurality of short-range sensors comprises an IR projector, two image sensors and an RGB sensor. 67. The method of claim 66,
wherein the plurality of short-range sensors are radar sensors. 67. The method of claim 66,
and the short-range sensors supply RGB-D data to the controller. 67. The method of claim 66,
and the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. 67. The method of claim 66,
wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects that are more than 4 meters away from the autonomous delivery vehicle. An autonomous delivery vehicle comprising:
a power base comprising at least two powered rear wheels, caster front wheels and an energy store, wherein the power base is configured to move at a commanded speed;
a cargo platform comprising a plurality of short-range sensors, the cargo platform mechanically attached to the power base;
a cargo container having a volume for receiving one or more objects to be shipped, the cargo container mounted on top of the cargo platform;
a long-range sensor chute comprising a LIDAR and one or more cameras, the long-range sensor chute mounted on top of the cargo container; and
a controller receiving data from the long range sensor suite and the plurality of short range sensors;
Autonomous delivery vehicle. 75. The method of claim 74,
wherein the plurality of short-range sensors detect at least one characteristic of a drivable surface. 75. The method of claim 74,
wherein the plurality of short-range sensors are stereo cameras. 75. The method of claim 74,
wherein the plurality of short-range sensors comprises an IR projector, two image sensors and an RGB sensor. 75. The method of claim 74,
wherein the plurality of short-range sensors are radar sensors. 75. The method of claim 74,
and the short-range sensors supply RGB-D data to the controller. 75. The method of claim 74,
and the controller determines the geometry of the road surface based on RGB-D data received from the plurality of short-range sensors. 75. The method of claim 74,
wherein the plurality of short-range sensors detect objects within 4 meters of the autonomous delivery vehicle and the long-range sensor suite detects objects that are more than 4 meters away from the autonomous delivery vehicle. 75. The method of claim 74,
and a second set of powered wheels capable of engaging the ground while the caster wheels are lifted from the ground. An autonomous delivery vehicle comprising:
a power base comprising at least two powered rear wheels, caster front wheels and an energy store, wherein the power base is configured to move at a commanded speed;
a cargo platform mechanically attached to the power base; and
a short-range camera assembly mounted to the cargo platform to detect at least one characteristic of a drivable surface, the short-range camera assembly comprising:
camera;
a first light; and
a first liquid cooling heat sink;
wherein the first liquid cooling heat sink cools the first light and the camera;
Autonomous delivery vehicle. 84. The method of claim 83,
wherein the short-range camera assembly further comprises a thermoelectric cooler between the camera and the liquid cooled heat sink. 84. The method of claim 83,
wherein the first light and the camera are recessed in a cover having openings that deflect illumination from the first light away from the camera. 84. The method of claim 83,
wherein the lights are angled downwards by at least 15° and are recessed at least 4 mm in the cover to minimize lighting distracting pedestrians. 84. The method of claim 83,
wherein the camera has a field of view and the first light comprises two LEDs with lenses for generating two beams of light that are diffused to illuminate the field of view of the camera. 88. The method of claim 87,
wherein the lights are angled approximately 50° apart and the lenses produce a 60° beam. 84. The method of claim 83,
and the short-range camera assembly includes an ultrasonic sensor mounted above the camera. 84. The method of claim 83,
and the short-range camera assembly is mounted in a central location on the front of the cargo platform. 84. The method of claim 83,
at least one corner camera assembly mounted on at least one corner of the front side of the cargo platform, the at least one corner camera assembly comprising:
ultrasonic sensor;
corner camera;
a second light; and
a second liquid cooling heat sink;
and the second liquid cooling heat sink cools the second light and the corner camera. 23. The method of claim 22,
wherein the historical data comprises surface data. 23. The method of claim 22,
wherein the historical data comprises discontinuous data.

Images