AU2021107433A4

AU2021107433A4 - Autonomous Bulldozer Control

Info

Publication number: AU2021107433A4
Application number: AU2021107433A
Authority: AU
Inventors: Elliot Nicholls
Original assignee: Stealth Tech Pty Ltd; Stealth Technologies Pty Ltd
Current assignee: Stealth Technologies Pty Ltd
Priority date: 2021-08-25
Filing date: 2021-08-25
Publication date: 2021-12-23
Anticipated expiration: 2029-08-25

Abstract

The present invention proposes a method of controlling a bulldozer, the method comprising: receiving a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being either forward or reverse; raising a blade of the bulldozer when the orientation is reverse; lowering the blade when the orientation is forward; driving the bulldozer from the start location in the orientation to the destination location. 16 110 112 114 Clesignated Working Are Reverse Operation 102 120 122 104 100 124 Fig. 4 122 154 152 Fig. 5 150

Description

110 112 114

Clesignated Working Are Reverse Operation 102

120 122

104

100 124 Fig. 4 122 154

152 Fig. 5

Autonomous Bulldozer Control

Field of the Invention

[001] The present invention relates to providing a method and system for autonomous control of a bulldozer.

Background

[002] The following discussion of the background art is intended to facilitate an understanding of the present invention only. It should be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was part of the common general knowledge as at the priority date of the application.

[003] A bulldozer (sometimes called a dozer for short) is a continuous tracked or wheeled tractor with a substantial wide metal plate (blade) that is mounted to the front of the tractor. Bulldozers are large and powerful tracked machinery and the tracks provide them with ground holding capability and mobility in all kinds of terrains. Bulldozers are integral part to a mining operation. They are used to move significant magnitude of soil for construction of new roads for mining pits, pushing ores for stock pile management and other applications that require large materials to be moved.

[004] Most bulldozers today either are operated manually with an operator in a cabin or with the latest developments today, tele-remote operations are being rolled out to the mining sector where operators can operate a bulldozer remotely from a central command centre, improving safety practices by removing operators from the cabin of the dozer.

[005] However, the benefits of full automation for bulldozer operations have yet to be realised as fully autonomous dozers still have technological challenges to solve as they operate within a dynamic and changing environment and work in a manner that is unique to the type of work they do.

[006] The present invention has been developed with one or more of these limitations in mind.

[007] Throughout the specification unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

[008] Throughout the specification unless the context requires otherwise, the word "include" or variations such as "includes" or "including", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

Summary of the Invention

[009] According to a first aspect there is provided a method of control of a bulldozer, comprising receiving a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being one of: forward or reverse; raising a blade of the bulldozer when the orientation is reverse; lowering the blade when the orientation is forward; driving the bulldozer from the start location in the orientation to the destination location.

[0010] In an embodiment the method comprises determining a path from the start location to the destination location according to the orientation. Preferably the path is a straight line.

[0011] In an embodiment the bulldozer is driven blade up to the start location when it is not currently located at the start location before commencing the command.

[0012] In an embodiment the bulldozer is turned at the start location, with the blade up, to be facing towards the destination location when the orientation is forward, if it is not facing in that direction.

[0013] In an embodiment the bulldozer is turned at the start location, with the blade up, to be facing opposite to towards the destination location when the orientation is backwards, if it is not facing in that direction.

[0014] In an embodiment the method further comprises receiving a geofence and ensuring the destination location is not outside of the geofence and the path does not cross the geofence.

[0015] In an embodiment the path takes account of the width (horizontal extent from one side of the bulldozer to the other) of the blade.

[0016] In an embodiment the command is part of a set of commands, where the set of commands comprise a push start location, and a push destination location and a number of pushes, wherein the number of pushes sets twice the number of movement commands in the set, wherein the push start location is used to set the start location in commands of odd numbered movements and the push destination is used to set the destination location in commands of odd numbered movements, wherein the push start location is used to set the destination location in commands of even numbered movements and the push destination is used to set the start location in commands of even numbered movements, and wherein in commands of odd numbered movements the orientation is set as forward and in commands of even numbered movements the orientation is set as reverse.

[0017] In an embodiment the method comprises receiving the set of commands.

[0018] In an embodiment the method also comprises receiving a navigation command for moving the bulldozer from its current position to the start location for the set of movements, and driving the bulldozer with the blade up to the start location.

[0019] In an embodiment the set of commands is part of a meta command, where the meta command comprises defining an area to be filling with mono-directional lines with a spacing according to the blade width, the number of lines in the fill defining the number of sets of commands, wherein the beginning of each line defining a push start location of each respective set of commands, the end of each line defining the push destination location of each respective set of commands.

[0020] In an embodiment the method comprises receiving the meta command.

[0021] In an embodiment the navigation command is for moving the bulldozer from one line to the next.

[0022] In an embodiment the number of pushes in each set of commands is determined by a height adjustment required of earth over which the bulldozer is moving in the set of movements.

[0023] In an embodiment the position of the area and orientation of the area is determined according to a required ground level change required.

[0024] In an embodiment the method comprises using a terrain detection system to detect obstacles on a potentially uneven ground surface.

[0025] In an embodiment the method comprises deciding whether a detection is an obstacle. In an embodiment the method comprises stopping movement when an obstacle is detected in the immediate path of the bulldozer.

[0026] In an embodiment the area defines the geofence.

[0027] In an embodiment the area is one of a plurality of areas, each one for defining the movement of a respective bulldozer, and each area being non-overlapping. In an embodiment each area is non-geographically overlapping for a given time period. In an embodiment areas may geographically overlap during non overlapping time periods.

[0028] According to another aspect there is provided a control system for autonomously controlling a bulldozer, comprising:| a receiver of a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being one of: forward or reverse; an actuator output for raising or lowering a blade of the bulldozer; a drive interface for controlling a drive mechanism of bulldozer to drive the bulldozer forward or in reverse; a location determining system; a processor for controlling the bulldozer according to the command, comprising controlling the actuator output to raise the blade of the bulldozer when the orientation is reverse, controlling the actuator output to lower the blade when the orientation is forward, controlling the drive interface to drive the bulldozer forward with the orientation is forward and when the bulldozer is not yet at the destination location, and controlling the drive interface to drive the bulldozer in reverse when the orientation is reverse and when the bulldozer is not yet at the destination location.

[0029] In an embodiment the processor is configured to determine a path from the start location to the destination location according to the orientation.

[0030] In an embodiment the processor is configured to control the actuator output to raise the blade of the bulldozer when the bulldozer is not currently located at the start location before commencing the command.

[0031] In an embodiment the processor is configured to control the drive interface to turn the bulldozer at the start location to be facing towards the destination location when the orientation is forward, if it is not facing in that direction and to control the actuator output to raise the blade.

[0032] In an embodiment the control system comprises a terrain scanning system and an obstacle detector for detecting when scanned terrain comprises an obstacle. In an embodiment the processor is configured to stop movement of the bulldozer when an obstacle is detected in the immediate path of the bulldozer.

[0033] In an embodiment the system comprises an input interface for defining areas that define the boundary for each of a plurality of bulldozers.

Description of Drawings

[0034] In order to provide a better understanding, embodiments will be descried in relation to the accompanying drawings, in which:

[0035] Figure 1 is a block diagram of a controller according to an embodiment of the present invention;

[0036] Figure 2 is a schematic side elevation of a bulldozer traversing a path according to an embodiment of the present invention;

[0037] Figure 3 is a schematic representation of a defined area with path lines according to an embodiment of the present invention;

[0038] Figure 4 is a schematic plan view showing a first designated work area and a second designated work area, each of which has an autonomously controlled bulldozer according to an embodiment of the present invention; and

[0039] Figure 5 is a screen shot of a command and control interface according to an embodiment of the present invention.

Detailed Description of Embodiments

[0040] While features may be described in combinations below, it is not intended that all of the features described are required in a particular combination. Fewer or different combinations are intended to be covered by the below examples.

[0041] Broadly, there is a method of control of a bulldozer 50 using a control system 10. The control system 10 comprises a processor 12 that receives a meta command which is comprised of sets of individual commands. The processor 12 interprets these commands and provides outputs to turn them into actions of the bulldozer 50. Each individual command comprises a start location, a destination location and an orientation of direction of movement of the bulldozer. The orientation is either forward 122 or reverse 112 movement.

[0042] The processor 12 controls an actuator 20 to raise or lower the blade 54 of the bulldozer 50. When the orientation is reverse 112 the actuator raises the blade 54 of the bulldozer and when the orientation is forward 122 the actuator lowers the blade 54.

[0043] The processor 12 controls a drive interface 22 to drive tracks 52 the bulldozer 50 from a start location 120 in the orientation (that is forwards or in reverse) to the destination location 122.

[0044] The drive interface 22 typically controls the tracks 52 (or drive wheels) of the bulldozer 50 individually in forward 122 or reverse 112 direction. When both move in the forward direction the bulldozer 50 moves forward. When both move in the reverse direction the bulldozer 50 moves in the reverse direction. The directions or speed are different the bulldozer 50 can turn either while moving or on the spot. For a wheeled bulldozer the steering wheels control turning in the standard manner.

[0045] A standard autonomous vehicle control system may be modified to operate as the control system of this invention.

[0046] The autonomous control comprises forward push movement, reverse movement and positioning movement to a start position.

[0047] The bulldozer 50 will begin the autonomous operation with positioning movement by driving itself along a path determined by navigation software with its blade 54 raised to a push starting point as indicated by the right hand end of the line 124 in Fig. 4. The bulldozer will be operated autonomously using an autonomous software platform, position information obtained by a position system 18 and sensors 16 set up around the bulldozer 50 for path planning as well as object detection and avoidance.

[0048] Next a path 124 from the start location to the destination location is determined in the forward direction 122. Preferably the path is a straight line. If the bulldozer 50 is not facing in direction facing the destination location, then the bulldozer is turned at the start location, with the blade up, to be facing towards the destination location.

[0049] The bulldozer 50 then is controlled to undertake a push movement with the blade 54 lowered, preferably along the straight line path to a destination location. Even though equal power may be provided to the tracks different loading on the blade or different traction or ground slope can cause the bulldozer to veer off course. The autonomous software platform, and a location detector 18 (such as including a GPS system) keeps the bulldozer 50 travelling along the straight line by correcting the course where needed.

[0050] At the destination the bulldozer 50 is then controlled to reverse 112 back along the line 114 with the destination location becoming the start location for this move and the original start position becoming the destination location for this movement. This is described in relation to bulldozer 110 in Fig. 4, as it is currently moving in reverse, although typically the same bulldozer 50 travels in reverse along the line 124. The blade 54 is raised and the track direction reversed. Again the autonomous software platform, and location detector keep the bulldozer 50 travelling along the straight line by correcting the course where needed.

[0051] The processor 12 then decides whether to repeat the push forward and reverse moves again, swapping back the start and destination locations, or whether to move to a new position. In the command a number of pushes can be provided. If the bulldozer has not completed the number of pushes it can again do the next push forward and reverse.

However, if the number of pushes for this set of commands have been completed then it can determine whether there is another push line or whether it has finished. If there is another push line the start location of the next line is determined, then this becomes the destination of a blade up location movement. Again, the autonomous software platform, and location detector keep the bulldozer 50 travelling to the destination of the location movement. When there, this location becomes the start position of the next push line and the destination location of that push line is received in another set of commands.

[0052] These set of push and reverse commends can then be executed as described above.

[0053] A command and control user interface 150 can be provided for a user to provide commands. The interface 150 may comprise controls 152 for changing views or type of information displayed and a display 154 for showing commands for each bulldozer and for showing what the respective bulldozer is doing. These commands may be to an individual movement level, or movement commands may be grouped into a set. For example, a push movement command made be grouped with a reverse command. Additionally, a number of repetitions of the push forward and reverse commend made be in the set of commands. Higher level commands may also be created, such as a meta command which may comprise sets of (push forward and reverse) commands, preferably with a reposition movement between each sets of commands.

[0054] In a particular embodiment the push or reverse command is part of a set of commands, where the set of commands comprise a push start location, and a push destination location and a number of pushes, wherein the number of pushes sets twice the number of movement commands in the set, wherein the push start location is used to set the start location in commands of odd numbered movements and the push destination is used to set the destination location in commands of odd numbered movements, wherein the push start location is used to set the destination location in commands of even numbered movements and the push destination is used to set the start location in commands of even numbered movements, and wherein in commands of odd numbered movements the orientation is set as forward and in commands of even numbered movements the orientation is set as reverse.

[0055] In an embodiment the set of commands comprises a navigation command for moving the bulldozer 50 from its current position to the start location for the set of movements, and driving the bulldozer with the blade up to the start location.

[0056] In an embodiment the set of commands is part of a meta command, where the meta command comprises defining an area such as 130 to be filling with mono-directional lines 134 with a spacing according to the blade width. The number of lines in the fill defines the number of sets of commends, wherein the beginning of each line defining a push start location of each respective set of commands, the end of each line defining the push destination location of each respective set of commands. By way of analogy, the meta command is similar to filling the area with raster scans like a raster scans lines on a cathode ray tube, with a scan being a blade down push, although each scan line might be'scanned over' a number of times and the bulldozer has to reverse, blade up, back over the push line back to the start of the scan.

[0057] In an embodiment the method comprises sending the meta command, set of commands or individual commands from the command and control interface 150 to the receiver 14 which receives the set of commands (etc) and provides them to the processor 12 for storage and execution.

[0058] In an embodiment the path takes account of the width of the blade so that the spacing between the paths is slightly less that the width of the blade so that overlap is sufficient to sideways overflow from the blade into the previous adjacent push of earth.

[0059] In an embodiment the navigation command is for moving the bulldozer 50 from one line 134 to the next.

[0060] The commands and control user interface 150 may be used to define a geofence 102/104 which is used to ensure the destination location of each movement is not outside of the geofence and the path of the movement does not cross the geofence. The area 130 may be used to define the geofence.

[0061] Multiple dozers 50, 110 are usually operated within a work site 100. With current bulldozer operations, bulldozer operators can actively communicate with each other and also have vision on other dozers to ensure that each are operated within their designated zones to prevent any unwanted collision.

[0062] However for bulldozer automation, multiple bulldozers will need to operate their respective Push and Reverse operations not only on a straight line path 114/124 but within a defined working area 102, 104. The autonomous bulldozers will utilize an onboard autonomous software platform executing on the processor 12 and sensors 16 to actively manage its path planning within its work site and utilize object detection and avoidance system to safely operate the dozers autonomous within a defined working area.

[0063] As seen on Fig. 2, the surface 60 of terrain traversed may be uneven such as with a mound 62 (or a depression (not shown). The sensors 16 may comprise a LIDAR emitter 56 which produces beam/s of laser where the reflection of the laser is used to define a 3D point cloud of the terrain. The LIDAR may detect terrain features such as the mound which are safe to travel over (forward or in reverse (not shown)) or push (forward only). To accommodate travel in reverse the sensors, including LIDAR may also be in the reverse direction, although this is not shown. Some obstacles such as 64 are not safe to push through. The LIDAR is processed to recognise the smooth transition of a mound

/ depression and the generally sharp changed in direction of ones that are not safe. Image recognition techniques may also be used.

[0064] An operator can manually drive the bulldozer around an area either under teleoperation or by being in the cabin and the system will record the path. The path can then be automatically configured by the system to become a perimeter for a geofencing area that will bound the operations of the autonomous bulldozer. This will prevent multiple bulldozers working the same work site (eg stockpile area) from interfering with each other, and will also prevent the operator from configuring autonomous missions and operations that will attempt to occur outside the geofenced area. Alternatively, a geofence area can be drawn as a shape on a map and this used to define the geofence area.

[0065] The system can allow multiple bulldozers to be working the same stockpile area as seen in Fig.s 4 and 5. The system will prevent collisions with other bulldozers and allow dozers to be logically separated by geofenced areas. A single operator can control multiple bulldozers using the command and control user interface 150. The bulldozers can be located at the same stockpile management area or at different stockpile management areas.

[0066] In an embodiment the area is one of a plurality of areas, each one for defining the movement of a respective bulldozer, and each area being non-overlapping. In an embodiment each area is non-geographically overlapping for a given time period. In an embodiment areas may geographically overlap during non overlapping time periods.

[0067] The command and control interface 150 may provide stockpile levels within the working area. When earth needs to be move from one part of the working area 130 to another the shape, dimensions, and orientation of the working area 130 may be determined according to the change in stockpile levels required. Preferably or instead, the level may be used to determine the starting end of the working area 130. Preferably or instead, the current level and the desired change in stockpile level may be used to determine the number of pushes in each set of commands.

[0068] In an embodiment there may also be a control system for autonomously controlling a bulldozer, comprising: a receiver of a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being one of: forward or reverse; an actuator output for raising or lowering a blade of the bulldozer; a drive interface for controlling a drive mechanism of bulldozer to drive the bulldozer forward or in reverse; a location determining system; a processor for controlling the bulldozer according to the command, comprising controlling the actuator output to raise the blade of the bulldozer when the orientation is reverse, controlling the actuator output to lower the blade when the orientation is forward, controlling the drive interface to drive the bulldozer forward with the orientation is forward and when the bulldozer is not yet at the destination location, and controlling the drive interface to drive the bulldozer in reverse when the orientation is reverse and when the bulldozer is not yet at the destination location.

[0069] In an embodiment the processor is configured to determine a path from the start location to the destination location according to the orientation.

[0070] In an embodiment the processor is configured to control the actuator output to raise the blade of the bulldozer when the bulldozer is not currently located at the start location before commencing the command.

[0071] In an embodiment the processor is configured to control the drive interface to turn the bulldozer at the start location to be facing towards the destination location when the orientation is forward, if it is not facing in that direction and to control the actuator output to raise the blade.

[0072] In an embodiment the control system comprises a terrain scanning system and an obstacle detector for detecting when scanned terrain comprises an obstacle. In an embodiment the processor is configured to stop movement of the bulldozer when an obstacle is detected in the immediate path of the bulldozer.

[0073] In an embodiment the system comprises an input interface for defining areas that define the boundary for each of a plurality of bulldozers.

[0074] A detailed map of an operating area incorporating knowledge of driveable areas and static obstacles using high resolution LiDARs can be created. Where required, these maps can be manually modified to, for example, designate physically driveable areas as restricted to autonomous operations. In addition, maps can be annotated with landmarks and other points of interest.

[0075] An Inertial Navigation System (INS) in conjunction with a stationary GPS base station enables centimetre-level accuracy in GPS enabled environments. Short GPS outages, such as from briefly passing underneath large obstacles, are handled through a highly accurate Inertial Measurement Unit (IMU) in conjunction with odometry information from the vehicle hardware. LiDAR localisation with sensor fusion can also be utilised if required.

[0076] Two behaviours can occur when dynamic obstacles - those that aren't present during the mapping process - are encountered: 1. The vehicle can be made to pause and raise a warning for action by the operator. The vehicle will remain paused until the dynamic obstacle is removed. The vehicle could be manually driven at this time to avoid the obstacle; 2. The vehicle's path can be updated dynamically to navigate a safe distance around the detected obstacle - dynamic obstacle avoidance.

[0077] LiDAR sensors can be arranged in a series of safety curtains, which detect any potential obstacles near the vehicle and can slow or stop the vehicle based on the position of any detected obstacles. In the simplest case, the vehicle can be made to slow or stop when obstacles come within a certain distance of the vehicle. More complex measures such as only taking into consideration objects within the vehicle's current trajectory are also available.

[0078] Given access to fuel or battery levels and range data, the command and control system can determine if executing a mission will come close to the vehicle's available range, and warn the operator before the mission starts. The processor may automatically trigger a return to base as the fuel or battery level nears the point where such a return would no longer be possible.

[0079] The Command-and-Control Centre allows controlling all aspects of the bulldozer. This includes:

• • Mission control (defining, starting, pausing, resuming, and cancelling missions) • • Mode control (moving between manual and autonomous control modes) • •Access control (opening charging and data ports) • • Diagnostics and fault reporting • • Remote emergency stop • • Mission reporting

[0080] Ground plane detection involves the processing of LiDAR point clouds and other sensor data in order to determine drivable areas. There are several complexities to this, especially in off-road, changing environments. The ability of the vehicle to detect and navigate negative obstacles (i.e., voids, such as steep downward slopes, ditches, or cliffs) will be dependent on the number and placement of sensors available for use in ground plane detection.

[0081] Excessively rough or unusual terrain may pose difficulties in determining what is and is not a part of the ground plane. Allowing rougher terrain to be driven over increases the risk that short objects lying on the ground will not be detected and may be inadvertently driven over.

[0082] Modifications may be made to the present invention within the context of that described and shown in the drawings. Such modifications are intended to form part of the invention described in this specification.

Claims

1. A method of control of a bulldozer, comprising receiving a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being one of: forward or reverse; raising a blade of the bulldozer when the orientation is reverse; lowering the blade when the orientation is forward; driving the bulldozer from the start location in the orientation to the destination location.

2. A control system for autonomously controlling a bulldozer, comprising:| a receiver of a command, the command comprising a start location, a destination location and an orientation of direction of movement of the bulldozer, the orientation being one of: forward or reverse; an actuator output for raising or lowering a blade of the bulldozer; a drive interface for controlling a drive mechanism of bulldozer to drive the bulldozer forward or in reverse; a location determining system; a processor for controlling the bulldozer according to the command, comprising controlling the actuator output to raise the blade of the bulldozer when the orientation is reverse, controlling the actuator output to lower the blade when the orientation is forward, controlling the drive interface to drive the bulldozer forward with the orientation is forward and when the bulldozer is not yet at the destination location, and controlling the drive interface to drive the bulldozer in reverse when the orientation is reverse and when the bulldozer is not yet at the destination location.

14 18

Receiver / Position Actuator Transmitter 20 2021107433

Sensors Processor Drive interface

16 12 22 10 Fig. 1

64 70 50 56

52

62 60 54

Fig. 2

132

134

130

Fig. 3

110 112 114

102

120 2021107433

122

50

104 124 Fig. 4 122 100 154

152 Fig. 5