GB2332415A

GB2332415A - Plural terrain scanning sensor arrangement for an earth working machine

Info

Publication number: GB2332415A
Application number: GB9825897A
Authority: GB
Inventors: Anthony Stentz; John Bares; Scott Boehmke; Chris Leger
Original assignee: Carnegie Mellon University
Current assignee: Carnegie Mellon University
Priority date: 1997-12-19
Filing date: 1998-11-27
Publication date: 1999-06-23
Anticipated expiration: 2018-11-27
Also published as: GB9825897D0; GB2332415B

Abstract

A plural terrain scanning sensor arrangement for an earth working machine comprises a plurality, preferably two 10, 12, of terrain scanning sensors mounted on the vehicle at different locations (20, 22, Figure 2), where at least one of the sensors scans in a horizontal plane and all the sensors scan in a vertical plane (Figure 3), and where the sensors are independently controlled by a computer 14. The sensors may be alternatively arranged such that at least one has a scan range which is unobtainable by at least one other (Figure 4). The sensors may be laser, radar, sonar, or any combination of two of these.

Description

2332415 SENSOR CONFIGURATION FOR AN EARTHMOVING MACHINE Technical Field

This invention relates generally to the use of scanning sensor systems on a mobile machine and, more particularly, to the use of two or more scanning sensor systems operating independently and cooperatively on an earthmoving machine.

Currently, the control systems in autonomous or semi-autonomous mobile machines use some form of scanning sensor system that construct range maps of the work environment for navigation and obstacle detection. The systems typically is include one sensor that is fixed to the machine and scans a fixed pattern. There are however, deficiencies in this type of arrangement in situations where the machines have multiple articulating parts that function simultaneously or where the machine performs tasks at a rate that cannot be supported by a single scanner. For example, in operations involving large earthmoving machines, it is desirable to monitor the progress of dumping the material while simultaneously scanning the dig site to plan the acquisition of the next load. One scanner with a fixed scan pattern cannot monitor two separate areas of terrain if one area is outside the sensor's field of view. It is also desirable to use the sensor to check for obstacles along the path of the equipment's moving parts. Ideally, the obstacle scan precedes movement of the equipment to allow adequate response time if an obstacle is detected. When one sensor is used, time delay is introduced because the current operation, such as digging or unloading, must finish before 2 another task, such as scanning for obstacles, can begin. Another situation where one scanner is deficient occurs if the structure of the equipment, obstructions at the work site, or limitations in the sensor scan pattern create blind spots that prevent the sensor from detecting portions of the work site.

It is an object of the present invention to attempt to alleviate the aforementioned problems and/or to provide improvements generally.

According to the present invention there is provided a scanning sensor configuration for earthmoving as claimed in the accompanying claims.

Also according to the present invention there is provided a method of configuring imaging sensors as claimed in the accompanying claims.

In one embodiment of the present invention two or more scanning sensor systems are integrated with a control system for independent, cooperative operation. When the control system operates the sensor systems independently, each sensor system provides information on different regions of the surroundings. This allows the control system to process information for multiple tasks concurrently, and determine optimal movement and timing of operation for controlling the equipment. When the sensors are used cooperatively, they provide information regarding the same area to allow a task to be performed more effectively. Whether operating independently or cooperatively, the sensors are positioned on the machine or at a location near a work site that allows the sensors to scan the desired portions of the environment. Depending on the application, sensors capable of providing omnidirectional information or alternatively, information limited to selected areas, may be incorporated in the present invention. The sensor scan patterns are variable and capable of being controlled independently and cooperatively for a variety of tasks such as navigation, obstacle detection, excavating, loading and dumping, 3 construction, exploration, and assembly line operations.

Brief Description of Drawinqs

Fig. 1 is a block diagram of a control system receiving input from multiple scanning sensors and generating output to the machinery's actuators; Fig. 2 is a front view of a typical arrangement of two scanning sensors on a piece of earthmoving equipment according to the present invention; Fig. 3 is a front view of the components of a type of scanning sensor used with the present invention; Fig. 4 is a top view of an application of the present invention at an excavating site; Fig. 5 is a side view of an alternate application of the present invention at an excavating site; Fig. 6 is a top view of an alternate application of the present invention on a wheel loader; and Fig. 7 is a side view of an alternate application of the present invention on a wheel loader.

Referring now to the drawings, Fig. 1 shows a block diagram of components of an autonomous piece of earthmoving machinery that incorporates the present invention. A first scanning sensor system 10 and a second scanning sensor system 12 provide inputs to a control system 14. More than two sensors can be used. Using data output by the scanning sensor systems 10, 12 along with input from other devices, the control system 14 generates 4 commands that are output to devices including actuators 16 for moving machinery and to the sensor systems 10, 12 for controlling their operation. The control system 14 includes processing means such as a computer having a microprocessor, read/write memory, data storage and retrieval devices, input/output capability, and capable of executing software pertinent to the particular application. The scanning sensor systems 10, 12 may also include processing means which accept inputs from the control system 14 to control operation of the sensor systems 10, 12 independently including each sensor's field of view and scan pattern. Processing means for image processing and recognition may interface with both the sensor systems 10, 12 and the control system 14. Data may be provided by a variety of sources including being preprogrammed in memory, entered by an operator, supplied by sensors, or generated by the processing means.

Fig. 2 illustrates a preferred embodiment of the present invention wherein left and right sensor systems 20, 22 are positioned on a piece of earthmoving machinery for monitoring task areas and objects at the worksite. The sensor systems 20, 22 are placed at approximately symmetrical locations on the machine and are driven independently to provide information on different, yet complementary regions of the work site. In this manner, one sensor system monitors the area around the task underway, while the other sensor system can monitor the next area where the equipment will operate. With the sensor systems 20, 22 operating independently in the present invention, tasks can be preplanned and time delays are reduced because the system does not have to wait for a single sensor system to become available. A particular application may require more than two sensor systems, in which case a plurality of sensor systems may be located in various positions to provide the required data.

The sensor systems 20, 22 may also be driven in a cooperative manner to provide more complete information regarding a specific area of a work site. For instance, during operations where a portion of a work site is occluded from a first sensor system's field of view by the structure of the earthmoving equipment, a second sensor system is positioned to have a different vantage point on or near the equipment to provide the information that cannot be obtained by the first sensor system.

Depending on the scanning capability of the sensor systems, the structure of the equipment, surrounding obstacles, and the task to be accomplished, two or more sensor systems of the same or different type can be utilized to provide the required information.

An exampl'e of a scanning sensor system suitable for use in the present invention is illustrated in Fig. 3. The sensor system 30 is shown mounted on a cutaway portion of a cab 32 of an earthmoving machine. The sensor's scanning mechanism consists of a circular reflector 33 mounted at an angle adjacent the sensor's distance measuring beam 34. The reflector 33 is attached to the shaft of a scanning motor 35 that rotates the reflector about a scanning axis 36 at a frequency of up to about 3600 rpm. This configuration reflects the outgoing distance measuring beam 34 in an unobstructed 180 degree scan and directs the return signal 37 from the terrain or an object 38 to the sensor system's receiver 39. The direction of the degree scan can be adjusted to provide information about a vertical or horizontal plane, depending on the task and the location of the worksite. For instance, if the work area is an excavation site below the horizon of the sensor, the 6 scan is directed downwardly from horizon to horizon. If the work area is in front of the equipment, such as unloading a bucket into a dump truck, the scan may be directed vertically. To achieve a 360 degree field of view, the sensor system 30 is mounted on the shaft of a motor 40 that rotates about a vertical axis 41. The rotation about the vertical axis 41 is referred to as "panning", and the distance measuring beam 34 of the sensor system 30 may thereby scan up to 360 degrees in a substantially horizontal plane while continuing to scan the vertical plane.

The distance measuring beam 34 may be produced by a sensor transmitter having the desired performance characteristics such as a radar, laser, or sonar transmitter. Environmental factors at the worksite such as blowing dust or snow, may dictate the suitability of the type of sensor employed. The sensor system produces range maps of the worksite and provides range data to objects and terrain within its field of view. The software associated with either the sensor system or the machinery's control system may include algorithms to recognize images and derive their location, size, and orientation from the data. The sensor system shown in Fig. 3 provides range data for a full 360 degree by 180 degree scan which is updated at a high rate. Current capabilities include a sensor rotation rate of up to 3600 rpm about the scan axis and 10 rpm about the pan axis. Higher scan rates are possible, depending on the components used and the task requirements. With this information, the worksite can be constantly monitored for obstacles. Efficient planning and execution of a machine's task is also facilitated by sensor systems having rapid scan rates and providing full field of view information. Each sensor's field of view and scan

7 pattern may also be independently adjusted to provide a more concentrated scan, and therefore more detailed and accurate data for a given area of interest.

There are several other types of scanning sensor systems that are suitable for use in the present invention including systems based on radar, laser, infrared, and sonar sensors. A plurality of sensors composed of the same or a combination of different types of sensor systems may be incorporated in the present invention.

Industrial Applicability

The present invention is applicable in situations where autonomous or semi-autonomous control of earthmoving machinery is desired in an earthmoving environment. Figs. 4 and 5 illustrate a typical excavation site with an excavator 46 positioned above a dig face 52 and a dump truck 48 located within reach of the excavator's bucket 50.

In order for the excavator 46 to operate autonomously, the location of objects and obstacles within the excavator's area of movement, and the location of terrain to be excavated must be known. The sensors used in the present invention must therefore be capable of providing current information regarding location of objects around the area of movement far enough in advance to provide the excavator with adequate response time.

Figure 4 illustrates an implementation of the present invention with left and right sensors 42, 43 mounted at approximately symmetrical locations to the left and right of a boom 44 on an excavator 46. A dump truck 48 is positioned near the excavator 46 for receiving the excavated materials. During the digging and loading cycle, the control system (not shown) commands the left and 8 right sensors 42, 43 to monitor the bucket 50 and adjacent areas. As the excavator 46 nears completion of the digging process, the control system commands the left sensor 42 to pan toward the dump truck 48 to check for obstacles in the path of movement of the excavator 46 and to determine the position and orientation of the dump truck 48.

After completing the loading cycle the scan speed of the sensor is coordinated with the pivotal rotation of the excavator 46 as it returns the boom 44 toward the dig face 52 to detect obstacles far enough in advance to allow the excavator 45 adequate time to respond.

The control system operates the left and is right sensors 42, 43 independently to improve efficiency. For example, as the excavator 46 swings toward the dump truck 48, the right sensor 43 retrogrades (i.e., pans in the opposite direction) to scan the excavated area to provide data for planning the next portion of the excavation. At the same time, the left sensor 42 scans the area around the dump truck 48. The scanning sensor 42 provides current information to the control system to allow it to determine an accurate location to unload the bucket 50, even if the dump truck 48 moved since the last loading cycle. While the bucket 50 is being unloaded, the right sensor 43 scans the area near and to the right of the bucket 50 to prepare for rotating toward the dig face 52. As the excavator 46 rotates to the right, the right sensor 43 pans ahead toward the dig face 52 to provide information for obstacle detection. When the excavator 46 begins to rotate toward the dig face 52 after unloading, the left sensor 42 retrogrades to view the distribution of soil in the bed of the dump truck 48 to determine the location in the bed to unload the next bucket of material. As the bucket 9 arrives near the dig face 52, the right sensor 43 scans the digging area. Once the left sensor 42 completes its scan of the dump truck 48, the control system commands the sensor 42 to also scan the digging area. The steps in the excavating process are repeated as outlined above until a dump truck bed is filled or the excavation is completed. The control system uses information provided by the sensor systems to determine whether operations should be halted such as when the dump truck is filled, the excavation is complete, or an obstacle is detected. The information is also used to navigate movement of the equipment.

A further application of the present invention with other large earthmoving machinery such as a wheel loader 60 digging into a soil face and unloading into a dump truck is shown in Figs. 6 and 7 with a left and right sensor 62, 64 mounted on either side of the cab. Alternatively, sensors may be located at front and rear of the machine. The role of the sensors 62, 64 is generally the same as it is with an excavator, but a key difference is that information from both the front and rear scan planes of the sensors are used to navigate the wheel loader 60 during operations. As the soil is scooped up in a bucket 66, both the left and right sensors 62, 64 monitor the bucket 66 and adjacent areas.

The area can therefore be monitored for obstacles as well as loading status of the bucket 66. When the bucket 66 is full, the rear sensor scans are used to monitor the area behind the wheel loader as it backs away from or moves toward the soil face 68. As the wheel loader 60 backs up, each sensor provides information to the left and right of the wheel paths so that motion can be halted if an object obstructs the path of the wheel loader 60. Once the wheel loader 60 reaches a pivot point to reverse directions, it stops as the bucket 66 is raised to full height as shown in Fig. 7. During this period, the scan plane from the right scanner 64 provides a range map of the soil face 68 that is used to plan the next dig. When the bucket 66 is fully raised, the left and right sensors 62, 64 scan either side of the front wheel paths as the wheel loader 60 moves toward or away from the dump truck 70. As the wheel loader 60 approaches the truck 70, information from both sensors 62, 64 is used to monitor clearance between the wheel loader 60 and the dump truck 70. once the bucket 66 is unloaded, both sensors 62, 64 are used to scan behind the wheel loader 60 as it backs up to the pivot point. The sensors 62, 64 once again scan the area in front of the wheel loader 60 as it approaches the soil face 68.

Upon arrival of a dump truck 70, the left s -sor 62 is used to scan the truck 70 and provide i--':ormation to determine its location and orientaticn. The truck 70 may be scanned while the wheel loa--er 60 is digging because the digging process can be adequately monitored with the right sensor 64. Both sensors 62, 64 are used during each approach toward the truck 70 so that clearance between the bucket 66 and truck 70 can be computed.

The application of the present invention to excavating and loading operations are illustrative of the utility of the two sensor configuration. There are many other methods of coordinating two or more sensors to complement one another depending on the specific tasks and required fields of view. Possible applications to machines in an earthmoving environment include excavators, wheel loaders, track-type tractors, compactors, motor graders, agricultural machinery, pavers, asphalt layers, and the like, which exhibit both (1) mobility over or through a work site, and (2) the capacity to alter the topography or geography of a work site with a tool or operative portion of the machine such as a bucket, shovel, blade, ripper, compacting wheel and the like. In an automated system, a sensing script can guide the scan pattern and scan rate for each sensor system as a function of the equipment's progress in the work cycle.

Other aspects, objects and advantages of the present invention can be obtained from a study of the drawings, the disclosure and the appended claims.

12

Claims

Claims

1. A scanning sensor configuration for earthmoving machinery comprising: a first scanning sensor system mounted at a first location on a piece of earthmoving machinery; and a second scanning sensor system mounted at a second location on the earthmoving machinery, and having independently controllable fields of view and scan patterns, at least one of the scanning sensor systems being operable to rotate to scan a field of view greater than zero degrees in a substantially horizontal plane around the earth moving machine, the scanning sensor systems being further operable to rotate to scan a field of view greater than zero degrees in a substantially vertical plane around the earthmoving machine, the first and second scanning sensor systems being operable to provide a first and a second set of range data for planning and executing respective first and second portions of an earthmoving task.

2. The configuration set forth in claim 1, further comprising: processing means operable to identify an object in the output range data from at least one of the first and second scanning sensor systems.

3. The configuration set forth in claim 2, further comprising: processing means operable to generate commands for controlling the fields of view and scan patterns for at least one of the first and second scanning sensor systems.

4. A scanning sensor configuration for an earthmoving machine comprising: a plurality of scanning sensors, each sensor being mounted at a different location on an earthmoving machine, at least one of the scanning sensors being operable to rotate to scan a field of view greater than zero degrees in a substantially horizontal plane around the earth moving machine, the scanning sensor systems being further operable to rotate to scan a field of view greater than zero degrees in a substantially vertical plane around the earthmoving machine, the scanning sensors being ne and having view and scan different areas planning and task.

positioned on the earthmoving mach independently adjustable fields of is patterns to provide range data for of the earthmoving environment for executing at least one earthmoving 5. The configuration set forth in claim 4, further comprising:

processing means operable to identify an object in the output image data from at least one of the scanning sensor systems.

6. The configuration set forth in claim 5, further comprising: processing means operable to generate commands for controlling the field of view and scan pattern for at least one of the scanning sensor systems.

14 7. A scanning sensor configuration for an earthmoving machine comprising: a plurality of scanning sensor systems mounted at spaced locations on an earthmoving machine, at least one of the scanning sensor systems being operable to rotate to scan a field of view greater than zero degrees in a substantially horizontal plane around the earth moving machine, to output image data in accordance therewith, at least one of the sensor systems having an independently adjustable field of view and scan pattern, at least one of the sensor systems positioned at a location to obtain range data for an area occluded by an obstruction from the scan of at least one of the other sensor systems; and processing means operable to identify an object in the output range data, and to generate commands for controlling the field of view and scan pattern for at least one of the sensor systems to provide information for planning and executing at least one earthmoving task.

8. The configuration set forth in claim 7, wherein at least one of the sensor systems includes a scanning radar sensor.

9. The configuration set forth in claim 7, wherein at least one of the sensor systems includes a scanning laser sensor.

10. The configuration set forth in claim 7, wherein at least one of the sensor systems includes one type of sensor and at least one of the other sensor systems includes another type of sensor.

11. A method of configuring imaging sensors for an earthmoving machine comprising: mounting a plurality of scanning sensor systems on an earthmoving machine, the scanning sensor systems operating to scan a field of view greater than zero degrees horizontally around the machine and to output range data in accordance therewith, each sensor system having an independently controllable field of view and scan pattern; positioning at least one of the sensor systems in a location to obtain range data for an area occluded by an obstruction from the scan of at least one of the other sensor systems; and processing range data to identify an object in the output range data and to plan and execute at least one earthmoving task.

12. The method of claim 11, further comprising the step of processing data to generate commands for controlling the field of view and scan pattern for at least one of the scanning sensor systems.

13. A method of configuring imaging sensors for an earthmoving machine substantially as described herein with reference to the accompanying drawings.

14. A scanning sensor configuration substantially as described herein with reference to the accompanying drawings.