WO2024039269A1 - Method and arrangement for localization of a work machine - Google Patents
Method and arrangement for localization of a work machine Download PDFInfo
- Publication number
- WO2024039269A1 WO2024039269A1 PCT/SE2022/050753 SE2022050753W WO2024039269A1 WO 2024039269 A1 WO2024039269 A1 WO 2024039269A1 SE 2022050753 W SE2022050753 W SE 2022050753W WO 2024039269 A1 WO2024039269 A1 WO 2024039269A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- lidar
- range readings
- radar
- work machine
- tramming
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000004807 localization Effects 0.000 title claims abstract description 31
- 238000005259 measurement Methods 0.000 claims abstract description 30
- 238000005065 mining Methods 0.000 claims description 39
- 238000012545 processing Methods 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 16
- 238000010276 construction Methods 0.000 claims description 8
- 230000004888 barrier function Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 abstract description 33
- 230000015654 memory Effects 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 6
- 238000005553 drilling Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000000428 dust Substances 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 239000011435 rock Substances 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000012080 ambient air Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000875 corresponding effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000001771 impaired effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 239000003570 air Substances 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000005422 blasting Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005507 spraying Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
- G01S13/865—Combination of radar systems with lidar systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
Definitions
- the present disclosure relates to a method and arrangement for a work machine.
- the disclosure relates to a computer-implemented method and tramming assist arrangement for localization of a work machine, when operating in an underground environment, using a plurality of range detection sensors.
- the disclosure also relates to corresponding computer programs configured to cause execution of the method and a work machine.
- Day-to-day operations of mining and tunnelling typically involve cycles of drilling, bolting, and blasting using work machines, e.g., mining machines configured for performing such operations.
- work machines e.g., mining machines configured for performing such operations.
- work machines such as trucks, loaders, drilling rigs and haulers, have been operated by an on-board operator present within the machine.
- trucks, loaders, drilling rigs and haulers have been operated by an on-board operator present within the machine.
- a work machine may be used in a fully automated, autonomous mode during some aspects of the mining/tunnelling operation, while other aspects call for operator control, e.g., through remote-control.
- LHD loading, hauling, and dumping machines.
- These mining machines represent transport vehicles that may be used to remove broken rock, haul it to a particular place where the broken rock is dumped, and to return to the initial (start) location to pick up a new load.
- LHD loading, hauling, and dumping machines.
- These vehicles often perform the same travel over and over, which makes the travel between load and dump locations well suited for automation.
- automation may prove beneficial.
- range detection techniques using one or more laser range detection sensors are used to support map generation, route determination and subsequent localization for a work machine in an underground environment.
- One or more range detection sensors, LIDAR scanners may be employed to determine a distance to the surrounding tunnel walls or other obstacles along the path, e.g., during autonomous tramming of a work machine and/or tramming in a remote-control mode.
- Range detection e.g., using laser technology
- the environmental conditions may be far from ideal, and the range readings obtained during operation in the mine may be highly uncertain.
- the uncertainties may in many cases depend on range detection sensor visibility. Dirt on a lens of the sensor or pollution in ambient air, e.g., from dust particles, are well-known sources of such uncertainties. There are several situations when these uncertainties affect the ability to localize a work machine in underground environment, resulting in the need to reduce the speed when performing autonomous tramming or tramming in a remote-control mode.
- a computer-implemented method for localizing a work machine is provided.
- the work machine is configured for autonomous tramming and/or remote-control tramming at an underground construction site or as a mining machine in an underground mining environment.
- the method comprises obtaining sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor.
- a subset of radar range readings are selected, the subset representing distance measurements within at least one segment of the coverage area.
- Localization of the work machine in a representation of the underground environment is determined based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
- a method that enables improvement in localization accuracy, e.g., when experiencing a scenario of impaired visibility. Improvements in the localization accuracy has a direct effect on the productivity of the work machine, since autonomous or remote operation of the work machine may only be allowed for as long as it is possible to determine a position of the work machine along a planned route. Sudden stops of the work machine may also result in spillage of material, e.g., from a bucket when the work machine is performing an ore removal operation back and forth from a draw point.
- obtaining of the sets of range readings from respective LIDAR and radar sensors is repeatedly performed, wherein the LIDAR range readings are obtained at a different, preferably higher, frequency than the subset of radar range readings.
- the range readings are stored and subjected to real-time processing, wherein the real-time processing comprises repetitively determining a distance measurement accuracy of the LIDAR range readings and selecting the subset of radar range readings based on the determined distance measurement accuracy.
- the repetitive obtaining of range readings from respective LIDAR and radar sensors and different frequencies and the subsequent real-time processing ensures high and safe production availability and adaptability, regardless of sensor capabilities at specific instances.
- safe localization may be performed to a greater extent e.g., by performing the localization based on range readings from the LIDAR sensors or from the radar sensors, or by combining range readings from the two types.
- a computer program product comprising a non-transitory computer readable medium having thereon a computer program comprising program instructions loadable into processing circuitry and configured to cause execution of the method according to the first aspect when the computer program is run by the processing circuitry.
- a tramming assist arrangement configured to be comprised in a work machine configured for autonomous tramming and/or remote-control tramming at a construction site or as a mining machine in a mine environment.
- the tramming assist system comprising at least one LIDAR sensor and at least one radar sensor configured to obtain range readings to determine a distance from the respective sensor to path barriers present along a path travelled by a tramming work machine.
- the tramming assist system comprising processing circuitry configured to obtain sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor.
- the processing circuitry is further configured to select a subset of radar range readings representing distance measurements within at least one segment of the coverage area; and determine a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
- a work machine is provided.
- the work machine is configured for autonomous tramming and/or remote-control tramming at a construction site or as a mining machine in a mine environment, the work machine comprising a tramming assist arrangement according to the third aspect.
- Figure 1 discloses a work machine comprising a tramming assist arrangement according to the present disclosure
- Figure 2 provides an example flowchart representation of method steps performed by the tramming assist arrangement of the mining machine
- Figure 3 a discloses an example block diagram of a tramming assist arrangement b. discloses an example block diagram of a tramming assist arrangement.
- work machine refers generally to a mobile work machine suitable for autonomous or remote-control tramming in an underground environment, e.g., a mining machine or tunnelling machine. More specific examples of such work machines are so called loader, hauler, and dumpers, also known as LHD machines, but also drilling rigs capable of performing autonomous or remotely controlled tramming between work sites.
- the functions or steps noted in the blocks can occur out of the order noted in the operational illustrations.
- two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality/acts involved.
- the functions or steps noted in the blocks can according to some aspects of the disclosure be executed continuously in a loop.
- Figures 1 discloses a work machine configured for operation in an underground environment, e.g., at a construction site or at an underground mining site.
- the work machine 10 is configured for autonomous or remotely controlled movement in an underground environment.
- the illustrated work machine 10 is a loader/hauler comprising a vehicle body 11, a bucket 12, and a tramming arrangement comprising range detection sensors 14, 15, and tramming assist circuitry 13.
- the work machine further comprises a plurality of range detection sensors, wherein the plurality of range detection sensors comprises at least one laser range scanner, LIDAR sensor, and at least one radar sensor.
- the range detection sensors are configured to perform scanning of the environment of the work machine by determining distances to obstacles surrounding the mining machine.
- the at least one LIDAR scanner is configured to measure distances using laser beam technology in given directions and with given angles.
- the LIDAR range detection sensor may be 2D scanner configured to monitor tunnel walls at predetermined heights.
- the one or more radar sensor is configured for determining distances in given directions and with given angles using radar technology.
- the mining machine comprises a front range detection LIDAR/radar sensor pair 14 and a rear range detection LIDAR/radar sensor pair 15, that are configured to determine a distance from the respective sensor to path barriers present along a path travelled by the mining machine during tramming.
- the present disclosure is in no way limited to the disclosed placing of the range detection sensors. Any type of sensor mounting that supports distance measuring/ range detection to surrounding walls and obstacles is within the scope of the present disclosure.
- Scanned data may be processed in the tramming assist arrangement and compared to reference profile data stored in an environmental model.
- a position of the work machine may be determined based on the finding of a match in the environment model to localize the work machine.
- the localization may be further corrected through dead-reckoning operations.
- a 3D LIDAR scanner may be applied, in which case 3D scanning data or point cloud data is produced and applied for positioning the mine vehicle.
- Point cloud data generated during a scanning operation may be applied for generating and updating an environment model, such as an underground tunnel model, which may be applied for positioning the mine vehicle at the worksite.
- the tramming assist arrangement may execute a point cloud matching functionality for matching operational (scanned) point cloud data (being scanned by the scanner(s) to environment model point cloud data, i.e., reference point cloud data.
- Position and direction of the scanning device and/or another interest point of the vehicle, such as the (leading edge of the) bucket may be determined in the mine coordinate system on the basis of detected matches between the operational point cloud data and the reference cloud data.
- a driving plan, or a route plan may define a route to be driven by the mine vehicle and may be used as an input for automatic control of the mine vehicle.
- the plan may be generated offline and off-site, for example in an office, or on-board the mine vehicle e.g., by a teaching drive.
- the plan may define a start point, an end point, and a set of route points for the automatic drive. Localization of the work machine performed according to the route plan.
- the route plan may be stored in a memory of the work machine or in memory of the tramming assist arrangement.
- the range detection sensors are used to measure distances to an object/barrier, e.g., a rock wall, a rock, or any other path barrier along the path travelled by the mining machine during tramming.
- the front range detection LIDAR/radar sensor pair 14 may be used to obtain range readings, e.g., from a laser scan over a range detection field or segment and from a radar scan over an overlapping or same range detection field or segment.
- the laser scan will provide range readings for each whole degree ⁇ 90 degrees from the respective longitudinal direction during a scan.
- the radar scan will provide range readings, e.g., for each whole degree, over an at least partly overlapping coverage area, e.g., ⁇ 75 degrees.
- laser or radar range scanners which measure distances, obtain range readings, at a significantly higher resolution or at a significantly lower resolution. It is also possible to use laser or radar range scanners which obtain range readings in a significantly wider direction, as well as those which measure distance in a narrower direction. It is also possible to use one or more single omnidirectional laser or radar range detection sensors to determine distance in any travelling direction of the vehicle or a rotating range detection sensor.
- the range detection sensor pairs are mounted on the mining machine.
- one sensor pair may be arranged on top of the mining machine, e.g., at a position maintaining a line of sight for the sensor pair from the vehicle to the surrounding environment also when the bucket is in a lowered position, in a partly lifted position and/or in a lifted position.
- Further range detection sensors may be provided at a lower part of the mining machine so that obstacles on the ground may be detected at times when the bucket is in a partly lifted position and/or in a lifted position, i.e., not obscuring the line of sight for range detection sensor mounted on a lower part of the mining machine. Consequently, the mounting of range detection sensors as visualized in Figure 1 is only for general understanding and the below proposed method will be equally applicable regardless of the where the range detection sensor is mounted on the mining machine.
- the work machine is configured to navigate within the underground environment using a pre-obtained representation of the mine.
- This kind of navigation has different challenges to that of navigating in an open-air environment.
- a proper representation of the mine is required, e.g., a navigation map that includes a topological structure of the mine, tunnels, and intersections, as well as measurements from range detection sensors, e.g., LIDAR sensors, to well-known reference points in the mine.
- a map of the operation area may be built during setup, using measurements from LIDAR sensors obtained during operator- controlled travel within the underground environment.
- the map is linked to a topological representation, wherein identified locations are perceived as nodes in the representation.
- the map allows for route planning and self-localization of the machine.
- Self-localization may be performed by scan matching between measurements obtained while a machine is moving in the represented environment, and landmarks stored in the map. Scan matching may be performed using a closest point algorithm and filtering.
- LHD Load-Haul-Dump
- the LHD is a center-articulated vehicle with a frontal bucket used to load and transport ore on the production levels of an underground mine. These vehicles are key components in an ore extraction production chain from the underground mine. The ore extraction rate from the mine will depend directly on the efficiency of the LHD.
- the work machine e.g., an underground mining machine
- the work machine is configured for autonomous tramming and/or remotecontrol tramming at an underground construction site or as a mining machine in an underground mining environment.
- the method comprises an optional step S21 of generating a representation of an underground environment in which the work machine is configured to operate whilst driving the work machine in the environment, e.g., using LIDAR sensors or a combination of LIDAR and radar sensors.
- the proposed method may be initiated at any location within an underground environment for which a representation has been generated.
- the work machine is configured to perform the method whilst performing an autonomous or partly autonomous tramming operation within the represented environment.
- the method comprises the step of obtaining S22 sets of range readings S L (t L ) G R ML , S R (t R ) G R MR from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances over a given period of time.
- the measured distances are obtained from respective sensor coverage area at least in part surrounding the respective sensor.
- the LIDAR and radarsensors are preferably simultaneously active when the work machine performs the tramming operation. Alternatively, the radar sensor could be activated when a predetermined number of uncertain LIDAR range readings have been detected.
- the obtaining S22 is repetitively performed while the work machine performs an automated, e.g., autonomous, or remotely controlled, tramming operation along a planned route.
- the obtained range readings may be stored S23, e.g., within a localization arrangement of the mining machine.
- a subset of radar range readings representing distance measurements within at least one segment of the coverage area is selected S25.
- the subset of radar range readings may be selected based on a determined distance measurement accuracy or lack of accuracy in the LIDAR range readings. As processing capacity increases, parallel processing of LIDAR range readings and radar range readings will increase and selecting the subset of radar range readings may imply selecting all obtained radar range readings.
- environmental conditions of the work machine may be far from ideal in the underground environment and the LIDAR range readings obtained during operation in the mine may be highly uncertain. The uncertainties may in many cases depend on range detection sensor visibility. Dirt on a lens of the sensor or pollution in ambient air, e.g., from dust particles, are well-known sources of such uncertainties.
- selecting S25 of the subset of radar range readings may be based on distance measurement accuracy of the LIDAR sensors.
- LiDAR data measurements from the radar sensor will also be processed. In such way the localization will maintain localization quality despite problems with the LIDAR data.
- range readings from the LIDAR sensors may be preferred and given priority for as long as the quality/accuracy of these range readings is deemed sufficient.
- the selecting follows upon real-time processing of range readings from one or more stored S23 sets of range readings.
- the range readings may be jointly or separately stored; the selecting being performed on range readings S(t) G /? Ms received or obtained from a buffer memory.
- the real-time processing may comprise a repeated determining S24 of distance measurement accuracy of the LIDAR range readings.
- radar range readings from the same time may be selected to be included in a localization processing.
- the selecting S25 of the radar range readings may be based on a determined distance measurement accuracy, e.g., an accuracy falling below a predetermined threshold.
- radar range readings may also be included when there is ample processing capacity.
- the implementation can be done in several ways where the data, i.e., range readings, from both LIDAR and radar may be used as a pool of data for navigation or only LIDAR data can be used until a threshold is reached (too many incorrect readings) and then the radar data complements the LIDAR data.
- the determining S24 of distance measurement accuracy comprises classifying at least one subset of the obtained LIDAR range readings as representing non- conclusive distance measurements.
- the selecting S25 then implies selecting a subset of radar range readings when at least one subset of the obtained LIDAR range readings are classified as representing non-conclusive distance measurements within said at least one segment of the sensor coverage area.
- a resulting set of range S(t) G R MF readings comprising a combination of radar range readings and LIDAR range readings is provided for subsequent processing to determine a localization x,y, 6 of the work machine.
- the at least one subset of obtained LIDAR range readings are classified as representing non-conclusive distance measurements when at least a predetermined number of distance measurements in the subset comprises measured distances shorter than a configurable minimum distance or longer than a configurable maximum distance.
- a localization of the work machine in a representation of the underground environment is determined S26 based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
- the LIDAR data may be obtained at a different frequency/rate than the radar data, e.g., at a higher frequency.
- the amount of LIDAR data may differ from the amount of radar data when performing the combination to arrive at the localization and there may be different needs in terms of processing capabilities for the obtained sets of range readings.
- a mutual distance is determined between the LIDAR and the radar sensors. Range readings of the LIDAR and radar sensors may then be correlated and the radar range readings within the at least one segment of the coverage area may be adjusted based on the determined mutual distance. This further improves the possibility of accurate localization of the work machine.
- the tramming motion of the work machine may be reduced when the at least one segment of the coverage area is greater than a predetermined portion of the coverage area.
- Reduction of the tramming motion may comprise reducing a speed of the work machine when travelling into a direction where other types of work machines, e.g., drill rigs, are known to perform drilling operations. Exiting the same environment, the earlier speed may be successively restored as the machine leaves the environment of impaired visibility. If there are no directional aspects of the reduced distance measurement accuracy, the velocity of the work machine may be reduced so that a same limitation to the velocity is obligated regardless of the travelling direction of the work machine.
- a tramming assist arrangement 30 is disclosed, e.g., the tramming assist arrangement 11 as comprised in the mining machine 10 of Figure 1.
- the work machine 10 may be running in an autonomous mode or in an unmanned, remotely operated mode.
- a user interface may be remote from the vehicle and the vehicle may be remotely controlled by an operator in the tunnel, or in control room at the mine area or even long distance away from the mine via communications network(s).
- a control unit outside the work machine 10 may be configured to perform some of the below disclosed features.
- the tramming assist arrangement 30 is configured to perform the above disclosed method.
- the tramming assist arrangement may be comprised in a control unit of the work machine, wherein the control unit comprises processing circuitry executing program code stored in a memory.
- the tramming assist arrangement 30 comprises at least one LIDAR sensor 32 and at least one radar sensor 31 mounted at a mutual distance from one another and configured to obtain range readings to determine a distance from the respective sensor to path barriers present along a path travelled by a tramming work machine.
- the processing circuitry of the tramming assist system is configured to perform the above disclosed method.
- the processing circuitry is configured to obtain S22 sets of range readings from respective LIDAR and radar sensors, select S25 a subset of radar range readings representing distance measurements within at least one segment of the coverage area and determine S27 a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
- Each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor.
- the LIDAR sensor and the radar sensor are mounted in adjacent positions along a symmetry line of the work machine.
- the LIDAR sensor and the radar sensor are arranged at an essentially same height.
- Said control unit of the work machine may be connected to further control units in the work machine, e.g., through a local network.
- the control unit is configured to control at least autonomous tramming and localization operations for the work machine. It is to be appreciated that the control unit may be configured to perform the above disclosed method steps. There may be further operations modules or functions performed by the control unit(s) to support a tramming assist localization functionality implemented in the mining machine.
- the tramming assist arrangement comprises processing circuitry 33 configured to determine a route for routing of the mining machine between a mapped location and an un-mapped draw point.
- the processing circuitry may comprise a processor 33a and a memory 33b.
- Figure 3a further illustrates an example computer program product 34 having thereon a computer program comprising instructions.
- the computer program product comprises a computer readable medium such as, for example a universal serial bus USB memory, a plug-in card, an embedded drive, or a read only memory ROM.
- the computer readable medium stores a computer program comprising program instructions that are loadable into the processing circuitry 33, e.g., into the memory 33b.
- the program instructions may be executed by the processor 33a to perform the above disclosed method.
- the computer program is loadable into data processing circuitry, e.g., into the processing circuitry 31 of Figure 3a, and is configured to cause execution of embodiments for diagnosing range detection capability of the at least one range detection sensor.
- Figure 3b an alternative schematic block diagram is disclosed for the tramming assist arrangement, e.g., the tramming assist arrangement 11 as comprised in the work machine 10 of Figure 1 and schematically disclosed in Figure 4a.
- One or more LIDAR sensors 32 are configured to obtain LIDAR range readings S L (t L ) G R ML .
- one or more radar sensors 31 may obtain radar range readings S R (t R ) G R MR .
- the LIDAR range readings and the radar range readings are obtained within a sensor coverage area at least in part surrounding the respective sensor.
- the radar and LIDAR sensor 31, 32 are preferably arranged at adjacent locations; thus, the coverage of the LIDAR sensor area and the radar sensor area will be overlapping. Overlapping coverage areas may of course also be obtained without physically arranging the LIDAR 32 and radar sensors 31 as sensor pairs, if their positions relative to one another, e.g., distance, may be determined and used in the processing of the obtained data.
- the block diagram may further comprise a memory arranged to store the obtained sets of range readings.
- the at least one radar sensor and the at least one LIDAR sensor are simultaneously active and provides range readings with a frequency determined by capabilities of the respective sensor.
- the radar sensor may be activated in response to one or more LIDAR range readings determined as inaccurate.
- the tramming assist arrangement further comprises a selector, i.e., processing circuitry configured to perform processing of the obtained set of range readings from the respective LIDAR and radar sensors (32, 31).
- the processing circuitry receives a combination of range readings S R , S L from the memory and performs processing of the received data.
- Range readings from the LIDAR sensor may be given priority for as long as the LIDAR range readings are not qualified as faulty. When the quality of the LIDAR range readings falls below a predetermined threshold, range readings from the radar sensor are used in the determining of the localization.
- the described embodiments and their equivalents may be realized in software or hardware or a combination thereof.
- the embodiments may be performed by general purpose circuitry. Examples of general-purpose circuitry include digital signal processors DSP, central processing units (CPU), co-processor units, field programmable gate arrays FPGA and other programmable hardware. Alternatively, or additionally, the embodiments may be performed by specialized circuitry, such as application specific integrated circuits ASIC.
- the general-purpose circuitry and/or the specialized circuitry may, for example, be associated with or comprised in an apparatus such as a wireless communication device or a network node.
- Embodiments may appear within an electronic apparatus comprising arrangements, circuitry, and/or logic according to any of the embodiments described herein. Alternatively, or additionally, an electronic apparatus may be configured to perform methods according to any of the embodiments described herein.
- the method embodiments described herein discloses example methods through steps being performed in a certain order. However, it is recognized that these sequences of events may take place in another order without departing from the scope of the claims. Furthermore, some method steps may be performed in parallel even though they have been described as performed in sequence. Thus, the steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step.
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Radar Systems Or Details Thereof (AREA)
- Geophysics And Detection Of Objects (AREA)
Abstract
The present disclosure relates to a method and arrangement for a work machine. In particular, the disclosure relates to a computer-implemented method and tramming assist arrangement for localization of a work machine, when operating in an underground environment, using a plurality of range detection sensors. The method comprises the steps of obtaining sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor and selecting a subset of radar range readings, wherein the subset represent distance measurements within at least one segment of the coverage area. The method further comprises the step of determining a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
Description
Method and arrangement for localization of a work machine
The present disclosure relates to a method and arrangement for a work machine. In particular, the disclosure relates to a computer-implemented method and tramming assist arrangement for localization of a work machine, when operating in an underground environment, using a plurality of range detection sensors. The disclosure also relates to corresponding computer programs configured to cause execution of the method and a work machine.
BACKGROUND
Day-to-day operations of mining and tunnelling typically involve cycles of drilling, bolting, and blasting using work machines, e.g., mining machines configured for performing such operations. Historically, work machines, such as trucks, loaders, drilling rigs and haulers, have been operated by an on-board operator present within the machine. However, in the constantly on-going process of improving safety, efficiency, and productivity; such machines are to an increasing extent being configured for autonomous operation and/or remote operation. In some examples, a work machine may be used in a fully automated, autonomous mode during some aspects of the mining/tunnelling operation, while other aspects call for operator control, e.g., through remote-control.
One example of mining machines where automated operation oftentimes is considered beneficial are so-called LHD (loading, hauling, and dumping) machines. These mining machines represent transport vehicles that may be used to remove broken rock, haul it to a particular place where the broken rock is dumped, and to return to the initial (start) location to pick up a new load. Thus, these vehicles often perform the same travel over and over, which makes the travel between load and dump locations well suited for automation. There are also various other situations where automation may prove beneficial.
To allow autonomous driving, it must be ensured that the vehicle is aware of its location in the environment in which it is traveling. However, in underground operation global navigation satellite systems are inaccessible, and no global navigation system is available for use. Instead, local navigation solutions must be utilized.
Autonomous operation underground requires a representation of the surroundings, such as a map representation, that the work machine can use for positioning and thereby be able to navigate from one location to another. Navigation rely highly on the representation of the environment of the vehicle. The map may be generated through a recording operation, whereby a work machine is driven along a path recording the surroundings during beneficial environmental conditions. Following the recording, one or more local map representations covering the region in which the work machine will be moving may be generated. A single route may be associated with one or several map representations, each covering a part of the route.
In recent years, range detection techniques using one or more laser range detection sensors, i.e., LIDAR scanners, are used to support map generation, route determination and subsequent localization for a work machine in an underground environment. One or more range detection sensors, LIDAR scanners, may be employed to determine a distance to the surrounding tunnel walls or other obstacles along the path, e.g., during autonomous tramming of a work machine and/or tramming in a remote-control mode.
Range detection, e.g., using laser technology, provides the advantage of enabling accurate readings. However, when operations are being performed in the underground environment, the environmental conditions may be far from ideal, and the range readings obtained during operation in the mine may be highly uncertain. The uncertainties may in many cases depend on range detection sensor visibility. Dirt on a lens of the sensor or pollution in ambient air, e.g., from dust particles, are well-known sources of such uncertainties. There are several situations when these uncertainties affect the ability to localize a work machine in underground environment, resulting in the need to reduce the speed when performing autonomous tramming or tramming in a remote-control mode.
Consequently, there is a need to improve localization of a work machine configured for autonomous tramming and/or remote-control tramming.
SUMMARY
It is therefore an object of the present disclosure to provide a method, a computer program product, a localization arrangement, and a mining machine that seeks to mitigate, alleviate,
or eliminate all or at least some of the above-discussed drawbacks of presently known solutions.
This and other objects are achieved by means of a method, a computer program product, a localization arrangement, and a mining machine as defined in the appended claims. The term exemplary is in the present context to be understood as serving as an instance, example or illustration.
According to a first aspect of the present disclosure, a computer-implemented method for localizing a work machine is provided. The work machine is configured for autonomous tramming and/or remote-control tramming at an underground construction site or as a mining machine in an underground mining environment. The method comprises obtaining sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor. A subset of radar range readings are selected, the subset representing distance measurements within at least one segment of the coverage area. Localization of the work machine in a representation of the underground environment is determined based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
Thus, a method is provided that enables improvement in localization accuracy, e.g., when experiencing a scenario of impaired visibility. Improvements in the localization accuracy has a direct effect on the productivity of the work machine, since autonomous or remote operation of the work machine may only be allowed for as long as it is possible to determine a position of the work machine along a planned route. Sudden stops of the work machine may also result in spillage of material, e.g., from a bucket when the work machine is performing an ore removal operation back and forth from a draw point.
In some examples, obtaining of the sets of range readings from respective LIDAR and radar sensors is repeatedly performed, wherein the LIDAR range readings are obtained at a different, preferably higher, frequency than the subset of radar range readings. The range readings are stored and subjected to real-time processing, wherein the real-time processing comprises repetitively determining a distance measurement accuracy of the
LIDAR range readings and selecting the subset of radar range readings based on the determined distance measurement accuracy.
The repetitive obtaining of range readings from respective LIDAR and radar sensors and different frequencies and the subsequent real-time processing, ensures high and safe production availability and adaptability, regardless of sensor capabilities at specific instances. By repetitively obtaining range readings from the LIDAR as well as the radar sensors, safe localization may be performed to a greater extent e.g., by performing the localization based on range readings from the LIDAR sensors or from the radar sensors, or by combining range readings from the two types.
According to a second aspect of the present disclosure, there is provided a computer program product comprising a non-transitory computer readable medium having thereon a computer program comprising program instructions loadable into processing circuitry and configured to cause execution of the method according to the first aspect when the computer program is run by the processing circuitry.
According to a third aspect of the present disclosure, there is provided a tramming assist arrangement. The tramming assist arrangement is configured to be comprised in a work machine configured for autonomous tramming and/or remote-control tramming at a construction site or as a mining machine in a mine environment. The tramming assist system comprising at least one LIDAR sensor and at least one radar sensor configured to obtain range readings to determine a distance from the respective sensor to path barriers present along a path travelled by a tramming work machine. The tramming assist system comprising processing circuitry configured to obtain sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor. The processing circuitry is further configured to select a subset of radar range readings representing distance measurements within at least one segment of the coverage area; and determine a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
According to a fourth aspect of the present disclosure, a work machine is provided. The work machine is configured for autonomous tramming and/or remote-control tramming at a construction site or as a mining machine in a mine environment, the work machine comprising a tramming assist arrangement according to the third aspect.
The above reflected advantages and others are provided also by the computer program code, the routing arrangement, and the mining machine.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 discloses a work machine comprising a tramming assist arrangement according to the present disclosure
Figure 2 provides an example flowchart representation of method steps performed by the tramming assist arrangement of the mining machine;
Figure 3 a. discloses an example block diagram of a tramming assist arrangement b. discloses an example block diagram of a tramming assist arrangement.
DETAILED DESCRIPTION
Aspects of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. The apparatus and method disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the aspects set forth herein. Like numbers in the drawings refer to like elements throughout.
The terminology used herein is for the purpose of describing aspects of the disclosure only and is not intended to limit the invention. It should be emphasized that the term "comprises/comprising" when used in this specification is taken to specify the presence of certain features, steps, or components, but does not preclude the presence or addition of one or more other features, steps, components, or groups thereof. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
The term work machine refers generally to a mobile work machine suitable for autonomous or remote-control tramming in an underground environment, e.g., a mining machine or tunnelling machine. More specific examples of such work machines are so called loader,
hauler, and dumpers, also known as LHD machines, but also drilling rigs capable of performing autonomous or remotely controlled tramming between work sites.
Embodiments of the present disclosure will be described and exemplified more fully hereinafter with reference to the accompanying drawings. The solutions disclosed herein can, however, be realized in many different forms and should not be construed as being limited to the embodiments set forth herein.
In some implementations and according to some aspects of the disclosure, the functions or steps noted in the blocks can occur out of the order noted in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality/acts involved. Also, the functions or steps noted in the blocks can according to some aspects of the disclosure be executed continuously in a loop.
It will be appreciated that when the present disclosure is described in terms of a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors.
In the following description of exemplary embodiments, the same reference numerals denote the same or similar components.
Figures 1 discloses a work machine configured for operation in an underground environment, e.g., at a construction site or at an underground mining site. The work machine 10 is configured for autonomous or remotely controlled movement in an underground environment. The illustrated work machine 10 is a loader/hauler comprising a vehicle body 11, a bucket 12, and a tramming arrangement comprising range detection sensors 14, 15, and tramming assist circuitry 13.
The work machine further comprises a plurality of range detection sensors, wherein the plurality of range detection sensors comprises at least one laser range scanner, LIDAR sensor, and at least one radar sensor. The range detection sensors are configured to perform scanning of the environment of the work machine by determining distances to
obstacles surrounding the mining machine. The at least one LIDAR scanner is configured to measure distances using laser beam technology in given directions and with given angles. The LIDAR range detection sensor may be 2D scanner configured to monitor tunnel walls at predetermined heights. Correspondingly, the one or more radar sensor is configured for determining distances in given directions and with given angles using radar technology. The skilled person will appreciate that other types of range detection or distance measuring sensors may also be applicable for the purpose of determining a distance between the mining machine and its surroundings and may be used in combination with said LIDAR and radar sensors. In the disclosed example, the mining machine comprises a front range detection LIDAR/radar sensor pair 14 and a rear range detection LIDAR/radar sensor pair 15, that are configured to determine a distance from the respective sensor to path barriers present along a path travelled by the mining machine during tramming. However, the present disclosure is in no way limited to the disclosed placing of the range detection sensors. Any type of sensor mounting that supports distance measuring/ range detection to surrounding walls and obstacles is within the scope of the present disclosure.
Scanned data may be processed in the tramming assist arrangement and compared to reference profile data stored in an environmental model. A position of the work machine may be determined based on the finding of a match in the environment model to localize the work machine. The localization may be further corrected through dead-reckoning operations.
In some examples, a 3D LIDAR scanner may be applied, in which case 3D scanning data or point cloud data is produced and applied for positioning the mine vehicle.
Point cloud data generated during a scanning operation may be applied for generating and updating an environment model, such as an underground tunnel model, which may be applied for positioning the mine vehicle at the worksite. The tramming assist arrangement may execute a point cloud matching functionality for matching operational (scanned) point cloud data (being scanned by the scanner(s) to environment model point cloud data, i.e., reference point cloud data. Position and direction of the scanning device and/or another interest point of the vehicle, such as the (leading edge of the) bucket, may be determined
in the mine coordinate system on the basis of detected matches between the operational point cloud data and the reference cloud data.
A driving plan, or a route plan, may define a route to be driven by the mine vehicle and may be used as an input for automatic control of the mine vehicle. The plan may be generated offline and off-site, for example in an office, or on-board the mine vehicle e.g., by a teaching drive. The plan may define a start point, an end point, and a set of route points for the automatic drive. Localization of the work machine performed according to the route plan. The route plan may be stored in a memory of the work machine or in memory of the tramming assist arrangement.
The range detection sensors are used to measure distances to an object/barrier, e.g., a rock wall, a rock, or any other path barrier along the path travelled by the mining machine during tramming. The front range detection LIDAR/radar sensor pair 14 may be used to obtain range readings, e.g., from a laser scan over a range detection field or segment and from a radar scan over an overlapping or same range detection field or segment. In some examples, the laser scan will provide range readings for each whole degree ± 90 degrees from the respective longitudinal direction during a scan. The radar scan will provide range readings, e.g., for each whole degree, over an at least partly overlapping coverage area, e.g., ± 75 degrees. As will be understood, it is possible to use laser or radar range scanners which measure distances, obtain range readings, at a significantly higher resolution or at a significantly lower resolution. It is also possible to use laser or radar range scanners which obtain range readings in a significantly wider direction, as well as those which measure distance in a narrower direction. It is also possible to use one or more single omnidirectional laser or radar range detection sensors to determine distance in any travelling direction of the vehicle or a rotating range detection sensor.
The range detection sensor pairs are mounted on the mining machine. When mounting the range detection sensor pairs on a machine comprising a bucket or scoop, one sensor pair may be arranged on top of the mining machine, e.g., at a position maintaining a line of sight for the sensor pair from the vehicle to the surrounding environment also when the bucket is in a lowered position, in a partly lifted position and/or in a lifted position. Further range detection sensors may be provided at a lower part of the mining machine so that obstacles
on the ground may be detected at times when the bucket is in a partly lifted position and/or in a lifted position, i.e., not obscuring the line of sight for range detection sensor mounted on a lower part of the mining machine. Consequently, the mounting of range detection sensors as visualized in Figure 1 is only for general understanding and the below proposed method will be equally applicable regardless of the where the range detection sensor is mounted on the mining machine.
The work machine is configured to navigate within the underground environment using a pre-obtained representation of the mine. This kind of navigation has different challenges to that of navigating in an open-air environment. In order to navigate autonomously inside the network of tunnels of an underground mine, a proper representation of the mine is required, e.g., a navigation map that includes a topological structure of the mine, tunnels, and intersections, as well as measurements from range detection sensors, e.g., LIDAR sensors, to well-known reference points in the mine. A map of the operation area may be built during setup, using measurements from LIDAR sensors obtained during operator- controlled travel within the underground environment. The map is linked to a topological representation, wherein identified locations are perceived as nodes in the representation. The map allows for route planning and self-localization of the machine.
When driving inside the tunnels, it is important to avoid collisions with the tunnel walls and to make appropriate decisions at tunnel intersections and access point, but self-localization at every point in a tunnel may not be necessary. Self-localization may be performed by scan matching between measurements obtained while a machine is moving in the represented environment, and landmarks stored in the map. Scan matching may be performed using a closest point algorithm and filtering.
The following disclosure will focus on a navigation system for a mining vehicle such as a Load-Haul-Dump (LHD) vehicle, but it will be appreciated that the principles are equally applicable to other type of work machines configured to perform autonomous movement in an underground or tunnel environment. Thus, the present disclosure is not limited to a loader/hauler type of mining machine 10 as disclosed in Figure 1, the disclosure is equally applicable to other types of mining machines, such as dumpers, concrete spraying machines, drilling rigs and/or bolting rigs.
The LHD is a center-articulated vehicle with a frontal bucket used to load and transport ore on the production levels of an underground mine. These vehicles are key components in an ore extraction production chain from the underground mine. The ore extraction rate from the mine will depend directly on the efficiency of the LHD.
Turning to Figure 2, aspects of a computer-implemented method for localizing a work machine in an underground environment is disclosed. The work machine, e.g., an underground mining machine, is configured for autonomous tramming and/or remotecontrol tramming at an underground construction site or as a mining machine in an underground mining environment.
The method comprises an optional step S21 of generating a representation of an underground environment in which the work machine is configured to operate whilst driving the work machine in the environment, e.g., using LIDAR sensors or a combination of LIDAR and radar sensors.
In its most general form, the proposed method may be initiated at any location within an underground environment for which a representation has been generated. The work machine is configured to perform the method whilst performing an autonomous or partly autonomous tramming operation within the represented environment.
The method comprises the step of obtaining S22 sets of range readings SL(tL) G RML , SR(tR) G RMR from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances over a given period of time. The measured distances are obtained from respective sensor coverage area at least in part surrounding the respective sensor. Thus, the work machine obtains range readings by different types of sensors and two different technologies. It has been shown that airborne dust has little impact on radar measurements. The LIDAR and radarsensors are preferably simultaneously active when the work machine performs the tramming operation. Alternatively, the radar sensor could be activated when a predetermined number of uncertain LIDAR range readings have been detected.
In some examples, the obtaining S22 is repetitively performed while the work machine performs an automated, e.g., autonomous, or remotely controlled, tramming operation
along a planned route. The obtained range readings may be stored S23, e.g., within a localization arrangement of the mining machine.
A subset of radar range readings representing distance measurements within at least one segment of the coverage area is selected S25. The subset of radar range readings may be selected based on a determined distance measurement accuracy or lack of accuracy in the LIDAR range readings. As processing capacity increases, parallel processing of LIDAR range readings and radar range readings will increase and selecting the subset of radar range readings may imply selecting all obtained radar range readings. As previously recognized, environmental conditions of the work machine may be far from ideal in the underground environment and the LIDAR range readings obtained during operation in the mine may be highly uncertain. The uncertainties may in many cases depend on range detection sensor visibility. Dirt on a lens of the sensor or pollution in ambient air, e.g., from dust particles, are well-known sources of such uncertainties. There are several situations when these uncertainties affect the ability to localize a work machine in underground environment, resulting in the need to reduce the speed when performing autonomous tramming or tramming in a remote-control mode. There are well known technologies to determine such uncertainties, such technologies not forming part of the present disclosure. However, selecting S25 of the subset of radar range readings may be based on distance measurement accuracy of the LIDAR sensors. Thus, when dust in the surroundings is detected, using LiDAR data, measurements from the radar sensor will also be processed. In such way the localization will maintain localization quality despite problems with the LIDAR data. When selecting range readings for the processing, range readings from the LIDAR sensors may be preferred and given priority for as long as the quality/accuracy of these range readings is deemed sufficient.
In some examples, the selecting follows upon real-time processing of range readings from one or more stored S23 sets of range readings. The range readings may be jointly or separately stored; the selecting being performed on range readings S(t) G /?Msreceived or obtained from a buffer memory. The real-time processing may comprise a repeated determining S24 of distance measurement accuracy of the LIDAR range readings. When the accuracy of the LIDAR range readings is determined to fall below an acceptable threshold
level during a given time, radar range readings from the same time may be selected to be included in a localization processing. Thus, the selecting S25 of the radar range readings may be based on a determined distance measurement accuracy, e.g., an accuracy falling below a predetermined threshold. As mentioned, radar range readings may also be included when there is ample processing capacity. The implementation can be done in several ways where the data, i.e., range readings, from both LIDAR and radar may be used as a pool of data for navigation or only LIDAR data can be used until a threshold is reached (too many incorrect readings) and then the radar data complements the LIDAR data.
In some examples, the determining S24 of distance measurement accuracy comprises classifying at least one subset of the obtained LIDAR range readings as representing non- conclusive distance measurements. The selecting S25 then implies selecting a subset of radar range readings when at least one subset of the obtained LIDAR range readings are classified as representing non-conclusive distance measurements within said at least one segment of the sensor coverage area. A resulting set of range S(t) G RMF readings comprising a combination of radar range readings and LIDAR range readings is provided for subsequent processing to determine a localization x,y, 6 of the work machine.
In some examples, the at least one subset of obtained LIDAR range readings are classified as representing non-conclusive distance measurements when at least a predetermined number of distance measurements in the subset comprises measured distances shorter than a configurable minimum distance or longer than a configurable maximum distance.
Thus, a localization of the work machine in a representation of the underground environment is determined S26 based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings. The LIDAR data may be obtained at a different frequency/rate than the radar data, e.g., at a higher frequency. Thus, the amount of LIDAR data may differ from the amount of radar data when performing the combination to arrive at the localization and there may be different needs in terms of processing capabilities for the obtained sets of range readings.
In some examples, a mutual distance is determined between the LIDAR and the radar sensors. Range readings of the LIDAR and radar sensors may then be correlated and the radar range readings within the at least one segment of the coverage area may be adjusted
based on the determined mutual distance. This further improves the possibility of accurate localization of the work machine.
In some examples, the tramming motion of the work machine may be reduced when the at least one segment of the coverage area is greater than a predetermined portion of the coverage area. Reduction of the tramming motion may comprise reducing a speed of the work machine when travelling into a direction where other types of work machines, e.g., drill rigs, are known to perform drilling operations. Exiting the same environment, the earlier speed may be successively restored as the machine leaves the environment of impaired visibility. If there are no directional aspects of the reduced distance measurement accuracy, the velocity of the work machine may be reduced so that a same limitation to the velocity is obligated regardless of the travelling direction of the work machine.
Turning to Figure 3a, a tramming assist arrangement 30 is disclosed, e.g., the tramming assist arrangement 11 as comprised in the mining machine 10 of Figure 1. As mentioned, the work machine 10 may be running in an autonomous mode or in an unmanned, remotely operated mode. Thus, a user interface may be remote from the vehicle and the vehicle may be remotely controlled by an operator in the tunnel, or in control room at the mine area or even long distance away from the mine via communications network(s). A control unit outside the work machine 10 may be configured to perform some of the below disclosed features.
The tramming assist arrangement 30 is configured to perform the above disclosed method. In the context of the present disclosure, the tramming assist arrangement may be comprised in a control unit of the work machine, wherein the control unit comprises processing circuitry executing program code stored in a memory. The tramming assist arrangement 30 comprises at least one LIDAR sensor 32 and at least one radar sensor 31 mounted at a mutual distance from one another and configured to obtain range readings to determine a distance from the respective sensor to path barriers present along a path travelled by a tramming work machine. The processing circuitry of the tramming assist system is configured to perform the above disclosed method. In its most general operational mode, the processing circuitry is configured to obtain S22 sets of range readings from respective LIDAR and radar sensors, select S25 a subset of radar range
readings representing distance measurements within at least one segment of the coverage area and determine S27 a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings. Each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor. In some examples, the LIDAR sensor and the radar sensor are mounted in adjacent positions along a symmetry line of the work machine. In some examples, the LIDAR sensor and the radar sensor are arranged at an essentially same height.
Said control unit of the work machine may be connected to further control units in the work machine, e.g., through a local network. The control unit is configured to control at least autonomous tramming and localization operations for the work machine. It is to be appreciated that the control unit may be configured to perform the above disclosed method steps. There may be further operations modules or functions performed by the control unit(s) to support a tramming assist localization functionality implemented in the mining machine. The tramming assist arrangement comprises processing circuitry 33 configured to determine a route for routing of the mining machine between a mapped location and an un-mapped draw point.
The processing circuitry may comprise a processor 33a and a memory 33b. Figure 3a further illustrates an example computer program product 34 having thereon a computer program comprising instructions. The computer program product comprises a computer readable medium such as, for example a universal serial bus USB memory, a plug-in card, an embedded drive, or a read only memory ROM. The computer readable medium stores a computer program comprising program instructions that are loadable into the processing circuitry 33, e.g., into the memory 33b. The program instructions may be executed by the processor 33a to perform the above disclosed method.
Thus, the computer program is loadable into data processing circuitry, e.g., into the processing circuitry 31 of Figure 3a, and is configured to cause execution of embodiments for diagnosing range detection capability of the at least one range detection sensor.
Turning to Figure 3b, an alternative schematic block diagram is disclosed for the tramming assist arrangement, e.g., the tramming assist arrangement 11 as comprised in the work machine 10 of Figure 1 and schematically disclosed in Figure 4a. One or more LIDAR sensors 32 are configured to obtain LIDAR range readings SL(tL) G RML . Simultaneously with the obtaining performed by the LIDAR sensor, one or more radar sensors 31 may obtain radar range readings SR(tR) G RMR. The LIDAR range readings and the radar range readings are obtained within a sensor coverage area at least in part surrounding the respective sensor. The radar and LIDAR sensor 31, 32 are preferably arranged at adjacent locations; thus, the coverage of the LIDAR sensor area and the radar sensor area will be overlapping. Overlapping coverage areas may of course also be obtained without physically arranging the LIDAR 32 and radar sensors 31 as sensor pairs, if their positions relative to one another, e.g., distance, may be determined and used in the processing of the obtained data.
The block diagram may further comprise a memory arranged to store the obtained sets of range readings. In some examples, the at least one radar sensor and the at least one LIDAR sensor are simultaneously active and provides range readings with a frequency determined by capabilities of the respective sensor. In other examples, the radar sensor may be activated in response to one or more LIDAR range readings determined as inaccurate.
The tramming assist arrangement further comprises a selector, i.e., processing circuitry configured to perform processing of the obtained set of range readings from the respective LIDAR and radar sensors (32, 31). The processing circuitry receives a combination of range readings SR, SL from the memory and performs processing of the received data. Range readings from the LIDAR sensor may be given priority for as long as the LIDAR range readings are not qualified as faulty. When the quality of the LIDAR range readings falls below a predetermined threshold, range readings from the radar sensor are used in the determining of the localization.
The description of the example embodiments provided herein have been presented for purposes of illustration. The description is not intended to be exhaustive orto limit example embodiments to the precise form disclosed; modifications and variations are possible within the scope of the above teachings or may be acquired from practice of various alternatives to the provided embodiments. The examples discussed herein were chosen
and described in order to explain the principles and the nature of various example embodiments and its practical application to enable one skilled in the art to utilize the example embodiments in various manners and with various modifications as are suited to the particular use contemplated. The features of the embodiments described herein may be combined in all possible combinations of source nodes, target nodes, corresponding methods, and computer program products. It should be appreciated that the example embodiments presented herein may be practiced in combination with each other.
The described embodiments and their equivalents may be realized in software or hardware or a combination thereof. The embodiments may be performed by general purpose circuitry. Examples of general-purpose circuitry include digital signal processors DSP, central processing units (CPU), co-processor units, field programmable gate arrays FPGA and other programmable hardware. Alternatively, or additionally, the embodiments may be performed by specialized circuitry, such as application specific integrated circuits ASIC. The general-purpose circuitry and/or the specialized circuitry may, for example, be associated with or comprised in an apparatus such as a wireless communication device or a network node.
Embodiments may appear within an electronic apparatus comprising arrangements, circuitry, and/or logic according to any of the embodiments described herein. Alternatively, or additionally, an electronic apparatus may be configured to perform methods according to any of the embodiments described herein.
Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used.
Reference has been made herein to various embodiments. However, a person skilled in the art would recognize numerous variations to the described embodiments that would still fall within the scope of the claims.
For example, the method embodiments described herein discloses example methods through steps being performed in a certain order. However, it is recognized that these sequences of events may take place in another order without departing from the scope of
the claims. Furthermore, some method steps may be performed in parallel even though they have been described as performed in sequence. Thus, the steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step.
In the same manner, it should be noted that in the description of embodiments, the partition of functional blocks into particular units is by no means intended as limiting. Contra rily, these partitions are merely examples. Functional blocks described herein as one unit may be split into two or more units. Furthermore, functional blocks described herein as being implemented as two or more units may be merged into fewer (e.g., a single) unit.
Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever suitable. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa.
In the drawings and specification, there have been disclosed exemplary aspects of the disclosure. However, many variations and modifications can be made to these aspects without substantially departing from the principles of the present disclosure. Thus, the disclosure should be regarded as illustrative ratherthan restrictive, and not as being limited to the aspects discussed above. Accordingly, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation.
Hence, the details of the described embodiments are merely examples brought forward for illustrative purposes, and all variations that fall within the scope of the claims are intended to be embraced therein.
Claims
1. A computer-implemented method for localizing a work machine (10) configured for autonomous tramming and/or remote-control tramming at an underground construction site or as a mining machine in an underground mining environment; the method comprising:
- obtaining (S22) sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor;
- selecting (S25) a subset of radar range readings representing distance measurements within at least one segment of the coverage area; and
- determining (S27) a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings.
2. The method of claim 1, the method further comprising:
- repetitively performing the obtaining (S22) of sets of range readings from respective LIDAR and radar sensors, wherein the LIDAR range readings are obtained at a higher frequency than the subset of radar range readings;
- storing (S23) the sets of range readings, and
- determining (S25) a distance measurement accuracy of the LIDAR range readings, wherein the selecting (S26) of the subset of radar range readings is based on the determined distance measurement accuracy.
3. The method of claim 2, wherein performing (S24) the real-time processing of the stored sets of range readings comprises:
- classifying at least one subset of the obtained LIDAR range readings as representing non-conclusive distance measurements.
4. The method of claim 2 or 3, wherein the subset of radar range readings are selected when at least one subset of the obtained LIDAR range readings are classified as
representing non-conclusive distance measurements within said at least one segment of the sensor coverage area.
5. The method of claim 4, wherein the at least one subset of the obtained LIDAR range readings are classified as representing non-conclusive distance measurements when at least a predetermined number of distance measurements in the subset comprises measured distances shorter than configurable minimum distance or longer than a configurable maximum distance.
6. The method of any of the preceding claims, further comprising:
- determining a mutual distance between the LIDAR and radar sensors,
- correlating range readings of the LIDAR and radar sensors; and
- adjusting the radar range readings within said at least one segment of coverage area based on the determined mutual distance.
7. The method of any of the preceding claims, further comprising:
- reducing a speed of the work machine when said at least one segment of the coverage area is greater than a predetermined portion of the coverage area.
8. A computer program product comprising a non-transitory computer readable medium having thereon a computer program comprising program instructions loadable into processing circuitry and configured to cause execution of the method according to any of claims 1-7 when the computer program is run by the processing circuitry.
9. A tramming assist arrangement (30) comprised in a work machine (10) configured for autonomous tramming and/or remote control tramming at a construction site or as a mining machine in a mine environment; the tramming assist arrangement comprising at least one LIDAR sensor (31) and at least one radar sensor (32 configured to obtain range readings to determine a distance from the respective sensor to path barriers present along a path travelled by a tramming work machine, the tramming assist arrangement comprising processing circuitry (33) configured to:
- obtain sets of range readings from respective LIDAR and radar sensors; wherein each set of range readings represents a plurality of measured distances within a sensor coverage area at least in part surrounding the respective sensor;
- select a subset of radar range readings representing distance measurements within at least one segment of the coverage area; and
- determine a localization of the work machine in a representation of the underground environment based on a combination of the obtained LIDAR range readings and the selected subset of radar range readings. A work machine (10) configured for autonomous tramming and/or remote-control tramming at a construction site or as a mining machine in a mine environment, the work machine comprising a tramming assist arrangement according to claim 11. The work machine according to claim 10, wherein a LIDAR sensor and a radar sensor are arranged to form a sensor pair (14, 15). The work machine according to claim 10 or 11, wherein the LIDAR sensor and the radar sensor are mounted in adjacent positions along a symmetry line of the work machine. The tramming assist arrangement according to any of claims 9 to 11, wherein the LIDAR sensor and the radar sensor are arranged at an essentially same height.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/SE2022/050753 WO2024039269A1 (en) | 2022-08-17 | 2022-08-17 | Method and arrangement for localization of a work machine |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/SE2022/050753 WO2024039269A1 (en) | 2022-08-17 | 2022-08-17 | Method and arrangement for localization of a work machine |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024039269A1 true WO2024039269A1 (en) | 2024-02-22 |
Family
ID=83280597
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/SE2022/050753 WO2024039269A1 (en) | 2022-08-17 | 2022-08-17 | Method and arrangement for localization of a work machine |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2024039269A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117849818A (en) * | 2024-03-08 | 2024-04-09 | 山西万鼎空间数字有限公司 | Unmanned aerial vehicle positioning method and device based on laser radar and electronic equipment |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170124862A1 (en) * | 2015-10-30 | 2017-05-04 | Komatsu Ltd. | Construction machine control system, construction machine, construction machine management system, and construction machine control method and program |
US20170307751A1 (en) * | 2016-04-22 | 2017-10-26 | Mohsen Rohani | Systems and methods for unified mapping of an environment |
US20170307746A1 (en) * | 2016-04-22 | 2017-10-26 | Mohsen Rohani | Systems and methods for radar-based localization |
-
2022
- 2022-08-17 WO PCT/SE2022/050753 patent/WO2024039269A1/en unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170124862A1 (en) * | 2015-10-30 | 2017-05-04 | Komatsu Ltd. | Construction machine control system, construction machine, construction machine management system, and construction machine control method and program |
US20170307751A1 (en) * | 2016-04-22 | 2017-10-26 | Mohsen Rohani | Systems and methods for unified mapping of an environment |
US20170307746A1 (en) * | 2016-04-22 | 2017-10-26 | Mohsen Rohani | Systems and methods for radar-based localization |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117849818A (en) * | 2024-03-08 | 2024-04-09 | 山西万鼎空间数字有限公司 | Unmanned aerial vehicle positioning method and device based on laser radar and electronic equipment |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6909726B2 (en) | Work machine control system, work machine, work machine management system, and work machine management method | |
CN108139755B (en) | Abnormality detection device for self-position estimation device, and vehicle | |
AU2011337752B2 (en) | Method of controlling travel within travel system for unmanned vehicle and travel system for unmanned vehicle | |
JP6672336B2 (en) | Work machine control system, work machine, work machine management system, and work machine management method | |
AU2016395616B2 (en) | Work machine management system and work machine | |
JP6271023B2 (en) | Work machine control system, work machine, and work machine control method | |
AU2016390303B2 (en) | Work machine management system, work machine, and work machine management method | |
JP6672339B2 (en) | Work machine management system and work machine | |
JP2017016477A (en) | Work machine travel assist system and transport vehicle | |
WO2024039269A1 (en) | Method and arrangement for localization of a work machine | |
WO2022024120A1 (en) | Guidance of a transport vehicle to a loading point | |
CN114391060A (en) | Positioning of mobile equipment in an underground worksite | |
WO2024005680A1 (en) | Method, arrangement, and underground mining machine for autonomous routing to an unmapped location | |
AU769593B2 (en) | Vehicle guidance system | |
CA3182963A1 (en) | Self-test method for a ranging sensor-arrangement of a work machine |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22768994 Country of ref document: EP Kind code of ref document: A1 |