WO2026019352A1 - Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment - Google Patents
Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environmentInfo
- Publication number
- WO2026019352A1 WO2026019352A1 PCT/SE2024/050696 SE2024050696W WO2026019352A1 WO 2026019352 A1 WO2026019352 A1 WO 2026019352A1 SE 2024050696 W SE2024050696 W SE 2024050696W WO 2026019352 A1 WO2026019352 A1 WO 2026019352A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mining machine
- mining
- material moving
- machine
- sensor measurements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- E—FIXED CONSTRUCTIONS
- E21—EARTH OR ROCK DRILLING; MINING
- E21F—SAFETY DEVICES, TRANSPORT, FILLING-UP, RESCUE, VENTILATION, OR DRAINING IN OR OF MINES OR TUNNELS
- E21F13/00—Transport specially adapted to underground conditions
- E21F13/02—Transport of mined mineral in galleries
- E21F13/025—Shuttle cars
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F3/00—Dredgers; Soil-shifting machines
- E02F3/04—Dredgers; Soil-shifting machines mechanically-driven
- E02F3/28—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets
- E02F3/36—Component parts
- E02F3/42—Drives for dippers, buckets, dipper-arms or bucket-arms
- E02F3/43—Control of dipper or bucket position; Control of sequence of drive operations
- E02F3/431—Control of dipper or bucket position; Control of sequence of drive operations for bucket-arms, front-end loaders, dumpers or the like
- E02F3/434—Control of dipper or bucket position; Control of sequence of drive operations for bucket-arms, front-end loaders, dumpers or the like providing automatic sequences of movements, e.g. automatic dumping or loading, automatic return-to-dig
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F3/00—Dredgers; Soil-shifting machines
- E02F3/04—Dredgers; Soil-shifting machines mechanically-driven
- E02F3/28—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets
- E02F3/36—Component parts
- E02F3/42—Drives for dippers, buckets, dipper-arms or bucket-arms
- E02F3/43—Control of dipper or bucket position; Control of sequence of drive operations
- E02F3/435—Control of dipper or bucket position; Control of sequence of drive operations for dipper-arms, backhoes or the like
- E02F3/437—Control of dipper or bucket position; Control of sequence of drive operations for dipper-arms, backhoes or the like providing automatic sequences of movements, e.g. linear excavation, keeping dipper angle constant
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/20—Drives; Control devices
- E02F9/2025—Particular purposes of control systems not otherwise provided for
- E02F9/2054—Fleet management
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/264—Sensors and their calibration for indicating the position of the work tool
- E02F9/265—Sensors and their calibration for indicating the position of the work tool with follow-up actions (e.g. control signals sent to actuate the work tool)
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/20—Drives; Control devices
- E02F9/2025—Particular purposes of control systems not otherwise provided for
- E02F9/205—Remotely operated machines, e.g. unmanned vehicles
-
- E—FIXED CONSTRUCTIONS
- E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
- E02F—DREDGING; SOIL-SHIFTING
- E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
- E02F9/26—Indicating devices
- E02F9/267—Diagnosing or detecting failure of vehicles
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/008—Registering or indicating the working of vehicles communicating information to a remotely located station
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
Definitions
- the disclosure relates to systems and methods for monitoring and controlling operation of a mining machine from a plurality of mining machine in a mining site.
- the disclosure further relates to a field computer device configured to be installed on the mining machine, a mining machine, a kit for the mining machine, a computer program product, and a computer-readable storage medium for monitoring and controlling operation of a mining machine.
- the LDH machines may be used to remove and transport broken rock/ore from a certain location, e.g., a position where blasting has been performed, to a particular place where the broken rock is dumped. After dumping their load, at the place that may be referred to as a dump point or location, the LHD machines typically return to an initial (start) location to pick up a new load. Thus, these machines often travel the same route over and over again, which makes the travel between load and dump locations well suited for automation. There are also various other situations where automation may prove beneficial. [0005]
- the mining machine may be operating in one out of different possible operating states, and a current state is typically recorded by a machine operator. This may however be a tiring and cumbersome process for a human.
- the method comprises, with at least one processor, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; applying a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determining at least one operation indicator from the identified at least one material moving cycle; and initiating an action in dependence on determining of the at least one operation indicator.
- the technical benefits and advantages comprise automatic prediction or detection of material moving cycles, which allows improved control over productivity of individual mining machines and of the entire mine. In this way, overall efficiency, safety and productivity of mining operations may be increased.
- the determining of the material moving cycles and operation indicators may be performed in real time, which allows implementation of timely control and intervention in fleet management systems. For a fleet of mining machines, decisions may be made on dynamic operational assignments.
- the provided approach is equipment agnostic, such that it may be integrated with various mining machinery and equipment without requiring extensive modifications or specialized hardware. This flexibility not only simplifies implementation, but also allows for wider adoption in different mining configurations.
- the provided system, mining machine, kit, and methods therein are developed and configured to operate with low computational resource requirements. This feature allows for the provided technique to be implemented in onboard controllers, e.g. in a field controller or computer device, which typically have limited processing capabilities.
- the provided technique may ensure optimal performance even in challenging computing environments.
- various events or actions may be triggered e.g. initiated in response to predicting the ore moving cycles and determining productivity indicators such as e.g., predictive maintenance of mining machines, optimizing routes traveled by the machines, defining and adjusting assignments for the mining machines, and others.
- the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by Docket No.: PS56142PC00/ P23087WO01 the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the method may further comprise determining a correctness of the identified at least one material moving cycle. The at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- the method may further comprise receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site. [0012] In some examples, the method may further comprise applying a second trained machine- learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. [0013] In some examples, the second trained machine-learning model is trained in dependency on one or more features selected from time series data previously acquired by measuring signals from one or more position beacons. The one or more position beacons may comprise the at least one position beacon. In some examples, the signals from one or more position beacons may comprise actual data and/or simulated data.
- the first trained machine-learning model is trained using inertial sensor measurements previously acquired by one or more IMU sensors.
- the one or more IMU sensors may include the at least one IMU sensor and/or by one or more other sensors.
- training inertial sensor measurements may be simulated data or a combination of actual data acquired in a real- world environment and the simulated data.
- initiating the action in dependence on determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; and adjusting a target requirement for performance of the mining machine.
- a computer device is provided that comprises at least one processor configured to perform the method in accordance with any embodiments of the present disclosure. Docket No.: PS56142PC00/ P23087WO01 [0018] Advantages and effects of the computer device are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer device are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.
- a computer program product comprises computer- executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with any embodiments of the present disclosure.
- the computer program product may be provided as part of a kit.
- Advantages and effects of the computer program product are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer program are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.
- a tangible computer-readable storage medium is provided that has stored thereon a computer program product comprising computer-executable instructions.
- the tangible computer-readable storage medium may be provided as part of a kit.
- Advantages and effects of the tangible computer-readable storage medium are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the tangible computer-readable storage medium are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.
- a mining machine for operation in a mining site is provided.
- the mining machine comprises a movable implement comprising a tool configured to load and unload material; at least one IMU sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site; and at least one processor and a memory comprising computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive the inertial sensor measurements acquired by the at least one IMU sensor, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the Docket No.: PS56142PC00/ P23087WO01 identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator.
- the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the computer-executable instructions when executed by the at least one processor, may further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- the computer-executable instructions when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site, and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
- initiating the action in dependence on the determining of the at least one operation indicator comprises prompting, by the at least one processor, a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
- a kit for installation on a mining machine is provided.
- the mining machine is configured to operate in a mining site.
- the kit comprises at least one IMU sensor configured to be associated with mining machine and computer program product comprising computer-executable Docket No.: PS56142PC00/ P23087WO01 instructions configured to be installed on the mining machine.
- the computer-executable instructions when executed by at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator.
- the at least one IMU sensor included in the kit comprises one e.g. single IMU sensor.
- the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.
- the inertial sensor measurements may comprise nine-axis measurements.
- the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the computer-executable instructions when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
- FIG.1A illustrates an example of a mining environment in which a method in accordance with examples of the present disclosure may be implemented.
- FIG.1B illustrates an example of a mining machine.
- FIG.2 is a block diagram illustrating an example of a system comprising a mining machine and a central control system, in which a method in accordance with examples of the present disclosure may be implemented.
- FIG.3 is a flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure.
- FIG.4 is a flowchart illustrating a method for generating and training a machine-learning model, in accordance with examples of the present disclosure.
- FIG.5 is another flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure.
- FIG.6 is another flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure.
- FIG.7 is a flowchart illustrating a method performed by a central control system, in accordance with examples of the present disclosure.
- FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a user interface of a computer device, in accordance with examples of the present disclosure.
- DETAILED DESCRIPTION [0047] Aspects of the present disclosure relate to a method performed by a field computer device for monitoring and controlling operation of a mining machine in a mining site, to the field computer device, a kit comprising at least one inertial measurement unit (IMU) sensor, and the mining machine comprising the field computer device.
- IMU inertial measurement unit
- the techniques described herein allow using inertial sensor measurements acquired by at least one inertial measurement unit (IMU) to identify one or more operating cycles for a mining machine that is operating in the mining site.
- IMU inertial measurement unit
- a trained machine-learning model may be used to identify the one or more operating cycles.
- the cycle may comprise a sequence of operating states comprising loading the mining machine at a loading point, moving of the loaded mining machine from the loading point to a dumping point, unloading the mining machine at the dumping point, and moving the unloaded mining Docket No.: PS56142PC00/ P23087WO01 machine from the dumping point to the loading point.
- an ore hauling cycle is detected as a whole, without separately identifying a sequence of operating states forming the cycle.
- the machine-learning model may be trained to detect the entire cycle rather than to detect separate operating states within an ore hauling cycle.
- the field computer device is configured to also acquire position sensor measurements also referred to herein as tag sensor measurements, e.g., from at least one position beacon or tag.
- the position beacon or tag may be positioned at a known location in the mine.
- the field computer device may acquire the position sensor measurements from at least one sensor that is configured to communicate with the at least one, typically multiple position tags.
- the at least one sensor may be installed in or otherwise associated with the field computer device, or it may be positioned in another location on the mining machine.
- the field computer device installed on the mining machine may be configured to acquire the position sensor measurements as the mining machine is moved through the mine, and as the inertial sensor measurements are acquired by the field computer device.
- the position or tag sensor measurements may be pre-processed and stored e.g. in a memory of the field computer device until an ore hauling cycle is detected. Once the cycle is identified and validated using the inertial sensor measurements, the position sensor measurements acquired during a time period when the inertial sensor measurements are acquired, are processed. The processing may be performed using a trained machine-learning model, referred to herein as a second machine-learning model.
- the second machine-learning model may be applied to the pre-processed position sensor measurements to identify and/or verify an origin or loading point from which the material is taken and a destination or dumping point to which the material is taken during the identification of the material hauling cycle.
- the second machine-learning model may provide an output such as a value indicating a probability that a given signal comes from a certain loading point. More than one loading point may be identified, each assigned a corresponding probability. In some examples, a loading point associated with a highest probability may be selected as an actual loading point from which the ore was taken during the identified ore hauling cycle. In some examples, the dumping point may be identified and/or verified, for the material hauling cycle, in a similar manner.
- the dumping point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a dumping time window.
- the computer-implemented method for monitoring and controlling operation of a mining machine in a mining site comprises, by a processor, as the mining machine is operating in the mining site, receiving inertial sensor Docket No.: PS56142PC00/ P23087WO01 measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; applying a first trained machine- learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determining at least one operation indicator from the identified at least one material moving cycle; and initiating an action in dependence on determining of the at least one operation indicator.
- FIG.1A depicts an example of a mining environment or site 10 comprising multiple, four in this example, mining vehicles or machines 12a, 12b, 12c, and 12d.
- the mining machines may be of the same or different types.
- the mining machines may be underground mining machines or surface mining machines.
- One or more of the mining vehicles 12a-12d may be load-haul-dump (LHD) machines or other types of machines configured to carry material from one location to another.
- the mining machine may be an excavator, a backhoe, or another type of a mining machine with a movable implement.
- the mining machines 12a-12d may be autonomous machines, such as e.g., fully or partially autonomous machines.
- the mining machines 12a-12d may be operating in a fleet of mining machines.
- the mining machines in the fleet may be assigned work assignments or tasks.
- the mining environment or site 10 may be a surface mining site or an underground mining site.
- the mining site 10 may be any type of a mine-like environment.
- the mining site 10 may be a subway mine.
- the mining site 10 may have a specific configuration. For example, if the mining site 10 is an underground mining site, it may include tunnels.
- a mining machine in the mining site 10 may have a moving or movable implement, such as, e.g., an arm having a bucket attached thereto.
- each of the mining machines 12a-12d may have a respective IMU sensor 14, also referred to herein as an inertial sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site.
- An IMU sensor or sensor unit is a sensor that provides motion data in a time-series format.
- the IMU sensor comprises an accelerometer configured to acquire acceleration measurements, a gyroscope configured to acquire angular velocity measurements, and a magnetometer configured to measure a magnitude and direction of the magnetic field at a location of the magnetometer.
- combined measurements acquired by the IMU sensor can be used to determine a position, velocity, acceleration, and orientation of an object in a three-dimensional space to which object the IMU sensor is attached.
- the measurements acquired by each of the accelerometer, gyroscope, and magnetometer are represented along a three-axis coordinate system.
- the IMU sensor may be configured to provide a nine-dimensional time series data.
- Docket No.: PS56142PC00/ P23087WO01 [0056]
- the mining machines 12a-12d comprise respective IMU sensors 14a, 14b, 14c, and 14d, coupled to a corresponding movable implement 15, such as movable implements 15a, 15b, 15c, and 15d of the mining machine.
- each of the inertial sensors 14a, 14b, 14c, and 14d may be a single i.e. one IMU sensor.
- inertial sensor measurements acquired by a single IMU sensor or sensor unit may be sufficient to perform the method in accordance with examples of the present disclosure, which makes the provided approach less costly and requiring less maintenance.
- the single IMU sensor unit comprises an accelerometer, a gyroscope, and a magnetometer.
- the movable implement may comprise a tool configured to load and unload material, which may be any suitable load picking and carrying implement or tool, e.g., a bucket or another work tool for digging and/or load picking and carrying.
- the movable implement may comprise an arm such as e.g. a hydraulic arm which may comprise the tool configured to load and unload material e.g. a bucket as in an LHD machine.
- the IMU sensor unit may be coupled to the arm, e.g, to a joint of the arm.
- the inertial sensor may be positioned in proximity to the joint of the arm and in proximity to a tool for digging and/or load picking and carrying, e.g., a bucket or another work tool of the movable implement.
- the IMU sensor unit may be positioned on the bucket, but such that the use of the bucket does not interfere with the accuracy of measurements acquired by the IMU sensor.
- the IMU sensor unit may be positioned to detect and report acceleration, orientation, angular rates, and other gravitational forces as the mining machine and/or its parts move and vibrate.
- At least one IMU sensor is positioned on the mining machine where the acquired sensor measurements are informative of a current state of the movable implement and of the entire machine. Also, an ease of installation of the inertial sensor is taken into consideration when selecting a specific location at which to associate the inertial sensor with the mining machine.
- the specific position of the inertial sensor on the mining machine may depend on a configuration of the mining machine and the machine’s movable implement, type of work performed by the mining machine, work environment conditions to which the mining machine is subjected, the feasibility of installation of the inertial sensor and/or on other factors. In any case, in examples herein, the inertial sensor is positioned outside of an operator compartment or cabin of the mining machine.
- each of the machines 12a-12d may comprise or may be associated with a respective control unit or a field computer device 16a, 16b, 16c, 16d that may be Docket No.: PS56142PC00/ P23087WO01 configured to perform the method in accordance with examples of the present disclosure.
- Each of the field computer devices 16a-16d includes various components not shown in this example, such as a memory device configured to store computer-executable instructions and processing circuitry e.g. one or more processors.
- Each of the field computer devices 16a-16d also has an input and output interface that is configured to communicate with other components, e.g., with the respective IMU sensor, as well as with a fleet controller or central control system 22 and with other external systems.
- one or more of the field computer devices 16a-16d may comprise or may otherwise be associated with one or more position sensors configured to communicate with at least one position beacon positioned in the mining site.
- the mining site 10 may comprise position beacons 11a, 11b, and 11c.
- Position beacons may be located in proximity to a material origin or loading point and in proximity to a material destination or dumping point, to assist in determining with more precision the origin of the material and a point of dumping the material. Position beacons may be useful in determining a location of the mining machine in the mine, as the machine is moving e.g. in a tunnel.
- the mining environment may in some cases be deep underground and tracking locations of mining machines operating in such environments may be a challenge, let alone automatically determining production of the machine.
- the position beacon may be an active position tag or a passive position tag.
- the position beacon be configured as a transmitter, in some cases as a transceiver, and the position of the position beacon may be known.
- the position beacons may be installed in any suitable locations in the mining site.
- position sensor measurements may be acquired based on detection of the position beacons.
- the one or more position sensors associated with the field computer device of the mining machine may detect signals, e.g. radio frequency (RF) or other types of signals, emitted by the position beacons.
- the position beacon may emit signals in response to corresponding signals from the position sensors associated with the mining machine.
- RF radio frequency
- the position sensors may be passive tag with a known position, and the one or more position sensors associated with the mining machine may read the position information carried by the passive position tag.
- the mining machine e.g. a field computer device associated with the mining machine, is Docket No.: PS56142PC00/ P23087WO01 configured to receive and process position sensor measurements to verify the loading point and the dumping point for at least one identified material moving cycle.
- Multiple mining machines 12a-12d may be operated in the mining site 10 for hauling material such as ore from draw points or locations to one or more dumping points where the ore is deposited.
- a mining machine may be operated in an operating or operational material moving cycle that may comprise a sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving or translating back from the dumping point to the loading point, without a load.
- the mining machine may also be in a standby mode or standby, when it is idling such that it is operating but not currently moving or performing any work.
- the material moving cycle may include another sequence of operating states.
- the material moving cycle may be repeated multiple times during, for example, a shift of the operation of the mining machine.
- the mining machine may perform a certain number of cycles during the shift.
- a kit 18 may be installed and deployed on a mining machine out of the plurality of mining machines 12a-12d in the mining site 10.
- the kit may comprise at least one inertial sensor and computer program product comprising computer-executable instructions which, when executed by processing circuitry e.g. at least one processor, cause the processor to perform a method in accordance with examples of the present disclosure.
- the computer-executable instructions may be stored in a memory e.g. in a memory of the field computer device that may be included as part of the kit.
- FIG. 1A illustrates very schematically, for the mining machines 12a and 12b, that the machines may comprise respective kits 18a and 18b, shown by a dot-dashed line.
- the kit 18a may comprise the inertial sensor 14a and computer program product that may be stored in the field computer device 16a.
- the kit 18b may comprise the inertial sensor 14b and computer program product that may be stored in the field computer device 16b.
- operation of mining machines in the mining site 10 may be controlled by the central control system 22 such as e.g. a fleet controller or fleet control system.
- the central control system 22 which is communicatively coupled with one or more of the mining machines 12a-12d, may track locations or positions of each of the mining machines 12a-12d, may receive data from the mining machines, and may send commands or instructions to the mining machines.
- the fleet control system may also be referred to as a traffic Docket No.: PS56142PC00/ P23087WO01 control system, and it may be any type of a control system that monitors multiple machines e.g.
- FIG.1B illustrates an example of a mining machine 32, such as e.g., any of the mining machines 12a-12d shown in FIG.1A, or another type of a mining machine.
- FIG.1B illustrates schematically example locations or positions 34p1 and 34p2 of where an inertial sensor may be coupled to a movable implement 25 of the mining machine 32.
- the inertial sensor may be configured to acquire accelerometer, magnetometer, and gyroscope measurements. The measurements may be acquired at a desired frequency and may be used to infer when the mining machine lifts the movable implement 25 such as a hydraulic arm with a bucket or another tool attached thereto.
- the inertial sensor may be installed on the hydraulic arm of the movable implement 25 in proximity to the bucket, as shown in FIG.1B.
- the example positions 34p1 and 34p2 may be evaluated for ease of installation and data quality of obtained sensor measurements.
- the approach described herein in accordance with examples of the present disclosure, comprises two workflows also referred to as pipelines, which in turn comprise multiple stages.
- One of the pipelines is a training pipeline, which is developed using historical or batch data, and includes pre- analysis stages such as e.g. sensor description and position, exploratory data analysis (EDA), and modeling stages such as data capture and pre-processing, feature extraction, modeling, and validation.
- pre- analysis stages such as e.g. sensor description and position
- exploratory data analysis (EDA) exploratory data analysis
- modeling stages such as data capture and pre-processing, feature extraction, modeling, and validation.
- FIG.2 illustrates an example of a system 100 in which example embodiments of the present disclosure may be implemented.
- the system 100 may be employed, at least in part, in the mining environment or site such as e.g. mining site 10 shown schematically in FIG.1A.
- the system 100 Docket No.: PS56142PC00/ P23087WO01 may comprise one or more mining machines, and a mining machine 12 is shown in FIG.2 as a representative mining machine.
- the system 100 may also comprise a central controller or control system 22 such as, e.g. a traffic control system which may be a fleet controller configured to perform coordinated control of machines in the mining site. It should be noted however that the central control system 22 may be remote and it may be positioned outside of the mining site 10.
- the mining machines may be e.g. load-haul-dump (LHD) trucks, shovels, etc.
- LHD load-haul-dump
- the mining machine 12 may comprise a control system or mining machine controller 28 and a field computer device 16 which may be configured to perform the method in accordance with examples of the present disclosure.
- the field computer device 16 may be in operable communication with the mining machine controller 28. It should be noted that, even though the field computer device 16 is shown as a separate device in FIG.2, in some examples, the field computer device 16 may be part of the mining machine controller 28. Thus, the functionality performed by the field computer device 16 may be performed by the mining machine controller 28. In some examples, this functionality may be imparted to the mining machine controller 28 by installing on the mining machine controller 28 computer-executable instructions which may be part of a kit as described herein.
- the mining machine 12 may comprise or may be associated with a display 33 which may be configured render a graphical user interface that is configured to display a representation of the identified one or more material moving cycles, one or more operation indicators, as well as other information related to operation of the mining machine 12 and analysis of the operation using the techniques described herein.
- the display 33 may be part of a dashboard display or another type of a built-in display. In some examples, the display 33 may be associated e.g. coupled to a suitable location within an operator compartment of the mining machine 12.
- the display 33 may be part of a remote device communicatively coupled to mining machine 12 and/or the mining machine controller 28.
- the display 33 may be presented in a smartphone, a tablet, or another computer device which may be positioned remotely relative to the mining machine.
- the machine controller 28 may be located onboard the machine 12.
- the machine controller 28 may be a main controller of the mining machine 12 which is configured to control operations of the mining machine 12.
- the machine controller 28 may comprise processing circuitry 30, such as at least one processor, and memory 31 which may comprise one or more memory units.
- the memory 31 comprises computer-executable instructions executable by the processing circuitry 30 of the machine controller 28.
- the memory 31 may be configured to store information, data, etc., and the Docket No.: PS56142PC00/ P23087WO01 computer-executable instructions to perform, when executed by the processing circuitry 30, various processes related to monitoring and control operation of the mining machine 12.
- the machine controller 28 may comprise or may be associated with various other components not shown herein.
- the field computer device 16 may be adapted to execute computer-executable instructions to perform the functions or processes described herein.
- the field computer device 16 may be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet.
- the field computer device 16 may comprise processing circuitry 210 and a memory device or memory 220 which may comprise one or more memory units.
- the memory 220 comprises computer-executable instructions which may be executed by the processing circuitry 210 to cause the processing circuitry 210 to perform the method in accordance with examples of the present disclosure.
- the memory 220 may store various data related to the method in accordance with examples of the present disclosure. As shown in FIG.2, the memory 220 may store inertial sensor measurements 221 which may be acquired by the IMU sensor 14 also shown in FIG.2. As also shown in FIG.2, the memory 220 may store position sensor measurements 222 which may be acquired by at least one position sensor 17 also shown in FIG.2.
- the position sensor 17 may be configured to communicate with one or more position beacons located in the mining site. In some implementations, the position sensor 17 may be included in the machine controller 28.
- the inertial sensor measurements and the position sensor measurements may be sent to an external system, e.g., to the control system 22 and/or to another system.
- the memory 220 may comprise a material moving cycles registry 224 storing one or more material moving cycles predicted or identified for the mining machine 12.
- the memory 220 may also comprise machine-learning (ML) model units that store at least one trained ML model as well as various information associated with the trained ML model e.g. data used to train the model, extracted/constructed and/or selected features, and other information.
- ML machine-learning
- the at least one ML model unit may comprise a first ML model unit 226 storing a trained ML model, referred to herein as a first ML model, that can be applied to the inertial sensor measurements to identify or predict at least one material moving cycle from the inertial sensor measurements.
- the at least one ML model unit may also comprise a second ML model unit 228 storing a trained ML model, referred to herein as a second ML model, that can be applied to the position sensor measurements to verify a position of the loading and/or dumping points for the at least one material moving cycle that involves moving material from the loading point to the dumping point.
- the ML model units 226, 228 may comprise one or more subunits or modules not shown here, e.g., exploratory data analysis (EDA) unit, a feature construction and selection unit, a model building unit, a model evaluation unit, and other units or subunits. Docket No.: PS56142PC00/ P23087WO01 [0078]
- the memory 220 may comprise an operation indicators registry 232 storing one or more operation indicators 234.
- the one or more operation indicators 234 may be determined from one or more cycles predicted or identified for the mining machine.
- Non-limiting examples of the operation indicators 234 comprise a duration of a certain material moving cycle e.g.
- an ore hauling cycle a number of material moving cycles performed by the mining machine 12 during a certain time period e.g. a shift, a day, a month, a quarter, etc.; a total operating time determined for the mining machine; a total non-productive time representing a time during which the machine is stopped and/or is not contributing to production; operational efficiency of the mining machine, an unplanned downtime experienced by the mining machine; one or more production delays, and various other operation indicators etc.
- the operation indicators 234 may be stored in association with other information such as e.g. an operator identifier identifying an operator of the mining machine 12, a machine identifier identifying the mining machine 12, locations in the mine where the one or more identified material moving cycles were identified, and other information.
- the processing circuitry 210 may comprise at least one ML model execution unit 240 that is configured to execute the trained ML models stored in the ML model units 226, 228.
- the first and second trained ML models may be executed by the processing circuitry 210 to perform the method in accordance with examples of the present disclosure.
- the processing circuitry 210 may also comprise model retraining unit 230 that allows the processing circuitry 210 to automatically train and retrain the first ML model for identifying and/or predicting a material moving cycle and the second ML model for identifying and/or verifying a loading point and a dumping point between which the mining machine has moved the material during the identified material moving cycle.
- the same module or unit may perform ML model processing and retraining, e.g. the ML model execution unit 240.
- the processing circuitry 210 may include various other modules and/units configured to perform actions and methods in accordance with examples herein. [0080]
- the processing circuitry 210 may include any number of hardware components for conducting data or signal processing or for executing computer code such as computer-executable instructions stored in the memory 220.
- the processing circuitry 210 may include a general-purpose processor, an application specific processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processing components, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.
- the memory 220 may be one or more devices for storing data and/or computer code such as computer-executable instructions for completing or facilitating methods described herein.
- the memory 220 may comprise random access memory (RAM), read-only memory (ROM), erasable Docket No.: PS56142PC00/ P23087WO01 programmable read-only memory (EPROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and/or computer instructions.
- the memory 604 may include database components, object code components, script components, and/or any other type of information structures for supporting the various processes and information structures described in the present disclosure.
- the memory 220 may be communicably connected to the processing circuitry 210, e.g., via a circuit or any other wired or wireless connection.
- the field computer device 16 may also include a communications interface 242 that may include wired and/or wireless communications interfaces, e.g., jacks, antennas, transmitters, receivers, transceivers, wire terminals, etc., for conducting data communications external systems or devices.
- the communications may be direct, e.g., local wired or wireless communications, or via a communications network, e.g., a WAN, the Internet, a cellular network, etc.
- the field computer device 16 may include various other components not shown herein.
- any of the data and computer-executable instructions stored in the memory 220 may be loaded onto the processing circuity 210 or used by the processing circuitry 210, to perform the method in accordance with examples of the present disclosure.
- the units of the field computer device 16 described herein may refer to a combination of analogue and digital circuits, and/or one or more processors configured with software and/or firmware, e.g., stored in the field computer device 16, that, when executed by respective one or more processors, may perform the methods in accordance with examples of the present disclosure.
- processors may be included in a single Application-Specific Integrated Circuitry (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip.
- ASIC Application-Specific Integrated Circuitry
- the field computer device 16 may comprise a communication interface for communication with the IMU sensor 14 to receive one or more inertial sensor measurements, and for communication with remote systems such as e.g. the control system 22.
- FIG. 2 illustrates that the central control system 22 comprises processing circuitry 24 and memory 260.
- the central control system 22 may be a central controller or fleet controller that may be positioned remotely from a plurality of mining machines operating in the mining site.
- the central control system 22 may be positioned in the mining environment or site, e.g., it may be part of or associated with one of the mining machines in the mining site. Regardless of its specific Docket No.: PS56142PC00/ P23087WO01 implementation and location, the central control system 22 is configured to receive information from and to send information and control commands to one or more mining machines out of the plurality of mining machines in the mining site.
- the central control system 22 may be configured to control, in a coordinated manner, movements and/or other functions of the mining machines.
- the mining machines may be fully autonomous, semi-autonomous, or manually controlled machines, and various type of signals and instructions may be received by the machines from the control system 22.
- the memory 260 of the central control system 22 may store computer-executable instructions that can be executed by the processing circuitry 24 to cause the processing circuitry 24 to perform monitoring and controlling of operations of the mining machine 12.
- the memory 260 of the central control system 20 may receive various information regarding the mining machine 12 e.g. from the field computer device 16 and/or the machine controller 28, such as one or more identified machine moving cycles, one or more operator indicators, and various other information. Based on the received information, the control system 22 may generate and send commands to the mining machine 12, as well as to one or more of other mining machines in the mining site.
- the control system 22 may comprise and/or may be communicatively coupled to a display that is configured to render a graphical user interface 280.
- the user interface 280 may display various information related to mining machines controlled via the control system 22.
- the user interface 280 may also be configured to receive user input e.g. with respect to the displayed information.
- the control system 22 may include an input and output device interface (not shown) such as e.g. a circuit for controlling input and output from and to peripheral devices including devices such as a mouse, a keyboard, joystick, touch-sensitive surface or pad, touch-sensitive screen, etc.
- the user interface 280 of the control system 22 may be configured to present various information based on the identified material moving cycles and the operation indicators.
- the information allows visualizing and assessing a status of mining machines in the fleet, as well as a status of the entire fleet. For example, at any point in time, one or more material moving cycles identified for one or more mining machines currently located in the mining site may be visualized. A representation of the machines in the mining site by material moving cycles may also be visualized such that it is possible to access a number of material moving cycles performed by the one or more machines. It may be also possible to determine whether a mining machine in the site is currently performing a cycle which has not yet been identified. It may be, for example, visualized how many mining machines are currently operative, delayed, reserved or out of service.
- the one or more representations of the status of the individual or groups of the machines, and of the entire fleet, may be generated and displayed in real time, such that a real time monitoring, assessment, and control of the fleet of mining machines may be Docket No.: PS56142PC00/ P23087WO01 performed.
- a user such as e.g. a fleet operator may be provided with information that can be used to assess the status of the mining machines and to make decisions regarding operation of the mining machines.
- the information may in some cases be analyzed automatically. Maintenance decisions, machine repositioning, task or job assignments, and other actions may be performed using the identified cycles and the operation indicators identified or predicted in accordance with examples of the present disclosure.
- Instructions or commands may be generated by the control system 22 and sent to the mining machine 12 and/or other machines, instructing the machines to initiate actions related to maintenance, repositioning, task or job assignments, and other types of actions.
- the approach described herein, in accordance with examples of the present disclosure may comprise two or more workflows also referred to as pipelines, which in turn may comprise multiple stages.
- One of the pipelines is a training pipeline, which is developed using historical or batch data, and includes pre-analysis stages such as e.g. sensor description and position, exploratory data analysis (EDA), and modeling stages such as data capture and processing, feature extraction, modeling, and validation.
- FIG.3 illustrates an example of a computer-implemented process or method 300 for monitoring and controlling operation of a mining machine in a mining site.
- the mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another.
- the method 300 may be performed by a computer device, e.g.
- the method 300 comprises, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time.
- the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer
- the received sensor measurements may comprise sensor data in 9 axes.
- patterns indicative of material moving cycles performed by the mining machine may be detected in the 9-axis inertial sensor data or signal.
- data acquired from one or more of the three-axis accelerometer, the three-axis gyroscope, and the three-axis magnetometer may be used to identify a material moving cycle.
- the IMU sensor may be positioned on the mining machine, e.g.
- the first period of time may be a shift comprising several hours, a day i.e.24 hours, a month, a quarter, or any other period of time. Any period of time may be selected to identify material moving cycles for the machine and to determine a number of the cycles performed by the machine during that period of time.
- the method 300 comprises applying a first trained machine-learning (ML) model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time.
- ML machine-learning
- the at least one material moving cycle out of a plurality of material moving cycles may be identified in real time, as the mining machine is operating in the mining site. In some examples, the material moving cycle may be identified at a later time, e.g., after the machine has completed the cycle. [0096] It should be noted that the material moving cycle may be identified with some probability of the identification because the cycle is identified as a pattern recognized in the inertial sensor measurements. The accuracy of the inertial sensor measurements and other factors may affect the correctness of the identification of the material moving cycle. [0097] In some examples, the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle.
- the sequence of the operating states performed by the mining machine may comprise (i) loading the mining machine with material at a loading point, (ii) moving the material by the mining machine from the loading point to a dumping point, (iii) unloading the material at the dumping point, and (iv) moving from the dumping point to the loading point or to another loading point.
- an LHD machine may pick a load of the material, e.g. in its bucket or other attachment, at the loading point and move the load to the dumping point where the load is deposited. The mining machine, without a load, may then return to the loading point to pick up another load, or the mining machine may move to a different loading point to pick up the different load.
- the material moving cycle may be considered completed once the machine returns to the same or different loading point in a state ready to pick another load. It should be noted that other sequence of the operating states may be considered to constitute the material moving cycle. [0098] In examples herein, separate operating states are not identified and the material moving cycle is identified and reported as a whole.
- the first ML model may be trained to recognize a pattern in Docket No.: PS56142PC00/ P23087WO01 the inertial sensor measurements that is indicative of a completed material moving cycle.
- the first ML model may be trained using known sub-patterns and time windows in IMU data corresponding to operating states forming the material moving cycle, but the separate operating states may not be detected for the purposes of the techniques described herein.
- the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer.
- the inertial sensor measurements may comprise data acquired in the nine corresponding axes, three per each of the accelerometer, gyroscope, and magnetometer.
- Data or signal from each axis may be pre-processed to generate new variables, e.g., one or more of signal enveloping, low-pass or high-pass filtering to eliminate noise, and moving average filtering to smooth the signal may be applied to the data.
- Feature extraction may be performed using these new variables.
- the trained first ML model may be agnostic to one or more out of a model of the mining machine, a brand of the mining machine, and a manufacturer of the mining machine.
- the first ML model may be trained in dependency on at least one property of the worksite e.g. mining environment, e.g. a layout of the mining environment, distances to be traveled by the mining machine, etc.
- the first ML model may be trained as described in connection with FIG. 4.
- a training pipeline described in more detail in connection with FIG.4, may be executed automatically, whereby the first ML model may be generated and trained, to be suitable for identifying material moving cycles in measurements acquired by the IMU sensor.
- IMU sensor data may be sufficient to identify material moving cycles, though position sensor data may additionally be used, as described below.
- no specific input from a user such as e.g. the operator of the mining machine or another user, may be required to identify at least one material moving cycle.
- the accuracy of the identification or detection of the material moving cycle is improved, which improves the way in which the operation of the mining machine is assessed and controlled.
- the method 300 may optionally comprise determining a correctness of the identified at least one material moving cycle.
- the at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- the process 300 may proceed to block 317.
- the process 300 may proceed to block 320.
- the at least one identified material moving cycle may be removed from further analysis and the process 300 may return to block 302 to receive further inertial sensor measurements acquired by the IMU sensor. It should be noted that more than one material moving cycles is typically identified for further analysis.
- the method 300 comprises determining at least one operation indicator from the identified at least one material moving cycle.
- the at least one material moving cycle may comprise two or more material moving cycles.
- the operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period.
- the operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine.
- the operation indicator may be expressed as a quantitative value, a qualitative value, or a combination thereof.
- Machine utilization times may be useful for establishing proactive maintenance plans for the mining machine and other machines in the mining environment.
- the operation indicators may comprise a total operating time representing a total time during which the mining machine is operational and productive e.g. moves the material such as ore from one or more loading points to one or more dumping points. The total operating time may be determined by summing all periods of time in which the mining machine is active and has been identified as generating value.
- the operation indicators may comprise a use of the mining machine per a certain duration of time.
- a use U of the mining machine per shift may be calculated by dividing the total operating time by a duration of the shift:
- a use of the mining machine during any other period of time may also be determined in a similar manner.
- the operation indicators may comprise an overall operational efficiency E which may be calculated by dividing the total operating time of the mining machine by a total available time indicating a duration of time period during which the mining machine is or was available.
- the operation indicators may be determined per shift, month, quarter, or per any other period of time, whereby a fleet performance over that period of time may be determined.
- the operation indicators may be represented on a user interface of a display, e.g., of the mining machine and/or a fleet control system, in the manner that allows assessing the fleet performance.
- the operation indicators may also include an overall equipment effectiveness (OEE) indicators determined for the mining machine. Real-time monitoring of the machines in the fleet may be performed, which allows e.g.
- the method 300 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the material moving cycle.
- the at least one operation indicator may comprise a plurality of operating indicators.
- initiating the action may comprise prompting a display of a representation of the identified material moving cycle and/or a representation of the operation indicator.
- the representation of the identified material moving cycle may be presented on a display such as e.g. a display of the central control system 22, a display associated with the mining machine and/or with the field computer device, and/or on a display associated with any other device or system.
- information on the material moving cycle and/or the at least one operation indicator may be displayed on user interface 280 rendered on the display communicatively coupled to the central control system 22. In this way, a fleet operator or another person may be enabled to access performance of the mining machine, as well as of other machines operating in the mine, and to determine if any actions need to be taken.
- information on the material moving cycle and/or the at least one operation indicator may be displayed Docket No.: PS56142PC00/ P23087WO01 on a user interface rendered on the display 33 associated with the mining machine 12.
- the driver or another operator of the mining machine 12 may be informed, in real time, about a current status of the mining machine and a location of the mining machine in the mine, a number of material moving cycles that have been performed by the mining machine, a time that it took the machine to perform each of the cycles, and/or any other suitable information.
- the driver may then adjust operation of the mining machine and/or initiate other actions based on the information on the material moving cycle and/or the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
- the instruction to control the mining machine may be obtained by the field computer device 16 and/or the mining machine controller 28, which may perform functionality of the field computer device 16. The instruction may be received from e.g. the central control system 22. In some examples, the instruction may be generated the field computer device 16 and/or the mining machine controller 28. In some examples, the field computer device 16 may generate and/or receive the instruction and to provide this instruction to the mining machine controller 28.
- such instructions may be a control instruction that is used to adjust one or more operating characteristics of the mining machine 12.
- the operating characteristics comprise one or more of a speed of material loading and/or unloading by the mining machine, a speed of movement of the mining machine, a position of the mining machine, specific loading and dumping points at which the mining machine can operate, timing and duration of idling by the mining machine, etc.
- the mining machine may be instructed to stop operation if it is determined that it is not currently capable of performing material moving e.g. due to a malfunction.
- the mining machine may be instructed to move to another location in the mine.
- the mining machine may be instructed to pick up a load of material from an alternative loading point and/or to carry the material to an alternative dumping point.
- the instruction to control the mining machine may be generated in dependence on the at least one operation indicator and additionally in dependence on other data regarding operation of the mining machine and/or other types of data.
- the adjustment of the target requirement for performance of the mining machine may involve adjusting operational parameters for the mining machine such that the machine is controlled to operate in dependence on its previously determined performance. For example, if the machine is Docket No.: PS56142PC00/ P23087WO01 determined to be underperforming, its target performance, e.g.
- a number of material moving cycles completed during a shift, a frequency of use of the machine, a total length of continuous periods of use of the machine, and/or other target performance parameters may be increased or otherwise adjusted.
- the target performance may be adjusted if it is additionally determined that the machine does not experience excessive downtime caused by unplanned failures or by other factors that affect machine performance.
- its target performance e.g. a number of material moving cycles completed during a shift, a frequency of use of the machine, a total length of continuous periods of use of the machine, and/or other target performance parameters, may be decreased or otherwise adjusted.
- Another type of adjustment for the mining machine may include a location of the mining machine in the worksite such as a mine.
- the mining machine with higher performance metrics may be moved to a more critical location in the mine.
- the mining machine with lower performance metrics may be moved to a location in the mine where less work would be required from the mining machine e.g. fewer material moving cycles are expected to be performed during a certain time period.
- one or more reasons of underperformance of the mining machine may be identified when the machine is determined to be underperforming e.g. due to component failures.
- a measure related to machine maintenance may be taken.
- a plan can be implemented to improve conditions in the work area.
- Some examples include power supply or water supply failures, which can extend machine downtime.
- a reliability ranking or another similar comparison measure may be used to rank the mining machines in the worksite based on their performance such as completion of one or more material moving cycles as identified in accordance with embodiments of the present disclosure.
- a mining machine may be considered to have a higher reliability when it has a higher utilization efficiency and a lower downtime than one or more of other mining machines.
- more reliable mining machines such as e.g. LHD machines
- LHD machines may be assigned to more critical production points in the worksite.
- Less reliable mining machines, such as e.g. LHD machines can be assigned to less critical points in the mine, to ensure planned production.
- Docket No.: PS56142PC00/ P23087WO01 [00129]
- FIG.4 illustrates an example of a training pipeline or process 400 for training a ML model e.g. the first ML model, in accordance with embodiments of the present disclosure.
- the training process 400 is performed to obtain an ML model that can be used to detect or predict or identify material moving cycles, also referred to herein as ore hauling cycles, that can be performed by the mining machine.
- the machine-learning model may be selected from one or more candidate machine-learning models.
- the material moving cycles may be identified and defined from patterns detected in signals acquired by one or more IMU sensor units.
- the training process 400 may be performed in advance, and a resulting trained machine-learning model may be provided, for example, as part of a kit provided in accordance with examples of the present disclosure.
- the training may be performed by a central control system e.g. central control system 22, and/or by another external control system.
- the training may be performed by the field computer device 16 and/or the mining machine controller 28.
- the execution of the training pipeline may be part of the method in accordance with examples of the present disclosure as performed by the field computer device 16 and/or the mining machine controller 28.
- the training pipeline may be executed automatically.
- the process 400 comprises obtaining training inertial sensor measurements acquired by one or more IMU sensors.
- the IMU sensors may be coupled to a mining machine e.g. to a movable implement comprising a controllable bucket or another similar tool.
- the inertial sensor measurements may be acquired directly or indirectly, from IMU sensors associated with respective one or more mining machines operating in a mining environment or mining site.
- the inertial sensor measurements which may be referred to as training data, are acquired as the one or more mining machines are operating in the mining environment.
- the one or more mining machines may be any suitable types of mining machines configured to load and upload material and to move with the material from one location to another, e.g., LHD machines.
- each of the mining machines used to acquire training data comprises one i.e. single IMU sensor unit coupled thereto, e.g., on the movable implement of the mining machine.
- the training inertial sensor data may be updated as more sensor measurements are acquired from the IMU sensors coupled to mining machines in the mining environment.
- the one or more inertial sensor measurements used for the training stage may be simulated data, or a combination of simulated data and actual inertial sensor measurements.
- exploratory data analysis may be performed on the acquired inertial sensor measurements.
- the EDA may be performed to ensure that the resulting model is agnostic to a type of the mining machine and a mining environment.
- the EDA stage may include pattern analysis, statistical analysis, frequency analysis, and other types of processing.
- the objective of the exploratory Docket No.: PS56142PC00/ P23087WO01 analysis of the inertial sensor measurements is to find patterns that allow identifying a time window in which the mining machines perform one or more material moving cycles.
- a material moving cycle may comprise sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the process 400 may comprise data preprocessing, which may involve various techniques to prepare the inertial sensor measurements for training. During this phase or stage, different data cleaning methods may be applied, such as e.g. outlier filtering, elimination of null or corrupted values, and signal smoothing using different techniques such as rolling, low-pass filtering or Fourier transform.
- calibration of the IMU sensor unit may be performed.
- Preprocessed IMU sensor data may be generated as a result of preprocessing of the one or more inertial sensor measurements.
- the process 400 may comprise performing feature construction and/or extraction to identity one or more features to be used in the ML model.
- the preprocessed IMU sensor data may be labeled according to observed and/or identified patterns, distinguishing between different material moving cycles.
- the most relevant signals may be selected to form variables, and sliding time windows may be designed or generated with overlapping, adjusted to a duration of the material moving cycles.
- statistical indicators may be calculated using the time windows as a basis, and a class is assigned with a label or name of the material moving cycles based on the previously labeled pattern.
- a binary classification may be used, with windows that correspond to a cycle and with windows that do not correspond to a cycle.
- a multiclass classification may be used such that more than two classes can be assigned to a material moving cycle e.g., a productive cycle, a non-productive cycle, a loaded cycle, an unloaded cycle, etc.
- the calculated characteristics and the class constitute the variables to be used for modeling.
- Non-limiting examples of the statistical indicators may comprise one or more of a mean, a standard deviation, a slope, a median, a maximum, a minimum, polynomial coefficients, kurtosis, skewness, and frequency response.
- Non-limiting examples of features obtained from the frequency response are gain, energy, and cutoff frequency.
- the process 400 may comprise training one or more candidate machine-learning models and selecting a model from the candidate models, which may be performed using any one or more of various approaches.
- An ML model may be Docket No.: PS56142PC00/ P23087WO01 selected from a set of the candidate ML models.
- training data may be fitted to different supervised classification models.
- the machine-learning models comprise decision tree models, which use a class variable as a target.
- One or more ML models may be trained and assessed, and a model that gives a most accurate performance and is less computationally expensive than other models may be selected.
- evaluation metrics such as e.g., one or more out of f1-score, recall, precision, accuracy, area under the curve (AUC), and others, may be used.
- techniques such as e.g. confusion matrices, classification reports, and others related to an importance of the variables may be used.
- an ML model may be selected which may deliver results above 90% for all evaluation metrics used.
- an ML model may be selected which may deliver results above 95% for all evaluation metrics used.
- the ML model may be selected based on other criteria.
- the process 400 may comprise validating the trained ML model.
- the validation may be performed using one or more of various validation techniques. For example, to ensure appropriate performance of the selected ML model, predictions may be made with unlabeled data, and metrics may be generated to evaluate and validate performance of the ML model using the unlabeled data.
- the validation state is used to access whether the model, selected at the training and model selection stage at block 411, is able to perform, i.e.
- a choice of a final ML model is an iterative process, subject to changes in the training data, so that monitoring, readjustment, and/or retraining may be performed in order to ensure that the ML model reflects an updated, most recent representation of reality. Accordingly, as shown schematically in FIG.4, by arrow a1, the process 400 may return, from block 413, to block 411, to continue training the model, which may be performed iteratively.
- the field computer device and/or the mining machine controller may be configured to perform the processing described in connection with FIG.3.
- the field computer device and/or the mining machine controller may be configured to perform, at least in part, the processing described in connection with FIG.4 which Docket No.: PS56142PC00/ P23087WO01 involves training a machine-learning model that is configured to be applied to inertial sensor measurements to predict or identify material moving cycles performed by the mining machine.
- the mining machine in addition to obtaining or receiving inertial sensor measurements acquired by at least one IMU sensor coupled to the mining machine, the mining machine also obtains or receives position sensor measurements acquired from at least one position beacon or tag positioned in the mining site externally to the mining machine. The position tag may be stationary at a predetermined location in the mine.
- FIG.1A shows schematically an example of the position tags or beacons 11a-11c, though it should be appreciated that multiple position beacons may be located in the mine, to e.g. mark locations of loading areas or points, dumping areas or points, and other locations in the mine.
- the position sensor measurements may be used to determine or verify locations of the loading point and the dumping point for one or more material moving cycles identified using the inertial sensor measurements data.
- the mining machine may be moving in a tight environment where multiple loading points and dumping points may be located, and the position sensor measurements may assist in determining with improved precision an origin and destination of the material. Accordingly, for each identified material moving cycle, it may be known where the material came from and where it was moved to.
- FIG.5 shows a computer-implemented process or method 500 for monitoring and controlling operation of a mining machine in a mining site, in accordance with some embodiments of the present disclosure.
- the mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another.
- the method 500 may be performed by a computer device, e.g.
- the method 500 comprises, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine The inertial sensor measurements may be acquired over a first period of time.
- the method 500 comprises, as the mining machine is operating in the mining site, receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site.
- the position sensor measurements may be acquired as time series data that may be stored in memory e.g. of the field computer device 16 and/or the mining machine controller 28.
- the position beacon or tag may be positioned at a known location in the mine.
- the computer device may receive the position sensor measurements from at least one sensor that is configured to communicate with the at least one, typically multiple position tags.
- the at least one position sensor e.g.
- the position sensor 17 shown in FIG.2 may be installed in or otherwise associated with the field computer device and/or mining machine controller 28.
- the position sensor measurements may be acquired as the mining machine is moved through the mine and as the mining machines receiving the inertial sensor measurements.
- the processing at block 503 may be performed simultaneously or substantially simultaneously with the processing at block 502.
- the method 500 comprises preprocessing of the inertial sensor measurements.
- the preprocessing may use known techniques for preparation raw sensor measurements data to further analysis, including smoothing and other techniques.
- the preprocessing at block 505 matches the preprocessing performed on the data in the model training pipeline.
- the method 500 comprises preprocessing of the position sensor measurements or data.
- the position sensor data may be time series data that may be preprocessed using one or more of a moving average filter, a median filter, and Kalman filter, to smooth the signals in the data and eliminate noise.
- the preprocessing of the position sensor measurements matches the preprocessing performed on the data in the training pipeline of the second model.
- the position sensor data may be preprocessed in accordance with characteristics of material moving cycles identified using embodiments of the present disclosure.
- a material moving cycle may have a respective duration, different from duration of other cycles, and may depend on factors such as e.g. machine operator behavior, zones in the mine, distances between loading and/or dumping points in the mine, a speed of the mining machine, etc.
- the time series of the position sensor data may be preprocessed by one or more scaling and dimensionality reduction methods so that the time series corresponding to a certain material moving cycle, i.e. acquired during the performance of that cycle by the mining machine, have the same size.
- the preprocessing of the position sensor measurements may be performed after applying the first trained ML model to the inertial sensor measurements to identify at least one material moving cycle as shown in connection with block 512.
- the method 500 comprises performing feature extraction from the preprocessed inertial sensor measurements and position sensor measurements. The same features may be extracted as those used to train the model.
- the preprocessed data may be accumulated in real time, e.g., in the memory of the field computer device or in another memory device, until a window size necessary to calculate the characteristics is met, and the data is then used to obtain the inference of the model.
- the feature extraction may involve applying a dimensionality reduction technique to extract meaningful features from the preprocessed inertial sensor measurements and position sensor measurements.
- the features which may be extracted from the preprocessed inertial sensor measurements are suitable for application to these features of the first ML model, to identify at least one material moving cycle.
- the features which may be extracted from the preprocessed position sensor measurements are suitable for application to these features of the second ML model, to verify the loading point and the dumping point for the identified at least one material moving cycle.
- the method 500 comprises applying the first trained ML model to the inertial sensor measurements to identify at least one material moving cycle.
- the processing at block 512 may be performed similar to the processing at block 312 of FIG.3.
- the method 500 comprises applying a second trained ML model to the position sensor measurements to verify the loading point and the dumping point for the identified material moving cycle.
- the second ML model may be trained similarly to the first ML model, e.g. as described in connection with FIG.4, but the training data would comprise position sensor measurements, actual and/or simulated, received from position beacons or tag positioned in the mine or in a simulator environment.
- the position sensor measurements or tag signals acquired during the same time period referred to herein as the first time period, may be processed e.g.
- the position sensor measurements or tag signals may be processed before the material moving cycle is identified.
- the tag signals may be processed and the processed data e.g. features may be input to the trained second ML model.
- the second ML model may provide an output such as a value indicating Docket No.: PS56142PC00/ P23087WO01 a probability that a given signal comes from a certain loading point. More than one loading point may be identified, each assigned a corresponding probability. In some examples, a loading point identified with a highest probability may be selected as an actual loading point from which the material was taken during the identified material moving cycle.
- the dumping point may be identified and/or verified, for the material moving cycle, in a similar manner in which the loading point may be identified.
- the loading point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a loading time window such as time window during which the mining machine is expected, from the training data, to perform loading of the material.
- the dumping point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a dumping time window such as time window during which the mining machine is expected, from the training data, to perform dumping or unloading of the material.
- the second ML model may be trained to select the loading and unloading points using time series data acquired of each tag signal.
- Previously acquired time series data may be labeled to indicate a correct source signal for a plurality of material moving cycles, and the second ML model is trained to differentiate and classify the time series data, to provide an output indicating a probability that position sensor data acquired from the positions beacons is indicative of actual material origin or loading points and/or of material unloading or dumping points.
- the method 500 comprises determining at least one operation indicator from the identified at least one material moving cycle.
- the at least one material moving cycle may comprise two or more material moving cycles.
- the operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period.
- the operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine.
- the processing at block 520 may be performed similarly to the processing at block 320 of FIG.3 and the description at block 320 is applicable herein. [00158]
- the method 500 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the at least one material moving cycle.
- initiating the action may comprise prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
- the Docket No.: PS56142PC00/ P23087WO01 processing at block 522 may be performed similarly to the processing at block 322 of FIG.3 and the description at block 322 is applicable herein.
- FIG.6 illustrates an example of a computer-implemented process or method 600 for monitoring and controlling operation of a mining machine in a mining site, in accordance with some embodiments of the present disclosure.
- the mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another.
- the method 600 may be performed by a computer device, e.g. the field computer device 16 and/or the mining machine controller 28, or by another suitable computer device which may be positioned in the mining machine or may be otherwise associated with the mining machine.
- the actions at blocks of FIG.6 do not have to be taken in the order stated below, but may be taken in any suitable order. Processing at some acts or blocks of FIG.6 is similar to processing at corresponding acts or blocks shown in FIGs.3 and 5. Also, blocks in FIG.6 have numerical references that are similar to numerical references of corresponding blocks of FIGs.3 and 5 at which similar processing is performed. The description of processing steps described in connection with FIGs.3 and 5 is not repeated in connection with FIG.6, but it should be appreciated that the description applies to the method 600 of FIG.6. [00161] At block 602, the method 600 may comprise, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine.
- the method 600 may also comprise, similar to processing at block 503 of FIG. 5, as the mining machine is operating in the mining site, receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site.
- the method 600 may comprise applying a first trained ML model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time.
- the method 600 may comprise determining a correctness of the identified at least one material moving cycle. The at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- the method 600 may determine whether the correctness of the identification of the at least one material moving cycle has been confirmed or verified. Responsive to Docket No.: PS56142PC00/ P23087WO01 determining that the correctness of the identification of the at least one material moving cycle has not been verified, the process 600 may proceed to block 617. Responsive to determining that the correctness of the identification of the at least one material moving cycle has been verified, the process 600 may proceed to block 618.
- the at least one identified material moving cycle may be removed from further analysis and the process 600 may return to block 602 to receive further inertial sensor measurements acquired by the IMU sensor. It should be noted that more than one material moving cycles is typically identified for further analysis.
- the method 600 may determine whether a required amount of position sensor measurements or data have been received. The required amount may be defined as an amount of data that is sufficient to verity locations of a loading point and/or a dumping point for the material moving cycle that involves transfer of material from the loading point to the dumping point.
- the position sensor measurements may not be available, e.g., due to absence or malfunction of a position sensor configured to communicate with one or more position beacons or tags that are external to the mining machine.
- the position sensor may be e.g. position sensor 17 shown in FIG.2, which may be part of or associated with the field computer device 16 and/or the machine controller 28. Such position sensor may be absent from some mining machines.
- one or more one or more position beacons or tags may be absent or not working properly.
- Various other factors may contribute to the lack of required amount of position sensor data, such as issues with transmission of communication signals between the position sensor on the mining machine and the one or more position tags, and others.
- the method 600 may perform preprocessing of the position sensor measurements, similar to the preprocessing of the position sensor measurements at block 507 of FIG. 5. [00169] At block 614, the method 600 comprises applying a second trained ML model to the position sensor measurements to verify the loading point and the dumping point for the identified material moving cycle. [00170] Referring back to decision block 618, regardless of the specific one or more reasons for the lack of the required amount of position sensor data, responsive to determining that the required amount of the position sensor measurements is not received, the process 609 proceeds to block 620 to determine at least one operation indicator from the identified at least one material moving cycle.
- a loading point and/or dumping point have not been verified for the material moving site.
- Other Docket No.: PS56142PC00/ P23087WO01 techniques may be used to verify the loading point and/or dumping point, without the use of position sensor measurements.
- cycles can be manually assigned to a load point by a fleet operator or mine coordinator, or can be manually selected by the operator e.g. on a screen of a computing device.
- the process 600 also follows to block 620 responsive to determining that the required amount of the position sensor measurements is received, in which case the loading point and/or the dumping point have been verified for the material moving site.
- the method 600 comprises determining at least one operation indicator from the identified at least one material moving cycle.
- the at least one material moving cycle may comprise two or more material moving cycles.
- the operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period.
- the operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine.
- the processing at block 620 may be performed similarly to the processing at block 320 of FIG.3 and the description at block 620 is applicable herein.
- the method 600 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the at least one material moving cycle.
- initiating the action may comprise prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
- the processing at block 622 may be performed similarly to the processing at block 322 of FIG.3 and the description at block 322 is applicable herein.
- FIG.7 illustrates an example of a method 700 that may be performed by a control system such as e.g. central control system 22 shown in FIGs.1A and 2.
- the central control system 22 which may be a traffic control system, comprises processing circuitry 24 e.g. at least one processor 24 that is configured to coordinate movements of one or more mining machines out of a plurality of mining machines in a mining site.
- the one or more of the mining machines e.g. a machine 12 may comprise a corresponding mining machine controller 28 and a field computer device 16 which may be part of the mining machine controller 28 or a separate hardware device.
- the field computer device 16 or the mining machine controller 28 may be configured to perform the method in accordance with examples of Docket No.: PS56142PC00/ P23087WO01 the present disclosure.
- the field computer device 16 and/or the mining machine controller 28 is configured to communicate with the central control system 22 via a communication interface, e.g. to send various data to the central control system 22 and to receive control instructions and information from the central control system 22.
- the central control system may receive, from the mining machine, information on the identified at least one material moving cycle and/or on the at least one operation indicator determined based on the identified at least one material moving cycle.
- the central control system may generate at least one command or instruction to the mining machine in dependence on the received representation.
- the instruction may be to control the mining machine in dependence on the at least one operation indicator.
- the instruction may instruct the mining machine to adjusting a target requirement for performance of the mining machine.
- the control system may send the at least one instruction to the mining machine.
- the generating and sending of the instruction may be performed in the same processing step or block.
- FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a user interface of a computer device, in accordance with examples of the present disclosure.
- the user interface may be rendered on a display associated with the mining machine e.g. the field computer device and/or the mining machine controller, and/or on a display associated with the central control device, e.g., a traffic controller device or system.
- FIGs.8, 9, 10, and 11 may be generated and presented as a result of identifying of one or more material moving cycle and determining at least one operator indicator based on the identified material moving cycle(s). The information may be displayed as part of initiating an action in dependence on determining of the at least one operation indicator in accordance with any methods of embodiments of the present disclosure.
- a bucket of the material is moved during each material moving cycle.
- FIG.8 illustrates, for a day and for shifts A, B, and C of the day, a number of buckets with the material that have been moved during a particular shift.
- a bucket refers to a completed material moving cycle.
- FIG.9 illustrates, for a shift A in this example, a total productivity in a number of buckets, per hours.
- FIG.9 also shows expected production as a line 901 and cumulative production as a dotted line 903.
- FIG.10 illustrates, for a shift A, production per an operator of a corresponding mining machine in a mine, expressed as a number of buckets moved per a specific operator. Docket No.: PS56142PC00/ P23087WO01
- FIG.11 illustrates details on performance, by time, of a mining machine, as various parameters and information.
- FIG.11 shows a date, equipment, operator, extraction point, dumping or dump point, cycle time, and detail.
- the information that can be displayed on the user interface of the computer device may be used to assess performance of the mining machine, determine further actions to be taken, control the mining machine, adjust various features of the mine, and perform any other suitable actions. It should be appreciated that various other information may be displayed as examples of FIGs.8-11 are shown for illustration purposes only.
- a kit for installation on a mining machine is provided, the mining machine being configured to operate in a mining site.
- the kit e.g.
- kit 18 shown in FIG.1A may comprise at least one IMU sensor configured to be associated with mining machine, and computer program product comprising computer-executable instructions configured to, when executed by at least one processor, to perform any of the methods in accordance with embodiments of the present disclosure.
- the at least one processor may be a processor of a field computer device, a processor of a mining machine controller, or any other suitable processor.
- the kit may be installed or deployed on the mining machine such that the at least one IMU sensor may be coupled to the mining machine e.g. on a movable implement of the mining machine and/or in another location where the IMU sensor may record inertial sensor measurements indicative of the mining machine performing a material moving cycle.
- the at least one IMU sensor comprises one i.e. single IMU sensor unit.
- Each IMU sensor unit may comprise an accelerometer, a gyroscope, and a magnetometer.
- the kit may comprise one or more IMU sensor unit each comprising a three-axis accelerometer, a three-axis gyroscope, and a three- axis magnetometer.
- the computer program product of the kit may be installed on the field computer device and/or the mining machine controller.
- the computer-executable instructions may be stored in memory of a field computer device e.g. field computer device 16 that may be provided as part of the kit.
- the computer-executable instructions may be stored in memory of a controller e.g. mining machine controller 28.
- the computer-executable instructions may be implemented as an application or app that can be installed in the memory of the mining machine controller 28.
- the computer program product of the kit may be implemented as the field computer device such that the field computer device may be installed and deployed on the mining machine in addition to existing one or more controllers of the mining machine.
- the computer-executable instructions when executed by at least one processor, may cause the at least one processor to, as the mining machine is operating in the mining site, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator.
- the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the computer-executable instructions when executed by at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
- the second trained machine-learning model may be trained in dependency on one or more features selected from time series data previously acquired by measuring signals from one or more position beacons.
- the one or more position beacons may comprise the at least one position beacon.
- the computer-executable instructions when executed by at least one processor, further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- initiating the action in dependence on the determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
- initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
- a mining machine for operation in a mining site comprises a movable implement comprising a tool configured to load and unload material; at least one IMU sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site; and at least one processor and a memory comprising computer-executable instructions.
- the computer-executable instructions when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive the inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator.
- the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point.
- the computer-executable instructions when executed by the at least one processor, further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle.
- the computer-executable instructions when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site, and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
- initiating the action in dependence on the determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. Docket No.: PS56142PC00/ P23087WO01 [00200] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. [00201] Operational steps described in any of the exemplary aspects herein are described to provide examples and discussion.
- the steps may be performed by hardware components, may be embodied in machine-executable instructions to cause a processor to perform the steps, or may be performed by a combination of hardware and software. Although a specific order of method steps may be shown or described, the order of the steps may differ. In addition, two or more steps may be performed concurrently or with partial concurrence. [00202]
- the terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
- Relative terms such as “below” or “above” or “upper” or “lower” may be used herein to describe a relationship of one element to another element as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present.
Landscapes
- Engineering & Computer Science (AREA)
- Mining & Mineral Resources (AREA)
- Mechanical Engineering (AREA)
- Civil Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Structural Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Paleontology (AREA)
- Geochemistry & Mineralogy (AREA)
- Geology (AREA)
- Operation Control Of Excavators (AREA)
Abstract
A method for monitoring and controlling operation of a mining machine (12) in a mining site, the mining machine, a field computer device (16), and a kit for installation on the mining machine are provided. As the mining machine is operating in the mining site, inertial sensor measurements are received over a first period of time by at least one inertial measurement unit (IMU) sensor associated with the mining machine. A first trained machine-learning model is applied to the inertial sensor measurements to identify at least one material moving cycle performed by the machine within the first period of time. At least one operation indicator is determined from the identified material moving cycle, and an action may be initiated in dependence on the operation indicator. Position sensor measurements may be acquired from position tags and used to verify a loading and/or dumping point for the material moving cycle.
Description
Docket No.: PS56142PC00/ P23087WO01 TITLE FIELD COMPUTER DEVICE, MINING MACHINE, KIT, AND METHODS FOR MONITORING AND CONTROLLING MINING MACHINES IN A MINING ENVIRONMENT TECHNICAL FIELD [0001] The disclosure relates to systems and methods for monitoring and controlling operation of a mining machine from a plurality of mining machine in a mining site. The disclosure further relates to a field computer device configured to be installed on the mining machine, a mining machine, a kit for the mining machine, a computer program product, and a computer-readable storage medium for monitoring and controlling operation of a mining machine. BACKGROUND [0002] In mining and tunnelling, developments are constantly underway to improve efficiency, productivity, and safety. One of the leading areas in which changes/improvements are increasingly taking place is automation, full or partial, of various processes occurring in mining/tunneling. [0003] Mining machines, e.g., trucks, for underground mining and tunneling can perform various tasks in environments that are dark and often inaccessible by foot and may generally be not comfortable for human drivers. Thus, it is often desirable that mining machines that operate in an underground environment can be driven in a fully autonomous mode, i.e., without an onboard operator being required to control the machines during machine operation. [0004] An example of mining machine where automated operation is typically considered to be beneficial are so-called load-haul-dump (LHD) machines. The LDH machines may be used to remove and transport broken rock/ore from a certain location, e.g., a position where blasting has been performed, to a particular place where the broken rock is dumped. After dumping their load, at the place that may be referred to as a dump point or location, the LHD machines typically return to an initial (start) location to pick up a new load. Thus, these machines often travel the same route over and over again, which makes the travel between load and dump locations well suited for automation. There are also various other situations where automation may prove beneficial. [0005] The mining machine may be operating in one out of different possible operating states, and a current state is typically recorded by a machine operator. This may however be a tiring and cumbersome process for a human. Also, such manual recording of the process may be error-prone, whereas accurate and timely recording of operating states of the mining machine may affect the productivity of the entire mining operation.
Docket No.: PS56142PC00/ P23087WO01 SUMMARY [0006] In an aspect, a method for monitoring and controlling operation of a mining machine in a mining site is provided. The method comprises, with at least one processor, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; applying a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determining at least one operation indicator from the identified at least one material moving cycle; and initiating an action in dependence on determining of the at least one operation indicator. [0007] The technical benefits and advantages comprise automatic prediction or detection of material moving cycles, which allows improved control over productivity of individual mining machines and of the entire mine. In this way, overall efficiency, safety and productivity of mining operations may be increased. Furthermore, the determining of the material moving cycles and operation indicators may be performed in real time, which allows implementation of timely control and intervention in fleet management systems. For a fleet of mining machines, decisions may be made on dynamic operational assignments. As a further advantage, the provided approach is equipment agnostic, such that it may be integrated with various mining machinery and equipment without requiring extensive modifications or specialized hardware. This flexibility not only simplifies implementation, but also allows for wider adoption in different mining configurations. [0008] Additionally, the provided system, mining machine, kit, and methods therein are developed and configured to operate with low computational resource requirements. This feature allows for the provided technique to be implemented in onboard controllers, e.g. in a field controller or computer device, which typically have limited processing capabilities. By using resources efficiently, the provided technique may ensure optimal performance even in challenging computing environments. Furthermore, various events or actions may be triggered e.g. initiated in response to predicting the ore moving cycles and determining productivity indicators such as e.g., predictive maintenance of mining machines, optimizing routes traveled by the machines, defining and adjusting assignments for the mining machines, and others. [0009] In some examples, the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by
Docket No.: PS56142PC00/ P23087WO01 the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [0010] In some examples, the method may further comprise determining a correctness of the identified at least one material moving cycle. The at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. [0011] In some examples, the method may further comprise receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site. [0012] In some examples, the method may further comprise applying a second trained machine- learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. [0013] In some examples, the second trained machine-learning model is trained in dependency on one or more features selected from time series data previously acquired by measuring signals from one or more position beacons. The one or more position beacons may comprise the at least one position beacon. In some examples, the signals from one or more position beacons may comprise actual data and/or simulated data. [0014] In some examples, the first trained machine-learning model is trained using inertial sensor measurements previously acquired by one or more IMU sensors. The one or more IMU sensors may include the at least one IMU sensor and/or by one or more other sensors. In some examples, training inertial sensor measurements may be simulated data or a combination of actual data acquired in a real- world environment and the simulated data. [0015] In some examples, initiating the action in dependence on determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. [0016] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; and adjusting a target requirement for performance of the mining machine. [0017] In some aspects, a computer device is provided that comprises at least one processor configured to perform the method in accordance with any embodiments of the present disclosure.
Docket No.: PS56142PC00/ P23087WO01 [0018] Advantages and effects of the computer device are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer device are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa. [0019] In some aspects, a computer program product is provided that comprises computer- executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with any embodiments of the present disclosure. In some examples, the computer program product may be provided as part of a kit. [0020] Advantages and effects of the computer program product are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer program are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa. [0021] In some aspects, a tangible computer-readable storage medium is provided that has stored thereon a computer program product comprising computer-executable instructions. The computer- executable instructions, when executed by at least one processor, cause the at least one processor to perform the method in accordance with any embodiments of the present disclosure. In some examples, the tangible computer-readable storage medium may be provided as part of a kit. [0022] Advantages and effects of the tangible computer-readable storage medium are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the tangible computer-readable storage medium are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa. [0023] In some aspects, a mining machine for operation in a mining site is provided. The mining machine comprises a movable implement comprising a tool configured to load and unload material; at least one IMU sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site; and at least one processor and a memory comprising computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive the inertial sensor measurements acquired by the at least one IMU sensor, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the
Docket No.: PS56142PC00/ P23087WO01 identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. [0024] Advantages and effects of the mining machine are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the mining machine are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa. [0025] In some examples, the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [0026] In some examples, the computer-executable instructions, when executed by the at least one processor, may further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. [0027] In some examples, the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site, and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. [0028] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises prompting, by the at least one processor, a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. [0029] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. [0030] In some aspects, a kit for installation on a mining machine is provided. The mining machine is configured to operate in a mining site. The kit comprises at least one IMU sensor configured to be associated with mining machine and computer program product comprising computer-executable
Docket No.: PS56142PC00/ P23087WO01 instructions configured to be installed on the mining machine. The computer-executable instructions, when executed by at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. [0031] In some examples, the at least one IMU sensor included in the kit comprises one e.g. single IMU sensor. [0032] In some examples of the kit, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. In such examples, the inertial sensor measurements may comprise nine-axis measurements. [0033] Advantages and effects of the kit are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the kit are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa. [0034] In some examples, wherein the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [0035] In some examples, the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. [0036] Additional features and advantages are disclosed in the following description, claims, and drawings. Furthermore, additional advantages will be readily apparent from the present disclosure to those skilled in the art or recognized by practicing the disclosure as described herein. There are also disclosed herein control units, computer program products, and computer-readable media associated with the above discussed technical effects and corresponding advantages.
Docket No.: PS56142PC00/ P23087WO01 BRIEF DESCRIPTION OF THE DRAWINGS [0037] With reference to the appended drawings, below follows a more detailed description of aspects of the disclosure cited as examples. [0038] FIG.1A illustrates an example of a mining environment in which a method in accordance with examples of the present disclosure may be implemented. [0039] FIG.1B illustrates an example of a mining machine. [0040] FIG.2 is a block diagram illustrating an example of a system comprising a mining machine and a central control system, in which a method in accordance with examples of the present disclosure may be implemented. [0041] FIG.3 is a flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure. [0042] FIG.4 is a flowchart illustrating a method for generating and training a machine-learning model, in accordance with examples of the present disclosure. [0043] FIG.5 is another flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure. [0044] FIG.6 is another flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure. [0045] FIG.7 is a flowchart illustrating a method performed by a central control system, in accordance with examples of the present disclosure. [0046] FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a user interface of a computer device, in accordance with examples of the present disclosure. DETAILED DESCRIPTION [0047] Aspects of the present disclosure relate to a method performed by a field computer device for monitoring and controlling operation of a mining machine in a mining site, to the field computer device, a kit comprising at least one inertial measurement unit (IMU) sensor, and the mining machine comprising the field computer device. [0048] The techniques described herein allow using inertial sensor measurements acquired by at least one inertial measurement unit (IMU) to identify one or more operating cycles for a mining machine that is operating in the mining site. A trained machine-learning model may be used to identify the one or more operating cycles. The cycle may comprise a sequence of operating states comprising loading the mining machine at a loading point, moving of the loaded mining machine from the loading point to a dumping point, unloading the mining machine at the dumping point, and moving the unloaded mining
Docket No.: PS56142PC00/ P23087WO01 machine from the dumping point to the loading point. However, in examples in accordance with the present disclosure, an ore hauling cycle is detected as a whole, without separately identifying a sequence of operating states forming the cycle. The machine-learning model may be trained to detect the entire cycle rather than to detect separate operating states within an ore hauling cycle. [0049] Furthermore, in examples herein, in addition to the inertial sensor measurements, the field computer device is configured to also acquire position sensor measurements also referred to herein as tag sensor measurements, e.g., from at least one position beacon or tag. The position beacon or tag may be positioned at a known location in the mine. The field computer device may acquire the position sensor measurements from at least one sensor that is configured to communicate with the at least one, typically multiple position tags. The at least one sensor may be installed in or otherwise associated with the field computer device, or it may be positioned in another location on the mining machine. The field computer device installed on the mining machine may be configured to acquire the position sensor measurements as the mining machine is moved through the mine, and as the inertial sensor measurements are acquired by the field computer device. [0050] In some examples, the position or tag sensor measurements may be pre-processed and stored e.g. in a memory of the field computer device until an ore hauling cycle is detected. Once the cycle is identified and validated using the inertial sensor measurements, the position sensor measurements acquired during a time period when the inertial sensor measurements are acquired, are processed. The processing may be performed using a trained machine-learning model, referred to herein as a second machine-learning model. The second machine-learning model may be applied to the pre-processed position sensor measurements to identify and/or verify an origin or loading point from which the material is taken and a destination or dumping point to which the material is taken during the identification of the material hauling cycle. [0051] The second machine-learning model may provide an output such as a value indicating a probability that a given signal comes from a certain loading point. More than one loading point may be identified, each assigned a corresponding probability. In some examples, a loading point associated with a highest probability may be selected as an actual loading point from which the ore was taken during the identified ore hauling cycle. In some examples, the dumping point may be identified and/or verified, for the material hauling cycle, in a similar manner. In some examples, the dumping point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a dumping time window. [0052] In aspects, the computer-implemented method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure comprises, by a processor, as the mining machine is operating in the mining site, receiving inertial sensor
Docket No.: PS56142PC00/ P23087WO01 measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; applying a first trained machine- learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determining at least one operation indicator from the identified at least one material moving cycle; and initiating an action in dependence on determining of the at least one operation indicator. [0053] FIG.1A depicts an example of a mining environment or site 10 comprising multiple, four in this example, mining vehicles or machines 12a, 12b, 12c, and 12d. The mining machines may be of the same or different types. The mining machines may be underground mining machines or surface mining machines. One or more of the mining vehicles 12a-12d may be load-haul-dump (LHD) machines or other types of machines configured to carry material from one location to another. For example, the mining machine may be an excavator, a backhoe, or another type of a mining machine with a movable implement. In some examples, the mining machines 12a-12d may be autonomous machines, such as e.g., fully or partially autonomous machines. In some examples, the mining machines 12a-12d may be operating in a fleet of mining machines. The mining machines in the fleet may be assigned work assignments or tasks. [0054] The mining environment or site 10 may be a surface mining site or an underground mining site. The mining site 10 may be any type of a mine-like environment. In some examples, the mining site 10 may be a subway mine. Although not shown in FIG.1A, The mining site 10 may have a specific configuration. For example, if the mining site 10 is an underground mining site, it may include tunnels. [0055] A mining machine in the mining site 10 may have a moving or movable implement, such as, e.g., an arm having a bucket attached thereto. As shown FIG.1A, each of the mining machines 12a-12d may have a respective IMU sensor 14, also referred to herein as an inertial sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site. An IMU sensor or sensor unit is a sensor that provides motion data in a time-series format. The IMU sensor comprises an accelerometer configured to acquire acceleration measurements, a gyroscope configured to acquire angular velocity measurements, and a magnetometer configured to measure a magnitude and direction of the magnetic field at a location of the magnetometer. Thus, combined measurements acquired by the IMU sensor can be used to determine a position, velocity, acceleration, and orientation of an object in a three-dimensional space to which object the IMU sensor is attached. The measurements acquired by each of the accelerometer, gyroscope, and magnetometer are represented along a three-axis coordinate system. Thus, the IMU sensor may be configured to provide a nine-dimensional time series data.
Docket No.: PS56142PC00/ P23087WO01 [0056] The mining machines 12a-12d comprise respective IMU sensors 14a, 14b, 14c, and 14d, coupled to a corresponding movable implement 15, such as movable implements 15a, 15b, 15c, and 15d of the mining machine. In some examples, each of the inertial sensors 14a, 14b, 14c, and 14d may be a single i.e. one IMU sensor. Thus, in some examples, inertial sensor measurements acquired by a single IMU sensor or sensor unit may be sufficient to perform the method in accordance with examples of the present disclosure, which makes the provided approach less costly and requiring less maintenance. The single IMU sensor unit comprises an accelerometer, a gyroscope, and a magnetometer. [0057] The movable implement may comprise a tool configured to load and unload material, which may be any suitable load picking and carrying implement or tool, e.g., a bucket or another work tool for digging and/or load picking and carrying. The movable implement may comprise an arm such as e.g. a hydraulic arm which may comprise the tool configured to load and unload material e.g. a bucket as in an LHD machine. In some examples, the IMU sensor unit may be coupled to the arm, e.g, to a joint of the arm. In some examples, the inertial sensor may be positioned in proximity to the joint of the arm and in proximity to a tool for digging and/or load picking and carrying, e.g., a bucket or another work tool of the movable implement. In some examples, the IMU sensor unit may be positioned on the bucket, but such that the use of the bucket does not interfere with the accuracy of measurements acquired by the IMU sensor. In any case, the IMU sensor unit may be positioned to detect and report acceleration, orientation, angular rates, and other gravitational forces as the mining machine and/or its parts move and vibrate. [0058] At least one IMU sensor is positioned on the mining machine where the acquired sensor measurements are informative of a current state of the movable implement and of the entire machine. Also, an ease of installation of the inertial sensor is taken into consideration when selecting a specific location at which to associate the inertial sensor with the mining machine. The specific position of the inertial sensor on the mining machine may depend on a configuration of the mining machine and the machine’s movable implement, type of work performed by the mining machine, work environment conditions to which the mining machine is subjected, the feasibility of installation of the inertial sensor and/or on other factors. In any case, in examples herein, the inertial sensor is positioned outside of an operator compartment or cabin of the mining machine. The IMU sensor cannot be positioned in the operator compartment or cabin because information about the movement of the arm and/or bucket would be lost in such case. [0059] It should be appreciated that the mining machine may comprise various other sensors. [0060] As further shown in FIG.1A, each of the machines 12a-12d may comprise or may be associated with a respective control unit or a field computer device 16a, 16b, 16c, 16d that may be
Docket No.: PS56142PC00/ P23087WO01 configured to perform the method in accordance with examples of the present disclosure. Each of the field computer devices 16a-16d includes various components not shown in this example, such as a memory device configured to store computer-executable instructions and processing circuitry e.g. one or more processors. The processors are configured to execute the computer-executable instructions to cause the processing circuitry to perform the method in accordance with examples of the present disclosure, as discussed in more detail below. Each of the field computer devices 16a-16d also has an input and output interface that is configured to communicate with other components, e.g., with the respective IMU sensor, as well as with a fleet controller or central control system 22 and with other external systems. [0061] Furthermore, in some examples, one or more of the field computer devices 16a-16d may comprise or may otherwise be associated with one or more position sensors configured to communicate with at least one position beacon positioned in the mining site. For example, as shown in FIG.1A in connection with the first mining vehicle 12a, the mining site 10 may comprise position beacons 11a, 11b, and 11c. Although the three position beacons are shown, multiple position beacons may be present in the mining site. Position beacons may be located in proximity to a material origin or loading point and in proximity to a material destination or dumping point, to assist in determining with more precision the origin of the material and a point of dumping the material. Position beacons may be useful in determining a location of the mining machine in the mine, as the machine is moving e.g. in a tunnel. The mining environment may in some cases be deep underground and tracking locations of mining machines operating in such environments may be a challenge, let alone automatically determining production of the machine. [0062] The position beacon may be an active position tag or a passive position tag. In some examples, the position beacon be configured as a transmitter, in some cases as a transceiver, and the position of the position beacon may be known. The position beacons may be installed in any suitable locations in the mining site. In some examples, as the mining machine is moving and/or handles material in the mining site, position sensor measurements may be acquired based on detection of the position beacons. For example, the one or more position sensors associated with the field computer device of the mining machine may detect signals, e.g. radio frequency (RF) or other types of signals, emitted by the position beacons. In some implementations, the position beacon may emit signals in response to corresponding signals from the position sensors associated with the mining machine. Furthermore, in some cases, the position sensors may be passive tag with a known position, and the one or more position sensors associated with the mining machine may read the position information carried by the passive position tag. Regardless of a specific implementation of position beacons in the mining site, the mining machine, e.g. a field computer device associated with the mining machine, is
Docket No.: PS56142PC00/ P23087WO01 configured to receive and process position sensor measurements to verify the loading point and the dumping point for at least one identified material moving cycle. [0063] Multiple mining machines 12a-12d may be operated in the mining site 10 for hauling material such as ore from draw points or locations to one or more dumping points where the ore is deposited. Thus, a mining machine may be operated in an operating or operational material moving cycle that may comprise a sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving or translating back from the dumping point to the loading point, without a load. The mining machine may also be in a standby mode or standby, when it is idling such that it is operating but not currently moving or performing any work. The material moving cycle may include another sequence of operating states. The material moving cycle may be repeated multiple times during, for example, a shift of the operation of the mining machine. The mining machine may perform a certain number of cycles during the shift. For example, in examples in which the mining machine is autonomous, the shift may be performed until, e.g., the mining machine runs out of fuel or power or until a planned production for a certain zone is met. [0064] In some examples, a kit 18 may be installed and deployed on a mining machine out of the plurality of mining machines 12a-12d in the mining site 10. The kit may comprise at least one inertial sensor and computer program product comprising computer-executable instructions which, when executed by processing circuitry e.g. at least one processor, cause the processor to perform a method in accordance with examples of the present disclosure. The computer-executable instructions may be stored in a memory e.g. in a memory of the field computer device that may be included as part of the kit. In some examples, the computer-executable instructions may be installed on the mining machine without the use of a separate field computer device, e.g., on a controller unit of the mining machine. [0065] FIG. 1A illustrates very schematically, for the mining machines 12a and 12b, that the machines may comprise respective kits 18a and 18b, shown by a dot-dashed line. The kit 18a may comprise the inertial sensor 14a and computer program product that may be stored in the field computer device 16a. Similarly, the kit 18b may comprise the inertial sensor 14b and computer program product that may be stored in the field computer device 16b. [0066] As shown in FIG.1A, operation of mining machines in the mining site 10, shown by way of example as the mining machines 12a-12d, may be controlled by the central control system 22 such as e.g. a fleet controller or fleet control system. The central control system 22, which is communicatively coupled with one or more of the mining machines 12a-12d, may track locations or positions of each of the mining machines 12a-12d, may receive data from the mining machines, and may send commands or instructions to the mining machines. The fleet control system may also be referred to as a traffic
Docket No.: PS56142PC00/ P23087WO01 control system, and it may be any type of a control system that monitors multiple machines e.g. receives information from the mining machines, and sends commands or instructions to the mining machines. The central control system 22 comprises processing circuitry 24 that is configured to execute computer-executable instructions which thereby cause the processing circuitry 24 to perform a method for controlling operation of one or more mining machines, in accordance with examples of the present disclosure. The central control system 22 may have various other components not shown herein for simplicity, e.g., memory, user interface, communication interface, etc. [0067] FIG.1B illustrates an example of a mining machine 32, such as e.g., any of the mining machines 12a-12d shown in FIG.1A, or another type of a mining machine. FIG.1B illustrates schematically example locations or positions 34p1 and 34p2 of where an inertial sensor may be coupled to a movable implement 25 of the mining machine 32. The inertial sensor may be configured to acquire accelerometer, magnetometer, and gyroscope measurements. The measurements may be acquired at a desired frequency and may be used to infer when the mining machine lifts the movable implement 25 such as a hydraulic arm with a bucket or another tool attached thereto. The inertial sensor may be installed on the hydraulic arm of the movable implement 25 in proximity to the bucket, as shown in FIG.1B. In the example illustrated in FIG.1B, the example positions 34p1 and 34p2 may be evaluated for ease of installation and data quality of obtained sensor measurements. It should be appreciated that the mining machine 32 is shown in FIG.1B as a non-limiting example only, as the inertial sensor may be associated with a mining machine in any suitable location. In some examples, two or more positions on the mining machine may be evaluated for placement of an inertial sensor. In some examples, the evaluation may not be performed. [0068] The approach described herein, in accordance with examples of the present disclosure, comprises two workflows also referred to as pipelines, which in turn comprise multiple stages. One of the pipelines is a training pipeline, which is developed using historical or batch data, and includes pre- analysis stages such as e.g. sensor description and position, exploratory data analysis (EDA), and modeling stages such as data capture and pre-processing, feature extraction, modeling, and validation. Another pipeline is a production pipeline, which is developed with data generated in real time, as the mining machine is operating in a mining environment or site, and includes such stages as e.g, queuing data, preprocessing, feature extraction, generating predictions, performing validation logic which includes confirming veracity or correctness of identification of a material moving cycle, determining operation indicators, and generating results representation. [0069] FIG.2 illustrates an example of a system 100 in which example embodiments of the present disclosure may be implemented. The system 100 may be employed, at least in part, in the mining environment or site such as e.g. mining site 10 shown schematically in FIG.1A. The system 100
Docket No.: PS56142PC00/ P23087WO01 may comprise one or more mining machines, and a mining machine 12 is shown in FIG.2 as a representative mining machine. The system 100 may also comprise a central controller or control system 22 such as, e.g. a traffic control system which may be a fleet controller configured to perform coordinated control of machines in the mining site. It should be noted however that the central control system 22 may be remote and it may be positioned outside of the mining site 10. The mining machines may be e.g. load-haul-dump (LHD) trucks, shovels, etc. [0070] As shown in FIG.2, the mining machine 12 may comprise a control system or mining machine controller 28 and a field computer device 16 which may be configured to perform the method in accordance with examples of the present disclosure. The field computer device 16 may be in operable communication with the mining machine controller 28. It should be noted that, even though the field computer device 16 is shown as a separate device in FIG.2, in some examples, the field computer device 16 may be part of the mining machine controller 28. Thus, the functionality performed by the field computer device 16 may be performed by the mining machine controller 28. In some examples, this functionality may be imparted to the mining machine controller 28 by installing on the mining machine controller 28 computer-executable instructions which may be part of a kit as described herein. In some examples, this functionality may be imparted to the mining machine controller 28 by installing on the mining machine controller 28 computer-executable instructions and at least one processor, which may be part of a kit as described herein. [0071] The mining machine 12 may comprise or may be associated with a display 33 which may be configured render a graphical user interface that is configured to display a representation of the identified one or more material moving cycles, one or more operation indicators, as well as other information related to operation of the mining machine 12 and analysis of the operation using the techniques described herein. The display 33 may be part of a dashboard display or another type of a built-in display. In some examples, the display 33 may be associated e.g. coupled to a suitable location within an operator compartment of the mining machine 12. Furthermore, in some examples, the display 33 may be part of a remote device communicatively coupled to mining machine 12 and/or the mining machine controller 28. For example, the display 33 may be presented in a smartphone, a tablet, or another computer device which may be positioned remotely relative to the mining machine. [0072] The machine controller 28 may be located onboard the machine 12. The machine controller 28 may be a main controller of the mining machine 12 which is configured to control operations of the mining machine 12. The machine controller 28 may comprise processing circuitry 30, such as at least one processor, and memory 31 which may comprise one or more memory units. The memory 31 comprises computer-executable instructions executable by the processing circuitry 30 of the machine controller 28. The memory 31 may be configured to store information, data, etc., and the
Docket No.: PS56142PC00/ P23087WO01 computer-executable instructions to perform, when executed by the processing circuitry 30, various processes related to monitoring and control operation of the mining machine 12. The machine controller 28 may comprise or may be associated with various other components not shown herein. [0073] The field computer device 16 may be adapted to execute computer-executable instructions to perform the functions or processes described herein. The field computer device 16 may be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet. The field computer device 16 may comprise processing circuitry 210 and a memory device or memory 220 which may comprise one or more memory units. The memory 220 comprises computer-executable instructions which may be executed by the processing circuitry 210 to cause the processing circuitry 210 to perform the method in accordance with examples of the present disclosure. [0074] The memory 220 may store various data related to the method in accordance with examples of the present disclosure. As shown in FIG.2, the memory 220 may store inertial sensor measurements 221 which may be acquired by the IMU sensor 14 also shown in FIG.2. As also shown in FIG.2, the memory 220 may store position sensor measurements 222 which may be acquired by at least one position sensor 17 also shown in FIG.2. The position sensor 17 may be configured to communicate with one or more position beacons located in the mining site. In some implementations, the position sensor 17 may be included in the machine controller 28. In some cases, the inertial sensor measurements and the position sensor measurements may be sent to an external system, e.g., to the control system 22 and/or to another system. [0075] The memory 220 may comprise a material moving cycles registry 224 storing one or more material moving cycles predicted or identified for the mining machine 12. [0076] The memory 220 may also comprise machine-learning (ML) model units that store at least one trained ML model as well as various information associated with the trained ML model e.g. data used to train the model, extracted/constructed and/or selected features, and other information. The at least one ML model unit may comprise a first ML model unit 226 storing a trained ML model, referred to herein as a first ML model, that can be applied to the inertial sensor measurements to identify or predict at least one material moving cycle from the inertial sensor measurements. The at least one ML model unit may also comprise a second ML model unit 228 storing a trained ML model, referred to herein as a second ML model, that can be applied to the position sensor measurements to verify a position of the loading and/or dumping points for the at least one material moving cycle that involves moving material from the loading point to the dumping point. [0077] The ML model units 226, 228 may comprise one or more subunits or modules not shown here, e.g., exploratory data analysis (EDA) unit, a feature construction and selection unit, a model building unit, a model evaluation unit, and other units or subunits.
Docket No.: PS56142PC00/ P23087WO01 [0078] The memory 220 may comprise an operation indicators registry 232 storing one or more operation indicators 234. The one or more operation indicators 234 may be determined from one or more cycles predicted or identified for the mining machine. Non-limiting examples of the operation indicators 234 comprise a duration of a certain material moving cycle e.g. an ore hauling cycle; a number of material moving cycles performed by the mining machine 12 during a certain time period e.g. a shift, a day, a month, a quarter, etc.; a total operating time determined for the mining machine; a total non-productive time representing a time during which the machine is stopped and/or is not contributing to production; operational efficiency of the mining machine, an unplanned downtime experienced by the mining machine; one or more production delays, and various other operation indicators etc. The operation indicators 234 may be stored in association with other information such as e.g. an operator identifier identifying an operator of the mining machine 12, a machine identifier identifying the mining machine 12, locations in the mine where the one or more identified material moving cycles were identified, and other information. [0079] As shown in FIG.2, the processing circuitry 210 may comprise at least one ML model execution unit 240 that is configured to execute the trained ML models stored in the ML model units 226, 228. The first and second trained ML models may be executed by the processing circuitry 210 to perform the method in accordance with examples of the present disclosure. The processing circuitry 210 may also comprise model retraining unit 230 that allows the processing circuitry 210 to automatically train and retrain the first ML model for identifying and/or predicting a material moving cycle and the second ML model for identifying and/or verifying a loading point and a dumping point between which the mining machine has moved the material during the identified material moving cycle. In some examples, the same module or unit may perform ML model processing and retraining, e.g. the ML model execution unit 240. The processing circuitry 210 may include various other modules and/units configured to perform actions and methods in accordance with examples herein. [0080] The processing circuitry 210 may include any number of hardware components for conducting data or signal processing or for executing computer code such as computer-executable instructions stored in the memory 220. The processing circuitry 210 may include a general-purpose processor, an application specific processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processing components, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. [0081] The memory 220 may be one or more devices for storing data and/or computer code such as computer-executable instructions for completing or facilitating methods described herein. The memory 220 may comprise random access memory (RAM), read-only memory (ROM), erasable
Docket No.: PS56142PC00/ P23087WO01 programmable read-only memory (EPROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and/or computer instructions. The memory 604 may include database components, object code components, script components, and/or any other type of information structures for supporting the various processes and information structures described in the present disclosure. The memory 220 may be communicably connected to the processing circuitry 210, e.g., via a circuit or any other wired or wireless connection. [0082] The field computer device 16 may also include a communications interface 242 that may include wired and/or wireless communications interfaces, e.g., jacks, antennas, transmitters, receivers, transceivers, wire terminals, etc., for conducting data communications external systems or devices. In various examples, the communications may be direct, e.g., local wired or wireless communications, or via a communications network, e.g., a WAN, the Internet, a cellular network, etc. The field computer device 16 may include various other components not shown herein. [0083] It should be appreciated that any of the data and computer-executable instructions stored in the memory 220 may be loaded onto the processing circuity 210 or used by the processing circuitry 210, to perform the method in accordance with examples of the present disclosure. [0084] Those skilled in the art will appreciate that the units of the field computer device 16 described herein may refer to a combination of analogue and digital circuits, and/or one or more processors configured with software and/or firmware, e.g., stored in the field computer device 16, that, when executed by respective one or more processors, may perform the methods in accordance with examples of the present disclosure. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuitry (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip. [0085] It should be appreciated that the field computer device 16, the machine controller 28, and the mining machine 12 may comprise various other components not shown in FIG.2 for the sake of simplicity. For example, the field computer device 16 may comprise a communication interface for communication with the IMU sensor 14 to receive one or more inertial sensor measurements, and for communication with remote systems such as e.g. the control system 22. [0086] FIG. 2 illustrates that the central control system 22 comprises processing circuitry 24 and memory 260. The central control system 22 may be a central controller or fleet controller that may be positioned remotely from a plurality of mining machines operating in the mining site. In some examples, the central control system 22 may be positioned in the mining environment or site, e.g., it may be part of or associated with one of the mining machines in the mining site. Regardless of its specific
Docket No.: PS56142PC00/ P23087WO01 implementation and location, the central control system 22 is configured to receive information from and to send information and control commands to one or more mining machines out of the plurality of mining machines in the mining site. The central control system 22 may be configured to control, in a coordinated manner, movements and/or other functions of the mining machines. The mining machines may be fully autonomous, semi-autonomous, or manually controlled machines, and various type of signals and instructions may be received by the machines from the control system 22. [0087] The memory 260 of the central control system 22 may store computer-executable instructions that can be executed by the processing circuitry 24 to cause the processing circuitry 24 to perform monitoring and controlling of operations of the mining machine 12. The memory 260 of the central control system 20 may receive various information regarding the mining machine 12 e.g. from the field computer device 16 and/or the machine controller 28, such as one or more identified machine moving cycles, one or more operator indicators, and various other information. Based on the received information, the control system 22 may generate and send commands to the mining machine 12, as well as to one or more of other mining machines in the mining site. [0088] As also shown in FIG.2, the control system 22 may comprise and/or may be communicatively coupled to a display that is configured to render a graphical user interface 280. The user interface 280 may display various information related to mining machines controlled via the control system 22. The user interface 280 may also be configured to receive user input e.g. with respect to the displayed information. The control system 22 may include an input and output device interface (not shown) such as e.g. a circuit for controlling input and output from and to peripheral devices including devices such as a mouse, a keyboard, joystick, touch-sensitive surface or pad, touch-sensitive screen, etc. [0089] The user interface 280 of the control system 22 may be configured to present various information based on the identified material moving cycles and the operation indicators. The information allows visualizing and assessing a status of mining machines in the fleet, as well as a status of the entire fleet. For example, at any point in time, one or more material moving cycles identified for one or more mining machines currently located in the mining site may be visualized. A representation of the machines in the mining site by material moving cycles may also be visualized such that it is possible to access a number of material moving cycles performed by the one or more machines. It may be also possible to determine whether a mining machine in the site is currently performing a cycle which has not yet been identified. It may be, for example, visualized how many mining machines are currently operative, delayed, reserved or out of service. The one or more representations of the status of the individual or groups of the machines, and of the entire fleet, may be generated and displayed in real time, such that a real time monitoring, assessment, and control of the fleet of mining machines may be
Docket No.: PS56142PC00/ P23087WO01 performed. In this way, a user such as e.g. a fleet operator may be provided with information that can be used to assess the status of the mining machines and to make decisions regarding operation of the mining machines. The information may in some cases be analyzed automatically. Maintenance decisions, machine repositioning, task or job assignments, and other actions may be performed using the identified cycles and the operation indicators identified or predicted in accordance with examples of the present disclosure. Instructions or commands may be generated by the control system 22 and sent to the mining machine 12 and/or other machines, instructing the machines to initiate actions related to maintenance, repositioning, task or job assignments, and other types of actions. [0090] The approach described herein, in accordance with examples of the present disclosure, may comprise two or more workflows also referred to as pipelines, which in turn may comprise multiple stages. One of the pipelines is a training pipeline, which is developed using historical or batch data, and includes pre-analysis stages such as e.g. sensor description and position, exploratory data analysis (EDA), and modeling stages such as data capture and processing, feature extraction, modeling, and validation. Another pipeline is a production pipeline, which is developed with data generated in real time, as the mining machine is operating in a mining environment or site, and includes such stages as e.g, queuing data, preprocessing, feature extraction, generating predictions, performing validation logic which includes confirming veracity of a state transition, determining operation indicators, and generating results representation. Other pipelines may be implemented as well. [0091] FIG.3 illustrates an example of a computer-implemented process or method 300 for monitoring and controlling operation of a mining machine in a mining site. The mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another. The method 300 may be performed by a computer device, e.g. the field computer device 16 or another suitable computer device which may be positioned in the mining machine or may be otherwise associated with the mining machine including remotely. In some examples, the method 300 may be performed by one or both the field computer device 16 and/or the mining machine controller 28. The actions at blocks of FIG.3 do not have to be taken in the order stated below, but may be taken in any suitable order. Dashed boxes indicate optional features. [0092] At block 302, the method 300 comprises, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time. In some examples, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, and the received sensor measurements may comprise sensor data in 9 axes. Thus, patterns indicative of material moving cycles performed by the mining machine may be detected in the 9-axis inertial sensor data or signal.
Docket No.: PS56142PC00/ P23087WO01 [0093] In some examples, data acquired from one or more of the three-axis accelerometer, the three-axis gyroscope, and the three-axis magnetometer may be used to identify a material moving cycle. For example, the IMU sensor may be positioned on the mining machine, e.g. on a movable implement of the mining machine, at a location where readings acquired by the IMU sensor are informative enough such that data from fewer than nine axes may be sufficient to accurately identify or detect or predict a material moving cycle. [0094] The first period of time may be a shift comprising several hours, a day i.e.24 hours, a month, a quarter, or any other period of time. Any period of time may be selected to identify material moving cycles for the machine and to determine a number of the cycles performed by the machine during that period of time. [0095] At block 312, the method 300 comprises applying a first trained machine-learning (ML) model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time. The at least one material moving cycle out of a plurality of material moving cycles may be identified in real time, as the mining machine is operating in the mining site. In some examples, the material moving cycle may be identified at a later time, e.g., after the machine has completed the cycle. [0096] It should be noted that the material moving cycle may be identified with some probability of the identification because the cycle is identified as a pattern recognized in the inertial sensor measurements. The accuracy of the inertial sensor measurements and other factors may affect the correctness of the identification of the material moving cycle. [0097] In some examples, the material moving cycle may comprise a sequence of operating states that the mining machine performs during the material moving cycle. The sequence of the operating states performed by the mining machine may comprise (i) loading the mining machine with material at a loading point, (ii) moving the material by the mining machine from the loading point to a dumping point, (iii) unloading the material at the dumping point, and (iv) moving from the dumping point to the loading point or to another loading point. For example, an LHD machine may pick a load of the material, e.g. in its bucket or other attachment, at the loading point and move the load to the dumping point where the load is deposited. The mining machine, without a load, may then return to the loading point to pick up another load, or the mining machine may move to a different loading point to pick up the different load. The material moving cycle may be considered completed once the machine returns to the same or different loading point in a state ready to pick another load. It should be noted that other sequence of the operating states may be considered to constitute the material moving cycle. [0098] In examples herein, separate operating states are not identified and the material moving cycle is identified and reported as a whole. The first ML model may be trained to recognize a pattern in
Docket No.: PS56142PC00/ P23087WO01 the inertial sensor measurements that is indicative of a completed material moving cycle. The first ML model may be trained using known sub-patterns and time windows in IMU data corresponding to operating states forming the material moving cycle, but the separate operating states may not be detected for the purposes of the techniques described herein. [0099] In some examples, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The inertial sensor measurements may comprise data acquired in the nine corresponding axes, three per each of the accelerometer, gyroscope, and magnetometer. Data or signal from each axis may be pre-processed to generate new variables, e.g., one or more of signal enveloping, low-pass or high-pass filtering to eliminate noise, and moving average filtering to smooth the signal may be applied to the data. Feature extraction may be performed using these new variables. [00100] In some examples, the trained first ML model may be agnostic to one or more out of a model of the mining machine, a brand of the mining machine, and a manufacturer of the mining machine. [00101] In some examples, the first ML model may be trained in dependency on at least one property of the worksite e.g. mining environment, e.g. a layout of the mining environment, distances to be traveled by the mining machine, etc. [00102] In some examples, the first ML model may be trained as described in connection with FIG. 4. In some examples, a training pipeline, described in more detail in connection with FIG.4, may be executed automatically, whereby the first ML model may be generated and trained, to be suitable for identifying material moving cycles in measurements acquired by the IMU sensor. In examples herein, IMU sensor data may be sufficient to identify material moving cycles, though position sensor data may additionally be used, as described below. In any case, no specific input from a user, such as e.g. the operator of the mining machine or another user, may be required to identify at least one material moving cycle. The accuracy of the identification or detection of the material moving cycle is improved, which improves the way in which the operation of the mining machine is assessed and controlled. [00103] At block 313, the method 300 may optionally comprise determining a correctness of the identified at least one material moving cycle. The at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. Other factors may be used to determine whether the material moving cycle that has been identified is indeed an actual material moving cycle performed by the mining machine and whether the cycle is completed. [00104] At decision block 315, it may be determined whether the correctness of the identification of the at least one material moving cycle has been confirmed or verified. Responsive to determining that
Docket No.: PS56142PC00/ P23087WO01 the correctness of the identification of the at least one material moving cycle has not been verified, the process 300 may proceed to block 317. Alternatively, responsive to determining that the correctness of the identification of the at least one material moving cycle has been verified, i.e. that the mining machine is estimated to have performed this cycle, the process 300 may proceed to block 320. [00105] At block 317, the at least one identified material moving cycle may be removed from further analysis and the process 300 may return to block 302 to receive further inertial sensor measurements acquired by the IMU sensor. It should be noted that more than one material moving cycles is typically identified for further analysis. [00106] At block 320, the method 300 comprises determining at least one operation indicator from the identified at least one material moving cycle. The at least one material moving cycle may comprise two or more material moving cycles. The operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period. The operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine. The operation indicator may be expressed as a quantitative value, a qualitative value, or a combination thereof. [00107] Machine utilization times may be useful for establishing proactive maintenance plans for the mining machine and other machines in the mining environment. In some examples, the operation indicators may comprise a total operating time representing a total time during which the mining machine is operational and productive e.g. moves the material such as ore from one or more loading points to one or more dumping points. The total operating time may be determined by summing all periods of time in which the mining machine is active and has been identified as generating value. [00108] In some examples, the operation indicators may comprise a use of the mining machine per a certain duration of time. For example, a use U of the mining machine per shift may be calculated by dividing the total operating time by a duration of the shift:
[00110] A use of the mining machine during any other period of time may also be determined in a similar manner. [00111] In some examples, the operation indicators may comprise an overall operational efficiency E which may be calculated by dividing the total operating time of the mining machine by a total available time indicating a duration of time period during which the mining machine is or was available. It shows what percentage of the available time is used for production:
Docket No.: PS56142PC00/ P23087WO01 [00112] ^ = ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 100% (2)
downtime indicating a time during which the mining machine is down due to unplanned failures. This indicator can help identify sources of faults that may occur in the mining machine. [00114] In some examples, the operation indicators may comprise unexpected delays during production, such as one or more of a lack of electrical power, quality problems, or unforeseen interruptions. Identifying and reducing such delays can improve efficiency of the operation of the mining machine, a group of the mining machines e.g. in a fleet, and of the entire mine. [00115] The operation indicators may be determined per shift, month, quarter, or per any other period of time, whereby a fleet performance over that period of time may be determined. [00116] The operation indicators may be represented on a user interface of a display, e.g., of the mining machine and/or a fleet control system, in the manner that allows assessing the fleet performance. The operation indicators may also include an overall equipment effectiveness (OEE) indicators determined for the mining machine. Real-time monitoring of the machines in the fleet may be performed, which allows e.g. making decisions during a shift, to ensure that the performance of the fleet during the shift conforms to target performance. The target performance may be defined as, e.g., one or more of a number of material moving cycles per a certain time duration, utilization of the mining machine such as an amount of time during which the machine is not used is minimized, etc. The target performance may be set for a shift, a day, a month, a quarter, a year, and/or any other period of time. [00117] At block 322, the method 300 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the material moving cycle. The at least one operation indicator may comprise a plurality of operating indicators. In some examples, initiating the action may comprise prompting a display of a representation of the identified material moving cycle and/or a representation of the operation indicator. The representation of the identified material moving cycle may be presented on a display such as e.g. a display of the central control system 22, a display associated with the mining machine and/or with the field computer device, and/or on a display associated with any other device or system. [00118] In some examples, information on the material moving cycle and/or the at least one operation indicator may be displayed on user interface 280 rendered on the display communicatively coupled to the central control system 22. In this way, a fleet operator or another person may be enabled to access performance of the mining machine, as well as of other machines operating in the mine, and to determine if any actions need to be taken. In some examples, additionally or alternatively, information on the material moving cycle and/or the at least one operation indicator may be displayed
Docket No.: PS56142PC00/ P23087WO01 on a user interface rendered on the display 33 associated with the mining machine 12. The driver or another operator of the mining machine 12 may be informed, in real time, about a current status of the mining machine and a location of the mining machine in the mine, a number of material moving cycles that have been performed by the mining machine, a time that it took the machine to perform each of the cycles, and/or any other suitable information. The driver may then adjust operation of the mining machine and/or initiate other actions based on the information on the material moving cycle and/or the at least one operation indicator. [00119] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. [00120] In some examples, the instruction to control the mining machine may be obtained by the field computer device 16 and/or the mining machine controller 28, which may perform functionality of the field computer device 16. The instruction may be received from e.g. the central control system 22. In some examples, the instruction may be generated the field computer device 16 and/or the mining machine controller 28. In some examples, the field computer device 16 may generate and/or receive the instruction and to provide this instruction to the mining machine controller 28. [00121] Regardless of the specific way in which the field computer device 16 and/or the mining machine controller 28 generates or receives or obtains the instruction, such instructions may be a control instruction that is used to adjust one or more operating characteristics of the mining machine 12. Non-limiting examples of the operating characteristics comprise one or more of a speed of material loading and/or unloading by the mining machine, a speed of movement of the mining machine, a position of the mining machine, specific loading and dumping points at which the mining machine can operate, timing and duration of idling by the mining machine, etc. For example, the mining machine may be instructed to stop operation if it is determined that it is not currently capable of performing material moving e.g. due to a malfunction. As another example, the mining machine may be instructed to move to another location in the mine. The mining machine may be instructed to pick up a load of material from an alternative loading point and/or to carry the material to an alternative dumping point. [00122] In some examples, the instruction to control the mining machine may be generated in dependence on the at least one operation indicator and additionally in dependence on other data regarding operation of the mining machine and/or other types of data. [00123] The adjustment of the target requirement for performance of the mining machine may involve adjusting operational parameters for the mining machine such that the machine is controlled to operate in dependence on its previously determined performance. For example, if the machine is
Docket No.: PS56142PC00/ P23087WO01 determined to be underperforming, its target performance, e.g. a number of material moving cycles completed during a shift, a frequency of use of the machine, a total length of continuous periods of use of the machine, and/or other target performance parameters, may be increased or otherwise adjusted. It should be noted that the target performance may be adjusted if it is additionally determined that the machine does not experience excessive downtime caused by unplanned failures or by other factors that affect machine performance. [00124] If the machine is determined to be overused, its target performance, e.g. a number of material moving cycles completed during a shift, a frequency of use of the machine, a total length of continuous periods of use of the machine, and/or other target performance parameters, may be decreased or otherwise adjusted. [00125] Another type of adjustment for the mining machine, based on the identified material moving cycle and the at least one operation indicator, may include a location of the mining machine in the worksite such as a mine. For example, the mining machine with higher performance metrics may be moved to a more critical location in the mine. As a related example, the mining machine with lower performance metrics may be moved to a location in the mine where less work would be required from the mining machine e.g. fewer material moving cycles are expected to be performed during a certain time period. [00126] In some examples, one or more reasons of underperformance of the mining machine may be identified when the machine is determined to be underperforming e.g. due to component failures. For example, it may be determined that the machine is underperforming due to excessive delays caused by mechanical failures, in which case a measure related to machine maintenance may be taken. [00127] In some examples, if the mining machine is determined to be underperforming due to excessive delays caused by environmental conditions, a plan can be implemented to improve conditions in the work area. Some examples include power supply or water supply failures, which can extend machine downtime. [00128] In some examples, a reliability ranking or another similar comparison measure may be used to rank the mining machines in the worksite based on their performance such as completion of one or more material moving cycles as identified in accordance with embodiments of the present disclosure. For example, a mining machine may be considered to have a higher reliability when it has a higher utilization efficiency and a lower downtime than one or more of other mining machines. In some examples, more reliable mining machines, such as e.g. LHD machines, may be assigned to more critical production points in the worksite. Less reliable mining machines, such as e.g. LHD machines, can be assigned to less critical points in the mine, to ensure planned production.
Docket No.: PS56142PC00/ P23087WO01 [00129] FIG.4 illustrates an example of a training pipeline or process 400 for training a ML model e.g. the first ML model, in accordance with embodiments of the present disclosure. The training process 400 is performed to obtain an ML model that can be used to detect or predict or identify material moving cycles, also referred to herein as ore hauling cycles, that can be performed by the mining machine. The machine-learning model may be selected from one or more candidate machine-learning models. The material moving cycles may be identified and defined from patterns detected in signals acquired by one or more IMU sensor units. The training process 400 may be performed in advance, and a resulting trained machine-learning model may be provided, for example, as part of a kit provided in accordance with examples of the present disclosure. In some examples, the training may be performed by a central control system e.g. central control system 22, and/or by another external control system. [00130] In some examples, the training may be performed by the field computer device 16 and/or the mining machine controller 28. Thus, the execution of the training pipeline may be part of the method in accordance with examples of the present disclosure as performed by the field computer device 16 and/or the mining machine controller 28. The training pipeline may be executed automatically. [00131] At block 402, the process 400 comprises obtaining training inertial sensor measurements acquired by one or more IMU sensors. The IMU sensors may be coupled to a mining machine e.g. to a movable implement comprising a controllable bucket or another similar tool. The inertial sensor measurements may be acquired directly or indirectly, from IMU sensors associated with respective one or more mining machines operating in a mining environment or mining site. The inertial sensor measurements, which may be referred to as training data, are acquired as the one or more mining machines are operating in the mining environment. The one or more mining machines may be any suitable types of mining machines configured to load and upload material and to move with the material from one location to another, e.g., LHD machines. In some examples, each of the mining machines used to acquire training data comprises one i.e. single IMU sensor unit coupled thereto, e.g., on the movable implement of the mining machine. [00132] In some examples, the training inertial sensor data may be updated as more sensor measurements are acquired from the IMU sensors coupled to mining machines in the mining environment. In some examples, the one or more inertial sensor measurements used for the training stage may be simulated data, or a combination of simulated data and actual inertial sensor measurements. [00133] At block 405, exploratory data analysis (EDA) may be performed on the acquired inertial sensor measurements. The EDA may be performed to ensure that the resulting model is agnostic to a type of the mining machine and a mining environment. The EDA stage may include pattern analysis, statistical analysis, frequency analysis, and other types of processing. The objective of the exploratory
Docket No.: PS56142PC00/ P23087WO01 analysis of the inertial sensor measurements is to find patterns that allow identifying a time window in which the mining machines perform one or more material moving cycles. A material moving cycle may comprise sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [00134] At block 407, the process 400 may comprise data preprocessing, which may involve various techniques to prepare the inertial sensor measurements for training. During this phase or stage, different data cleaning methods may be applied, such as e.g. outlier filtering, elimination of null or corrupted values, and signal smoothing using different techniques such as rolling, low-pass filtering or Fourier transform. In some examples, calibration of the IMU sensor unit may be performed. Also, secondary signals and/or additional transforms may be obtained that provide relevant information for a feature extraction stage, such as, e.g., envelope calculation, Fourier transform, logarithmic transform, wavelets, and empirical mode decomposition, among others. Preprocessed IMU sensor data may be generated as a result of preprocessing of the one or more inertial sensor measurements. [00135] At block 409, the process 400 may comprise performing feature construction and/or extraction to identity one or more features to be used in the ML model. For example, the preprocessed IMU sensor data may be labeled according to observed and/or identified patterns, distinguishing between different material moving cycles. The most relevant signals may be selected to form variables, and sliding time windows may be designed or generated with overlapping, adjusted to a duration of the material moving cycles. In some examples, for each of the axes e.g.9 axes of an IMU sensor, statistical indicators may be calculated using the time windows as a basis, and a class is assigned with a label or name of the material moving cycles based on the previously labeled pattern. In some examples, a binary classification may be used, with windows that correspond to a cycle and with windows that do not correspond to a cycle. In some examples, a multiclass classification may be used such that more than two classes can be assigned to a material moving cycle e.g., a productive cycle, a non-productive cycle, a loaded cycle, an unloaded cycle, etc. The calculated characteristics and the class constitute the variables to be used for modeling. Non-limiting examples of the statistical indicators may comprise one or more of a mean, a standard deviation, a slope, a median, a maximum, a minimum, polynomial coefficients, kurtosis, skewness, and frequency response. Non-limiting examples of features obtained from the frequency response are gain, energy, and cutoff frequency. [00136] At block 411, at a training and model selection stage, the process 400 may comprise training one or more candidate machine-learning models and selecting a model from the candidate models, which may be performed using any one or more of various approaches. An ML model may be
Docket No.: PS56142PC00/ P23087WO01 selected from a set of the candidate ML models. For example, training data may be fitted to different supervised classification models. In some examples, the machine-learning models comprise decision tree models, which use a class variable as a target. One or more ML models may be trained and assessed, and a model that gives a most accurate performance and is less computationally expensive than other models may be selected. For the evaluation, evaluation metrics such as e.g., one or more out of f1-score, recall, precision, accuracy, area under the curve (AUC), and others, may be used. Also, techniques such as e.g. confusion matrices, classification reports, and others related to an importance of the variables may be used. In some examples, an ML model may be selected which may deliver results above 90% for all evaluation metrics used. In some examples, an ML model may be selected which may deliver results above 95% for all evaluation metrics used. The ML model may be selected based on other criteria. [00137] At block 413, the process 400 may comprise validating the trained ML model. The validation may be performed using one or more of various validation techniques. For example, to ensure appropriate performance of the selected ML model, predictions may be made with unlabeled data, and metrics may be generated to evaluate and validate performance of the ML model using the unlabeled data. The validation state is used to access whether the model, selected at the training and model selection stage at block 411, is able to perform, i.e. recognize or identify material moving cycles in inertial sensor measurements acquired by an IMU sensor or IMU sensor unit coupled to the mining machine, with precision similar to that exhibited by that model at the training and model selection stage. [00138] It should be noted that a choice of a final ML model is an iterative process, subject to changes in the training data, so that monitoring, readjustment, and/or retraining may be performed in order to ensure that the ML model reflects an updated, most recent representation of reality. Accordingly, as shown schematically in FIG.4, by arrow a1, the process 400 may return, from block 413, to block 411, to continue training the model, which may be performed iteratively. [00139] Further, as shown in FIG.4, at decision block 415, it may be determined whether the validation of the trained ML model is complete. If this is the case, the trained ML model may be output at block 417. Otherwise, as shown in FIG.4, the process 400 may return to block 413 to continue the validation process of the ML model. [00140] To perform the method steps of the method for monitoring and controlling operation of a mining machine out of a plurality of mining machines in a mining site, the field computer device and/or the mining machine controller may be configured to perform the processing described in connection with FIG.3. In some examples, the field computer device and/or the mining machine controller may be configured to perform, at least in part, the processing described in connection with FIG.4 which
Docket No.: PS56142PC00/ P23087WO01 involves training a machine-learning model that is configured to be applied to inertial sensor measurements to predict or identify material moving cycles performed by the mining machine. [00141] In some examples, in addition to obtaining or receiving inertial sensor measurements acquired by at least one IMU sensor coupled to the mining machine, the mining machine also obtains or receives position sensor measurements acquired from at least one position beacon or tag positioned in the mining site externally to the mining machine. The position tag may be stationary at a predetermined location in the mine. For example, FIG.1A shows schematically an example of the position tags or beacons 11a-11c, though it should be appreciated that multiple position beacons may be located in the mine, to e.g. mark locations of loading areas or points, dumping areas or points, and other locations in the mine. The position sensor measurements may be used to determine or verify locations of the loading point and the dumping point for one or more material moving cycles identified using the inertial sensor measurements data. In the mine, the mining machine may be moving in a tight environment where multiple loading points and dumping points may be located, and the position sensor measurements may assist in determining with improved precision an origin and destination of the material. Accordingly, for each identified material moving cycle, it may be known where the material came from and where it was moved to. This advantageously allows monitoring processes in the mine with improved accuracy, which allows managing and controlling operations of the mining machines and of the entire mine with improved performance. The material moving cycles performed by the mining machine as well as related information may be determined automatically, without any input from a driver or another person, which improves efficiency of the mine operation and reduces a risk of errors in assessment and control of the mine operations. [00142] FIG.5 shows a computer-implemented process or method 500 for monitoring and controlling operation of a mining machine in a mining site, in accordance with some embodiments of the present disclosure. The mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another. The method 500 may be performed by a computer device, e.g. the field computer device 16 and/or the mining machine controller 28, or by another suitable computer device which may be positioned in the mining machine or may be otherwise associated with the mining machine. The actions at blocks of FIG.5 do not have to be taken in the order stated below, but may be taken in any suitable order. Processing at some acts or blocks of FIG.5 is similar to corresponding acts or blocks shown in FIG.3 in connection with the method 300, and their description is therefore not repeated in connection with FIG.5. [00143] At block 502, the method 500 comprises, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine The inertial sensor measurements may be acquired over a first period of time.
Docket No.: PS56142PC00/ P23087WO01 [00144] At block 503, the method 500 comprises, as the mining machine is operating in the mining site, receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site. The position sensor measurements may be acquired as time series data that may be stored in memory e.g. of the field computer device 16 and/or the mining machine controller 28. [00145] The position beacon or tag may be positioned at a known location in the mine. The computer device may receive the position sensor measurements from at least one sensor that is configured to communicate with the at least one, typically multiple position tags. The at least one position sensor, e.g. position sensor 17 shown in FIG.2, may be installed in or otherwise associated with the field computer device and/or mining machine controller 28. The position sensor measurements may be acquired as the mining machine is moved through the mine and as the mining machines receiving the inertial sensor measurements. In other words, the position sensor measurements may be received simultaneously or substantially simultaneously with receiving the inertial sensor measurements. Thus, the processing at block 503 may be performed simultaneously or substantially simultaneously with the processing at block 502. [00146] At block 505, the method 500 comprises preprocessing of the inertial sensor measurements. The preprocessing may use known techniques for preparation raw sensor measurements data to further analysis, including smoothing and other techniques. The preprocessing at block 505 matches the preprocessing performed on the data in the model training pipeline. As data is received from the sensors, the data may be continuously processed in real time and stored in the memory of the field computer device 16 and/or the mining machine controller 28. [00147] At block 507, the method 500 comprises preprocessing of the position sensor measurements or data. The position sensor data may be time series data that may be preprocessed using one or more of a moving average filter, a median filter, and Kalman filter, to smooth the signals in the data and eliminate noise. The preprocessing of the position sensor measurements matches the preprocessing performed on the data in the training pipeline of the second model. [00148] In some examples, the position sensor data may be preprocessed in accordance with characteristics of material moving cycles identified using embodiments of the present disclosure. A material moving cycle may have a respective duration, different from duration of other cycles, and may depend on factors such as e.g. machine operator behavior, zones in the mine, distances between loading and/or dumping points in the mine, a speed of the mining machine, etc. Thus, the time series of the position sensor data may be preprocessed by one or more scaling and dimensionality reduction methods so that the time series corresponding to a certain material moving cycle, i.e. acquired during the performance of that cycle by the mining machine, have the same size.
Docket No.: PS56142PC00/ P23087WO01 [00149] It should be noted that the data preprocessing at blocks 505 and 507 may be performed in any suitable order or simultaneously. In some examples, the preprocessing of the position sensor measurements may be performed after applying the first trained ML model to the inertial sensor measurements to identify at least one material moving cycle as shown in connection with block 512. [00150] At block 510, the method 500 comprises performing feature extraction from the preprocessed inertial sensor measurements and position sensor measurements. The same features may be extracted as those used to train the model. The preprocessed data may be accumulated in real time, e.g., in the memory of the field computer device or in another memory device, until a window size necessary to calculate the characteristics is met, and the data is then used to obtain the inference of the model. [00151] The feature extraction may involve applying a dimensionality reduction technique to extract meaningful features from the preprocessed inertial sensor measurements and position sensor measurements. The features which may be extracted from the preprocessed inertial sensor measurements are suitable for application to these features of the first ML model, to identify at least one material moving cycle. Similarly, the features which may be extracted from the preprocessed position sensor measurements are suitable for application to these features of the second ML model, to verify the loading point and the dumping point for the identified at least one material moving cycle. [00152] At block 512, the method 500 comprises applying the first trained ML model to the inertial sensor measurements to identify at least one material moving cycle. The processing at block 512 may be performed similar to the processing at block 312 of FIG.3. [00153] At block 514, the method 500 comprises applying a second trained ML model to the position sensor measurements to verify the loading point and the dumping point for the identified material moving cycle. The second ML model may be trained similarly to the first ML model, e.g. as described in connection with FIG.4, but the training data would comprise position sensor measurements, actual and/or simulated, received from position beacons or tag positioned in the mine or in a simulator environment. [00154] In some examples, once the material moving cycle is identified and its correctness is validated or confirmed, e.g. as shown in connection with blocks 313 and 315 of FIG.3, the position sensor measurements or tag signals acquired during the same time period, referred to herein as the first time period, may be processed e.g. as shown at blocks 507 and 510. In some examples, the position sensor measurements or tag signals may be processed before the material moving cycle is identified. [00155] The tag signals may be processed and the processed data e.g. features may be input to the trained second ML model. The second ML model may provide an output such as a value indicating
Docket No.: PS56142PC00/ P23087WO01 a probability that a given signal comes from a certain loading point. More than one loading point may be identified, each assigned a corresponding probability. In some examples, a loading point identified with a highest probability may be selected as an actual loading point from which the material was taken during the identified material moving cycle. In some examples, the dumping point may be identified and/or verified, for the material moving cycle, in a similar manner in which the loading point may be identified. The loading point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a loading time window such as time window during which the mining machine is expected, from the training data, to perform loading of the material. The dumping point may be identified and/or verified based on a portion of the position sensor measurements that were acquired during a dumping time window such as time window during which the mining machine is expected, from the training data, to perform dumping or unloading of the material. [00156] The second ML model may be trained to select the loading and unloading points using time series data acquired of each tag signal. Previously acquired time series data may be labeled to indicate a correct source signal for a plurality of material moving cycles, and the second ML model is trained to differentiate and classify the time series data, to provide an output indicating a probability that position sensor data acquired from the positions beacons is indicative of actual material origin or loading points and/or of material unloading or dumping points. [00157] At block 520, the method 500 comprises determining at least one operation indicator from the identified at least one material moving cycle. The at least one material moving cycle may comprise two or more material moving cycles. The operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period. The operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine. The processing at block 520 may be performed similarly to the processing at block 320 of FIG.3 and the description at block 320 is applicable herein. [00158] At block 522, the method 500 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the at least one material moving cycle. In some examples, initiating the action may comprise prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. The
Docket No.: PS56142PC00/ P23087WO01 processing at block 522 may be performed similarly to the processing at block 322 of FIG.3 and the description at block 322 is applicable herein. [00159] In some examples, position sensor measurements may not be available, or acquired position sensor measurements may not be sufficient to verify positions of loading and/or dumping points for a corresponding material moving cycle. [00160] FIG.6 illustrates an example of a computer-implemented process or method 600 for monitoring and controlling operation of a mining machine in a mining site, in accordance with some embodiments of the present disclosure. The mining machine may be e.g. machine 12 such as an LHD machine configured to move or haul material such as ore from one location to another. The method 600 may be performed by a computer device, e.g. the field computer device 16 and/or the mining machine controller 28, or by another suitable computer device which may be positioned in the mining machine or may be otherwise associated with the mining machine. The actions at blocks of FIG.6 do not have to be taken in the order stated below, but may be taken in any suitable order. Processing at some acts or blocks of FIG.6 is similar to processing at corresponding acts or blocks shown in FIGs.3 and 5. Also, blocks in FIG.6 have numerical references that are similar to numerical references of corresponding blocks of FIGs.3 and 5 at which similar processing is performed. The description of processing steps described in connection with FIGs.3 and 5 is not repeated in connection with FIG.6, but it should be appreciated that the description applies to the method 600 of FIG.6. [00161] At block 602, the method 600 may comprise, as the mining machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the mining machine. The method 600 may also comprise, similar to processing at block 503 of FIG. 5, as the mining machine is operating in the mining site, receiving, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site. [00162] At block 612, the method 600 may comprise applying a first trained ML model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time. [00163] At block 613, the method 600 may comprise determining a correctness of the identified at least one material moving cycle. The at least one material moving cycle may be determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. Other factors may be used to determine whether the material moving cycle that has been identified is indeed an actual material moving cycle performed by the mining machine and whether the cycle is completed. [00164] At decision block 615, the method 600 may determine whether the correctness of the identification of the at least one material moving cycle has been confirmed or verified. Responsive to
Docket No.: PS56142PC00/ P23087WO01 determining that the correctness of the identification of the at least one material moving cycle has not been verified, the process 600 may proceed to block 617. Responsive to determining that the correctness of the identification of the at least one material moving cycle has been verified, the process 600 may proceed to block 618. [00165] At block 617, the at least one identified material moving cycle may be removed from further analysis and the process 600 may return to block 602 to receive further inertial sensor measurements acquired by the IMU sensor. It should be noted that more than one material moving cycles is typically identified for further analysis. [00166] At decision block 618, the method 600 may determine whether a required amount of position sensor measurements or data have been received. The required amount may be defined as an amount of data that is sufficient to verity locations of a loading point and/or a dumping point for the material moving cycle that involves transfer of material from the loading point to the dumping point. In some examples, the position sensor measurements may not be available, e.g., due to absence or malfunction of a position sensor configured to communicate with one or more position beacons or tags that are external to the mining machine. The position sensor may be e.g. position sensor 17 shown in FIG.2, which may be part of or associated with the field computer device 16 and/or the machine controller 28. Such position sensor may be absent from some mining machines. [00167] As another possible reason for which sufficient position sensor measurements may not be acquired is that one or more one or more position beacons or tags may be absent or not working properly. Various other factors may contribute to the lack of required amount of position sensor data, such as issues with transmission of communication signals between the position sensor on the mining machine and the one or more position tags, and others. [00168] At block 607, responsive to determining that the required amount of the position sensor measurements is received, the method 600 may perform preprocessing of the position sensor measurements, similar to the preprocessing of the position sensor measurements at block 507 of FIG. 5. [00169] At block 614, the method 600 comprises applying a second trained ML model to the position sensor measurements to verify the loading point and the dumping point for the identified material moving cycle. [00170] Referring back to decision block 618, regardless of the specific one or more reasons for the lack of the required amount of position sensor data, responsive to determining that the required amount of the position sensor measurements is not received, the process 609 proceeds to block 620 to determine at least one operation indicator from the identified at least one material moving cycle. In this case, a loading point and/or dumping point have not been verified for the material moving site. Other
Docket No.: PS56142PC00/ P23087WO01 techniques may be used to verify the loading point and/or dumping point, without the use of position sensor measurements. For example, cycles can be manually assigned to a load point by a fleet operator or mine coordinator, or can be manually selected by the operator e.g. on a screen of a computing device. [00171] The process 600 also follows to block 620 responsive to determining that the required amount of the position sensor measurements is received, in which case the loading point and/or the dumping point have been verified for the material moving site. [00172] At block 620, the method 600 comprises determining at least one operation indicator from the identified at least one material moving cycle. The at least one material moving cycle may comprise two or more material moving cycles. The operation indicator may be or may indicate a duration of the identified material moving cycle, a number of material moving cycles performed by the mining machine during the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any other time period. The operation indicator may comprise any suitable one or more performance metrics indicating a status, a need for service, and other characteristics of the mining machine. The processing at block 620 may be performed similarly to the processing at block 320 of FIG.3 and the description at block 620 is applicable herein. [00173] At block 622, the method 600 comprises initiating an action in dependence on determining of the at least one operation indicator and/or in dependence on the identifying of the at least one material moving cycle. In some examples, initiating the action may comprise prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. The processing at block 622 may be performed similarly to the processing at block 322 of FIG.3 and the description at block 322 is applicable herein. [00174] FIG.7 illustrates an example of a method 700 that may be performed by a control system such as e.g. central control system 22 shown in FIGs.1A and 2. The central control system 22, which may be a traffic control system, comprises processing circuitry 24 e.g. at least one processor 24 that is configured to coordinate movements of one or more mining machines out of a plurality of mining machines in a mining site. The one or more of the mining machines e.g. a machine 12 may comprise a corresponding mining machine controller 28 and a field computer device 16 which may be part of the mining machine controller 28 or a separate hardware device. The field computer device 16 or the mining machine controller 28 may be configured to perform the method in accordance with examples of
Docket No.: PS56142PC00/ P23087WO01 the present disclosure. The field computer device 16 and/or the mining machine controller 28 is configured to communicate with the central control system 22 via a communication interface, e.g. to send various data to the central control system 22 and to receive control instructions and information from the central control system 22. [00175] At block 702, the central control system may receive, from the mining machine, information on the identified at least one material moving cycle and/or on the at least one operation indicator determined based on the identified at least one material moving cycle. [00176] At block 704, the central control system may generate at least one command or instruction to the mining machine in dependence on the received representation. The instruction may be to control the mining machine in dependence on the at least one operation indicator. In some examples, the instruction may instruct the mining machine to adjusting a target requirement for performance of the mining machine. [00177] At block 706, the control system may send the at least one instruction to the mining machine. The generating and sending of the instruction may be performed in the same processing step or block. [00178] FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a user interface of a computer device, in accordance with examples of the present disclosure. The user interface may be rendered on a display associated with the mining machine e.g. the field computer device and/or the mining machine controller, and/or on a display associated with the central control device, e.g., a traffic controller device or system. The information shown in FIGs.8, 9, 10, and 11 may be generated and presented as a result of identifying of one or more material moving cycle and determining at least one operator indicator based on the identified material moving cycle(s). The information may be displayed as part of initiating an action in dependence on determining of the at least one operation indicator in accordance with any methods of embodiments of the present disclosure. In the examples of FIGs.8-11, during each material moving cycle, a bucket of the material is moved. [00179] FIG.8 illustrates, for a day and for shifts A, B, and C of the day, a number of buckets with the material that have been moved during a particular shift. In this and other examples, a bucket refers to a completed material moving cycle. The amount of the moved material is also shown in FIG.8, in tons. [00180] FIG.9 illustrates, for a shift A in this example, a total productivity in a number of buckets, per hours. FIG.9 also shows expected production as a line 901 and cumulative production as a dotted line 903. [00181] FIG.10 illustrates, for a shift A, production per an operator of a corresponding mining machine in a mine, expressed as a number of buckets moved per a specific operator.
Docket No.: PS56142PC00/ P23087WO01 [00182] FIG.11 illustrates details on performance, by time, of a mining machine, as various parameters and information. Thus, FIG.11 shows a date, equipment, operator, extraction point, dumping or dump point, cycle time, and detail. [00183] The information that can be displayed on the user interface of the computer device, in accordance with examples of the present disclosure, may be used to assess performance of the mining machine, determine further actions to be taken, control the mining machine, adjust various features of the mine, and perform any other suitable actions. It should be appreciated that various other information may be displayed as examples of FIGs.8-11 are shown for illustration purposes only. [00184] In an aspect, a kit for installation on a mining machine is provided, the mining machine being configured to operate in a mining site. The kit, e.g. kit 18 shown in FIG.1A, may comprise at least one IMU sensor configured to be associated with mining machine, and computer program product comprising computer-executable instructions configured to, when executed by at least one processor, to perform any of the methods in accordance with embodiments of the present disclosure. The at least one processor may be a processor of a field computer device, a processor of a mining machine controller, or any other suitable processor. [00185] The kit may be installed or deployed on the mining machine such that the at least one IMU sensor may be coupled to the mining machine e.g. on a movable implement of the mining machine and/or in another location where the IMU sensor may record inertial sensor measurements indicative of the mining machine performing a material moving cycle. In some examples, the at least one IMU sensor comprises one i.e. single IMU sensor unit. Each IMU sensor unit may comprise an accelerometer, a gyroscope, and a magnetometer. In some examples, the kit may comprise one or more IMU sensor unit each comprising a three-axis accelerometer, a three-axis gyroscope, and a three- axis magnetometer. [00186] In some examples, the computer program product of the kit may be installed on the field computer device and/or the mining machine controller. Thus, the computer-executable instructions may be stored in memory of a field computer device e.g. field computer device 16 that may be provided as part of the kit. In some examples, the computer-executable instructions may be stored in memory of a controller e.g. mining machine controller 28. In some examples, the computer-executable instructions may be implemented as an application or app that can be installed in the memory of the mining machine controller 28. [00187] In some examples, the computer program product of the kit may be implemented as the field computer device such that the field computer device may be installed and deployed on the mining machine in addition to existing one or more controllers of the mining machine.
Docket No.: PS56142PC00/ P23087WO01 [00188] The computer-executable instructions, when executed by at least one processor, may cause the at least one processor to, as the mining machine is operating in the mining site, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. [00189] In some examples, the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [00190] In some examples, the computer-executable instructions, when executed by at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. The second trained machine-learning model may be trained in dependency on one or more features selected from time series data previously acquired by measuring signals from one or more position beacons. The one or more position beacons may comprise the at least one position beacon. [00191] In some examples, the computer-executable instructions, when executed by at least one processor, further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. [00192] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. [00193] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine.
Docket No.: PS56142PC00/ P23087WO01 [00194] In an aspect, a mining machine comprising the kit in accordance with examples of the present disclosure is provided. [00195] In an aspect, a mining machine for operation in a mining site is provided. The mining machine comprises a movable implement comprising a tool configured to load and unload material; at least one IMU sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site; and at least one processor and a memory comprising computer-executable instructions. The computer-executable instructions, when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site, receive the inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. [00196] In some examples, the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. [00197] In some examples, the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. [00198] In some examples, the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site, and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. [00199] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
Docket No.: PS56142PC00/ P23087WO01 [00200] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of obtaining an instruction to control the mining machine in dependence on the at least one operation indicator, providing the instruction to the mining machine, and adjusting a target requirement for performance of the mining machine. [00201] Operational steps described in any of the exemplary aspects herein are described to provide examples and discussion. The steps may be performed by hardware components, may be embodied in machine-executable instructions to cause a processor to perform the steps, or may be performed by a combination of hardware and software. Although a specific order of method steps may be shown or described, the order of the steps may differ. In addition, two or more steps may be performed concurrently or with partial concurrence. [00202] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including” when used herein specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. [00203] It will be understood that, although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the scope of the present disclosure. [00204] Relative terms such as “below” or “above” or “upper” or “lower” may be used herein to describe a relationship of one element to another element as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements present. [00205] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having
Docket No.: PS56142PC00/ P23087WO01 a meaning consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. [00206] It is to be understood that the present disclosure is not limited to the aspects described above and illustrated in the drawings; rather, the skilled person will recognize that many changes and modifications may be made within the scope of the present disclosure and appended claims. In the drawings and specification, there have been disclosed aspects for purposes of illustration only and not for purposes of limitation, the scope of the inventive concepts being set forth in the following claims.
Claims
Docket No.: PS56142PC00/ P23087WO01What is claimed is: 1. A computer-implemented method (300) for monitoring and controlling operation of a mining machine in a mining site, the method comprising, by a processor: as the mining machine is operating in the mining site, receiving (302) inertial sensor measurements acquired by at least one inertial measurement unit, IMU, sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; applying (312) a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determining (320) at least one operation indicator from the identified at least one material moving cycle; and initiating (322) an action in dependence on determining of the at least one operation indicator. 2. The method according to claim 1, wherein the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. 3. The method according to any of claims 1 to 2, comprising: determining (315) a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. 4. The method according to any of claims 2 to 3, further comprising receiving (503), over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site. 5. The method according to claim 4, comprising applying (514) a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
Docket No.: PS56142PC00/ P23087WO01 6. The method according to claim 5, wherein the second trained machine-learning model is trained in dependency on one or more features selected from time series data previously acquired by measuring signals from one or more position beacon. 7. The method according to any of claims 1 to 6, wherein the first trained machine-learning model is trained using inertial sensor measurements previously acquired by one or more IMU sensors. 8. The method according to any of claims 1 to 7, wherein initiating (322) the action in dependence on determining of the at least one operation indicator comprises, by the processor, prompting a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator. 9. The method according to any of claims 1 to 8, wherein initiating (322) the action in dependence on the determining of the at least one operation indicator comprises one or more of: obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; providing the instruction to the mining machine; and adjusting a target requirement for performance of the mining machine. 10. A computer device (16, 28) comprising at least one processor (210, 30) that is configured to perform the method according to any one of claims 1 to 9. 11. A computer program product comprising computer-executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method according to any of claims 1 to 9. 12. A tangible computer-readable storage medium, having stored thereon a computer program product comprising computer-executable instructions which, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 9. 13. A mining machine (12) for operation in a mining site, the mining machine (12) comprising: a movable implement (15) comprising a tool configured to load and unload material; at least one inertial measurement unit, IMU, (14) sensor associated with mining machine and configured to acquire inertial sensor measurements as the mining machine is operating in the mining site; and at least one processor (210, 30) and a memory (220, 31) comprising computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to:
Docket No.: PS56142PC00/ P23087WO01 as the mining machine is operating in the mining site, receive the inertial sensor measurements acquired by the at least one IMU sensor, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. 14. The mining machine (12) according to claim 13, wherein the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. 15. The mining machine (12) according to any of claims 13 to 14, wherein the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to determine a correctness of the identified at least one material moving cycle, wherein the at least one material moving cycle is determined to be correctly identified when the at least one material moving cycle has a duration that is within a threshold range and/or when the at least one material moving cycle is an operational cycle. 16. The mining machine (12) according to any of claims 14 to 15, wherein the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to: receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle. 17. The mining machine (12) according to any of claims 13 to 16, wherein initiating the action in dependence on the determining of the at least one operation indicator comprises prompting, by the at least one processor, a display of a representation of the identified at least one material moving cycle and/or a representation of the at least one operation indicator.
Docket No.: PS56142PC00/ P23087WO01 18. The mining machine (12) according to any of claims 13 to 17, wherein initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of: obtaining an instruction to control the mining machine in dependence on the at least one operation indicator; providing the instruction to the mining machine; and adjusting a target requirement for performance of the mining machine. 19. A kit (18) for installation on a mining machine (12) configured to operate in a mining site, the kit comprising: at least one inertial measurement unit, IMU, (14) sensor configured to be associated with mining machine; computer program product comprising computer-executable instructions configured to be installed on the mining machine, the computer-executable instructions, when executed by at least one processor, cause the at least one processor to: as the mining machine is operating in the mining site, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the mining machine, wherein the inertial sensor measurements are acquired over a first period of time; apply a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycle performed by the mining machine within the first period of time; determine at least one operation indicator from the identified at least one material moving cycle; and initiate an action in dependence on determining of the at least one operation indicator. 20. The kit (18) according to claim 19, wherein the material moving cycle comprises a sequence of operating states that the mining machine performs during the material moving cycle, the sequence of operating states comprising loading the mining machine with material at a loading point, moving the material by the mining machine from the loading point to a dumping point, unloading the material at the dumping point, and moving from the dumping point to the loading point or to another loading point. 21. The kit (18) according to any of claims 19 to 20, wherein the computer-executable instructions, when executed by the at least one processor, further cause the at least one processor to:
Docket No.: PS56142PC00/ P23087WO01 receive, over the first period of time, position sensor measurements acquired from at least one position beacon positioned in the mining site; and apply a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping point for the identified at least one material moving cycle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/SE2024/050696 WO2026019352A1 (en) | 2024-07-18 | 2024-07-18 | Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/SE2024/050696 WO2026019352A1 (en) | 2024-07-18 | 2024-07-18 | Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2026019352A1 true WO2026019352A1 (en) | 2026-01-22 |
Family
ID=92106878
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/SE2024/050696 Pending WO2026019352A1 (en) | 2024-07-18 | 2024-07-18 | Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2026019352A1 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210216889A1 (en) * | 2020-01-09 | 2021-07-15 | Caterpillar Inc. | Predicting Worksite Activities of Standard Machines Using Intelligent Machine Data |
| US20220003116A1 (en) * | 2019-02-15 | 2022-01-06 | Bewimo Pty Ltd | Improvements relating to underground mining |
| US20220099533A1 (en) * | 2020-09-25 | 2022-03-31 | Caterpillar Inc. | System and method for monitoring machine operations at a worksite |
| US20220335352A1 (en) * | 2021-04-15 | 2022-10-20 | Invento, Inc | System and method for managing construction and mining projects using computer vision, sensing and gamification |
-
2024
- 2024-07-18 WO PCT/SE2024/050696 patent/WO2026019352A1/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220003116A1 (en) * | 2019-02-15 | 2022-01-06 | Bewimo Pty Ltd | Improvements relating to underground mining |
| US20210216889A1 (en) * | 2020-01-09 | 2021-07-15 | Caterpillar Inc. | Predicting Worksite Activities of Standard Machines Using Intelligent Machine Data |
| US20220099533A1 (en) * | 2020-09-25 | 2022-03-31 | Caterpillar Inc. | System and method for monitoring machine operations at a worksite |
| US20220335352A1 (en) * | 2021-04-15 | 2022-10-20 | Invento, Inc | System and method for managing construction and mining projects using computer vision, sensing and gamification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10872302B2 (en) | Automatically determining construction worksite operational zones based on received construction equipment telemetry data | |
| CN111435464B (en) | Targeted testing and machine learning systems for detecting and identifying machine behavior | |
| WO2025018925A1 (en) | System, vehicle controller, traffic control system, mining vehicle, kit, and methods for monitoring and controlling vehicles in a mining environment | |
| US10163033B2 (en) | Vehicle classification and vehicle pose estimation | |
| CN114938670A (en) | Predicting worksite activity of a standard machine using intelligent machine data | |
| EP3303772B1 (en) | Adaptation of mining operations satellite coverage | |
| US11726007B2 (en) | System and method for monitoring machine operations at a worksite | |
| US10325424B2 (en) | Machine time usage determination system and method | |
| AU2019387326B2 (en) | Model generation for route planning or positioning of mobile object in underground worksite | |
| WO2026019352A1 (en) | Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment | |
| US20170185906A1 (en) | System and Method for Determining Machine Operational State | |
| WO2026019351A1 (en) | Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment | |
| US12400487B2 (en) | System and method for suggesting operational zones for a worksite on a device | |
| EP4459099A1 (en) | Controlling mesh installation based on a mesh deformation parameter | |
| EP4227869A1 (en) | Management systems for evaluation and continuous improvement of workflows involving heavy-duty vehicles | |
| AU2024332240A1 (en) | Computer implemented method and system for suppressing false dump events | |
| CN117868249A (en) | Computer-implemented method for determining that an attachment tool is attached to a stick or boom of a work machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24748463 Country of ref document: EP Kind code of ref document: A1 |