EP3660453A1 - Positioning of mobile object in underground worksite - Google Patents
Positioning of mobile object in underground worksite Download PDFInfo
- Publication number
- EP3660453A1 EP3660453A1 EP18209502.6A EP18209502A EP3660453A1 EP 3660453 A1 EP3660453 A1 EP 3660453A1 EP 18209502 A EP18209502 A EP 18209502A EP 3660453 A1 EP3660453 A1 EP 3660453A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- mobile object
- basis
- floor
- horizontal
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 31
- 239000002245 particle Substances 0.000 claims description 53
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 238000005259 measurement Methods 0.000 claims description 7
- 230000002265 prevention Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 13
- 238000010295 mobile communication Methods 0.000 description 7
- 239000011435 rock Substances 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 239000000463 material Substances 0.000 description 5
- 238000005065 mining Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000009412 basement excavation Methods 0.000 description 3
- 238000005553 drilling Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003014 reinforcing effect Effects 0.000 description 2
- 238000000342 Monte Carlo simulation Methods 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011960 computer-aided design Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000012517 data analytics Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 238000003801 milling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000009423 ventilation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/20—Control system inputs
- G05D1/22—Command input arrangements
- G05D1/228—Command input arrangements located on-board unmanned vehicles
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/60—Intended control result
- G05D1/646—Following a predefined trajectory, e.g. a line marked on the floor or a flight path
Definitions
- the present invention relates to positioning of a mobile object in an underground worksite.
- Underground worksites such as hard rock or soft rock mines, typically comprise a variety of operation zones intended to be accessed by different types of mobile work machines, herein referred to as mobile vehicles.
- An underground mobile vehicle may be an unmanned, e.g. remotely controlled from a control room, or a manned mobile vehicle, i.e. operated by an operator sitting in a cabin of the mobile vehicle.
- Mobile vehicles operating in underground work sites may be autonomously operating, i.e. automated or semi-automated mobile vehicles, which in their normal operating mode operate independently without external control but which may be taken under external control at certain operation areas or conditions, such as during states of emergencies.
- Location tracking of mobile objects, such as mobile vehicles and persons is required at many worksites.
- WO2015106799 discloses a system for scanning surroundings of a vehicle for producing data to determining position and orientation of the vehicle.
- the vehicle is provided with a reference point cloud data of the mine.
- the control unit is configured to match second point cloud data produced by a scanning device of the vehicle to the reference point cloud data in order to determine position data of the vehicle.
- an apparatus comprising: means for determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object, determining horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and a three-dimensional model of the tunnel.
- the means comprises at least one processor; and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.
- a method for mobile object positioning in underground worksite comprising: determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object, determining horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and a three-dimensional model of the tunnel.
- an apparatus comprising at least one processing core, at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processing core, cause the apparatus at least to carry out the method or an embodiment of the method.
- a computer program or a computer program storage medium comprising code for, when executed in a data processing apparatus, to cause the method of the second aspect or any embodiment thereof.
- the three-dimensional position indicator is generated on the basis of a sub-set of three-dimensional floor point locations within a threshold distance.
- the three-dimensional model comprises three-dimensional point cloud data generated on the basis of scanning the tunnel and the floor model comprises a sub-set of points extracted from the three-dimensional model.
- the mobile object is a vehicle
- the horizontal progression of the vehicle is determined on the basis of a dead reckoning algorithm configured to accumulate the vehicle's travelled distance and heading on the basis of an input signal indicative of vehicle wheel rotation and relative heading.
- proximity of the mobile object to a location reference unit is determined, and the position estimate is updated on the basis of location of the location reference unit and estimated distance of the mobile object to the location reference unit.
- the location reference unit may be a wireless signal emission unit at a tunnel wall, a vehicle location tracking unit of a vehicle, or a pedestrian location tracking unit a carried by a pedestrian.
- the apparatus is a server or part of a control system configured to visualize the at least one monitored feature on at least one display device.
- Figure 1 illustrates an underground worksite 1 comprising a network 2 of underground tunnels (illustrated without roof).
- a plurality of mobile objects such as persons or pedestrians 3 and/or mobile vehicles 4, 5, 6, 7 may be present in and move between different areas or operation zones of the worksite 1.
- underground worksite herein is intended to include a variety of underground worksites, including for example different kinds of underground excavation worksites, such as mines, roadwork sites, and railroad worksites.
- mobile object in this specification and claims is intended to include all mobile objects which may have an access into an operation zone of a worksite, such as mobile vehicles and human beings being at work in the worksite.
- mobile vehicle herein refers generally to mobile work machines suitable to be used in the operation of different kinds of underground mining or construction excavation worksites, such as lorries, dumpers, vans, mobile rock drilling or milling rigs, mobile reinforcement machines, and bucket loaders or other kind of mobile work machines which may be used in different kinds of excavation worksites.
- the mobile vehicles may be autonomously operating mobile vehicles, which herein refers to automated or semi-automated mobile vehicles.
- the worksite 1 comprises a communications system, such as a wireless access system comprising a wireless local area network (WLAN), comprising a plurality of wireless access nodes 8.
- the access nodes 8 may communicate with wireless communications units comprised by the vehicles or carried by the pedestrians and with further communications devices 9, such as network device(s) configured to facilitate communications with an on-site (underground or above-ground) or remote control and/or monitoring system server.
- WLAN wireless local area network
- the worksite 1 may further comprise various other entities not shown in Figure 1 , such as various system elements for power supply, ventilation, communications, and automation.
- the worksite may comprise a passage control system comprising passage control units (PCU) separating operation zones, some of which may be for autonomously operating vehicles.
- PCU passage control units
- the passage control system and associated PCUs may be configured to allow or prevent movement of one or more vehicles and/or pedestrians between zones.
- the mobile object 3-7 may comprise a sensor device configured to provide information of at least horizontal movement of the mobile object.
- a pedestrian 3 may be equipped with a sensor device 10 for movement tracking 11, such as a pedestrian dead reckoning (PDR) capable device, such as an inertial measurement unit (IMU).
- PDR refers generally to the process for determining the traveled distance of a pedestrian by counting steps based on accelerometer impacts, along with an estimate of the user's step length.
- the PDR process may be configured to learn based on the movement of the mobile object and adapt the step length.
- the direction of travel also known as heading, may also be included in the definition of dead reckoning, i.e.
- the sensor device 10 may be connected to a mobile communications device 20, such as a personal mobile phone or other type of user device, configured to communicate with the access node 8.
- the sensor device 10 may also communicate with the access node 8 directly or by a proxy of another sensor device 10. It will be appreciated that the sensor device 10 and the mobile communications device 20 may be integrated as single device.
- Figure 3 illustrates a method for mobile object positioning in an underground worksite.
- the method may be implemented by at least one device of a mobile object positioning system, such as a worksite server or other control device, a mobile unit such as a mobile communications device carried by the pedestrian 3 in the tunnel, or an vehicle on-board control device.
- a mobile object positioning system such as a worksite server or other control device, a mobile unit such as a mobile communications device carried by the pedestrian 3 in the tunnel, or an vehicle on-board control device.
- Horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object is determined 30.
- the horizontal progression is determined based on information received by a signal indicating length and direction changes by a sensor unit of the mobile unit, such as an IMU carried out by a pedestrian.
- Horizontal position of the mobile object is determined 31 on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object.
- a 3D position indicator is generated 32 on the basis of the horizontal position of the mobile object and a 3D model of the tunnel.
- the 3D position indicator may refer to a data record comprising horizontal and vertical (x, y, z) position value stored in a memory and/or to a display indication.
- the term floor model refers generally to a model comprising a set of points indicative of the tunnel floor at least in horizontal plane, i.e. 2D or x, y coordinates. Such points may also be referred to as floor points. It is to be noted that the floor model may be in the form of a point cloud or mesh representation, for example. The floor model may be applied as a map for the mobile object movement tracking as presently disclosed, and the floor points may be considered as map points.
- the floor model may comprise also vertical plane, i.e. height or z coordinate data and/or supplementary data for at least some of the floor points.
- the floor model does not necessarily define the absolutely lowest bottom level of the underground tunnel, but it may instead be more feasible to extract the floor model and floor points defining accessible areas at some height from the tunnel bottom.
- the floor model may include all points where it is possible for the tracked object to be, and exclude all locations where it is impossible for the object to be located. For example, a person can be located on the floor of the mine, but cannot be located inside the rock or walls of the mine.
- the floor model may be automatically generated by detecting and extracting a sub-set of (floor) points from the 3D model of the tunnel.
- the extraction of the floor points may comprise calculating surface normal for each of points in a sub-set of points of the points defined in the 3D model, and selecting the floor points on the basis of e.g. the surface normal directions of the points in the sub-set of points of the 3D model.
- the 3D model of the tunnel may comprise 3D point cloud data, which may be received as an output of scanning the tunnel by a surveyor or by a mining vehicle.
- the 3D model may be or generated on the basis of a design model, such as a CAD model, created by a mine designing software or a 3D model created on the basis of tunnel lines and profiles designed in a drill and blast design software, such as iSURE®.
- a design model such as a CAD model
- a mine designing software or a 3D model created on the basis of tunnel lines and profiles designed in a drill and blast design software, such as iSURE®.
- mobile object movement tracking may be performed on the basis of the floor model comprising the extracted set of points of an initial 3D model of the tunnel environment.
- Figure 3 illustrates general features related to the underground mobile object position tracking and various additions and amendments may be applied, some further embodiments being illustrated below.
- block 31 is based on particle filtering and comprises:
- the fusing comprises blocks 42 to 44:
- the updated positions of the particles are compared to a sub-set of the floor point locations.
- particles that have no floor points within a threshold distance are filtered out. This indicates that the estimation has become impossible, for example object moved through a wall.
- block 44 a location of a closest floor point within a threshold distance is assigned for each particle. It is to be noted that in case of 2D implementations, block 44 may be omitted.
- the location estimate particles of the mobile object may be updated 41 only with horizontal direction estimate and movement (step or wheel rotation) length estimate.
- all particles have an initial location in 3D (x, y, z) and are based on the estimated horizontal progression moved to new locations in xy plane while the z plane/position remains unchanged.
- their updated positions are compared 42 to the positions of the floor model. For each particle in the set, a floor point closest to the respective particle is determined and the particle is moved to, or assigned with the closest floor point location, thus causing the particles to move along the floor model.
- the 3D position indicator may be selected on the basis of (and among) a sub-set of 3D floor point locations within a threshold distance from a reference point, such as previous 3D position or the horizontal position defined in block 31.
- the threshold distance may be set for z plane and floor points only close enough to the previous 3D position indicator's z plane position are considered. This reduces processing and enables limiting the search into a relevant sub-set of floor model data, thus also facilitating avoiding problems caused by overlapping tunnels.
- the comparison in block 42 and/or 44 to the floor model is carried out in 3D, i.e. the particle locations are compared to the floor model points in horizontal and vertical planes. If no floor model points are at given 3D radius, the particle is considered to be outside of the floor model (and map) and is removed.
- the positioning system is capable to ensure both that the particle does not go out of tunnel in horizontal direction but does not also jump too much in height direction. If for example two tunnels go above each other, this prevents the particle from jumping from one tunnel to the other which would be problematic in case of 2D maps.
- the particle is moved to the closest map point making it move along the map in 3D while not jumping to above or below tunnels.
- a floor model or map matching algorithm may thus be applied, correcting the dead-reckoning based estimate by fitting the shape of the estimated trajectory to the shape of the areas defined accessible in 3D in the floor model.
- the method thus uses the floor model of the underground space.
- the point cloud of the floor or road can be automatically extracted from a full 3D survey point cloud.
- Particle movement calculation may thus be simplified and done in 2D while the results are transferred to 3D.
- Such method may also be referred to as mobile object dead reckoning along a 3D map.
- the solution does not require for example altitude meters or similar additional sensors. It is adequate to equip the mobile object 3, 4-7 only with a sensor device capable for step detection or wheel rotation measurement with a relative heading estimate and optionally a length estimate in horizontal direction.
- the presently disclosed features for mobile object location tracking hence facilitate less complicated calculations and thus faster position updating or reduce required computational resources. As compared to position update computation completely in a 3D environment, further error caused with vertical dimension calculation may be avoided. Further, since the floor model comprising the directly extracted set of initial 3D model points is applied, a special type of map need not to be generated and the features may be applied directly on point clouds or surface meshes with no special map generation or vectorization needed.
- proximity of the mobile object 3-7 to a location reference unit is detected.
- the horizontal position and/or the 3D position indicator of the mobile object may be updated on the basis of location of the location reference unit and estimated distance of the mobile object to the location reference unit.
- the location reference unit is a mobile object, such as a sensor device carried by a pedestrian 3 or a vehicle positioning unit.
- the location of the reference unit may be received from the reference unit or a system maintaining up-to-date position information based on information from location reference unit(s).
- location information of a plurality of reference units is applied for updating the 3D position indicator.
- the location reference unit may be a wireless signal emission unit at a tunnel wall, a mining vehicle location tracking unit of a mining vehicle 4-7, or a PDR or another position tracking unit a carried by a pedestrian 3.
- An RF tag, an access point, a visually readable code or another fixed unit the location of which is accurately known may serve as the location reference.
- US7899599 disclosing that such identifier may be applied to update dead reckoning based location.
- Bluetooth or Bluetooth Low Energy based communication is applied for the location reference unit detection and location.
- the 3D position indicator may be updated in response to detecting the mobile object 3-7 at (the proximity of) the fixed location reference unit.
- the horizontal position obtained on the basis of dead reckoning or the particle states are updated based on the received location reference information.
- the mobile communications device 20 may be configured to monitor visible Bluetooth devices in the environment.
- Known beacons or vehicles may be detected based on the MAC address of the Bluetooth devices, while disregarding all other Bluetooth devices. This facilitates to initialize the location tracking or starting position for the mobile object on the basis of known beacon locations in access points to mine, initializing the location information in case of errors in strategic points in mine, detecting person to person interaction, detecting person to vehicle interaction, and/or resetting person's location to vehicle location in case person vehicle interaction is detected.
- a vehicle may report a detected Bluetooth address belonging to a person.
- the person's location may be updated to correspond to the location of the vehicle.
- An additional status may be added for the person location information to indicate that the person is at vehicle with an identifier 'xxx'.
- the position of the mobile object may be updated on the basis of the reference unit position and estimated distance to the reference unit. The method may be applied to update location of all persons traveling in a car in the worksite as well as walking past vehicles.
- the mobile object is a pedestrian 3.
- the horizontal progression of the pedestrian may be estimated on the basis of a dead reckoning algorithm configured to accumulate the pedestrian's travelled distance and heading on the basis of an input signal indicative of pedestrian steps by an IMU.
- the IMU may be configured to perform processing of raw data from sensors and generate the signal indicating estimated travelled distance and heading change is sent to a location tracking unit, such as a server, carrying out the method of Figure 3 .
- the signal may be transmitted at set time intervals of in response to detecting a step.
- PDR and a particle filter (PF) is provided below.
- PDR is achieved by utilizing the kinematics of human gait.
- the pedestrian's step count, step length and heading are estimated and, after that, dead reckoning algorithm is used to provide the current location of a pedestrian.
- Heading i.e., yaw angle, ⁇
- ⁇ is estimated using gyroscopes and/or digital compasses (magnetometers).
- full 3-dimensional attitude of the IMU typically needs to be estimated. This can be done by using rotation matrices or quaternions, for example.
- attitude estimation we can simplify the problem by writing that we have (virtual) gyroscope reading, ⁇ , measuring the yaw angle.
- step detection There exist various different methods for step detection, including peak detection, zero-crossing, autocorrelation, fast Fourier transform.
- E k and N k are the East and North components, respectively, and ⁇ S k is the estimated step length of the step k.
- PDR is a relative position method
- the initial position of the pedestrian must be also known by some means. For example, this may be done by integrating PDR with GNSS in outdoors and WLAN-based positioning in indoors where the initial or reference position can be acquired.
- PF is applied to fuse the map information and the PDR output.
- Basic idea of the particle filter is based Monte Carlo simulation. This means that position solution distributions are presented as random samples. Each sample (i.e. in this case position) has also weight, which presents the likelihood of the sample. When the sample (position) is propagated using PDR method and position ends up going through wall or inside wall, the weight of the respective sample is decreased.
- Each map point pi should present only the floor points of the tunnel (i.e., areas where people are able to walk) and map points should be spaced with appropriate spacing, such as spacing selected in the range of 0.2 to 1 meter, in an embodiment approximately 0.5 meter spacing.
- spacing selected in the range of 0.2 to 1 meter, in an embodiment approximately 0.5 meter spacing.
- distance between points two closest points pi and pj should be at maximum 0.5 meters but smaller than 0.4 meters. The maximum and minimum values may be varied, but larger number of map points directly affects calculation time/computational load of algorithm.
- ⁇ heading (direction of travel of a pedestrian)
- b bias of the gyroscope (or virtual gyroscope) measurement.
- w i for particle i.
- the particle filter has also a resampling phase, which avoids degeneracy problem, where only a few of the particles will have a significant weight.
- the mobile object is a vehicle 4-7
- the horizontal progression of the vehicle is estimated on the basis of a dead reckoning algorithm configured to accumulate the vehicle's travelled distance and heading on the basis of an input signal indicative of vehicle wheel rotation and relative heading.
- the vehicle may comprise a positioning unit configured to perform at least some of the presently disclosed features.
- a dead reckoning unit of the vehicle transmits a signal indicative of the estimated position to a server or another unit configured to perform at least the method of Figure 3 .
- the system may comprise further operational modules supplementing dead reckoning based position tracking, such as a tyre slipping and/or wear compensation module.
- a location tracking kit comprising the dead-reckoning unit is attachable to a vehicle when taking into use underground.
- the vehicle 4-7 provided with a scanning device is serving as a mobile surveying device.
- the vehicle may execute the surveying continuously when carrying out dedicated normal operations of the vehicle. If the vehicle is a rock drilling rig or a reinforcing rig, it may scan the surroundings when it stops at a work site for executing drilling or feeding reinforcing elements or material. It may also be defined that the scanning is executed at least once each time when the vehicle is not moving. Thanks to this procedure, the mine may be surveyed repeatedly and in parallel to the normal operational process without any need for extra resources. The 3D model of the mine may thus be accurate and updated.
- the vehicle 4-7 may be a semi-autonomous or autonomous vehicle and comprise a control unit with a collision prevention feature.
- the collision prevention system may prevent collision to surrounding surfaces such as rock walls.
- the collision prevention system may prevent collision to other vehicles, other booms, auxiliary devices, rock blocks, persons or any other physical objects which may be located close to the vehicle or are entering to the proximity.
- the 3D position indicator may be applied in various ways, only some examples being illustrated herein.
- the mobile object is displayed based on the 3D position indicator on a 3D map based on the 3D model.
- the 3D position indicator is provided as an input for a collision avoidance system. This facilitates to prepare for a probable or possible collision risk beyond line of sight.
- the 3D position indicator is provided as an input for updating position of other mobile object(s).
- Figure 6 illustrates an example of a system for underground worksite.
- the system comprises a wireless access network 60 comprising a plurality of access nodes 8 for wireless communication with communication devices 10 of mobile objects 3-7 in the tunnels.
- the system comprises a server 61, which may comprise one or more above or underground computing units.
- the server 61 is configured to perform at least some of the above illustrated features related to mobile object positioning, such as the methods of Figures 3 and 4 on the basis of signals received from mobile object(s) via the access network 60.
- Figure 6 illustrates operational modules 62-68 of the server 61 according to some embodiments.
- An object tracking module 63 is configured to perform the method of Figure 3 and provide the generated 3D position indicator to further modules, in some embodiments to a position service module 62.
- the server 61 may comprise a task manager or management module 64, which is configured to manage at least some operations at the worksite.
- the task manager may be configured to assign work tasks for a fleet of vehicles and update and/or monitor task performance and status, which is indicated at a task management GUI.
- the server 61 may comprise a model processing module 65, which may maintain one or more models of the underground worksite, such as the 3D model.
- the model processing module 65 is configured to extract the floor model and store it to the database or storage 67.
- the server 61 may comprise a visualizer GUI module 66, which is configured to generate at least some display views for an operator (locally and/or remotely).
- the visualizer GUI module 66 is configured to generate, on the basis of the 3D model or floor model, a 3D (and/or 2D) view indicating the current position of the mobile object on the basis of the 3D indicator generated in block 32.
- the server 61 may comprise further module(s) 68, such as a remote monitoring process and UI, and/or a cloud dispatcher component configured to provide selected worksite information, such as the mobile object position information to a cloud service.
- module(s) 68 such as a remote monitoring process and UI
- cloud dispatcher component configured to provide selected worksite information, such as the mobile object position information to a cloud service.
- the system and server 61 may be connected to a further system 70 and/or network 69, such a worksite management system, a cloud service, an intermediate communications network, such as the internet, etc.
- the system may further comprise or be connected to a further device or control unit, such as a handheld user unit, a vehicle unit, a worksite management device/system, a remote control and/or monitoring device/system, data analytics device/system, sensor system/device, etc.
- the object tracking 63 may be implemented as part of another module, such as the position service module 62.
- the position service 62 is configured to provide, upon request or by push transmission, mobile object position information obtained from or generated on the basis of information from the object tracking 63 for relevant other modules or functions, such as the database 67, the visualizer graphical user interface 66, and/or remote units or systems 70 via one or more networks 69.
- relevant other modules or functions such as the database 67, the visualizer graphical user interface 66, and/or remote units or systems 70 via one or more networks 69.
- the modules are illustrated as inter-connected, but it is to be appreciated that not all modules need to be connectable.
- the system may comprise or be connected to a vehicle control unit or module provided with the 3D position indicator.
- vehicle control unit may be provided in each autonomously operating vehicle and be configured to control at least some autonomous operations of the vehicle on the basis of their 3D location indicators. For example, in response to detecting a person to enter a zone comprising an autonomously operating vehicle, the control unit may be configured to send a control command to stop the vehicle.
- An electronic device comprising electronic circuitries may be an apparatus for realizing at least some embodiments of the present invention, such as the main operations illustrated in connection with Figure 3 .
- the apparatus may be comprised in at least one computing device connected to or integrated into a control system which may be part of a worksite control or automation system.
- Figure 7 illustrates an example apparatus capable of supporting at least some embodiments of the present invention. Illustrated is a device 80, which may be configured to carry out at least some of the embodiments relating to the mobile object position tracking illustrated above.
- the device 80 comprises or implements the server 61 and/or the object tracking module 63 of Figure 6 .
- the device is comprised or carried by the mobile object 3-7, such as a mobile communications device or a vehicle control unit, configured to carry out at least some of the embodiments relating to the mobile object position tracking illustrated above.
- a processor 81 which may comprise, for example, a single- or multi-core processor.
- the processor 81 may comprise more than one processor.
- the processor may comprise at least one application-specific integrated circuit, ASIC.
- the processor may comprise at least one field-programmable gate array, FPGA.
- the processor may be configured, at least in part by computer instructions, to perform actions.
- the device 80 may comprise memory 82.
- the memory may comprise random-access memory and/or permanent memory.
- the memory may be at least in part accessible to the processor 81.
- the memory may be at least in part comprised in the processor 81.
- the memory may be at least in part external to the device 80 but accessible to the device.
- the memory 82 may be means for storing information, such as parameters 84 affecting operations of the device.
- the parameter information in particular may comprise parameter information affecting the mobile object positioning, such as threshold values and timing parameters.
- the memory 82 may comprise computer program code 83 including computer instructions that the processor 81 is configured to execute.
- computer instructions configured to cause the processor to perform certain actions are stored in the memory, and the device in overall is configured to run under the direction of the processor using computer instructions from the memory, the processor and/or its at least one processing core may be considered to be configured to perform said certain actions.
- the processor may, together with the memory and computer program code, form means for performing at least some of the above-illustrated method steps in the device.
- the device 80 may comprise a communications unit 85 comprising a transmitter and/or a receiver.
- the transmitter and the receiver may be configured to transmit and receive, respectively, information in accordance with at least one cellular or non-cellular standard.
- the transmitter and/or receiver may be configured to operate in accordance with global system for mobile communication, GSM, wideband code division multiple access, WCDMA, long term evolution, LTE, 3GPP new radio access technology (N-RAT), wireless local area network, WLAN, Ethernet and/or worldwide interoperability for microwave access, WiMAX, standards, for example.
- the device 80 may comprise a near-field communication, NFC, transceiver.
- the NFC transceiver may support at least one NFC technology, such as NFC, Bluetooth, or similar technologies.
- the device 80 may comprise or be connected to a UI.
- the UI may comprise at least one of a display 86, a speaker, an input device 87 such as a keyboard, a joystick, a touchscreen, and/or a microphone.
- the UI may be configured to display views on the basis of the worksite model(s) and the mobile object position indicators.
- a user may operate the device and control at least some features of a control system, such as the system illustrated in Figure 6 .
- the user may control a vehicle 4-7 and/or the server via the UI, for example to change operation mode, change display views, modify parameters 84 in response to user authentication and adequate rights associated with the user, etc.
- the device 80 may further comprise and/or be connected to further units, devices and systems, such as one or more sensor devices 88 sensing environment of the device 80.
- the sensor device may comprise an IMU or another type of sensor device configured to determine movements of a mobile object. For example, heading information may be obtained directly from an electronic compass.
- the processor 81, the memory 82, the communications unit 85 and the UI may be interconnected by electrical leads internal to the device 80 in a multitude of different ways.
- each of the aforementioned devices may be separately connected to a master bus internal to the device, to allow for the devices to exchange information.
- this is only one example and depending on the embodiment various ways of interconnecting at least two of the aforementioned devices may be selected without departing from the scope of the present invention.
- At least some embodiments of the present invention find industrial application at least in underground mining.
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Business, Economics & Management (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Game Theory and Decision Science (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Navigation (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
Description
- The present invention relates to positioning of a mobile object in an underground worksite.
- Underground worksites, such as hard rock or soft rock mines, typically comprise a variety of operation zones intended to be accessed by different types of mobile work machines, herein referred to as mobile vehicles. An underground mobile vehicle may be an unmanned, e.g. remotely controlled from a control room, or a manned mobile vehicle, i.e. operated by an operator sitting in a cabin of the mobile vehicle. Mobile vehicles operating in underground work sites may be autonomously operating, i.e. automated or semi-automated mobile vehicles, which in their normal operating mode operate independently without external control but which may be taken under external control at certain operation areas or conditions, such as during states of emergencies. Location tracking of mobile objects, such as mobile vehicles and persons is required at many worksites.
-
WO2015106799 discloses a system for scanning surroundings of a vehicle for producing data to determining position and orientation of the vehicle. The vehicle is provided with a reference point cloud data of the mine. The control unit is configured to match second point cloud data produced by a scanning device of the vehicle to the reference point cloud data in order to determine position data of the vehicle. - The invention is defined by the features of the independent claims. Some specific embodiments are defined in the dependent claims.
- According to a first aspect of the present invention, there is provided an apparatus, comprising: means for determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object, determining horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and a three-dimensional model of the tunnel.
- In some embodiments, the means comprises at least one processor; and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.
- According to a second aspect of the present invention, there is provided a method for mobile object positioning in underground worksite, comprising: determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object, determining horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and a three-dimensional model of the tunnel.
- According to a third aspect, there is provided an apparatus comprising at least one processing core, at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processing core, cause the apparatus at least to carry out the method or an embodiment of the method.
- According to a fourth aspect, there is provided a computer program or a computer program storage medium comprising code for, when executed in a data processing apparatus, to cause the method of the second aspect or any embodiment thereof.
- According to an embodiment of any of the aspects, the three-dimensional position indicator is generated on the basis of a sub-set of three-dimensional floor point locations within a threshold distance.
- According to an embodiment of any of the aspects, the three-dimensional model comprises three-dimensional point cloud data generated on the basis of scanning the tunnel and the floor model comprises a sub-set of points extracted from the three-dimensional model.
- According to an embodiment of any of the aspects, the mobile object is a vehicle, and the horizontal progression of the vehicle is determined on the basis of a dead reckoning algorithm configured to accumulate the vehicle's travelled distance and heading on the basis of an input signal indicative of vehicle wheel rotation and relative heading.
- According to an embodiment of any of the aspects, proximity of the mobile object to a location reference unit is determined, and the position estimate is updated on the basis of location of the location reference unit and estimated distance of the mobile object to the location reference unit. For example, the location reference unit may be a wireless signal emission unit at a tunnel wall, a vehicle location tracking unit of a vehicle, or a pedestrian location tracking unit a carried by a pedestrian.
- According to an embodiment of any of the aspects, the apparatus is a server or part of a control system configured to visualize the at least one monitored feature on at least one display device.
-
-
FIGURE 1 illustrates an example of an underground work site; -
FIGURE 2 illustrates pedestrian tracking in underground tunnel; -
FIGURES 3 and4 illustrate methods according to at least some embodiments; -
FIGURE 5 illustrates particle map fusion; -
FIGURE 6 illustrates an example system according to at least some embodiments; and -
FIGURE 7 illustrate an apparatus capable of supporting at least some embodiments. -
Figure 1 illustrates anunderground worksite 1 comprising anetwork 2 of underground tunnels (illustrated without roof). A plurality of mobile objects, such as persons orpedestrians 3 and/ormobile vehicles worksite 1. - The term underground worksite herein is intended to include a variety of underground worksites, including for example different kinds of underground excavation worksites, such as mines, roadwork sites, and railroad worksites. The term mobile object in this specification and claims is intended to include all mobile objects which may have an access into an operation zone of a worksite, such as mobile vehicles and human beings being at work in the worksite. The term mobile vehicle herein refers generally to mobile work machines suitable to be used in the operation of different kinds of underground mining or construction excavation worksites, such as lorries, dumpers, vans, mobile rock drilling or milling rigs, mobile reinforcement machines, and bucket loaders or other kind of mobile work machines which may be used in different kinds of excavation worksites. The mobile vehicles may be autonomously operating mobile vehicles, which herein refers to automated or semi-automated mobile vehicles.
- The
worksite 1 comprises a communications system, such as a wireless access system comprising a wireless local area network (WLAN), comprising a plurality ofwireless access nodes 8. Theaccess nodes 8 may communicate with wireless communications units comprised by the vehicles or carried by the pedestrians and withfurther communications devices 9, such as network device(s) configured to facilitate communications with an on-site (underground or above-ground) or remote control and/or monitoring system server. - The
worksite 1 may further comprise various other entities not shown inFigure 1 , such as various system elements for power supply, ventilation, communications, and automation. For example, the worksite may comprise a passage control system comprising passage control units (PCU) separating operation zones, some of which may be for autonomously operating vehicles. The passage control system and associated PCUs may be configured to allow or prevent movement of one or more vehicles and/or pedestrians between zones. - The mobile object 3-7 may comprise a sensor device configured to provide information of at least horizontal movement of the mobile object. With reference to the example of
Figure 2 , apedestrian 3 may be equipped with asensor device 10 formovement tracking 11, such as a pedestrian dead reckoning (PDR) capable device, such as an inertial measurement unit (IMU). PDR refers generally to the process for determining the traveled distance of a pedestrian by counting steps based on accelerometer impacts, along with an estimate of the user's step length. In some embodiments, the PDR process may be configured to learn based on the movement of the mobile object and adapt the step length. The direction of travel, also known as heading, may also be included in the definition of dead reckoning, i.e. distance added with direction is combined to a displacement vector and those vectors are summed. Thesensor device 10 may be connected to amobile communications device 20, such as a personal mobile phone or other type of user device, configured to communicate with theaccess node 8. Thesensor device 10 may also communicate with theaccess node 8 directly or by a proxy ofanother sensor device 10. It will be appreciated that thesensor device 10 and themobile communications device 20 may be integrated as single device. - A problem with known PDR based methods is that they only work in 2D. In 3D environment the calculation would be complicated. There is now provided an improved system for tracking position of a mobile object.
-
Figure 3 illustrates a method for mobile object positioning in an underground worksite. The method may be implemented by at least one device of a mobile object positioning system, such as a worksite server or other control device, a mobile unit such as a mobile communications device carried by thepedestrian 3 in the tunnel, or an vehicle on-board control device. - Horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object is determined 30. In some embodiments, the horizontal progression is determined based on information received by a signal indicating length and direction changes by a sensor unit of the mobile unit, such as an IMU carried out by a pedestrian.
- Horizontal position of the mobile object is determined 31 on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object. A 3D position indicator is generated 32 on the basis of the horizontal position of the mobile object and a 3D model of the tunnel.
- The 3D position indicator may refer to a data record comprising horizontal and vertical (x, y, z) position value stored in a memory and/or to a display indication. The term floor model refers generally to a model comprising a set of points indicative of the tunnel floor at least in horizontal plane, i.e. 2D or x, y coordinates. Such points may also be referred to as floor points. It is to be noted that the floor model may be in the form of a point cloud or mesh representation, for example. The floor model may be applied as a map for the mobile object movement tracking as presently disclosed, and the floor points may be considered as map points. The floor model may comprise also vertical plane, i.e. height or z coordinate data and/or supplementary data for at least some of the floor points. It is to be appreciated that the floor model does not necessarily define the absolutely lowest bottom level of the underground tunnel, but it may instead be more feasible to extract the floor model and floor points defining accessible areas at some height from the tunnel bottom. The floor model may include all points where it is possible for the tracked object to be, and exclude all locations where it is impossible for the object to be located. For example, a person can be located on the floor of the mine, but cannot be located inside the rock or walls of the mine.
- The floor model may be automatically generated by detecting and extracting a sub-set of (floor) points from the 3D model of the tunnel. The extraction of the floor points may comprise calculating surface normal for each of points in a sub-set of points of the points defined in the 3D model, and selecting the floor points on the basis of e.g. the surface normal directions of the points in the sub-set of points of the 3D model.
- The 3D model of the tunnel may comprise 3D point cloud data, which may be received as an output of scanning the tunnel by a surveyor or by a mining vehicle. In other embodiments, the 3D model may be or generated on the basis of a design model, such as a CAD model, created by a mine designing software or a 3D model created on the basis of tunnel lines and profiles designed in a drill and blast design software, such as iSURE®. Thus, mobile object movement tracking may be performed on the basis of the floor model comprising the extracted set of points of an initial 3D model of the tunnel environment.
- It will be appreciated that
Figure 3 illustrates general features related to the underground mobile object position tracking and various additions and amendments may be applied, some further embodiments being illustrated below. - With reference to
Figure 4 , in some embodiments block 31 is based on particle filtering and comprises: - establishing 40 a set of particles representative of position estimate options for the mobile object, the particles being associated with a position value,
- updating 41 the position of each of the particles on the basis of the estimated horizontal progression of the mobile object, and
- fusing the updated positions of the particles with the floor model.
- In some embodiments, the fusing comprises
blocks 42 to 44: Inblock 42 the updated positions of the particles are compared to a sub-set of the floor point locations. Inblock 43 particles that have no floor points within a threshold distance are filtered out. This indicates that the estimation has become impossible, for example object moved through a wall. In block 44 a location of a closest floor point within a threshold distance is assigned for each particle. It is to be noted that in case of 2D implementations, block 44 may be omitted. - All calculation for the particle movement may thus be carried out in 2D. The location estimate particles of the mobile object may be updated 41 only with horizontal direction estimate and movement (step or wheel rotation) length estimate.
- In some embodiments, all particles have an initial location in 3D (x, y, z) and are based on the estimated horizontal progression moved to new locations in xy plane while the z plane/position remains unchanged. After all particles have been moved (in 2D), their updated positions are compared 42 to the positions of the floor model. For each particle in the set, a floor point closest to the respective particle is determined and the particle is moved to, or assigned with the closest floor point location, thus causing the particles to move along the floor model.
- The 3D position indicator may be selected on the basis of (and among) a sub-set of 3D floor point locations within a threshold distance from a reference point, such as previous 3D position or the horizontal position defined in
block 31. For example, the threshold distance may be set for z plane and floor points only close enough to the previous 3D position indicator's z plane position are considered. This reduces processing and enables limiting the search into a relevant sub-set of floor model data, thus also facilitating avoiding problems caused by overlapping tunnels. - According to some embodiments, the comparison in
block 42 and/or 44 to the floor model is carried out in 3D, i.e. the particle locations are compared to the floor model points in horizontal and vertical planes. If no floor model points are at given 3D radius, the particle is considered to be outside of the floor model (and map) and is removed. By carrying out such map comparison in 3D the positioning system is capable to ensure both that the particle does not go out of tunnel in horizontal direction but does not also jump too much in height direction. If for example two tunnels go above each other, this prevents the particle from jumping from one tunnel to the other which would be problematic in case of 2D maps. The particle is moved to the closest map point making it move along the map in 3D while not jumping to above or below tunnels. A floor model or map matching algorithm may thus be applied, correcting the dead-reckoning based estimate by fitting the shape of the estimated trajectory to the shape of the areas defined accessible in 3D in the floor model. - In the example of
Figure 5 , the updated position of theupper particle 50 is acceptable but thelower particle 51 goes outside the floor model and must be removed as faulty. - The method thus uses the floor model of the underground space. The point cloud of the floor or road can be automatically extracted from a full 3D survey point cloud. Particle movement calculation may thus be simplified and done in 2D while the results are transferred to 3D. Such method may also be referred to as mobile object dead reckoning along a 3D map. The solution does not require for example altitude meters or similar additional sensors. It is adequate to equip the
mobile object 3, 4-7 only with a sensor device capable for step detection or wheel rotation measurement with a relative heading estimate and optionally a length estimate in horizontal direction. - The presently disclosed features for mobile object location tracking hence facilitate less complicated calculations and thus faster position updating or reduce required computational resources. As compared to position update computation completely in a 3D environment, further error caused with vertical dimension calculation may be avoided. Further, since the floor model comprising the directly extracted set of initial 3D model points is applied, a special type of map need not to be generated and the features may be applied directly on point clouds or surface meshes with no special map generation or vectorization needed.
- In some embodiments, proximity of the mobile object 3-7 to a location reference unit is detected. The horizontal position and/or the 3D position indicator of the mobile object may be updated on the basis of location of the location reference unit and estimated distance of the mobile object to the location reference unit. In some embodiments, the location reference unit is a mobile object, such as a sensor device carried by a
pedestrian 3 or a vehicle positioning unit. The location of the reference unit may be received from the reference unit or a system maintaining up-to-date position information based on information from location reference unit(s). In some embodiments location information of a plurality of reference units is applied for updating the 3D position indicator. - The location reference unit may be a wireless signal emission unit at a tunnel wall, a mining vehicle location tracking unit of a mining vehicle 4-7, or a PDR or another position tracking unit a carried by a
pedestrian 3. An RF tag, an access point, a visually readable code or another fixed unit the location of which is accurately known may serve as the location reference. Reference is also made toUS7899599 disclosing that such identifier may be applied to update dead reckoning based location. - In some example embodiments, Bluetooth or Bluetooth Low Energy based communication is applied for the location reference unit detection and location. The 3D position indicator may be updated in response to detecting the mobile object 3-7 at (the proximity of) the fixed location reference unit. In another embodiment, the horizontal position obtained on the basis of dead reckoning or the particle states are updated based on the received location reference information.
- For example, the
mobile communications device 20 may be configured to monitor visible Bluetooth devices in the environment. Known beacons or vehicles may be detected based on the MAC address of the Bluetooth devices, while disregarding all other Bluetooth devices. This facilitates to initialize the location tracking or starting position for the mobile object on the basis of known beacon locations in access points to mine, initializing the location information in case of errors in strategic points in mine, detecting person to person interaction, detecting person to vehicle interaction, and/or resetting person's location to vehicle location in case person vehicle interaction is detected. - For example, a vehicle may report a detected Bluetooth address belonging to a person. Upon detecting that the person is very closely located to or at the same location as the vehicle, the person's location may be updated to correspond to the location of the vehicle. An additional status may be added for the person location information to indicate that the person is at vehicle with an identifier 'xxx'. Alternatively, the position of the mobile object may be updated on the basis of the reference unit position and estimated distance to the reference unit. The method may be applied to update location of all persons traveling in a car in the worksite as well as walking past vehicles.
- In some embodiments, the mobile object is a
pedestrian 3. The horizontal progression of the pedestrian may be estimated on the basis of a dead reckoning algorithm configured to accumulate the pedestrian's travelled distance and heading on the basis of an input signal indicative of pedestrian steps by an IMU. The IMU may be configured to perform processing of raw data from sensors and generate the signal indicating estimated travelled distance and heading change is sent to a location tracking unit, such as a server, carrying out the method ofFigure 3 . For example, the signal may be transmitted at set time intervals of in response to detecting a step. A further implementation example applying PDR and a particle filter (PF) is provided below. - PDR is achieved by utilizing the kinematics of human gait. Typically the pedestrian's step count, step length and heading are estimated and, after that, dead reckoning algorithm is used to provide the current location of a pedestrian. The steps may be detected using the norm of accelerometer triad, a as
- Heading, i.e., yaw angle, Ψ, is estimated using gyroscopes and/or digital compasses (magnetometers). In order to calculate the heading, full 3-dimensional attitude of the IMU typically needs to be estimated. This can be done by using rotation matrices or quaternions, for example. When the attitude estimation is done, we can simplify the problem by writing that we have (virtual) gyroscope reading, ω, measuring the yaw angle. Thus, heading Ψn at time instant n can be estimated as
- There exist various different methods for step detection, including peak detection, zero-crossing, autocorrelation, fast Fourier transform. The position propagation in two dimension North-East-frame for PDR be written as
- Because PDR is a relative position method, the initial position of the pedestrian must be also known by some means. For example, this may be done by integrating PDR with GNSS in outdoors and WLAN-based positioning in indoors where the initial or reference position can be acquired.
- PF is applied to fuse the map information and the PDR output. Basic idea of the particle filter is based Monte Carlo simulation. This means that position solution distributions are presented as random samples. Each sample (i.e. in this case position) has also weight, which presents the likelihood of the sample. When the sample (position) is propagated using PDR method and position ends up going through wall or inside wall, the weight of the respective sample is decreased.
-
- There are two main requirements for the map: Each map point pi should present only the floor points of the tunnel (i.e., areas where people are able to walk) and map points should be spaced with appropriate spacing, such as spacing selected in the range of 0.2 to 1 meter, in an embodiment approximately 0.5 meter spacing. For example, distance between points two closest points pi and pj should be at maximum 0.5 meters but smaller than 0.4 meters. The maximum and minimum values may be varied, but larger number of map points directly affects calculation time/computational load of algorithm.
- The particle filter is implemented using following state vector x for each particle i
- 1) Propagation phase, where the PDR equations (4, 5) are applied to propagate the states of each particle. For example, an attitude quaternion is received from the IMU with every step. Heading Ψ is extracted from the quaternion.
- 2) Update phase, where the states are updated. In the present example it is map update or Bluetooth beacon received signal strength update
- In map update we first find the closest point of the map and if the distance from the particle to this closest point is greater than certain threshold (e.g. 1.5 meters) then we set particle weight wi to zero (another update would be just to decrease the weight)
- In Bluetooth update received signal strength RSSI (measured in dBm) of Bluetooth beacons with known positions is applied. The following formula may be applied to change RSSI to distance d (in meters)
- If distance is to particle (from Bluetooth beacon) is smaller than d with respective RSSIdBm , particle weight is untouched. However, if the distance is greater, the particles are moved or forced to be inside the distance. Also, there should be line of sight visibility between the particle and Bluetooth beacon. In addition to above two steps, the particle filter has also a resampling phase, which avoids degeneracy problem, where only a few of the particles will have a significant weight.
- In some embodiments, the mobile object is a vehicle 4-7, and the horizontal progression of the vehicle is estimated on the basis of a dead reckoning algorithm configured to accumulate the vehicle's travelled distance and heading on the basis of an input signal indicative of vehicle wheel rotation and relative heading. The vehicle may comprise a positioning unit configured to perform at least some of the presently disclosed features. In another embodiment, a dead reckoning unit of the vehicle transmits a signal indicative of the estimated position to a server or another unit configured to perform at least the method of
Figure 3 . It is to be appreciated that the system may comprise further operational modules supplementing dead reckoning based position tracking, such as a tyre slipping and/or wear compensation module. In an embodiment, a location tracking kit comprising the dead-reckoning unit is attachable to a vehicle when taking into use underground. - According to an embodiment, the vehicle 4-7 provided with a scanning device is serving as a mobile surveying device. The vehicle may execute the surveying continuously when carrying out dedicated normal operations of the vehicle. If the vehicle is a rock drilling rig or a reinforcing rig, it may scan the surroundings when it stops at a work site for executing drilling or feeding reinforcing elements or material. It may also be defined that the scanning is executed at least once each time when the vehicle is not moving. Thanks to this procedure, the mine may be surveyed repeatedly and in parallel to the normal operational process without any need for extra resources. The 3D model of the mine may thus be accurate and updated.
- The vehicle 4-7 may be a semi-autonomous or autonomous vehicle and comprise a control unit with a collision prevention feature. The collision prevention system may prevent collision to surrounding surfaces such as rock walls. In addition, the collision prevention system may prevent collision to other vehicles, other booms, auxiliary devices, rock blocks, persons or any other physical objects which may be located close to the vehicle or are entering to the proximity.
- The 3D position indicator may be applied in various ways, only some examples being illustrated herein. In some embodiments, the mobile object is displayed based on the 3D position indicator on a 3D map based on the 3D model. In some embodiments, the 3D position indicator is provided as an input for a collision avoidance system. This facilitates to prepare for a probable or possible collision risk beyond line of sight. In some embodiments, the 3D position indicator is provided as an input for updating position of other mobile object(s).
-
Figure 6 illustrates an example of a system for underground worksite. The system comprises awireless access network 60 comprising a plurality ofaccess nodes 8 for wireless communication withcommunication devices 10 of mobile objects 3-7 in the tunnels. The system comprises aserver 61, which may comprise one or more above or underground computing units. Theserver 61 is configured to perform at least some of the above illustrated features related to mobile object positioning, such as the methods ofFigures 3 and4 on the basis of signals received from mobile object(s) via theaccess network 60. -
Figure 6 illustrates operational modules 62-68 of theserver 61 according to some embodiments. Anobject tracking module 63 is configured to perform the method ofFigure 3 and provide the generated 3D position indicator to further modules, in some embodiments to aposition service module 62. - The
server 61 may comprise a task manager ormanagement module 64, which is configured to manage at least some operations at the worksite. For example, the task manager may be configured to assign work tasks for a fleet of vehicles and update and/or monitor task performance and status, which is indicated at a task management GUI. - The
server 61 may comprise amodel processing module 65, which may maintain one or more models of the underground worksite, such as the 3D model. In some embodiments, themodel processing module 65 is configured to extract the floor model and store it to the database orstorage 67. - The
server 61 may comprise avisualizer GUI module 66, which is configured to generate at least some display views for an operator (locally and/or remotely). In some embodiments, thevisualizer GUI module 66 is configured to generate, on the basis of the 3D model or floor model, a 3D (and/or 2D) view indicating the current position of the mobile object on the basis of the 3D indicator generated inblock 32. - The
server 61 may comprise further module(s) 68, such as a remote monitoring process and UI, and/or a cloud dispatcher component configured to provide selected worksite information, such as the mobile object position information to a cloud service. - The system and
server 61 may be connected to afurther system 70 and/ornetwork 69, such a worksite management system, a cloud service, an intermediate communications network, such as the internet, etc. The system may further comprise or be connected to a further device or control unit, such as a handheld user unit, a vehicle unit, a worksite management device/system, a remote control and/or monitoring device/system, data analytics device/system, sensor system/device, etc. - The object tracking 63 may be implemented as part of another module, such as the
position service module 62. Theposition service 62 is configured to provide, upon request or by push transmission, mobile object position information obtained from or generated on the basis of information from the object tracking 63 for relevant other modules or functions, such as thedatabase 67, the visualizergraphical user interface 66, and/or remote units orsystems 70 via one ormore networks 69. In the example ofFigure 6 the modules are illustrated as inter-connected, but it is to be appreciated that not all modules need to be connectable. - The system may comprise or be connected to a vehicle control unit or module provided with the 3D position indicator. The vehicle control unit may be provided in each autonomously operating vehicle and be configured to control at least some autonomous operations of the vehicle on the basis of their 3D location indicators. For example, in response to detecting a person to enter a zone comprising an autonomously operating vehicle, the control unit may be configured to send a control command to stop the vehicle.
- An electronic device comprising electronic circuitries may be an apparatus for realizing at least some embodiments of the present invention, such as the main operations illustrated in connection with
Figure 3 . The apparatus may be comprised in at least one computing device connected to or integrated into a control system which may be part of a worksite control or automation system. -
Figure 7 illustrates an example apparatus capable of supporting at least some embodiments of the present invention. Illustrated is adevice 80, which may be configured to carry out at least some of the embodiments relating to the mobile object position tracking illustrated above. In some embodiments, thedevice 80 comprises or implements theserver 61 and/or theobject tracking module 63 ofFigure 6 . In another embodiment, the device is comprised or carried by the mobile object 3-7, such as a mobile communications device or a vehicle control unit, configured to carry out at least some of the embodiments relating to the mobile object position tracking illustrated above. - Comprised in the
device 80 is aprocessor 81, which may comprise, for example, a single- or multi-core processor. Theprocessor 81 may comprise more than one processor. The processor may comprise at least one application-specific integrated circuit, ASIC. The processor may comprise at least one field-programmable gate array, FPGA. The processor may be configured, at least in part by computer instructions, to perform actions. - The
device 80 may comprisememory 82. The memory may comprise random-access memory and/or permanent memory. The memory may be at least in part accessible to theprocessor 81. The memory may be at least in part comprised in theprocessor 81. The memory may be at least in part external to thedevice 80 but accessible to the device. Thememory 82 may be means for storing information, such asparameters 84 affecting operations of the device. The parameter information in particular may comprise parameter information affecting the mobile object positioning, such as threshold values and timing parameters. - The
memory 82 may comprisecomputer program code 83 including computer instructions that theprocessor 81 is configured to execute. When computer instructions configured to cause the processor to perform certain actions are stored in the memory, and the device in overall is configured to run under the direction of the processor using computer instructions from the memory, the processor and/or its at least one processing core may be considered to be configured to perform said certain actions. The processor may, together with the memory and computer program code, form means for performing at least some of the above-illustrated method steps in the device. - The
device 80 may comprise acommunications unit 85 comprising a transmitter and/or a receiver. The transmitter and the receiver may be configured to transmit and receive, respectively, information in accordance with at least one cellular or non-cellular standard. The transmitter and/or receiver may be configured to operate in accordance with global system for mobile communication, GSM, wideband code division multiple access, WCDMA, long term evolution, LTE, 3GPP new radio access technology (N-RAT), wireless local area network, WLAN, Ethernet and/or worldwide interoperability for microwave access, WiMAX, standards, for example. Thedevice 80 may comprise a near-field communication, NFC, transceiver. The NFC transceiver may support at least one NFC technology, such as NFC, Bluetooth, or similar technologies. - The
device 80 may comprise or be connected to a UI. The UI may comprise at least one of adisplay 86, a speaker, aninput device 87 such as a keyboard, a joystick, a touchscreen, and/or a microphone. The UI may be configured to display views on the basis of the worksite model(s) and the mobile object position indicators. A user may operate the device and control at least some features of a control system, such as the system illustrated inFigure 6 . In some embodiments, the user may control a vehicle 4-7 and/or the server via the UI, for example to change operation mode, change display views, modifyparameters 84 in response to user authentication and adequate rights associated with the user, etc. - The
device 80 may further comprise and/or be connected to further units, devices and systems, such as one ormore sensor devices 88 sensing environment of thedevice 80. The sensor device may comprise an IMU or another type of sensor device configured to determine movements of a mobile object. For example, heading information may be obtained directly from an electronic compass. - The
processor 81, thememory 82, thecommunications unit 85 and the UI may be interconnected by electrical leads internal to thedevice 80 in a multitude of different ways. For example, each of the aforementioned devices may be separately connected to a master bus internal to the device, to allow for the devices to exchange information. However, as the skilled person will appreciate, this is only one example and depending on the embodiment various ways of interconnecting at least two of the aforementioned devices may be selected without departing from the scope of the present invention. - It is to be understood that the embodiments of the invention disclosed are not limited to the particular structures, process steps, or materials disclosed herein, but are extended to equivalents thereof as would be recognized by those ordinarily skilled in the relevant arts. It should also be understood that terminology employed herein is used for the purpose of describing particular embodiments only and is not intended to be limiting.
- Reference throughout this specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Where reference is made to a numerical value using a term such as, for example, about or substantially, the exact numerical value is also disclosed.
- As used herein, a plurality of items, structural elements, compositional elements, and/or materials may be presented in a common list for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary. In addition, various embodiments and example of the present invention may be referred to herein along with alternatives for the various components thereof. It is understood that such embodiments, examples, and alternatives are not to be construed as de facto equivalents of one another, but are to be considered as separate and autonomous representations of the present invention.
- Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the preceding description, numerous specific details are provided, such as examples of lengths, widths, shapes, etc., to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
- While the forgoing examples are illustrative of the principles of the present invention in one or more particular applications, it will be apparent to those of ordinary skill in the art that numerous modifications in form, usage and details of implementation can be made without the exercise of inventive faculty, and without departing from the principles and concepts of the invention. Accordingly, it is not intended that the invention be limited, except as by the claims set forth below.
- The verbs "to comprise" and "to include" are used in this document as open limitations that neither exclude nor require the existence of also un-recited features. The features recited in depending claims are mutually freely combinable unless otherwise explicitly stated. Furthermore, it is to be understood that the use of "a" or "an", that is, a singular form, throughout this document does not exclude a plurality.
- At least some embodiments of the present invention find industrial application at least in underground mining.
-
- ASIC
- Application-specific integrated circuit
- CAD
- Computer-aided design
- FPGA
- Field-programmable gate array
- GNSS
- Global navigation satellite system
- GSM
- Global system for mobile communication
- GUI
- Graphical user interface
- IMU
- Inertial measurement unit
- LTE
- Long term evolution
- MAC
- Medium access control
- NFC
- Near-field communication
- N-RAT
- 3GPP new radio access technology
- PDR
- Pedestrian dead reckoning
- UI
- User interface
- WCDMA
- Wideband code division multiple access
- WiMAX
- Worldwide interoperability for microwave access
- WLAN
- Wireless local area network
Claims (15)
- An apparatus, comprising means for performing:- determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object,- determining a horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and- generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and on the basis of a three-dimensional model of the tunnel.
- The apparatus of claim 1, wherein the three-dimensional position indicator is generated on the basis of a sub-set of three-dimensional floor point locations within a threshold distance.
- The apparatus of claim 1 or 2, wherein the mobile object is a pedestrian, and the horizontal progression of the pedestrian is determined on the basis of a dead reckoning algorithm configured to accumulate the pedestrian's travelled distance and heading on the basis of an input signal from an inertial measurement unit.
- The apparatus of claim 3, wherein the apparatus is connected to or comprises a mobile user device comprising or being locally connectable to the inertial measurement unit wearable by the pedestrian.
- The apparatus of claim 1 or 2, wherein the mobile object is a vehicle, and the horizontal progression of the vehicle is determined on the basis of a dead reckoning algorithm configured to accumulate the vehicle's travelled distance and heading on the basis of an input signal indicative of vehicle wheel rotation and relative heading.
- The apparatus of any preceding claim, the apparatus being further configured for performing:- establishing a set of particles representative of horizontal position options,- updating positions of the particles on the basis of the determined horizontal progression of the mobile object,- fusing the updated positions of the particles with the floor model.
- The apparatus of claim 6, wherein the fusing comprises:- comparing the updated positions of the particles to a sub-set of floor point positions of the floor model,- filtering out particles that have no floor points within a threshold distance, and- assigning, for each particle, a position of a closest floor point within a threshold distance.
- The apparatus of any preceding claim, the apparatus being further configured for performing:- indicating, on the basis of the three-dimensional position indicator, the mobile object on a three-dimensional map based on the three-dimensional model, and/or- providing the three-dimensional position indicator as an input for a collision prevention system.
- The apparatus of any preceding claim, wherein the three-dimensional model comprises three-dimensional point cloud data generated on the basis of scanning the tunnel and the floor model comprises a sub-set of points extracted from the three-dimensional model.
- The apparatus of any preceding claim, the apparatus being further configured for performing- detecting proximity of the mobile object to a location reference unit, and- updating the position estimate on the basis of location of the location reference unit and estimated distance of the mobile object to the location reference unit.
- A method for positioning a mobile object in an underground tunnel, comprising:- determining horizontal progression of a mobile object in an underground tunnel from a preceding position estimate or an initial position of the mobile object,- determining a horizontal position of the mobile object on the basis of a floor model of the tunnel and the estimated horizontal progression of the mobile object, and- generating a three-dimensional position indicator on the basis of the horizontal position of the mobile object and a three-dimensional model of the tunnel.
- The method of claim 11, further comprising:- establishing a set of particles representative of horizontal position options,- updating positions of the particles on the basis of the determined horizontal progression of the mobile object,- fusing the updated positions of the particles with the floor model.
- The method of claim 12, wherein the fusing comprises:- comparing the updated positions of the particles to a sub-set of floor point positions of the floor model, and- filtering out particles that have no floor points within a threshold distance
- The method of any preceding claim 11 to 13, wherein the mobile object is a pedestrian, and the horizontal progression of the pedestrian is estimated on the basis of a dead reckoning algorithm configured to accumulate the pedestrian's travelled distance and heading on the basis of an input signal indicative of pedestrian steps by an inertial measurement unit.
- A computer program comprising code for, when executed in a data processing apparatus, to cause a method in accordance with any one of claims 11 to 14 to be performed.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP18209502.6A EP3660453B1 (en) | 2018-11-30 | 2018-11-30 | Positioning of mobile object in underground worksite |
PCT/EP2019/082921 WO2020109473A1 (en) | 2018-11-30 | 2019-11-28 | Positioning of mobile object in underground worksite |
CA3120810A CA3120810A1 (en) | 2018-11-30 | 2019-11-28 | Positioning of mobile object in underground worksite |
US17/297,591 US11874115B2 (en) | 2018-11-30 | 2019-11-28 | Positioning of mobile object in underground worksite |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP18209502.6A EP3660453B1 (en) | 2018-11-30 | 2018-11-30 | Positioning of mobile object in underground worksite |
Publications (3)
Publication Number | Publication Date |
---|---|
EP3660453A1 true EP3660453A1 (en) | 2020-06-03 |
EP3660453B1 EP3660453B1 (en) | 2023-11-29 |
EP3660453C0 EP3660453C0 (en) | 2023-11-29 |
Family
ID=64564667
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18209502.6A Active EP3660453B1 (en) | 2018-11-30 | 2018-11-30 | Positioning of mobile object in underground worksite |
Country Status (4)
Country | Link |
---|---|
US (1) | US11874115B2 (en) |
EP (1) | EP3660453B1 (en) |
CA (1) | CA3120810A1 (en) |
WO (1) | WO2020109473A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112862879A (en) * | 2021-02-18 | 2021-05-28 | 中国矿业大学(北京) | Method for constructing subway tunnel three-dimensional model based on TIN model |
WO2024096764A1 (en) * | 2022-10-31 | 2024-05-10 | Epiroc Rock Drills Aktiebolag | Automated road maintenance prediction |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112762928B (en) * | 2020-12-23 | 2022-07-15 | 重庆邮电大学 | ODOM and DM landmark combined mobile robot containing laser SLAM and navigation method |
US20220289237A1 (en) * | 2021-03-10 | 2022-09-15 | Gm Cruise Holdings Llc | Map-free generic obstacle detection for collision avoidance systems |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060235609A1 (en) * | 2003-03-25 | 2006-10-19 | Maekelae Hannu | Method and control system for positioning a mine vehicle |
JP2008185506A (en) * | 2007-01-31 | 2008-08-14 | Mobile Mapping Kk | Apparatus and method for navigation, and automobile |
US7899599B2 (en) | 2003-07-03 | 2011-03-01 | Sandvik Mining And Construction Oy | Arrangement for monitoring the location of a mining vehicle in a mine |
WO2015106799A1 (en) | 2014-01-14 | 2015-07-23 | Sandvik Mining And Construction Oy | Mine vehicle, mine control system and mapping method |
US20180075643A1 (en) * | 2015-04-10 | 2018-03-15 | The European Atomic Energy Community (Euratom), Represented By The European Commission | Method and device for real-time mapping and localization |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7069124B1 (en) * | 2002-10-28 | 2006-06-27 | Workhorse Technologies, Llc | Robotic modeling of voids |
DE602006021188D1 (en) * | 2005-07-26 | 2011-05-19 | Macdonald Dettwiler & Associates Inc | CAR |
US9292017B2 (en) * | 2013-04-22 | 2016-03-22 | Dan Alan Preston | System and method for real-time guidance and mapping of a tunnel boring machine and tunnel |
EP3023577A1 (en) * | 2014-11-20 | 2016-05-25 | Sandvik Mining and Construction Oy | A control system for a drilling apparatus |
US20170234129A1 (en) * | 2016-02-11 | 2017-08-17 | Eagle Harbor Holdings, Llc | System and method for real-time guidance and mapping of a tunnel boring machine and tunnel |
JP7051864B2 (en) * | 2016-12-21 | 2022-04-11 | ヴァイタル アラート コミュニケーション インコーポレイテッド | Magnetic positioning system |
-
2018
- 2018-11-30 EP EP18209502.6A patent/EP3660453B1/en active Active
-
2019
- 2019-11-28 CA CA3120810A patent/CA3120810A1/en active Pending
- 2019-11-28 WO PCT/EP2019/082921 patent/WO2020109473A1/en active Application Filing
- 2019-11-28 US US17/297,591 patent/US11874115B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060235609A1 (en) * | 2003-03-25 | 2006-10-19 | Maekelae Hannu | Method and control system for positioning a mine vehicle |
US7899599B2 (en) | 2003-07-03 | 2011-03-01 | Sandvik Mining And Construction Oy | Arrangement for monitoring the location of a mining vehicle in a mine |
JP2008185506A (en) * | 2007-01-31 | 2008-08-14 | Mobile Mapping Kk | Apparatus and method for navigation, and automobile |
WO2015106799A1 (en) | 2014-01-14 | 2015-07-23 | Sandvik Mining And Construction Oy | Mine vehicle, mine control system and mapping method |
US20180075643A1 (en) * | 2015-04-10 | 2018-03-15 | The European Atomic Energy Community (Euratom), Represented By The European Commission | Method and device for real-time mapping and localization |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112862879A (en) * | 2021-02-18 | 2021-05-28 | 中国矿业大学(北京) | Method for constructing subway tunnel three-dimensional model based on TIN model |
CN112862879B (en) * | 2021-02-18 | 2023-07-07 | 中国矿业大学(北京) | Subway tunnel three-dimensional model construction method based on TIN model |
WO2024096764A1 (en) * | 2022-10-31 | 2024-05-10 | Epiroc Rock Drills Aktiebolag | Automated road maintenance prediction |
Also Published As
Publication number | Publication date |
---|---|
WO2020109473A1 (en) | 2020-06-04 |
CA3120810A1 (en) | 2020-06-04 |
US20220026215A1 (en) | 2022-01-27 |
EP3660453B1 (en) | 2023-11-29 |
EP3660453C0 (en) | 2023-11-29 |
US11874115B2 (en) | 2024-01-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11874115B2 (en) | Positioning of mobile object in underground worksite | |
CN109059942B (en) | Underground high-precision navigation map construction system and method | |
US11783248B2 (en) | United states construction management system and method | |
AU2014274647B2 (en) | Determining terrain model error | |
AU2011202456B2 (en) | Traffic Management System for a Passageway Environment | |
US10591640B2 (en) | Processing of terrain data | |
WO2018102320A1 (en) | System for incremental trajectory estimation based on real time inertial sensing | |
EP3881031B1 (en) | Systems and methods for direction estimation in indoor and outdoor locations | |
AU2014274649B2 (en) | System and method for modelling worksite terrain | |
US20220026236A1 (en) | Model generation for route planning or positioning of mobile object in underground worksite | |
WO2021053111A1 (en) | Positioning of mobile device in underground worksite | |
EP3754157A1 (en) | Underground worksite passage and model generation for path planning | |
US10378907B1 (en) | Methods and systems for determining vertical location in enclosed spaces | |
Sun et al. | Internet of things based 3D assisted driving system for trucks in mines | |
AU2014274648B2 (en) | Determining terrain of a worksite | |
US20220292782A1 (en) | Modelling of underground worksite | |
KR102613634B1 (en) | Construction equipment mobile navigation system | |
Rodriguez et al. | Evaluation Study of Inertial Positioning in Road Tunnels for Cooperative ITS Applications | |
Martin Rodriguez et al. | Evaluation Study of Inertial Positioning in Road Tunnels for Cooperative ITS Applications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20201203 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20220425 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230603 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20230721 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602018061751 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
U01 | Request for unitary effect filed |
Effective date: 20231220 |
|
U07 | Unitary effect registered |
Designated state(s): AT BE BG DE DK EE FI FR IT LT LU LV MT NL PT SE SI Effective date: 20240122 |
|
P04 | Withdrawal of opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20240117 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240301 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240329 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240329 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240301 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
U20 | Renewal fee paid [unitary effect] |
Year of fee payment: 6 Effective date: 20240422 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20240229 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20231130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20231129 |