RU2815329C1 - Method of mining equipment inspection - Google Patents

Abstract

FIELD: inspection of mining equipment.

SUBSTANCE: examples of mining equipment are mill, crusher, grinder. Disclosed is a computer-implemented method for virtual inspection of the interior of mining equipment, which includes: obtaining a first data set, wherein the first data set contains data points with coordinates of the interior of the mining equipment; converting the obtained first data set into a second data set, wherein the second data set is adapted for use by the virtual or augmented reality device to represent the interior of the mining equipment; directing at least one user during the virtual inspection of the inside of the mining equipment based on the second data set by moving the visual perception of said at least one user of the virtual or augmented reality device to one or more points of interest of the inner part of the mining equipment. Also disclosed is a non-transitory computer-readable medium for storing information, storing a computer program, which, when executed by a computer, causes the computer to implement the above method.

EFFECT: high accuracy and information content of the obtained data on mining equipment during inspection with simultaneous reduction of the duration of the inspection.

7 cl, 12 dwg

Description

Field of technology

The present invention generally relates to systems, methods and apparatus for inspecting mining equipment. Examples of mining equipment are a mill, crusher, shredder. The present invention also relates to systems, methods and devices for virtual inspection of mining equipment.

State of the art

Mining equipment, such as a horizontal or vertical mill, crusher or shredder, is used to reduce the size of rock materials, such as minerals or ores, through a mining process such as grinding, crushing or grinding. Typically, the mining process applies stress forces to the rock until it breaks into smaller pieces. To aid the mining process, metal objects such as rods or balls may be placed in mining equipment. In addition, mining equipment can perform dry or wet mining, with the wet mining process providing higher efficiency and suppressing dust. However, during the mining process (especially wet mining), mining equipment and metal objects will wear out.

To protect mining equipment from excessive wear, linings may be installed on the surface of the mining equipment. This lining can be made of ceramic, rubber, polymer or composite material and is typically installed in such a way that it can be replaced when it reaches the end of its life, for example using screws, latches or hooks. The lining may be shaped to support the mining and/or release process, for example by having projections and recesses that function as shovels. Regular inspections are required to assess lining wear and/or damage and determine the need for replacement.

However, to conduct an inspection, the inspection personnel, that is, the specialist, is required to enter a hostile environment, such as a room or space located inside the mining equipment, or a space located near the place where the mining process occurs. To reduce the likelihood of harm to inspection personnel during an inspection, two safety precautions are typically followed. First, they turn off the power to the mining equipment and clean it of contaminants, and then inspect it during its downtime, that is, when the mining equipment is not working, that is, not performing any mining process. Second, the inspection should be completed as quickly as possible to minimize downtime and, more importantly, reduce the length of time inspection personnel are exposed to the hostile environment. Once the inspection is completed, work can begin, and the data obtained during this inspection can be analyzed to determine the wear status of the mining equipment, as well as its lining and exhaust system.

Prior Art

AU 2016 200024 A1 relates to a system and method for monitoring the surface wear condition of mining equipment. In this document, to determine whether a surface (eg lining) needs repair or replacement, surfaces are measured and compared to historical data. The measurement is performed on (and possibly cleaned of contaminants from) mining equipment with the power turned off, using a scanner that runs along the horizontal and vertical axis to create 3D point cloud data. The scanner is attached to the end of a rod, crossbar or boom that is inserted into the mining equipment, or mounted on the top of a tripod placed inside the mining equipment. The idea here is to mount the sensor in a stationary and rigid manner to prevent its position within the mining equipment from changing while scanning is performed. However, since the scanner used here remains stationary, part of the surface to be inspected may still be covered with rock material, i.e. crushed ore or other residues such as sludge. In addition, AU 2016 200024 A1 suggests scanning the entire surface by positioning the scanner as close to the center of the mining equipment as possible. However, an uneven surface, such as one containing protrusions, can result in scanning dead spots where part of the surface to be scanned is covered by said protrusions. In addition to the rock material and other debris present in the mining equipment during scanning, these scanning dead zones further degrade the quality and completeness of the point cloud data used for analysis and inspection.

Technical problem

From the above, it appears that there is a technical challenge in how to obtain sufficient and complete data on mining equipment during inspection, while at the same time minimizing the inspection time.

In addition, there is a technical challenge of how to improve inspection to identify specific points of interest within mining equipment and improve notification of these points of interest.

In addition, since mining equipment that has just been shut down may still contain mining material such as minerals, ore, or slurry, parts of the structure or surface of the mining equipment may be coated with this rock material, preventing them from being inspected. Traditionally, mining equipment must be cleaned, emptied, cleared of contaminants, and shut down for inspection, which further increases the inspection time. Therefore, another technical challenge is to avoid, or at least reduce the need for, emptying, decontamination and/or power outage of mining equipment.

Solution

This technical problem is solved by the present invention in accordance with the independent claims. Additional preferred embodiments are described in the dependent claims.

Advantageous Effects

Systems, methods and devices in accordance with the subject matter of the independent claims improve the performance of inspection of mining equipment. In particular, inspection times are reduced, which improves the safety of inspection personnel and reduces downtime for mining equipment. In addition, data acquisition during an inspection is performed in such a way that the data set obtained during an inspection is complete, thereby avoiding subsequent inspections. In addition, inspections can be carried out without completely shutting off power to mining equipment, allowing continuous operation. In addition, virtual inspection improves the identification of specific points of interest within mining equipment, particularly by providing virtual inspection to multiple local and remote users simultaneously inspecting that mining equipment.

Brief description of drawings

In fig. 1A-1E show examples of mining equipment.

In fig. 2A-2D show a lined drum mill and data sets obtained to perform a drum mill inspection.

In fig. 3A and 3B show the process of obtaining data, the process of identifying missing data.

In fig. 4A-4E show the movement of the sensor during inspection of a drum mill in operation.

In fig. 5A-5C show a different method of obtaining data when performing a drum mill inspection.

In fig. 6 shows a computer-implemented method for virtually inspecting the interior of mining equipment.

In fig. 7 is a block diagram illustrating an exemplary hardware configuration of a computing device for implementing the methods described herein.

In fig. 8 shows a computer-implemented method for guiding a procedure for scanning mining equipment for inspection analysis.

In fig. 9A-9B show the inspection system without and with a remote display.

In fig. 10 illustrates a computer-implemented feedback procedure for scanning mining equipment based on input from a person with expert knowledge directing said scanning.

In fig. 11A-11C show an example of calculating the change in the gradient of acquired data points, whereby an area of insufficient data and/or high measurement uncertainty can be determined.

In fig. 12A and 12B show an example of guiding the user and/or causing the sensor to move in a direction to (re)scan an identified area or an area of insufficient data.

Carrying out the invention

Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the following description contains examples only and should not be construed as limiting the present invention. Below, similar or identical reference numerals indicate similar or identical elements or functions.

[Mining equipment]

With reference to FIG. 1A-1D explains the functionality of various mining equipment. Each mining equipment shown in FIG. 1A-1D is configured to reduce the size of the rock material 10 to produce a material or product 20 having a smaller size.

In fig. 1A shows a crush crusher 100 as a mining equipment. Such a crusher can be a jaw crusher, a gyrator or a cone crusher. In this case, the rock material 10 is compressed between the first surface 110 and the second surface 120. Either one or both surfaces can move in direction A towards and away from each other, allowing the rock material 10 to enter the space between these two surfaces and its compression and crushing. The resulting product 20 is ejected from the crusher 100 under the influence of gravity and pressure from the rock material 10 introduced into the crusher 100. The size of the product 20 is controlled by controlling the amount of movement in direction A. During operation, the surfaces 110 and 120 wear and therefore can be protected by a lining.

In fig. 1B, an impact crusher 200 is shown as a mining equipment. Rock material 10 is fed to the impact crusher 200 and is thrown onto the internal surfaces 220 of the impact crusher 200 through the use of rotating blades 210. In the example of FIG. 1B, the blades 210 rotate in direction B. The impact of the rock material 10 on the internal surfaces 220 breaks it into smaller pieces, resulting in a product 20 ejected from the impact crusher 200. Although the impact crusher 200 shown in FIG. 1B illustrates a horizontal type impact crusher when viewed from the side, while a vertical type impact crusher has a similar structure when viewed from above. Then, in a vertical type impact crusher, the inner surfaces 220 will be part of the peripheral walls. By moving the internal surfaces 220 closer to or further away from the rotating blades 210, the size of the product 20 is adjusted. Since both the blades 210 and the internal surfaces 210 wear during operation, they may be covered with or made from a lining that can be replaced if necessary. .

In fig. 1C shows a high-pressure roller crusher 300 as a mining equipment. In this case, the first roller 310 and the second roller 320 rotate in the first direction B and the second direction B', that is, in opposite directions. Thus, the rock material 10 is fed into the nip between the two rolls 310, 320 and reduced in size, resulting in an ejected product 20. The size of the product 20 is adjusted by bringing the two rolls 310, 320 closer together or further apart. To accommodate various rock materials 10 and to provide wear protection, the rolls 310, 320 may be lined or made of liners.

In fig. 1D shows a mill 400 with mixing of grinding media as a mining equipment. The media agitation mill 400 rotates the shaft 410 in direction B, thereby rotating the cross members 420 which agitate the rock material 10. This agitation reduces the size of the rock material 10 through impact, compression, shear and abrasion forces between the pieces of rock material 10. Because the rock material 10 of different sizes and weights settle or separate at different heights within the mixing vessel 430, the desired product can be removed from the mixing vessel 430 at the appropriate height 401, 402. Screening or filtering the recovered product can further reduce the size variation of the recovered product. The inner surface of the mixing vessel 430 and the crossbar 420 wear during operation. Therefore, they can also be lined.

In fig. 1E, a drum mill 500 is shown as mining equipment. Rods or balls made of metal may be inserted into the drum mill 500 to support mining operations. The mining operation is performed by rotating the drum container 510 in a certain direction B. Thus, the rock material 10 is subjected to impact, compression, shear and abrasion forces, thereby reducing its size. Through an outlet structure (not shown in FIG. 1E), a finished product of a given size or distribution into specific sizes can be removed from drum vessel 510. A liner may be installed to protect the interior surface of drum vessel 510 during mining operations. Mining operations can even be supported by shaping the lining into proper shape.

Other examples of mining equipment are the horizontal mill and the vertical mill.

In addition to the various types of crushers and mills described above, mining equipment may also contain screening machines, conveyor belts, power lines, pipelines, or flotation machines.

Screening machines receive granular ore material and separate it into several grades based on particle size. Applying the inspection methodology described below to the screening machines, that is, obtaining point cloud data for these machines or at least a portion of these machines, supports their performance in separating ore material. This reduces both the risk of damage to the screening machine itself (e.g. due to wear, fatigue and material damage) and the risk of damage to other mining equipment (e.g. due to improperly screened material with incompatible particle sizes being released into downstream mining equipment).

Conveyor belts can be several kilometers long and serve to move rock material between mining equipment. Applying the inspection methodology described below, that is, obtaining point cloud data for the conveyor belts or at least a portion of the conveyor belts, ensures their proper operation. Instead of using depth analysis, as described below, to analyze wear and deformation of mining equipment, a thermal sensor can measure the temperature of bearings and other moving components of mining equipment, particularly the conveyor belt. Areas of equipment indicating above average temperatures provide information such as insufficient lubrication and/or severe material wear that is likely to result in failure. A flying drone can be used to transport an inspection sensor along mining equipment while searching such areas. This eliminates the need for inspection personnel to manually scan mining equipment, reducing health and safety risks, and also improves scanning accuracy as there is less chance of human error.

Power lines can be several kilometers long and serve to provide mining equipment with electricity to carry out mining work. By applying the point cloud data inspection methodology described below to power lines, breaks, power losses, and faults can be detected. Additionally, the temperature sensor can measure the temperature of power lines and power equipment containing transformers, generators, and power electronic equipment to identify areas where power lines or power equipment may be overheating. These areas may indicate a short circuit, component deterioration, or equipment overload. A flying drone can be used to transport an inspection sensor along power lines and power equipment while searching such areas. In addition, deformation or damage to power lines and power equipment can be inspected using in-depth analysis similar to the one described below to ensure compliance with operating standards and insulation standards.

Pipelines are used to move mining materials, such as ores containing coal or iron, or mining waste, called tailings, over long distances. The length of such pipelines can range from several tens to hundreds of kilometers. By applying the inspection methodology described below, pipelines can detect deformation and wear leading to the risk of breakage and leakage. This detection allows preventative maintenance to be performed before a pipeline failure occurs and before that failure can cause damage to mining equipment connected to the pipeline. A flying drone can be used to transport the sensor along the pipeline while searching for areas of deformation and wear. This search may be expanded to include pumps, filters, compressors or other piping equipment used in combination with that piping.

Flotation machines are designed to selectively separate hydrophobic materials from hydrophilic materials. By applying the inspection methodology described below to flotation machines, their operation is maintained and the output product is of a composition suitable for further processing.

[Receiving Data]

To better illustrate and explain the embodiment(s) described herein, the drum mill 500 and its geometric shape are used to explain the mining equipment inspection process. However, other types of mining equipment (as described above) can be equally used.

In fig. 2A shows a drum mill 500 as a mining equipment, cut away for illustrative purposes. Drum mill 500 includes a lining 520 with projections and/or recesses and an exhaust system 530. During mining operation, during drum mill operation, drum mill 500 is rotated, for example, in direction B to reduce the size of rock material 20 similar to the mining operation shown in fig. 1E.

In preparation for inspection, drum mill 500 is slowed down and stopped to allow inspection personnel to insert sensor 30 into drum mill 510 and begin inspection. A sensor 30 (e.g., a depth sensor that senses the distance from the sensor to the surface as depth and thus performs a depth analysis) may be located within the drum mill 510 to scan the interior surface of the drum mill 500. In this case, the sensor 50 outputs sets data containing data points 540* with corresponding coordinates.

The sensor 30 may be portable, requiring the drum mill 500 to be completely stopped and decontaminated and/or cleaned before the drum mill 500 can be entered by inspection personnel. Alternatively, the sensor 30 may be supported by a rod, robot, or flying drone entering the drum bowl 510. For this last alternative, it may not be necessary to stop, decontaminate, and/or empty the drum mill 500. The inspection method for the latter option will be described below.

In fig. 2B shows a point cloud data set containing the above-mentioned data points 540* that can be represented and/or stored as coordinate vectors. The coordinate vector may be a three-dimensional vector in a Cartesian coordinate system defining points in space related to the mining equipment (eg, related to the position of the surface or lining of the mining equipment). One skilled in the art will appreciate that the distance between the sensor and points on the mining equipment can be obtained from time-of-flight information using a laser and the like. (depth information data). Alternatively, the coordinate vector may be a four-dimensional vector, in which the first three values represent a three-dimensional vector in a Cartesian coordinate system (as just explained), and the fourth value refers to the reflectance property of the scanned points of the mining equipment (for example, the reflectance intensity of the laser radiation of the sensor on surface texture or surface composition at appropriate points in the mining equipment). Alternatively or additionally, the fourth value may relate to a thermal property of the scanned points of the mining equipment (eg, surface temperature measured by a thermal sensor used in conjunction with the previously mentioned scanner).

For clearer illustration, the asterisk "*" added to certain reference designations and used in the following description indicates the data representation or point cloud representation of reality of the corresponding component. For example, reference identifier 500* denotes mining equipment point cloud data, reference identifier 520* denotes a feature of point cloud data 500*, reference identifier 540* denotes point cloud data point 500*, and reference identifier 550* denotes an area of insufficient data in cloud data 500* points.

As shown in FIG. 2C, sensor 30 may have a limited scanning range 31 and may be prone to scanning dead zones 32. In this case, scanning dead zones 32 may result from the presence of protrusions or recesses, such as liner 520, blocking a portion of the surface to be scanned that would be in the line of sight of sensor 30, such as within the scanning range 31 of sensor 30, where there are no protrusions. or recesses. An enlargement of such a dead zone 32 for scanning in section AA in FIG. 2C is shown in FIG. 2D for illustrative purposes. Information about the surface located within such scanning dead zone 32 may be reflected as gaps in the point cloud data or missing/inaccurate data, which may lead to inaccurate analysis of mining equipment. Using the method described below, a complete point cloud data set 500* of the mining equipment (eg, drum mill 500) can be obtained for analysis, eg, wear and/or failure analysis, while also obtaining missing surface information. Examples of forms of wear or damage resulting from such analysis include defects, porosity, cracks, voids, discontinuities, missing or faulty parts, corrosion, impact damage, separation (eg, linings), etc.

In fig. 3A shows a data acquisition process for a method for obtaining a complete 500* point cloud data set. As shown in FIG. 3A, first a first data set 501* and then a second data set 502* are acquired by the sensor 30. In this case, each data set contains data points 540* with corresponding coordinates, for example, on the surface of mining equipment. Such data acquisition is accomplished by moving the sensor 30 relative to the surface of the drum container 510 located inside the drum container 510, that is, from position (a) to position (b), as shown in FIG. 3A.

With reference to FIG. 3A, letters (a)-(f) indicate various times and locations of sensor 30 subsequent to the acquisition of the first and second sets of data 501*, 502*. In this case, the letters (a) - (f) indicate various positions of the sensor 30 within the drum mill 500 during scanning while moving (from left to right) substantially along the axis of rotation of the drum mill 500. The sensor 30 can also (or alternatively) rotate, for example , in direction C, as shown in FIG. 2C. Therefore, movement of the sensor 30 may include translational and/or rotational movement. Therefore, the sensor 30 may be a movable sensor 30, preferably portable, flying, hovering or suspended. In this regard, for example, a flying drone, a rod or a robotic suspension device can be used. In addition, as sensor 30 gradually acquires separate data sets 501*, ... 506*, a second data set 502* is acquired after the first data set 501* and after movement of the sensor 30. Likewise, a third data set 503* is obtained after the second and first sets 501*, 502* data and after moving the sensor 30, etc. This process of incrementally acquiring data sets 501*, ... 506* to create point cloud data 500* is also called "virtual painting" of the interior of mining equipment.

To create data sets 501*, ... 506* as depth information, sensor 30 may obtain distance information from sensor 30 to a surface located within mining equipment, such as drum mill 500. Laser, radar, or radar may be used to obtain said depth information. , sonar, stereoscopic image, time-of-flight sensor, etc. or a combination thereof. The data received by sensor 30 may also include information related to surface texture or surface composition. For example, while the time of flight of the emitted signal and its reflection may indicate the distance to the surface being measured, the intensity-related property of the reflected and measured signal may indicate the type of material to which that distance is being measured. For example, rubber can absorb more light from a laser scanner than a metal surface.

In addition, the sensor 30 may obtain information about the orientation and/or position (eg, using odometry) of the sensor 30. In this case, the orientation information may include information about the roll of the sensor 30, its rotation about its lateral axis, and/or its vertical axis , and the position information may include information about the x, y and z coordinates of the sensor 30. To obtain orientation and position information, selective measurements of the gyroscope and accelerometer at high frequencies, for example, at least 100 Hz, can be used. Since information about the position and orientation of the sensor 30 is thereby known, and since depth information is also obtained, the corresponding coordinates of data points 540* representing the scanned surface can be retrieved as acquired data sets 501*, ... 506*. Therefore, the data points 540* contain coordinates indicating the location of the surface sensed by the sensor 30. The point cloud data 500* is used for analysis because it is a collection of data points 540*, each of which represents location information or intensity and location information points on the surface located inside the 500* mining equipment as mentioned above. Thus, each data point 540* represents information about the structure of the internal surface of the mining equipment 500.

However, since the scanning range 31 may limit the size of each data set 501*, ... 506* received by the sensor 30, and since the position and location of the sensor 30 cannot be tracked with perfect accuracy, to create point cloud data 500* representing the structure of the inside of the drum mills 500 that are needed for analysis, it may be necessary to combine or "stitch" the data sets 501*, ... 506*. As described below, positioning and alignment of each data set 501*, ... 506* is first performed before combining them into point cloud data 500* used for analysis.

Returning to the generation of the 500* point cloud data shown in FIG. 2B, merging of data sets 501*, 502* is performed as follows. After receiving two data sets 501*, 502*, features 520* are extracted from the first and second data sets 501*, 502*. In this case, features 520* may be marks, edges, a surface texture pattern, and/or a surface reflectance pattern of the scanned surfaces and are illustrated by a dotted surface for simplicity in FIG. 2B. That is, the combination of data sets can be performed based on positional or structural features and/or reflection property features.

Although the first and second data sets 501*, 502* may have already been approximately aligned during the scanning process described above, discrepancies resulting from intermittent or erroneous sampling of orientation and position information (eg, due to integration errors) still need to be corrected. Moreover, if either or both of the orientation and position information cannot be obtained, positioning and alignment of the data sets 501*, 502* cannot be accomplished during the scanning process described above and may need to be performed in a different manner. Therefore, the first and second data sets 501*, 502* are positioned and aligned using the extracted features 520* and combined into point cloud data 500*. In particular, when a portion of the scanned surface is represented by both data sets 501*, 502* (eg, when data sets 501*, 502* overlap, as shown in FIG. 3A), the data sets 501*, 502* can be scaled, moved and/or or rotate to overlay their respective 520* features until they are correctly positioned and aligned. Thus, positioning and alignment of the two data sets 501*, 502* are performed and/or improved before they are combined into point cloud data 500*.

In this regard, the above-mentioned features 520* may be marks placed at predetermined locations within the drum container 510, they may be in the shape of the liner 520 itself, and/or they may be a pattern of surface texture, surface composition, or surface material indicated by intensity information 540* data points. Since the tags, linings 520 and materials located within the mining equipment have certain identifiable shapes and properties, which are also represented in the resulting data sets 501*, 502*, feature detection, edge detection, line tracking can be performed to extract these features 520* or fitting using a spline to data points 540 * or a surface represented by data points 540 * in data sets 501 * , 502 * . In this regard, positioning and alignment may involve linear transformation, preferably rotation, scaling and/or translation of the first and/or second data set 501*, 502* to maximize correlation, alignment and/or feature matching. Based on the difference between the features 520* of the first and second data sets 501*, 502*, the correlation, alignment, and/or correspondence of the features can be quantified. For example, the least squares error may be used as a quantifier for feature correlation, alignment, and/or matching.

Based on the point cloud data 500*, changes in the design or shape of the internal surface of the mining equipment, such as the drum vessel 510, can be detected. For example, changes in the original design or shape, etc. can be detected. compared to the previous inspection. These changes may indicate wear and/or deformation of the mining equipment, lining 520, inlet system (not shown) and/or exhaust system 530. As a result, it is possible to evaluate and determine whether the mining equipment, lining 520, inlet system or exhaust system 530 to maintain the operation and safety of mining equipment, such as the drum mill 500. To perform this assessment and identify point clouds from the data 500*, visual inspection techniques described below may be used.

To enable assessment of changes in design or shape, the entire structure of the drum mill 500 must be scanned. Alternatively, if only a specific area of the mining equipment is of interest, a corresponding portion of the entire structure must be scanned. In any case, the scan must be a complete scan of the structure or shape of interest and must be of a certain quality/resolution. Otherwise, the assessment may be inadequate due to incomplete data or low quality/resolution data. To ensure that the scan is complete and at least the specified quality/resolution, based on the point cloud data 500*, the geometric shape 510* of the mining equipment, such as the drum mill 500, is estimated, and by using the point cloud data 500*, an area 550* of the area being assessed is identified. geometric shape 510* that indicates insufficient data, or an area of interest (see also FIG. 3B). Additionally, by providing information about the position and orientation of sensor 30, an indication (or direction thereof) may be provided to an area indicating insufficient data, or an area of interest without initially having a geometric shape estimate.

With respect to geometric shape estimation, this may be performed by positioning a base geometric shape 510*, such as a cylinder in the case of drum mill 500, in point cloud data 500* (composed of data sets 501*, ... 506* in FIG. 3b). A basic geometric shape can be thought of as a collection of three-dimensional points that can be connected by edges to form faces between those edges. To minimize the computational load, the number of points of the underlying geometry 510* should be kept as small as possible while still achieving the underlying 500* data representation of the point cloud. In this case, the error in the coefficient of determination between the base geometry 510* and the data points 540* from the point cloud data 500* may be 0.75, preferably 0.8 or 0.9.

The base geometry 510* may be translated, rotated, and scaled until the difference between the base geometry 510* and the data points 540* from the point cloud data 500* is minimized (e.g., using least squares regression or correlation maximization). .

It should be noted that the impact crusher 200, high-pressure roller crusher 300, media agitation mill 400 (shown in FIGS. 1B-1D), and the horizontal or vertical mill may also use one or more cylinders as the base geometry 510*. whereas the crusher 100 (shown in FIG. 1A) may use a pair of planes as the base geometry 510*. Here, to ensure that manipulation and comparison with the base geometry 510* does not require large computational costs when estimating the geometric shape of the mining equipment, a base geometry 510* with a small number of edges is used, e.g., less than a certain number of edges, e.g. 100.

It should also be noted that the point cloud data 500* may first be meshed, and then the underlying geometry 510* may be placed within that mesh and compared to it. During meshing, the coordinate information of each data point 540* may be considered a vertex and connected to adjacent vertices (eg, based on the coordinate information of other data points 540*) via edges to form edges. By applying smoothing filters and/or vertex merging filters to the mesh, the number of edges, vertices, and faces is reduced. Therefore, comparing the base geometry 510* with the filtered mesh to align the base geometry 510* becomes even less computationally intensive.

By using the underlying geometry 510*, gaps or missing data in point cloud data 500* can be identified and reported. For example, when there are insufficient data points 540* located on or near the surface of the base geometry 510*, an area 550* indicating insufficient data can be determined. In other words, such gaps indicate missing data that can be identified using the underlying geometry 510*.

The region 550* indicating insufficient data can be identified as follows. The data sets 501*, ... 506* may be mapped onto the surface of the base geometry 510*, and areas of the base geometry 510* not covered by the data sets 501*, ... 506* may be identified as an area 550* indicating insufficient data. In this case, a circle of each data set 501*, ... 506* based on the scanning range 31 of the sensor 30 can be used to track the scanned surface and the non-scanned surface of the underlying geometry, allowing the area of (missing or) insufficient data to be determined. Likewise, the number of data points 540* located within an area, i.e., “data point density,” can also be used to indicate whether the data set 501*, ... 506* is sufficiently populated, allowing the area of insufficient data to be determined. Likewise, by calculating the absolute values of gradients across a 500* point cloud data surface, values greater than a certain threshold indicate an area of high measurement uncertainty, allowing the identification of an area of (poor or) insufficient data.

In fig. 11A-11C show an example of how the absolute values of gradients over a surface of 500* point cloud data can be calculated. In this case, FIG. 11A corresponds to the cross section of liner 520 shown in FIG. 2D. For comparison, the shape of the lining 520 is shown in FIG. 11A. Scanning the liner 520 will result in a finite number of data points 540* that can be plotted as shown in FIG. 11B. As described above, each data point 540* represents a point in space corresponding to the interior surface of the mining equipment, such as the liner 520 shown in FIG. 2D or illustrated in FIG. 11A. In fig. 11B, data points 540* are represented as a distance y from sensor 30 to that point in space, with each distance y measured for a different orientation angle x of sensor 30. The combination of data points 540* in FIG. 11B, representing the y distance and x angles, is called the xy function, which also represents the inner surface of the mining equipment.

The gradient or mathematical differentiation of this function xy is calculated by dividing the change in distance to the inner surface dy by the change in angle dx (that is, dy/dx). For simplicity, assume that the change in angle is constant, then the gradient for the two data points 540* in FIG. 11V can be calculated as dy1/dx and dy2/dx. In this case, dy1/dx returns a small and negative value compared to dy2/dx which returns a large and positive value, as shown in FIG. 11C. Based on the absolute value of these gradients, large values greater than a certain threshold (eg, a statistical value of all y values, such as the top quartile) may be determined to indicate inaccuracies and/or gaps in the point cloud data. As shown in FIG. 11B, due to the scanning dead zone 32 shown in FIG. 2D, a gap of size dy2 appears, thereby indicating that part of the surface has not been completely scanned.

Instead of comparing the absolute value of gradients (or mathematical differentiation of the xy function), to save computational load, one can compare the absolute value of the change in distance dy with a threshold value.

In the above, distance y is used as the distance from the sensor 30 to the inner surface. If the sensor does not remain in the same position inside the mining equipment when rotating, the distance y can be retrieved using the basic geometry 510*. In particular, the distance y may be the distance from each data point 540* to the nearest surface of the underlying geometry 510*. As such, the y values represent the Euclidean distance from each point 540* of the point cloud data 500* to the surface of the underlying geometry 510*. The x value may then be based on the circumferential location of the data point 540* of said base geometry, for example, simulating a stationary and rotating sensor 30. In other words, the xy function may be determined by normalizing the point cloud data 500* based on the base geometry 510*.

When applying any of the above methods, one skilled in the art will understand that when the change in distance dy or gradient/mathematical differentiation of function xy exceeds a certain threshold value, the region is defined as a region of high measurement uncertainty. This area of high measurement uncertainty may also be displayed on the surface of the base geometry 510* as an area 550* indicating insufficient data. Equally, scanning dead zones 32 causing discontinuities in data sets 501*, ... 506* and point cloud data 500* can also be identified on the surface of the underlying geometry 510* and defined as an area 550* indicating insufficient data.

Considering the area 550* indicating insufficient data, if inspection personnel perform a "virtual painting" using the sensor 30, the scanning information may be transmitted to these personnel to correct the "virtual painting." For example, the display may indicate where an area 550* indicating insufficient data is located. In this case, the area 550* indicating insufficient data may be referred to as an identified area 550* (e.g., that has been identified on the surface of the underlying geometry 510*) and may be color coded or otherwise highlighted to notify inspection personnel of its location. regarding the position and orientation of the sensor 30. Thus, the display instructs and/or directs inspection personnel to move the sensor 30 such that the identified area 550* is (re)scanned. In addition, the identified area 550* may also be used to automatically reorient and/or move the sensor 30 to (re)scan the identified area 550*.

Below with reference to FIG. 8 explains the procedure for directing a user or inspection personnel to an identified area 550*. In this case, if the (total) area of the identified area 550* is equal to or greater than a predetermined size (eg, above a predetermined value), the guiding procedure is repeated to continue scanning the identified area 550*. Once the area of the identified region 550* falls below a predetermined size (or below a predetermined value), the guiding procedure concludes that sufficient data of sufficient quality have been obtained to perform the analysis.

Specifically, during the guiding procedure, the display displays a portion of the base geometry 510* (S11) and identifies the area of the identified region 550* (S12). However, based on the orientation of the displayed view, for example, due to the position and rotation of the sensor 30, the identified area 550* may be outside the scanning range 31 of the sensor 30. Therefore, if the area of the identified area 550* is greater than or equal to the specified size (S13: YES), the guiding procedure determines the “next coordinate” (S14). The next coordinate is associated with or within the identified region 550* and is preferably located on the surface of the base geometry 510*. This next coordinate is retrieved and highlighted (S15), for example, by displaying a portion of the surface of the base geometry 510* located near the next coordinate in a different color or by animating an arrow on the display indicating the direction in which the sensor 30 should rotate and/or move, that is, move so that its scanning area 31 overlaps the specified next coordinate.

In fig. 12A and 12B illustrate an example of a visual display of the following coordinate. In fig. 12A performed a scan comparable to that of FIG. 2B, wherein a scan of the identified area 550* is required to complete the scanning process. The next coordinate 541* has been identified within the identified area 550* and serves as a target for continued scanning. Let's assume that the scanner is currently pointed towards the area labeled C-C in FIG. 12A, then the scanning range of this scanner does not cover the specified next coordinate. Specifically, an exemplary display corresponding to the scanning area as shown in FIG. 12B, within the field of view does not contain either the identified area 550* or the next coordinate. Therefore, to indicate the direction in which the sensor must be moved to cover the next coordinate 541* and scan the identified area 550*, the display in FIG. 12V can superimpose arrow 542*.

Thus, the guiding procedure causes the sensor 30 to move in the direction to the next coordinate until the next coordinate falls within the scanning range 31 of the sensor 30 (S16). As a result, the identified range 550* is (re)scanned, whereby more data is added to the point cloud data 500*, and the area size of the identified region 550* (eg, the insufficient data region 550*) is reduced.

It should be noted that by locating the next coordinate within the identified area 550* as close as possible to the edge of the scanning range 31 of the sensor 30, the required movement to (re)scan the identified range 550* is minimized. Determining the next coordinate in this manner speeds up the scanning process because movement of the sensor 30 is minimized. This can be particularly useful when the sensor 30 is controlled automatically or autonomously, for example, using a robot or drone that has limited battery life and time to complete the scanning procedure. Alternatively, when the sensor 30 is moved manually, the user or inspection personnel are notified of the position of the next coordinate and instructed to move the sensor 30 in the direction to the next coordinate until the next coordinate falls within the scanning range 31 of the sensor 30, while minimizing said required movement , as described above, physical work and stress on the user or inspection personnel are reduced.

When the next coordinate falls within the scanning range 31 of the sensor 30, a further, for example, third, data set 503* containing data points 540* with coordinates is obtained from the sensor 30, as described above (S17). To further combine this third data set 503* with the already acquired point cloud data 500*, features 520* are extracted from the point cloud data 500* and the third data set 503* as described above (S18). The third set of data 503* is then positioned and aligned with the point cloud data 500* and combined into the point cloud data 500* as described above (S19). The guiding routine then re-evaluates the base geometry 510* and returns such a result to guide scanning of the remaining identified region 550* (S19-S11).

Because the 500* point cloud data captures and represents more geometric information and details of the mining equipment, the following estimate of the underlying 510* mining equipment geometry is improved because it is based on the updated 500* point cloud data. In addition, as the point cloud data 500* has increased or changed, the region 550* indicating insufficient data may also have decreased or changed. Therefore, the identified region 550* located on the (re)estimated geometry 510* and associated with insufficient or low-quality data is also re-identified as described above (S12), but using the updated point cloud data 500*. As a result, the area of the identified region 550* may decrease or change.

The above guiding procedure is repeated from re-estimating and displaying the base geometry (S11) to updating the point cloud data 500* until the identified area of the identified region 550* (S12) is greater than or equal to a predetermined value (S13: YES) . If multiple identified regions 550* are identified, the above guiding procedure is repeated until the sum of all areas of the identified regions 550* is greater than or equal to the specified target value.

After the area of the identified region 550* becomes smaller than a predetermined value (S13: NO), analysis (eg, fault, wear and/or degradation analysis) is performed based on the point cloud data 500*, and the scanning procedure ends (S20). Thus, since the insufficient data region 550* is below the threshold, the completeness and quality/resolution of the point cloud data 500* is ensured. As a result, inspection times are reduced, improving the safety of inspection personnel and reducing downtime for mining equipment.

Without this human-machine guidance procedure, data may be missing, insufficient or erroneous, which will only be noticed during subsequent analysis of the resulting 500* point cloud data. Consequently, unnecessary repetitions of the scanning procedure are avoided.

For man-machine procedure guidance, the above-mentioned display with sensor 30 may be installed, for example, if the scanning device including the display and sensor 30 is portable. When remotely controlling the position and orientation of the sensor 30 from outside the mining equipment, the display may be part of an augmented reality (AR) or virtual reality (VR) suite worn by personnel. VR may also be used if the sensor 30 is supported by a flying drone or robotic gimbal embedded in mining equipment. As a result, inspection personnel can still perform "virtual painting" but are not required to enter the hazardous environment found in or around mining equipment.

If the movement of the sensor 30 is controlled autonomously, that is, with the help of a control system controlling the flying drone or gimbal, the flight or movement plan can be dynamically adjusted based on the identified coordinates.

[Alignment of main/independent components]

When combining a new data set 501*,... 506* with another data set 501*,... 506* or point cloud data 500* to align the new data set 501*,... 506*, principal components may be extracted from features 520* (OC) or independent components (IC) (hereinafter OC may also include IC). Ultimately, OCs are indicators that are unique to each 520* feature and indicate the orientation and scale of the corresponding 520* feature. Therefore, when data and processing requirements are low, the OC can be linearly transformed to align the data, allowing for faster alignment of features 520* and hence a new data set 501*, ... 506*.

For example, the alignment between data sets 501*, ... 506* and/or point cloud data 500* may be indicated by the dot product of features 520* and, preferably, OK. If multiple features 520* and/or OK have been extracted, the alignment of data sets 501*, ... 506* and/or point cloud data 500* can be specified by convolution and/or correlation of features 520*, and preferably OK. Because dot product, convolution, and/or correlation can (in addition to the least squares regression mentioned above) provide an indication of alignment or correspondence, operations on OK can be performed with significantly less computational burden compared to the same operations performed on points 540* extracted feature data 520*.

In this regard, as already described above, the OK alignment also includes a linear transformation, preferably rotation, scaling and/or translation of the data set 501*, ... 506* and/or point cloud data 500* to ensure maximum alignment and/or fit.

[Receiving data while working]

As described above with reference to FIG. 2A-2D and FIG. 3A and 3B, point cloud data may be acquired by moving the sensor 30 relative to the mining equipment (e.g., drum mill 500) to obtain complete, high quality point cloud data using an automated or continuously guided human-machine interaction process (e.g., the guiding process of FIG. 8). The embodiment described above is aimed at inspecting mining equipment during downtime. An additional option for inspecting mining equipment during operation is described below.

To better illustrate and explain the following embodiment, the drum mill 500 and its geometric shape are used to explain the process of operating mining equipment. However, other types of mining equipment (as described above or shown in FIGS. 1A-1E) may equally be used.

In fig. 4A-4E show a sensor 30 located inside the drum mill 500, that is, its drum container 510. In this case, the drum mill 500 operates and rotates, for example, in direction B, thereby performing its job of extracting the rock material 10 using the lining 520 installed inside the drum mill 500. The sensor 30 is moved along the interior of the mining equipment, for example, by rotating it, for example, in direction C (see FIGS. 4A-4C) and/or by moving it substantially parallel to the axis of rotation of the container 510 mining equipment (see Fig. 4D (non-rotating sensor) and Fig. 4E (rotating sensor)). In this case, the mining equipment, that is, the drum mill 500, rotates around its rotation axis during operation. By using the sensor 30, the first point cloud data and the second mining equipment point cloud data are obtained in a similar manner to the procedure described above and shown in FIG. 3A and 3B, where multiple partially overlapping data sets 501*, ... 506* are collected, positioned, aligned, and merged into each of the point cloud data 500*.

In the present embodiment, the first and second point cloud data represent various passes or scans of a surface located within the mining equipment. This surface may be the mining equipment structure (or its lining 520) or may be rock material 10 (eg, rock or cuttings) located on or covering the surface of the mining equipment structure. The overlap of the first and second point cloud data represents the same piece of mining equipment, but at different times and/or in different orientations.

Fig. 4A-4C are provided to illustrate how different point cloud data (representing different passes or scans of the same surface of mining equipment) can be obtained. For example, to obtain the first point cloud data, sensor 30a may begin scanning the interior of drum container 510 with scan range 31 oriented as shown in FIG. 4A. The sensor 30 is then rotated 360°*n+90° in direction C (where n is a natural number including zero) and the drum 30 is rotated at least 270° in direction B until the situation shown in FIG. 4B. In this case, the scanning range 31 of the sensor 30 is rotated by at least (n+1)*360° relative to the mining equipment. By continuously scanning, such as acquiring, positioning, aligning, and merging data sets during this rotation, sensor 30 can scan the entire interior surface of drum container 510 at least once. Therefore, based on the output of the sensor 30 during the transition from FIG. 4A to 4B obtain first point cloud data. The same procedure is repeated when moving from FIG. 4B to 4C and obtain the second point cloud data. In other words, the second point cloud data is obtained after the first point cloud data and after the sensor 30 and/or the mining equipment has been/has been moved. As a result, the resulting first and second point cloud data represent the same parts of the mining equipment structure (eg, drum mill 500), but with different obstructions on the scanned surface caused by the mining material 10, such as rocks and cuttings (the latter not shown).

It is worth noting that the sensor 30 may also be rotated faster to achieve a full rotation of 90°+n*360° (eg, where n is a natural number greater than zero) in FIG. 4B relative to FIG. 4A. Thus, sensor 30 seamlessly scans the interior surface of the mining equipment (at least partially) more than once. In this way, higher quality data of the first point cloud can be obtained, in particular since the appearance of dead zones 32 (see FIGS. 2C and 2D) can be omitted. In particular, dead spots can be avoided when the sensor 30 is not located at the center of rotation of the mining equipment (eg, drum vessel 510), but in an offset location. In this arrangement, the sensor 30 scans surfaces at various angles, allowing scanning dead zones, i.e., those resulting from obstructive lining 520 protruding into drum vessel 510 at a certain angle, to be compensated. For example, in FIG. 4A-4C, the sensor may be positioned further to the upper right to extend the scan range 31 into the area behind the liner protrusions 520. Thus, sufficient and high quality data of the first point cloud can be obtained. The above also applies to the second point cloud data.

In addition, if there are no significant dead spots for scanning, for example, if the liner 520 is missing or its protrusions are insignificant for the scanning operation, the sensor 30 may not be rotated at all, and only the drum container 510 may be rotated during acquisition of the first and second point clouds. To minimize the likelihood that the surface of the mining equipment scanned by the sensor 30 will be covered or obstructed by rock material 10, the sensor 30 may be directed in the direction where the majority of the rock material 10 is most likely to have separated from or moved away from said surface (e.g. , Fig. 4A). Additionally, when using such a “fixed” or non-rotating sensor 30, a stationary rock protection 10, such as a cage, may be installed around the sensor 30, but without protrusion into the scanning range 31. To avoid synchronizing the rotation of the sensor 30 and the drum vessel 510, the sensor 30 is preferably rotated in a direction C opposite to the rotation direction B of the mining equipment. When rotating in opposite directions, the sensor 30 can rotate at any angular speed, for example, faster, as fast, or slower than the angular speed of the mining equipment. However, point cloud data acquisition is completed after the interior of the mining equipment has been scanned at least once.

In addition, the method explained in [Acquiring Data] can also be used to obtain 500* point cloud data. In this case, the identified area 550* may be scanned when the surface of the mining equipment associated with the identified area 550* is less likely to be covered or obstructed. For example, it is less likely that the surface of the mining equipment located within the scanning range 31, as shown in FIG. 4A is covered by the mining material 10, while it is more likely that the surface of the mining equipment within the scanning range 31, as shown in FIG. 4C is covered by rock material 10. Consequently, the rotation of the sensor 30 may be automatically adjusted so that the alignment of the scanning range 31 with the identified area 550* (or the above-mentioned next coordinate) occurs at the time when the surface of the mining equipment associated with the identified area 550* , is less likely to be covered or obstructed by rock material 10 (eg, FIG. 4A). Additionally, to prevent the rock material 10 from moving excessively and/or unnecessarily blocking the scanning range 31, the mining equipment 500 may be rotated at an angular velocity equal to or less than the angular velocity during normal operation. In this case, the mining equipment 500 can be continuously moved slowly (eg, moved forward at a very low angular velocity), avoiding the need to stop and secure the mining equipment 500 between each scanning iteration. To protect the sensor 30 from rock material, such as that caught and falling from the liner 520, a cage may be provided surrounding the sensor 30.

After receiving the first and second point cloud data from the sensor 30, data points constituting the surface of the mining equipment are extracted from the first and second (ie, plurality) point cloud data. This takes advantage of the fact that the mining equipment continues to rotate while scanning, since its rotation exposes the surface of the mining equipment that would be covered with rock or cuttings if the mining equipment were scanned while stationary.

An example of a method for determining the data points that make up the surface of mining equipment is described. First, based on data from each of the first and second point clouds, surfaces located inside the mining equipment are determined. Based on the identified surfaces, the location of data points representing the surface located inside the mining equipment is then estimated, and point cloud data is generated based on this estimate.

For example, for each point cloud data, the surface of the interior of the mining equipment is determined and stored. The surfaces determined from each point cloud data are then aligned and compared to classify or evaluate which portions of the surfaces represent the mining equipment structure and which portions of the surfaces represent the rock material 10 (eg, rock or cuttings) covering the mining equipment structure. In this case, the alignment can be performed in a similar manner to the feature extraction and alignment explained above. However, information related to the relative rotation of the sensor 30 and the mining equipment can be used to perform an initial alignment, while the alignment procedure mentioned above using extracted features is used for fine alignment. For example, to assist with alignment, a reference mark located on the mining equipment and on the sensor 20 may provide information regarding their respective orientations. Additionally, control commands to rotate the mining equipment and/or sensor 30 may be used to determine their current orientation. Therefore, the acquired first and second point cloud data are rotated based on the rotation angles of the sensor and mining equipment.

Based on this classification or evaluation, only those portions of the surfaces that represent the design of the mining equipment are extracted, combined, and used to evaluate the geometric shape or data points corresponding to the interior surface of the mining equipment. An example of such an assessment is explained below.

With reference to FIG. 5A, assume that the first and second point cloud data (eg, Fig. 5A(a) and (b)) are obtained based on the method described above. For simplicity, the rock material and lining have been omitted and FIG. 5A depicts only a portion of the rotating drum vessel 510 and the cuttings 11. However, the same procedure is equally applicable to rock material 10 other than cuttings 11. Because the drum tank 510 rotates and is subject to varying or alternating forces from the rock material 10 and/or slurry 11, the coating located on the inner surface of the drum container 510 changes from the first scan to the second scan. Therefore, the corresponding data from the first point cloud and the data from the second point cloud indicate different internal surfaces, although the same surface of the mining equipment was scanned (at least in part). Based on the first and second point cloud data (and/or additional point cloud data, such as the third through fifth point cloud data shown in FIG. 5A(c)-(e)), a plurality of scanned surfaces corresponding to the same surface are aligned and compared. mining equipment.

For example, as shown in FIG. 5B, from a plurality of point cloud data indicating the frequency or probability that a surface is at a certain distance from the sensor 30, a frequency or probability density 11* may be obtained. In particular, when focusing on the region B-B in FIG. 5B, a Weibull-like distribution similar to that shown in FIG. 5C. In this case, with a change in the distance d to the sensor 30, the frequency or probability density p of the surface changes. In the example shown in FIG. 5C, the highest probability density is at distance d(1). In this case, it is assumed that distances close to distance d(2) could be caused by cuttings 11 getting stuck on the surface of the mining equipment structure. It is also suggested that distances close to the d(0) distance could be the result of measurement errors or measurements where the surface of the mining equipment structure was cleaner than normal. In the case where the measurement error is sufficiently small, for example, when the permissible measurement error reaches a value in the range of ±0 mm to ±50 mm, preferably from ±1 mm to ±10 mm or from ±1 mm to ±5 mm, the distance between d (0) and d(1) can be reduced to zero.

Based on this probability density distribution, the distance from the sensor 30 to the surface of the mining equipment can be classified as falling between or on the distances d(0) to d(1). When the measurement error is negligible, the distance from the sensor 30 to the surface can be determined as d(1). When applying such a classification not only to the B-B area, but to the entire periphery or surface of the mining equipment, that surface of the mining equipment and therefore the geometric shape or data points representing the surface of the interior of the mining equipment may be assessed, despite the fact that said mining equipment is in progress. The resulting point cloud data containing these evaluated data points can then be used for subsequent analysis. As a result, assessments can be performed without stopping mining equipment. In this way, not only is downtime for mining equipment reduced, but it can even be avoided.

[Inspection System]

As mentioned above, in order to facilitate the inspection of mining equipment, the display part of a computer monitor, AR or VR kit can be used. Below with reference to FIG. 9A provides more detailed information related to inspection through the use of inspection system 50 with display 60.

In this case, the inspection system 50 also includes a sensor 30 configured to sense distance to a surface (also referred to as "scanning"), such as one located on the interior of mining equipment, a tracking device 51 configured to track the location and orientation of the sensor 30, and a point cloud generator 52 configured to generate point cloud data based on the specified perceived distance and location and orientation tracked by the tracker 51. In this case, the tracker 51 may be configured in conjunction with the sensor 30 and may output tracking information of the sensor 30, or tracker the device 51 may not be implemented together with the sensor 30, but, for example, is implemented adjacent to the point cloud generator 52. Most importantly, the sensor 30 and tracker 51 are conical in shape, configured to output their data to a point cloud generator, which may be configured to sample, for example at regular sampling intervals, coordinate values and angle values related to location and orientation of the sensor 30. In addition, the point cloud generator 52 is configured to calculate the coordinates at which the surface sensed by the sensor 30 is located. These calculated coordinates constitute the individual point cloud data points. When sensor 30 is configured to sense multiple distances to a surface, such as by generating a depth image or depth map, each sensed distance can be used to calculate a coordinate of said surface, allowing multiple coordinates to be calculated simultaneously, increasing scanning speed. In other words, if a depth image is created, each pixel location of the depth image corresponds to a horizontal and vertical angle from the central sensing axis of the sensor, and each pixel value corresponds to a distance from the sensor 30 to the surface (the sensor 30 in accordance with other embodiments may similarly use the image depths). Therefore, the point cloud generator 52 may be configured to generate a data point associated with each pixel while calculating the coordinates constituting the data points from the point cloud data.

The inspection system 50 also includes a surface estimator 53 configured to evaluate one or more surfaces based on point cloud data, and a geometry estimator 54 configured to estimate the (basic) geometric shape of mining equipment based on the surface(s) being evaluated. . The display 60 is configured to display the surface being evaluated and/or the geometric shape being evaluated. In this case, the display 60 may be combined with a point cloud generator 52, a surface estimator 53, and a geometric shape estimator 54, but may alternatively be a stand-alone device. Surface evaluation and geometric shape evaluation can be performed similarly to those described above, for example, in the [Data acquisition] section.

To ensure the completeness and/or quality of the acquired data, the point clouds may use the (underlying) geometric shape of the mining equipment to indicate an area 550* indicative of insufficient data (eg, an identified area 550*). Preferably, various shading, outlining, coloring, and the like may be used as visual feedback to indicate differences in point cloud data density and/or differences in the fidelity of the estimated surfaces and/or geometric shape. In this case, the uncertainty may be based on the mixed correlation coefficient or the R ² value of the surface portion and/or geometric portion being evaluated. In particular, this visual feedback provides information to the user or inspection personnel, such as information about an area 550* indicating insufficient or poor quality data, which enables the user or inspection personnel to quickly identify the specified area and ensure that the resulting point cloud data sufficiently populated with high-quality data to perform subsequent analysis; for example, by re-scanning pieces of mining equipment corresponding to a specified area, as described above. Displaying the surface and/or geometry being evaluated in this manner is also useful when inspecting mining equipment remotely, for example when it is in operation and the user or inspection personnel responsible for the inspection cannot enter the mining equipment.

In addition, the tracking device 51 may be configured to track the location and orientation of the display 60, and the display 60 may be configured to display the surface (mining equipment) being assessed based on the location and orientation of the display 60. Therefore, the use of AR or VR kits provides the user with or inspection personnel, the ability to better control or manipulate the location and rotation of sensor 30 to obtain point cloud data that is sufficient to perform subsequent analysis.

However, in some cases, the user or inspection personnel inspecting and/or operating the sensor 30 to obtain point cloud data may rely on the expertise of others to ensure that the resulting point cloud data is sufficient to perform subsequent analysis. Equally, others may wish to focus subsequent analysis on areas of interest. Areas of interest may include, for example, those areas that are expected to be subject to excessive wear or that have not been inspected for an extended period of time. Therefore, the point cloud data may be transmitted to a terminal, computer, or VR/AR suite of the person providing the expertise, allowing that person to indicate where the area(s) of interest is located in the mining equipment. To color an area located on a (basic) geometric shape and mark that area as an identified region 550* and/or a region of interest, the person providing the expertise may implement and use a virtual torch. Information related to this area/areas of interest is then returned to the inspection system and displayed on the display 60, similar to an area 550* indicating insufficient data (eg, an identified area 550*). The user or inspection personnel can then be informed of this area and can perform scanning based on expert knowledge.

However, there may not be an option to transmit point cloud data of all mining equipment, for example, when the communication channel between the inspection system and the expert is insufficient to transmit large volumes of data. Therefore, to reduce the amount of data transmitted, at least the area of interest of the mining equipment must be scanned and its data must be transmitted.

Therefore, as shown in FIG. 9B, the inspection system 50 may also include a remote display 61, for example, which hosts an expert, a mining equipment database 55, a sub-cloud locator 56, and a data transceiver 57. In this case, the mining equipment database 55 is configured to store a template geometric shape of the mining equipment, for example, based on CAD data of the mining equipment and an area of interest of the mining equipment. Subcloud determiner 56 is configured to extract from the point cloud data a subset of data in the form of subcloud data based on a region of interest. The mining equipment database 55 and subcloud identifier 56 may be located adjacent to tracker 51, point cloud generator 52, surface estimator 53, and geometry estimator 54, as shown in FIG. 9B. In this case, the data transceiver 57 is configured to transmit sub-cloud data to and from the remote display 61. Thus, the area of interest is transmitted to the person providing the expertise, and input from that person (e.g., input through an input device located adjacent to the remote display 61) is returned to allow said input to be displayed on the display 60.

Alternatively, the mining equipment database 55 and sub-cloud locator 56 may not be located adjacent to the tracker 51, etc., as described above. For example, if large servers are required to store the template geometry of all mining equipment, it may not be feasible to include them in the device along with the tracking device 51, etc. In this case, the transmitter 57 transmits data containing the estimated geometry and an indication of the scanned surface to the sub-cloud identifier 56. In this case, the indication of the scanned surface may be a color difference or a parameterization of the estimated geometry in accordance with the scanned point cloud data. Thus, in order to inform which area/part of the estimated geometry has been scanned, not all of the point cloud data needs to be transmitted. After receiving this data, the sub-cloud determiner 56 retrieves the template geometric shape of the mining equipment from the mining equipment database 55 and overlays the obtained data. Subcloud locator 56 then causes the display 61 to display a template geometric shape with the data overlaid to inform the person at the remote display 61 which piece(s) of mining equipment have been scanned.

In addition, the person providing the expertise may enter into the mining equipment database 55 information about what portion of the mining equipment template geometry constitutes the area of interest. This input can be done in advance, before the inspection or during it, but remotely. In the latter embodiment, the input to indicate an area of interest and the transmission of information related to the specified area of interest require a relatively small amount of data, which can still be transmitted over the above-mentioned communication link. Therefore, the remote computer may be co-located with the remote display 61 and configured to receive input to define another area of interest and to store said area of interest in the mining equipment database 55 .

In the case where the mining equipment database 55 and the sub-cloud locator 56 are located together with the tracker 51, etc., the sub-cloud locator 56, based on the comparison between the template geometry and the estimated geometry, can determine the location of a region of interest on the estimated geometry and can extract a subset of the 500* point cloud data as sub-cloud data to be transmitted. In this case, the sub-cloud data may be transmitted each time it is updated by scanning (eg, continuously) or after scanning is completed, such as when the identified area 550* is small enough. By transmitting the sub-cloud data, each time it is updated, new information about the mining equipment can be displayed on the remote display 61, allowing the expert to determine whether a new point of interest can be added to the mining equipment database 55. Therefore, collaborative scanning by the user or inspection personnel can be achieved based on feedback from the person providing the expertise to ensure that the scan covers each area of interest before performing analysis of the mining equipment. In addition, the amount of data transmitted by the transceiver 57 is reduced.

Additionally, the display 60 may be configured to highlight an area/areas of interest on the display 60 to indicate an area of interest to the user or inspection personnel receiving the point cloud data 500*. Therefore, special attention may be paid to obtaining the 500* point cloud data that constitutes the subcloud data. As a result, the user is directed to perform scanning or retrieving data in a manner that ensures that the 500* point cloud data constituting the sub-cloud data is of sufficient quality and completeness. Such special attention may be especially important if different qualitative thresholds are applied to different regions of interest (for example, by defining different maximum variances of data points for different regions of interest).

If the point cloud data 500* does not contain data points in a region of interest, a surface and/or geometric shape to be estimated may be extracted at the location of that region of interest. This highlight may be displayed on the display 60 using a color or an arrow, as explained above. To determine which regions do not contain (sufficient) data points, a gap detector can be used, which is configured to detect coordinates on the geometric shape being estimated for which the number of points of the specified point cloud data is below a given value. Additionally or alternatively, the surface gradient is calculated as described above, and a gap is determined where the gradient exceeds a predetermined value. Therefore, these detected coordinates constitute "gaps" in the resulting point cloud data 500*, which can be highlighted on the estimated geometry displayed on the display 60.

The above is summarized in the feedback procedure shown in FIG. 10. In this case, the inspection system 50 obtains one or more areas of interest from the mining equipment database 55 (S101). Then, the mining equipment is scanned (eg, according to the procedure explained above) and 500* mining equipment point cloud data is generated (S102). Then, the surface to be evaluated and/or the geometric shape to be evaluated is evaluated and displayed on the display 60 indicating the region of interest (S103). In this case, this indication can be done by color or an arrow indicating the direction of the area of interest. Then, the point cloud data 500* is transmitted to the remote display 61 (S104). In this case, all point cloud data 500* or the aforementioned sub-cloud data (corresponding to point cloud data 500* in the region(s) of interest) may be transmitted. The area of interest may then be updated and entered into the mining equipment database 55 (S105), for example, if the expert identifies new areas of interest during scanning. If it is determined that the point cloud data 500* are sufficient (eg, sufficiently dense) and of sufficiently high quality (eg, low variance of data points) in each region of interest (S106: YES), perform an inspection analysis (S107). Otherwise (S106: NO), the procedure returns to extracting or acquiring POIs (including new POIs) and guided scanning is continued.

[Virtual inspection]

According to another embodiment, a computer-implemented method in FIG. 1 is proposed for virtual inspection of the interior of mining equipment. 6 (as described above). Virtual inspection refers to a method and technology for maintaining, inspecting, testing, monitoring, recognizing failures, providing guidance regarding the interior of mining equipment from one or more locations outside said mining equipment using virtual reality technology, also called augmented reality kit (AR) or virtual reality (VR) worn by the personnel mentioned above, for example, VR headsets or head-mounted devices such as Oculus Rift, or VR glasses or VR headsets, which can provide a stereoscopic display to the user. This kind of analysis may be performed during step S20 in FIG. 8 or step S107 in FIG. 10, or after them.

According to the first step (S1) of the computer-implemented method in FIG. 6, the first set of data (point cloud data as described above) is obtained. As described above, the first data set contains coordinate data points of the interior of a mining equipment such as a mill, such as a horizontal or vertical mill, crusher, shredder, or mining equipment, as described above. Preferably, the data points define in three-dimensional space the interior of the mining equipment, that is, geometric shapes, surfaces, directions, orientations, alignments, etc., defining the physical appearance of the interior of the mining equipment. The data points may also include data points for linings installed on appropriate surfaces of the mining equipment to protect it from excessive wear. The data points may further contain information related to the reflectance property of scanned points of the mining equipment (e.g., the intensity of reflection of the sensor's laser light from the surface texture or composition of the surface at corresponding points of the mining equipment), that is, the reflectance property of the material forming the surface of the mining equipment.

Preferably, the first set of data may be obtained using one or more sensors, such as sensor 30 described above, a three-dimensional scanning device, a mobile scanning device such as a mobile or flying drone, or a robotic gimbal as described above, or from another source. The first data set may also contain data points of a specific production data set, that is, a definition of the interior of the mining equipment as it was originally manufactured or planned to be manufactured, for example, as defined by a CAD data set or the like. One of ordinary skill in the art will appreciate that any of these exemplary data sets may be obtained by inputting said data sets into a computing device, such as a laptop, a computer workstation, a cloud computer, a computer data server, and the like.

According to the second step (S2) of the computer-implemented method of FIG. 6, the resulting first set of data is subsequently converted into a second set of data that is adapted (eg, has a suitable format) for use by a virtual or augmented reality device, such as a VR headset, VR glasses, VR headset, or other VR head-mounted device, as described above. One skilled in the art will appreciate that this transformation engine generates a virtual (software-based) geometric data set to be used by a virtual or augmented reality device such that the user of said virtual or augmented reality device is provided with the sensation of inspecting the interior of the mining equipment, that is, he has a visual perception of it. In other words, the wearer of the virtual or augmented reality device is provided with a three-dimensional virtual reality view of the interior of the mining equipment and can virtually inspect or navigate within the interior of the mining equipment by moving the virtual or augmented reality device or an external input device for the virtual or augmented reality device to inspect the interior of mining equipment. Additionally, to better illustrate which piece of mining equipment is composed of which material/materials, shading or color coordination of the virtual reality view can be implemented based on the reflectivity information of the data points. For example, each pixel of the second data set may be provided with three-dimensional information identifying points in space related to mining equipment, as well as a color or shading value related to said reflectance information. This improves visual perception during virtual inspection of mining equipment.

Such external input device may be an external motion controller (such as a joystick), a haptic input controller, recordable movement and rotation of a head-mounted device, or the like.

In this case, the transformation step described above, which can also be considered a post-processing step of the resulting first data set, can automatically include checking whether all required or relevant (eg for virtual inspection purposes) data points of the interior of the mining equipment have actually been obtained. Such a check may be performed regarding the identification of insufficient/sufficient data as described above. That is, the transformation can be combined with the method described above for obtaining point cloud data. In other words, based on the known physical geometry of the mining equipment as it is manufactured, for example, post-processing may already identify missing data points and thus request the provision of these missing data points, for example, by requesting a sensor, a 3D scanning device , mobile scanning device or robotic pendant to obtain missing data points. This prevents delays in providing virtual inspection to users.

According to the third step (S3) of the computer-implemented method of FIG. 6, one or more users are guided through a virtual inspection of the interior of a mining facility based on a second set of data by moving the visual perception of said user(s) to one or more points of interest (POIs), also referred to above as areas of interest, within mining equipment.

In this case, the POIs of the interior of the mining equipment may refer to certain areas of the interior of the mining equipment that represent critical wear areas, areas that are particularly susceptible to wear, certain areas of the lining, areas that have already been inspected in the past and for which the user requires obtain updated information regarding the current state of wear before deciding on replacement or the like.

Moving the user's visual experience may be achieved such that the user controls a virtual or augmented reality device or an external input device for a virtual or augmented reality device such that the user has the visual impression of moving or viewing the internal environment of mining equipment and, in particular, inspecting or pointing at various POIs.

In virtual or augmented reality, many users can also be represented in the form of corresponding virtual representatives (avatar, etc.). To navigate the interior of mining equipment, virtual representatives can follow input from users, for example following changes in visual perception. This provides an improved ability to recognize whether other users are looking at and/or moving towards other pieces of mining equipment.

The second data set may further contain data that refines the POIs such that the generated VR environment for the virtual inspection already contains pointers to the POIs in visual perception. That is, the second set of data may identify regions of POIs that are to be highlighted for visual perception in a manner similar to the selection of the identified region 550* described above. Such identification may be accomplished, for example, by adding flags or labels to the POIs defined in the second data set, as well as by determining how the pointers should be generated in the VR environment, for example, the shape of the pointers, the color of the pointers, the orientation of the pointers, etc. P.

When multiple users are logged into virtual or augmented reality, changes in the appearance of a piece of mining equipment displayed in virtual reality can be initiated, for example using a virtual flashlight to enable individual users to quickly point out and communicate additional (possibly unmarked) POIs other users. The user(s) can then be guided throughout the virtual inspection by following the signs provided for the virtual experience. The user(s) can then quickly inspect critical areas of the mining equipment, such as areas of particular concern regarding excessive wear.

In addition, the user can shift the visual perception to other areas of the interior of the mining equipment identified by the user. For example, by virtually inspecting the interior of mining equipment, a user can use an external input device for a virtual or augmented reality device to add pointers to a second set of data. One skilled in the art will appreciate that these added indicators identify three-dimensional locations of specific areas in the VR environment that should be highlighted in a particular manner (a particular color, a particular symbol, etc.). Such additional indicators may be provided, for example, for areas where the user identifies an initial wear process (eg by observing new cracks, etc.) and should be monitored more closely in the future or should be inspected more closely by other users ( for example, remote users) who are simultaneously guided throughout the virtual inspection.

In addition, through the use of an external input device for the virtual or augmented reality device, the user can further input text information (notes, etc.), image data, audio or voice data in connection with identifying specific areas or areas of the interior of the mining equipment. One skilled in the art will appreciate that this added data (eg, text data, image data, audio data, voice data, etc.) can be enhanced for users to perceive when virtually inspecting the interior of mining equipment. Thus, such additional data can be entered to expand the capabilities of the virtual inspection.

Such additional data may also be entered or recorded during the virtual inspection. In particular, multiple users (eg, on-site and off-site/remote users) may enter various additional data during a virtual inspection regarding specific POIs.

This added data, which may be added to the second set of data and thus complement the visual perception, may also contain identification marks regarding specific pieces of mining equipment, part numbers, installation date information, batch information such as when manufactured or part replaced, inventory levels or order status of specific parts, part weight information, etc. In addition, such expanded additional parameter information may include information about tests performed on specific pieces of mining equipment, i.e. test dates, test parameters, etc.

According to another embodiment, virtual inspection of the interior of mining equipment may be coordinated between two or more users. In this case, by using a motion tracking mechanism for the first user, for example using a head tracking sensor (for example, using an accelerometer, gyroscope, etc.) and thus identifying the virtual path of the first user in the virtual perception of the virtual inspection of the interior of mining equipment, i.e. movement from a first POI to a second POI, for example in the context of tracking the progression of specific lining cracks or other forms of wear as described above, to a virtual or augmented reality device, the second user's devices also provide this same virtual path. In other words, by coordinating the virtual inspection for users, these users are provided with the same visual experience of the interior of the mining equipment, i.e. they look at the same POIs at the same point in time. This provides one user with the ability to guide other users through a virtual inspection of mining equipment, as well as the ability to augment other users' visual experience by pointing to specific POIs, adding additional text, image, audio, and voice data as described above.

One skilled in the art will appreciate that this coordination mechanism can be implemented such that the input from the motion tracking sensor to the first user's virtual or augmented reality device is also used in the other users' virtual or augmented reality device. This can be achieved in such a way that virtual or augmented reality devices communicate with each other. For example, a first virtual or augmented reality device transmits motion tracking sensor input data (which is received by the device) to other virtual or augmented reality devices via a wired or wireless connection such that the other virtual or augmented reality devices access the second set of data in accordance with with these motion tracking sensor inputs.

One skilled in the art will also appreciate that coordination between the two users can be equally applied, with the first user performing the scanning/retrieval of the first set of data and the second user performing the real-time inspection. Thus, missing or insufficient data can be indicated by the second user using the POI, directing the first user to the POI to obtain additional data. At the same time, inspection of areas of mining equipment provided with sufficient data can already take place.

According to another embodiment, the second set of data (as explained above) may be transmitted to one or more remote users. Remote (or off-site) users may be, for example, users who are not present at the actual geographic location of the mining equipment. In this way, a second set of data can be generated locally, for example by acquiring the first set of data using 3D scanning, sensors and/or mobile devices for mining equipment, and then shared with other remote users. Thus, a virtual inspection as described here can be performed remotely in a manner that does not require the physical presence of technical experts and engineers.

According to another embodiment, the virtual inspection may be further enhanced by obtaining additional data points of the first set of data of the interior of the mining equipment based on the first virtual inspection of one or more points of interest of the interior of the mining equipment. This defines a feedback mechanism, preferably initiated by a remote user, to obtain additional information regarding the actual interior of the mining equipment, such as geometric shapes, surfaces, directions, orientations, alignments, etc., defining the physical appearance of the interior of the mining equipment. For example, this feedback mechanism could be used if a virtual inspection identifies an area of potential wear on internal mining equipment that requires a more detailed investigation that requires higher (spatial) resolution data. This feedback mechanism can also be used if the virtual inspection identifies specific areas for which the actual appearance (obtained by the first data set) must be obtained differently, for example if the sensor must measure at a different angle due to a dead zone or the like ., as described above. One skilled in the art will appreciate that upon obtaining additional data points, these additional data points of the first data set may be provided with a transformation to the second data set, and thus the user may be provided with visual feedback of the additional data points in visual perception while performing the virtual inspections. In other words, an updated virtual inspection can easily indicate whether sufficient additional data points (with higher spatial resolution, etc.) have been obtained.

In another embodiment, the first set of data (as described above) may be acquired at various points in time, for example, during the life of the mining equipment, one month or one year. By defining the first data set as DS1, thus, the first data set can be obtained at different times t, for example, at three times, that is, DS1(t1), DS1(t2) and DS1(t3). Because of this, physical parameters of the actual interior of the mining equipment are obtained, such as geometric shapes, surfaces, directions, orientations, alignments, etc., which determine the physical appearance of the interior of the mining equipment over time. In addition, the first data set at these different times is converted into a plurality of second data sets DS2, ie in this example DS2(t1), DS2(t2) and DS2(t3).

Based on this, a virtual inspection of the interior of the mining equipment can be provided in such a way that the development of a physical parameter and/or modeling of a physical parameter at one or more points of interest is added to the visual perception. Specifically, based on changes in data sets over multiple points in time, and when and where POIs were identified, artificial intelligence (AI) can be trained to classify under what degree or pattern of change in data sets POIs are likely to occur. . Therefore, a first estimate of POIs may be provided during scanning, but without requiring a second user to provide input of possible additional POIs.

For example, for one or more specific points of interest, a temporal wear profile or a temporal trend profile for lining dimensions, a heat map, or the like may be determined, which may be provided during a virtual inspection as the user navigates to the specific point of interest. Thus, virtual inspection can be provided such that the current condition of the interior of the mining equipment can be virtually inspected along with a real-time visual perception of the temporal evolution of specific POIs located in the interior of the mining equipment over time. This can also be improved by comparison with the original design of specific POIs (eg using comparison of CAD models, alternative designs, etc.).

Modeling of a physical parameter can be carried out based on the known development of the physical parameter, for example, measured dimensions of the lining, cross sections of the lining, etc., and also through the use of a modeling algorithm that predicts the further development of the physical parameter. For example, by defining a time constant that identifies how a physical parameter (lining thickness at certain POIs, etc.) has decreased over time, a modeling algorithm can be used to predict the likely evolution of that physical parameter. One skilled in the art will appreciate that this provides an improved mechanism for informing the user of the predicted time when specific pieces of mining equipment will need to be replaced. This improves the coordination of mining equipment downtime, which takes a relatively long time and leads to significant operating costs.

In addition, the second set of data may also be supplemented with additional physical parameter information regarding said one or more POIs of the interior of the mining equipment. This additional physical parameter information may be, for example, a video clip of how pieces of mining equipment look or behave in a real-life operating situation, i.e., when grinding, crushing or crushing minerals or ores. This may also include additional modeling of motion, material flow, crushing, grinding, etc., which provides the user with additional understanding of the distribution of shear, impact, drive power, etc. throughout the interior of the mining equipment and thus the service life of that mining equipment. This additional physical parameter information may also include a comparison of the current state of the lining with the original lining design.

Virtual inspection can be further enhanced by providing a virtual measuring device or virtual tape measure. The virtual measuring device can be used by the user to determine the dimension of the VR environment as represented in the user's visual perception. For example, a user may place a virtual tape measure along an identified crack (or other wear patterns as described above) located on the interior of mining equipment and indicate the dimension (eg, length) of that crack. The computing device then processes said dimension data (in the virtual space defined by the second data set) and performs a conversion to the real space dimension as defined by the first data set. Because of this, the user can be provided with direct feedback regarding the actual size of a newly identified crack on the interior of the mining equipment.

In addition, virtual inspection can be further enhanced by providing a virtual cross-sectional analyzer that displays the cross-section or contour of the surface of the mining equipment. For example, a user may draw or project a line on the surface of mining equipment, and the protrusions or depressions identified on that line may be diagrammed in a virtual representation of the VR environment. Thus, deformations or damage to the surface of the mining equipment become easily identifiable.

In fig. 7 is a schematic representation of a computing device 40 which, as in the above embodiments, may be configured to implement the computer-implemented methods described above and defined in the claims, and thus be capable of operating as a device for inspection of mining equipment. Computing device 40, which may also be referred to as firmware signal processing hardware 40, in embodiments such as the present embodiments, includes a communications interface (I/F) 41 for receiving mining equipment data from sensor 30, scanner, or mobile device. as described above. Computing device 40 also includes a processor (e.g., a central processing unit, CPU, or graphics processing unit, GPU) 42, a random access memory 43, and an instruction store 44 that stores a computer program containing computer-readable instructions that, when executed by the processor 42, cause the processor 42 to execute various functions, including those defined in the computer-implemented methods described above and defined in the claims. Instruction store 44 may include ROM (eg, in the form of electrically erasable programmable read-only memory (EEPROM) or flash memory) that is preloaded with said computer-readable instructions. Alternatively, instruction store 44 may comprise RAM or a similar type of storage device, and computer readable computer program instructions may be input thereto from a computer program product such as a non-transitory computer readable storage medium 45 in the form of a CD-ROM, DVD-ROM, and etc. or a computer readable signal 46 carrying said computer readable instructions. In either case, the computer program, when executed by a processor, causes the processor to perform at least one of these computer-implemented methods of obtaining point cloud data, inspecting the interior of an operating mining equipment, and virtually inspecting the interior of a mining equipment, as described herein. However, it should be noted that device 40 may alternatively be implemented in non-programmable hardware such as an application specific integrated circuit (ASIC).

Those skilled in the art will appreciate that various modifications and changes may be made to the objects and methods of the present invention, as well as the construction of the present invention, without departing from the scope of the present invention.

The present invention has been described with respect to specific embodiments and examples, which in all respects are intended to be illustrative and not limiting. Those skilled in the art will appreciate that many different combinations of hardware, software and/or firmware are suitable for the practice of the present invention.

In accordance with aspects of the present invention, the following is provided:

A1. A computer-implemented method for obtaining point cloud data for obtaining point cloud data of mining equipment, preferably an interior or region of mining equipment, comprising:

receiving from the sensor a first set of data and a second set of data, each data set comprising data points with coordinates;

extracting features from the first and second data sets;

aligning the first and second data sets using the extracted features;

combining the aligned first and second data sets into point cloud data.

A2. A computer-implemented method for obtaining point cloud data according to A1, in which the specified features contain structural features of mining equipment and/or features of the reflection properties of mining equipment.

A3. A computer-implemented method for obtaining point cloud data for any of A1 - A2, additionally including:

assessment of the geometric shape of mining equipment based on point cloud data.

A4. A computer-implemented method for obtaining point cloud data for any of A1 - A3, additionally including:

using point cloud data using a virtual or augmented reality device to provide visual perception of mining equipment.

A5. A computer-implemented method for obtaining point cloud data for any of A1 - A4, additionally including:

identifying, using point cloud data, a region of estimated geometric shape that indicates insufficient data.

A6. A computer-implemented method for obtaining point cloud data for any of A1 - A5, in which

if the area of the identified area is greater than the specified area,

the next coordinate is extracted from the identified area, where the next coordinate is preferably the coordinate closest to the scanning direction of the sensor, and

cause the sensor to move towards the next coordinate until the next coordinate falls within the scan range of the specified sensor, or

the user is notified of the next coordinate and is instructed to move the sensor in the direction of the next coordinate until the next coordinate falls within the scan range of said sensor.

A7. A computer-implemented method for obtaining point cloud data along A6, in which

if the next coordinate falls within the scanning range of the sensor, the specified method further includes:

receiving from the sensor a third data set containing data points with coordinates;

feature extraction from point cloud data and the third dataset;

aligning the third data set with the point cloud data;

merging the aligned third dataset with the point cloud data;

re-evaluating the geometric shape of the mining equipment as an estimated geometric shape based on the point cloud data;

re-identifying, using the point cloud data, a region of the estimated geometry indicative of insufficient data as a region indicative of insufficient data.

A8. A computer-implemented method for obtaining point cloud data for any of A1 - A7, in which

If the area of the identified area is smaller than the specified area, a fault analysis is performed based on the point cloud data.

A9. A computer-implemented method for obtaining point cloud data for any of A1 - A8, in which

the sensor is a movable sensor, preferably portable, flying or suspended.

A10. A computer-implemented method for obtaining point cloud data for any of A1 - A9, in which

The sensor is a depth sensor that perceives the distance from the sensor to the surface as depth.

A11. A computer-implemented method for obtaining point cloud data for any of A1 - A10, in which

as depth information, the sensor senses information about the distance from the sensor to the surface of the mining equipment, preferably the interior or said area of the mining equipment.

A12. A computer-implemented method for obtaining point cloud data for any of A1 - A11, in which

the sensor senses information about an intensity-related property of the reflected and measured signal.

A13. A computer-implemented method for obtaining point cloud data for any of A1 - A12, in which

the second set of data is obtained after the first set of data and after moving the sensor.

A14. A computer-implemented method for obtaining point cloud data for any of A1 - A13, in which

the sensor receives information about the orientation and/or odometry of the sensor.

A15. A computer-implemented method for obtaining point cloud data for any of A1 - A14, in which

orientation information contains information about the roll of the sensor, its rotation around its transverse axis and/or around its vertical axis; And

Odometry information contains information about the x, y and z coordinates of the sensor.

A16. A computer-implemented method for obtaining point cloud data for any of A1 - A15, in which

data points are coordinates indicating the location of the surface sensed by the sensor.

A17. A computer-implemented method for obtaining point cloud data for any of A1 - A16, in which

features are extracted using one of feature detection, edge detection, line tracking, or spline fitting to the surface represented by the data points.

A18. A computer-implemented method for obtaining point cloud data for any of A1 - A17, in which

In this extraction, the principal components of features for alignment are extracted.

A19. A computer-implemented method for obtaining point cloud data for any of A1 - A8, in which

the alignment comprises a linear transformation, preferably rotating, scaling and/or translating the first, second and/or third set of data to ensure maximum alignment and/or fit.

A20. A computer-implemented method for obtaining point cloud data along A19, in which

the alignment between the first, second and/or third data sets and/or point cloud data is indicated by a dot product of features, preferably principal components.

A21. A computer-implemented method for obtaining point cloud data from A19 or A20, in which

the alignment of the first, second and/or third data sets and/or point cloud data is indicated by convolution and/or correlation of features, preferably principal components.

A22. A computer-implemented method for obtaining point cloud data for any of A1 - A21, in which

The point cloud data is combined into a mesh before the geometric shape of the mining equipment is estimated.

IN 1. A computer-implemented inspection method for inspecting mining equipment, comprising:

moving the sensor inside or along mining equipment;

obtaining, using the sensor, first point cloud data and second point cloud data, wherein the point cloud data represents a surface located within or along the mining equipment;

determining, based on each of the first and second point cloud data, surfaces located inside or along the mining equipment;

estimating, based on certain surfaces, the geometric shape of the mining equipment, preferably the geometric shape of the interior of the mining equipment.

AT 2. A computer-based inspection method according to B1, in which

During said production, the mining equipment is rotated or moved.

AT 3. A computer-based inspection method according to B1 or B2, in which

the sensor rotates in the opposite direction to the direction of rotation of the mining equipment.

AT 4. A computer-implemented inspection method according to any of B1 - B3, in which

the sensor rotates at a higher angular speed than the angular speed of the mining equipment.

AT 5. A computer-implemented inspection method according to any of B1 - B4, in which

mining equipment rotates at an angular velocity equal to or less than the angular velocity during normal operation.

AT 6. A computer-implemented inspection method according to any of B1 - B5, in which

the first point cloud data and the second point cloud data are obtained in accordance with the method of any one of A1 to A22.

AT 7. A computer-implemented inspection method according to any of B1 - B6, in which

the sensor is moved substantially parallel to the axis of rotation of the mining equipment.

AT 8. A computer-implemented inspection method according to any of B1 - B7, in which

In operation, mining equipment rotates around its axis of rotation.

AT 9. A computer-implemented inspection method according to any of B1 - B8, in which

the second point cloud data is obtained after the first point cloud data and after the sensor and/or mining equipment has been/has been moved.

AT 10 O'CLOCK. A computer-implemented inspection method according to any of B1 - B9, in which

the resulting first and second point cloud data are rotated based on the rotation angles of the sensor and mining equipment.

C1. A man-machine guide system for inspecting mining equipment, preferably the interior or area of mining equipment, comprising:

display;

a sensor configured to sense the distance to the surface of the mining equipment;

a tracking device configured to track the location and orientation of the sensor;

a point cloud generator configured to generate point cloud data based on the specified perceived distance and the tracked location and orientation of the sensor;

a surface estimation device configured to estimate a surface based on point cloud data, and

a geometric shape estimation device configured to estimate the geometric shape of mining equipment based on said surface.

C2. Man-machine guiding system according to C1, in which

the display is configured to display the surface being assessed based on the location and orientation of the sensor.

C3. Man-machine guiding system according to C1 or C2, in which

the tracking device is further configured to track the location and/or orientation of the display; A

the display is configured to display the surface being evaluated based on the location and/or orientation of said display.

C4. A man-machine guiding system according to any one of C1 - C3, in which the sensor is additionally configured to sense the reflection property of mining equipment.

C5. The man-machine guidance system according to any one of C1 to C4, wherein the sensor is further configured to sense an intensity-related property of the reflected and measured signal.

C6. Man-machine guiding system according to any of C1 - C5, additionally containing:

a mining equipment database configured to store a template geometric shape and an area of interest for mining equipment;

a subcloud determiner configured to extract from the point cloud data subsets in the form of subcloud data based on the specified region of interest; And

a data transceiver configured to transmit sub-cloud data to a remote display.

C7. Man-machine guiding system according to any of C1 - C6, in which

if the point cloud data does not contain data points in the region of interest,

at the location of the specified area of interest, the surface and/or geometric shape to be evaluated is isolated.

C8. Man-machine guiding system according to any of C1 - C7, additionally containing:

a remote computer, preferably co-located with the remote display and configured to:

accept input to define another area of interest and

saving the specified area of interest in the mining equipment database.

C9. Man-machine guiding system according to any of C1 - C8, additionally containing:

a gap detector configured to detect coordinates on the geometric shape being evaluated for which

the number of point cloud data points is less than a specified value, or

the gradient exceeds the specified value.

C10. Man-machine guiding system according to any of C1 - C9, in which

The display is a virtual or augmented reality display.

D1. A computer-implemented method for virtually inspecting mining equipment, preferably the interior or area of mining equipment, comprising:

obtaining a first set of data, the first set of data containing data points with coordinates of the mining equipment;

converting the resulting first set of data into a second set of data, the second set of data being adapted for use by a virtual or augmented reality device;

guiding at least one user through a virtual inspection of the mining equipment based on the second set of data by moving the visual perception of the at least one user of the virtual or augmented reality device to one or more points of interest of the mining equipment.

D2. A computer-implemented method according to D1, in which the first set of data contains the reflection property of mining equipment with the coordinates of the mining equipment.

D3. The computer-implemented method of any one of D1 to D2, wherein the first set of data contains an intensity-related property of the reflected and measured signal with the coordinates of the mining equipment.

D4. A computer-implemented method according to any one of D1 to D3, further comprising:

coordination of the virtual inspection between at least two users.

D5. The computer-implemented method of any one of D1 to D4, further comprising:

transmitting the second set of data to one or more remote users.

D6. A computer-implemented method according to any one of D1 to D5, further comprising:

obtaining additional data points of the first mining equipment data set based on the first virtual inspection of one or more mining equipment points of interest.

D7. A computer-implemented method according to any one of D1 to D6, further comprising:

obtaining multiple first sets of data at different points in time;

converting said resulting set of first data sets into a set of second data sets,

providing a virtual inspection of the mining equipment based on said plurality of second data sets, wherein the virtual inspection provides an evolution of a physical parameter and/or a simulation of a physical parameter and/or a cross-section and/or contour at said one or more points of interest of the mining equipment.

D8. A computer-implemented method according to any one of D1 to D7, further comprising:

augmenting the second set of data with additional information about a physical parameter in relation to said one or more mining equipment points of interest.

D9. A computer-implemented method according to any one of D1 to D7, further comprising:

using a virtual flashlight for visual perception.

E1. A computer program which, when executed by a computer (40), causes said computer to perform a method according to any one of A1 - A22 or B1 - B10 or D1 - D9.

E2. A non-transitory computer-readable information storage medium (45) storing a computer program in accordance with E1.

E3. Signal (46) carrying a computer program in accordance with E1.

Claims (17)

1. A computer-based method for virtual inspection of the interior of mining equipment, including: obtaining a first set of data, the first set of data containing data points with coordinates of the interior of the mining equipment; converting the resulting first set of data into a second set of data, the second set of data adapted for use by a virtual or augmented reality device to represent the interior of the mining equipment; directing at least one user through a virtual inspection of the interior of the mining equipment based on the second set of data by moving the visual perception of the at least one user of the virtual or augmented reality device to one or more points of interest of the interior of the mining equipment. 2. The computer-implemented method according to claim 1, additionally including: coordination of the virtual inspection between at least two users. 3. Computer-implemented method according to any one of claims. 1, 2, additionally including: transmitting the second set of data to one or more remote users. 4. Computer-implemented method according to any one of claims. 1-3, further including: obtaining additional data points of the first set of data of the interior of the mining equipment based on the first virtual inspection of one or more points of interest of the interior of the mining equipment. 5. Computer-implemented method according to any one of claims. 1-4, additionally including: obtaining multiple first sets of data at different points in time; converting said resulting set of first data sets into a set of second data sets, providing a virtual inspection of the interior of the mining equipment based on said plurality of second sets of data, wherein the virtual inspection provides development of a physical parameter and/or modeling of a physical parameter at said one or more points of interest of the interior of the mining equipment. 6. Computer-implemented method according to any one of claims. 1-5, additionally including: supplementing the second set of data with additional information about a physical parameter with respect to said one or more points of interest of the interior of the mining equipment. 7. A non-transitory computer-readable information storage medium (45) storing a computer program that, when executed by a computer (40), causes said computer to carry out a method according to any one of claims. 1-6.