WO2023062407A1

WO2023062407A1 - Method and system for determining and selecting rock breaking target poses for a rock breaker

Info

Publication number: WO2023062407A1
Application number: PCT/IB2021/059373
Authority: WO
Inventors: Javier RUIZ DEL SOLAR SAN MARTÍN; Mauricio Alejandro MASCARÓ MUÑOZ; Sebastián Isao PARRA TSUNEKAWA; Daniel Germán CÁRDENAS NAHUEL
Original assignee: Universidad De Chile
Priority date: 2021-10-12
Filing date: 2021-10-12
Publication date: 2023-04-20
Also published as: AR127337A1; AU2021468870A1

Abstract

The invention refers to a method and a system for determining and selecting rock breaking target poses, the method comprising acquiring images and range data of rock elements in a pile of material to carry out a segmentation process using an watershed method adapted for surfaces structured as point clouds, that allows to accurately identify rock elements within the pile of material, and the system comprising a plurality of sensors configured to acquire images and range data of rock elements in a pile of material and a processor operatively connected to said plurality of sensors to receive and process the sensor data and configured to carry on the steps of said method.

Description

METHOD AND SYSTEM FOR DETERMINING AND SELECTING ROCK BREAKING TARGET POSES FOR A ROCK BREAKER

FIELD OF THE INVENTION

The present invention relates to a method and a system for determining rock breaking target poses. More particularly, the present invention relates to a method and a system that, based on sensor data comprising images and a point cloud, automatically determines a rock breaking target pose, which is used by a rock breaker for the fragmentation of oversized rocks within a pile of material.

BACKGROUND

In different types of mining operations, rock fragmentation is a necessary process for the correct operation of the subsequent stages of mineral recovery. Commonly, this task is carried out through the use of hydraulic hammers, also known as rock breakers.

In underground mining, the material is extracted from different extraction points and transported to the plant by means of LHD (Load-Haul-Dump) vehicles, which transport the material to discharge shafts or directly to a primary crusher. In both cases, the material falls on structures called transfer grates or grizzlies, which retain rocks that are larger than the appropriate size for the next stage of the process. In order to pass the retained rocks through the grizzly, it is necessary to carry out a fragmentation process with rock breakers.

In the case of open pit mining, rock breakers are used in all primary crusher facilities, where rock breakers are used to reduce the size of the rocks that are not suitable for crushing.

In the state of the art, technologies are developed to remotely control rock breakers (tele-operation), by means of a remote operating station located in a more comfortable and safer place. However, several drawbacks are identified in current technologies for remote operation of rock breakers. Commonly a single operator must handle more than one rock breaker, due to the intermittent nature of their operation. This situation frequently causes the fragmentation process to take longer than normal, generating a delay in the mine's production chain. It is also common that more than one LHD vehicle discharges material into a single shaft, and when the rock breaker is operating or the grizzly is obstructed, LHD vehicles must wait to discharge the material, generating a costly site stoppage of LHD equipment. On the other hand, due to the deepening of the mining operations, the material to be extracted becomes increasingly hard, and therefore, difficult to break, which causes larger rocks to reach the grizzly, thereby increasing the work of the hydraulic hammers and consequently the need for a more efficient process for breaking the rocks.

An exemplary technology of the prior art is described in document US 2019/0152038 A1 , which discloses a hydraulic hammer for a working machine configured for digging a surface including a chisel, sensors, and a controller. Each of the sensors is configured for generating a signal indicative of a projecting distance between one of the sensors and the surface, and the controller is configured for receiving the signals, determining an angle between the chisel and a plane substantially tangent to the contact location, and reorienting the chisel so that the chisel is substantially orthogonal to the contact location with the angle at substantially ninety degrees. However, the measurements made to the work surface are limited to the determination of a projecting distance in order to identify a contact surface, but this information is insufficient to provide a proper and efficient operation of rock fragmentation as it is unable to achieve an accurate determination of the pose of the rock breaker, as well as to accurately identify rock elements in a worksite.

In view of the abovementioned problems, it is necessary to provide more efficient methods and systems for the rock fragmentation process, by reducing the human intervention required and increasing the automation in rock breaker operations, particularly by providing a method and system to automatically determine rock- breaking target poses, which are used by rock breakers for the fragmentation of oversized rocks within a pile of material. BRIEF SUMMARY OF THE INVENTION

The invention refers to a method and a system to automatically determine rock breaking target poses, which are used by a hydraulic hammer when fragmenting oversized rocks within a pile of material. The method steps and complete understanding of the invention may be obtained using the following drawings, descriptions, and claims.

In a first aspect of the invention, a method for determining and selecting rock breaking target poses for a rock breaker is provided, which is able to assist a user to operate rock breakers for the fragmentation of oversized rocks within a pile of material. In a general implementation, the method comprises acquiring images and range data of rock elements in a pile of material to carry out a segmentation process that allows to accurately identify rock elements within the pile of material. A point cloud of the pile of material is generated using the range data obtained by the sensors, which is processed to carry out a first segmentation process of rock elements, using a modified watershed method, thereby obtaining a segmented point cloud of rock elements. A second segmentation process of rock elements is performed based on the image data obtained by the camera sensors, thereby obtaining image-based rock elements data. The results of the first and second segmentation processes are combined to carry out a third segmentation process, which includes combining the segmented point cloud and the image-based rock elements data for obtaining a corrected segmented point cloud of rock elements.

The results of the corrected segmented point cloud of rock elements are used to generate sub-regions in the rock elements by means of a processor, which allows to obtain rock breaking target pose candidates that are subsequently validated and selected based on a hierarchical organization of the candidates, according to predefined criteria.

The selected pose for breaking rock may be used for assisting in the operation of a rock breaker, for example, by providing a tool implemented in a user interface that helps a user to visualize the determined target pose and target pose candidates in a virtual environment. In other embodiments, the means to control the movement of the rock breaker may be connected to actuation means, allowing to directly operate the movement of the rock breaker according to the target pose, in an automated, or semi-automated, operation of the rock breaker.

In another aspect of the invention, a system for determining and selecting rock breaking target poses for a rock breaker, which provides the elements and the support to implement the steps of the method described above. Particularly, the system comprises a plurality of sensors configured to acquire images and range data of rock elements in a pile of material, and a processor operatively connected to said plurality of sensors to receive and process the sensor data and configured to carry out the steps of the method. The selected pose of the rock breaker determined by the processor may be subsequently used by means for breaking rocks.

The method and system operate permanently and continuously in real time, thereby allowing to analyze continuously the environment and determining the next rock breaking target pose for the rock breaker.

The system and method described above may allow a fully autonomous operation of rock breakers or may be used as an assistance tool in the teleoperation for rock breaker operators. The system described allows to detect and recognize automatically any kind of rocks and determining the pose for the hydraulic rock breaker to perform the breaking process.

The automation of rock breakers has important benefits for the mining operation, such as the following:

Reducing waiting times between mineral discharges and the start of the fragmentation process with rock breakers.

Preventing the clogging of transfer grizzly due to excess material with oversize.

Reducing the variability of the process.

Facilitating the training process for personnel associated with the rock fragmentation process. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a preferred embodiment of the system for determining and selecting rock breaking target poses for a rock breaker according to the present invention.

FIGs. 2A, 2B, 2C, and 2D illustrate exemplary results in the different stages of the combined segmentation process.

FIGs. 3A and 3B illustrate an exemplary implementation of the correction process of a point cloud segmentation using image-based detections.

FIG. 4 illustrates an exemplary embodiment of the generation of a horizontal plane projected over the grizzly.

FIG. 5 illustrates the projection of an exemplary rock element in the projected horizontal plane.

FIG. 6 illustrates the projection of sub-regions over the surface of exemplary rock elements.

FIG. 7 illustrates the generation of a bounding-box in exemplary rock elements.

FIG. 8 illustrates an exemplary implementation of point grouping in rock elements according to the segmentation process and considering their position in the grid of sub-regions.

FIGs. 9A, 9B, and 9C illustrate exemplary embodiments of the validation of sub-regions for evaluating breaking poses on rock elements, using a first validation criteria and selecting different thresholds for the angular error.

FIGs. 10A, 10B, and 10C illustrate exemplary embodiments of the validation of sub-regions for evaluating breaking poses, using a second validation criteria and selecting different thresholds for a curvature tolerance.

FIG. 1 1 illustrates the generation of edges of exemplary rock elements in a bi-dimensional projection, considering the corresponding points of the segmentation process. FIG. 12 illustrates an exemplary implementation of a process of detection of narrow regions in rock elements.

FIGs. 13A and 13B illustrate exemplary implementations of the hierarchical organization of target pose candidates, considering the most accessible pose (less time required) as a criterion. The chisel tip is illustrated in a white circle, the arrows with a clear color illustrate the generated breaking poses, and the dark arrow illustrates the first selected pose, circled in black. In numbers, the ranking of each breaking pose.

FIG. 14 illustrates another exemplary implementation of the hierarchical organization of target pose candidates, considering a combination of criterion 1 and criterion 2.

FIG. 15 illustrates an exemplary embodiment in which the grizzly is divided into regions, and all the pose detected corresponding to rock elements outside the selected region are removed from the model.

FIGs. 16A, 16B, and 16C illustrate three sequences of results obtained in a simulated environment, illustrating the results of point cloud segmentation and the combined segmentation with image data.

FIGs. 17A and 17B illustrate the results obtained in a real environment, illustrating the results of a point cloud obtained when the rock breaker strikes a rock element, and a visible image of the material on the grizzly.

FIG. 18 illustrates the results of the segmentation process in moment of the operation in which there were no rocks on the grizzly.

FIGs. 19A and 19B illustrate the results of the segmentation process in the grizzly of FIG. 18, in a different moment of the operation with rock elements detected over the grizzly.

DETAILED DESCRIPTION AND BEST MODES OF IMPLEMENTATION

The following is a detailed description of exemplary embodiments to illustrate the principles of the invention. The embodiments are provided to illustrate aspects of the invention, but the invention is not limited to any embodiment. The scope of the invention encompasses numerous alternatives, modifications and equivalent, only limited by the embodiments of the claims.

According to FIG. 1 , in a first aspect of one embodiment of the invention a method for determining and selecting rock breaking target poses for a rock breaker (100) is disclosed, the method comprising the steps of: acquiring, by means of a plurality of sensors (300), images and range data of rock elements in a pile of material (200); generating, by a processor, a point cloud of the pile of material using the range data; identifying, by said processor, rock elements in the pile of material performing a segmentation process including: a first segmentation process of rock elements based on the point cloud of the pile of material, using a watershed method adapted for surfaces structured as point clouds, for obtaining a segmented point cloud of rock elements; a second segmentation process of rock elements based on the image data of the pile of material, using image data processing techniques, for obtaining image-based rock elements data; a third segmentation process of rock elements by combining the segmented point cloud and the image-based rock elements data for obtaining a corrected segmented point cloud of rock elements; generating, by said processor, sub-regions in the rock elements from the corrected segmented point cloud of rock elements for obtaining rock breaking target pose candidates; validating said sub-regions according to predefined rules; selecting, by said processor, rock breaking target poses for the rock breaker based on a hierarchical organization of the rock breaking target pose candidates, according to predefined criteria; wherein the selected pose is used for breaking rock. As used herein, range data refers to the data obtained by means of range sensors, which may include 2D LIDAR, 3D LIDAR, RADAR, binocular camera, TOF (Time of Flight) camera or 1 D laser scanners, among others. The point cloud refers to the tridimensional representation in a virtual environment of the elements or objects detected by the range sensors, which consists of a set of data points obtained by processing the range data.

The method generates an appropriate representation of the rock pile, in which each rock element is individualized. The individualization of rock elements, referred hereinafter as segmentation, is performed in a three-dimensional representation of the rocks detected by the sensors, which allows the control of the rock breaker to strike rocks. To achieve this goal, the range data obtained by the sensors is processed to generate the point cloud in order to model a three-dimensional workspace. This information is combined with the images obtained of the rock elements, comprising color and texture information that is used to characterize the workspace.

In order to provide a clear understanding of the operation of the method for determining and selecting rock breaking target poses for a rock breaker, the main features of the previous steps are briefly described below.

Generation of the Point Cloud

Preferably, the first segmentation process of rock elements includes a first step of removing from the point clouds the captured points that should not be considered in the model. For example, these points may be related to the mine infrastructure and/or parts of the rock breaker that can be captured by the range sensors, which must be filtered in order to obtain a point cloud containing only data related to rock elements. Different approaches may be considered in order to achieve this goal such as, for example, defining a specific working area, using a geometrical tridimensional representation of the parts of the hydraulic rock breaker and the grizzly, using a previously captured environment model to identify the mine infrastructure (the floor, grill, railings, etc.), to identify and remove the points that are not part of the rock elements, among others. Once all the points that are not related to rock elements are filtered out, all data from the range sensors are combined to represent in a unique and simplified way the spatial information. Preferably, to complete the generation of the point cloud, a step of resolution reduction is carried out in order to allow a faster processing. In this embodiment, the sensors data is subsampled using voxels, where the centroid of the points within the voxel (C_vox) simplifies the spatial location of all points {P₁, P₂> ->Pk contained within the voxel. The centroid of the voxel is defined as:

After applying the previous steps, a point cloud containing only the material on the grizzly is obtained.

Rock Segmentation using the Point Cloud

In order to carry out the first segmentation process of rock elements based on the point clouds, the inventors have developed a version of the watershed method adapted to be applied to a point cloud defining a surface. This adapted version allows to apply the concept of descent and initial labeling directly on a point cloud, performing a surface analysis, without the need to convert this point cloud to a polygon mesh, which is where this type of methods is applied (since it allows to generalize several concepts to surfaces). By not needing a polygon mesh, this method is able to generate the necessary information to segment directly without the need for additional processing (such as calculating triangulations). In general, the use of surface processing with polygonal meshes is usually convenient when an object is well defined, and it is required to generate a three-dimensional visualization of it. Since in this case the information obtained from the surface is more relevant than the visualization itself, it is more convenient to use the point cloud directly, which is what the proposed method does. After generating the point cloud, the first segmentation process is performed to allow the segmentation of the point clouds of rock elements, which comprises the steps of: an initial labeling of points of the point cloud for identifying points related to convex surfaces that are most similar to rock elements, including: obtaining a normal vector and a curvature of each point; determining a height value of each point using a first height function, which uses the normal vector and curvature previously obtained; assigning a label to all the points with a height value above a threshold; performing morphological operations of dilation and erosion to the labeled points, for connecting labeled points that are near according to a predefined proximity radius; grouping the connected points based on proximity, assigning to the points of each group a second label, which correspond to the seed points; applying a flooding process to the seed points, in order to propagate the second labels of each group; and labeling the points that were not processed previously by associating them to a label of the nearest point, thereby obtaining a completely segmented point cloud, with each point containing the normal vector and a label corresponding to each rock element.

To obtain the seed points in the initial labeling, a previous step is performed directed to determine a normal vector and a curvature of each point. This step is carried out by estimating a plane at the location of each point, preferably using the least squares algorithm, and estimating the curvature in that point, preferably using the eigenvalues of the covariance matrix obtained with the nearby points. Then, a height value of each point is calculated, which allows determining which of the values will be the initial labels and how the labels will be propagated within the algorithm. Preferably, the height value is determined by a first height function, which is defined by means of the deviation of the plane at the location of a point with respect to a desired orientation. Considering that the goal is to find planes whose normal component is in the vertical direction, the height function can be represented using the dot product with r = {0, 0, 1} as follows:

H(p) = r ■ [n_x, n_y, n_z] where H(p) is the height function for the point p, and n_£ is the i component of the normal vector in the tridimensional space.

By using this function, a higher value is provided to convex points or valleys considered from a top view. This is advantageous as it allows differentiating valleys on different levels in the vertical axis (because it does not use this information). If the vertical axis were used directly, the result would prioritize valleys that are higher than others, which is not convenient when segmenting rocks with different heights.

The seed points are obtained by labeling the points with the highest height value, preferably using a single threshold. In preferred embodiments, a specific label value is assigned to label the points, assigning for example L(p) = 1 to all points with height value above the threshold, where L(p) represents the label value of point P-

Then, morphological operations of dilation and erosion are performed to the labeled points of the point cloud, connecting nearest labeled points according to a pre-defined proximity radius to reduce the number of incorrect detections. These morphological operations of dilation and erosion allow removing connected components of few points as well as to join very close components into one. Finally, the labeled points are separated into groups according to the proximity of the points, which is done using traditional methods of associating connected components using the proximity of the points. The result of these steps of dilation and erosion generates labeled points L(p) > 1, which correspond to the seed points to be propagated.

An exemplary implementation to perform the label dilation may be implemented as follows: analyzing each point of the point cloud by iteratively selecting a point (p) to be analyzed; selecting a set of points (Q) that are close to point (p) according to a distance radio (r); determining if point (p) has been previously labeled with a label value in the labeling process described above; in case that point (p) has been previously labeled with a label value, adding the set of points (Q) to a list of points (L); labeling the list of points (L) according to the label of point (p).

Similarly, an exemplary implementation to perform the label erosion process may be implemented as follows: analyzing each point of the point cloud by iteratively selecting a point (p) to be analyzed; selecting a set of points (Q) that are close to point (p) according to a distance radio r; identifying if point (p) has been previously labeled with a label value in the labeling process; in case that point (p) has been previously labeled with a label value and the set of points (Q) contains any point with a label different from the label of point (p), adding points (p) to a list of points (L); labeling the list of points (L) according to the point found having the different label.

The flooding process allows a propagation of the labels from the seeds to points of lower height in the rock elements, allowing to delimit the points of contact between different rocks.

Preferably, to carry out the flooding process a second height function is defined, different from the first height function defined in the labeling process. This is due to the fact that the requirements of both processes are different. In the labeling process, the height function is defined with the object of detecting rocks by using a single threshold, so it is preferred to detect the valleys of the rocks without giving relevance to the height of these valleys. In contrast, in the flooding process the goal is to propagate the points in such a way as to correctly limit the separations between rocks as previously detected in the labeling process. Inventors have found that using the convexity of the surface (as the labeling process) is not convenient in the flooding process because it is not possible to generalize a surface as a single plane in the intersections between rocks, since these points do not correspond to surfaces. Therefore, in the flooding process the vertical axis value is chosen as the propagation criterion, since this allows generalizing the intersections of the rocks as a point such that it is the minimum vertical axis value between two different surfaces. It is to be considered that this approach does not work well in cases in which a rock rests on another, but this is not a problem in our context, since to carry out the rock breaking process it is of interest to detect the rocks that are at the upper level of the material, since those are the rocks that needs to be broken.

When implementing this methodology on a point cloud, it is necessary to generalize some concepts. On one hand, the distance between points is used as a proximity criterion as in the previously described algorithms. In addition to this, points are grouped according to a certain predefined range according to the height value they have. This consideration allows reducing the noise level when propagating the labels using the value in the vertical axis.

In the flooding process, a hierarchical ranking is performed according to the height values, assigning a height level N(p) to the height values and then propagating by proximity of the points and by level of points.

An exemplary implementation of the flooding process may be implemented as follows: determining a height value of each point of the point cloud based on a second height function; assigning a height level to each height value; performing a hierarchical ranking to the height values according to a hierarchical parameter, based on the height level; assigning the points to a hierarchical list of points, which organize the set of points according to their hierarchical parameter; and analyzing each point of the hierarchical list to carry out the following steps: in case a point of the list is labeled from the dilation and erosion process, performing a propagation to neighbor points in the inferior level of hierarchy; in case a point of the list is not labeled from the dilation and erosion process, searching for the nearest point that is labeled and performing a propagation to neighbor points in the inferior level of hierarchy.

In preferred embodiments, for each point p of the point cloud, the height value H p) is calculated, and then a hierarchal value is assigned to the height value according to a parameter step A//, as follows:

H(p)~

/V(p) =

A// where /V(p) is the height level, and preferably the parameter A// corresponds to the same one used in the subsampling, in this preferred embodiment A// = 0.0375. This results in such a way that the hierarchical value is equivalent to the level in the vertical axis of the voxel corresponding to the point.

Preferably, after this hierarchy assignation, the points are analyzed from highest to lowest according to the hierarchy, then, if the point already has a level, this label is propagated to all the neighbor points corresponding to a lower level hierarchy. Conversely, if the point is not labeled, it takes the label of the nearest labeled point, wherein the neighbor searching process is performed using a distance that is greater than the one used in the descending stage. This allows propagating the label to the nearest local maxima (or catchment basins of the surface) that are incorrectly generated by noisy vertical axis values of the points.

Therefore, this approach based on label propagation by level allows labels to propagate uniformly through all points within the same hierarchy, thus producing the flooding effect of watershed for the point cloud.

In preferred embodiments, an additional step is considered in the flooding process to label the points that were not processed, by associating them to the label of the nearest point. As a result, a segmented point cloud is obtained in which each point contains a normal to the surface, as well as a label corresponding to one of the rocks.

FIGs. 2A, 2B, 2C and 2D illustrate an exemplary embodiment of the invention, by using the rock segmentation process according to the previous steps. FIG. 2A discloses a representation of a raw point cloud, FIG. 2B discloses a representation of the process after initial labeling, FIG. 2C discloses a representation of an intermediate stage of the flooding process, and FIG. 2D discloses a representation of the final result of the flooding process.

Second and Third Segmentation Processes

After the steps described above, second and third segmentation processes are performed. The second segmentation process includes processing image data obtained by the image sensors, which preferably includes representing the rock elements detected by the image sensors as ellipses, represented by coordinates in an image space, as for example, a central position (x_z,y_z), a major axis a minor axis b and orientation 0,. Calibration information of the image sensor, preferably a camera, includes its focal point f_x,fy) and principal point (c_x, c_y).

The third segmentation process includes combining the processed image data and the segmented point cloud, projecting the centers of the ellipses on the point cloud containing the rock elements already segmented, thereby generating coordinates in the space of the point cloud, in the camera reference system: centroids (x_P,y_P,z_P), major axis a_P, minor axis b_P and orientation 6_P. The centroids (x_P,y_P,z_P) are located on the rock surface and are determined by selecting points from the point cloud that are projected into the coordinates (x y_z) in the image space, and then selecting the one nearest to the camera. Then, the following equation holds for the centroid:

Note that the coordinates of the centroid are proportional to the depth z_p of the centroid. Once the centroids (x_P,y_P,z_P) have been calculated, the other parameters of the ellipse in the point cloud space, the semi-major axis a_P, semi-minor axis b_P, and the orientation 0_P, are calculated, by assuming that both a_P and b_P are proportional to the depth z_p of the centroid, and that they are projected into a_z

in the image space. They can be computed as follows:

with f_a and f_b being corrected focal distances:

In this embodiment, a correction step is performed in order to assure that each segmented rock contains only one centroid, which includes identifying if a rock element determined in the first segmentation process contains more than one centroid determined in the second segmentation process, and subdividing the rock element into smaller rock elements so that each rock element has a single centroid associated. To carry out this division, the method comprises finding the ellipse that is closest to each of the points on the rock element and assigning that point to said closest ellipse. These operations constitute some of the mayor improvements of the invention with respect to the prior art, as they allow to improve the segmentation process by correcting the initial segmentation, combining this information with the image data, in particular using the ellipses corresponding to the rocks detected in the images.

An exemplary implementation of this combined segmentation process may be described as follows: identifying rock elements with an incorrect segmentation process; identifying an ellipse of the incorrectly segmented rock element, the ellipse comprising ellipse variables (x_P,y_P,z_P, a_P,b_P, 0_P); determining a normal vector of the rock element; correcting the depth of the centroid in order to position the centroid in the center of the rock element; calculating a Mahalanobis distance to the centroid, which includes using the mayor axis of the ellipse as the first diagonal component of the covariance matrix, and using the minor axis as the second diagonal component of the covariance matrix; once the Mahalanobis distance for all ellipses and points have been calculated, associating each point on the rock element to the ellipse having the smallest distance to the point; and clustering the points associated to the same ellipse.

The result of the previous steps is a set of clusters, each associated with an ellipse. As each point was assigned to an ellipse individually, the spatial continuity information was not used, so some small clusters may remain misassigned. To correct this situation, in preferred embodiments the method may include an additional step of analyzing whether there are multiple clusters associated with each ellipse. The analyzing process considers that, if it is identified that an ellipse has only one associated cluster, then that cluster is considered correct. On the other hand, if is determined that an ellipse has multiple clusters associated, the method considers the largest cluster as correct, and the other clusters are merged using the proximity of the points of the incorrect cluster with the closest points of the correct cluster.

FIGs. 3A and 3B disclose an exemplary implementation of the third segmentation process. FIG. 3A discloses the rock segmentation in a point cloud, and the center projections of the ellipses detected in images, including multiple centroids. FIG. 3B discloses the segmentation of the point cloud after a correction, using the information provided by the centers of the ellipses.

Rock Breaking Target Pose Evaluation Process

Once the third segmentation process is achieved, a rock breaking target pose evaluation process is performed, which generates a set of feasible and hierarchical rock breaking target pose candidates. The aim of this process is to assure that all the poses obtained are able to fracture the rock when the hydraulic rock breaker impacts on that position with the corresponding angle. All the poses obtained by the process are hierarchically organized and ordered with the goal of minimizing the operation time of the rock breaker in order to leave the grid without rock material, thereby allowing the evaluation process to select the first pose of this set to break the rock. The method includes a set of rules and criteria to select and prioritize the rock breaking target poses, which are preferably defined according to the rules that are commonly used in this kind of operations, for example, by means of official rock breaker operation manuals.

According to FIGs. 4 to 6, the step of determining a target pose for the hydraulic rock breaker includes generating sub-regions (215), which are obtained by dividing the surface of the rock elements into sub-regions (215) of the same size on a projected horizontal plane parallel to the grizzly (210). Each sub-region (215) is then processed to validate whether it is feasible to position the rock breaker on this region, for example, considering the material composition and rock breaker motion constraints

The step of generating sub-regions comprises grouping each point of the segmented point cloud according to its location within a predefined space on an area of interest, for example, a projected horizontal plane parallel to the grizzly, as disclosed in FIGs. 4 and 5. As disclosed in FIG. 6, the sub-regions are generated for each rock element and preferably, the points grouped in this space correspond to the points that are at the upper level of the voxel representation of the segmented point cloud of the rock element (i.e., the height where the rock has its highest value on the vertical axis).

In preferred embodiments, the size of each sub-region is selected considering the size of a chisel (1 10) of the rock breaker, to ensure that the tip makes contact only with the points within the sub-region. In general, the chisel of rock breakers used in mining facilities have a conical tip whose thickness reaches a diameter of about 8 cm to 12 cm. Accordingly, the tip of a rock breaker may reach a maximum area corresponding to a circle with a radius of 6 cm. In this case, each sub-region is preferably represented as a square of size 12 x 12 cm². The generation of the sub-regions may be obtained by evaluating whether a rock element comply with a size requirement, for example, in relation to the size of the grizzly on which the rock elements are deposited. This step comprises estimating an enveloping volume of each rock element, in order to identify if a rock element is able to pass through the grid. To represent the rock enveloping volume, an oriented bounding-box is defined using the points of each rock element, as disclosed in FIG. 7.

The cases of rocks of a low volume, capable of passing through the grizzly, are not considered in the breaking target pose search. For example, there are rocks that are of such a size that they are able to pass through the grizzly without being fragmented, but they are stuck. These cases are dealt without the need of breaking these rocks, just by redistributing the material on the grizzly. Another possible case are rocks that due to lack of visibility are considered by the system as small rocks, since only a part of them is visible, which causes the system to classify them as a "small rock". In this case, the lack of visibility is due to the fact that there are visible rocks that hide other rocks. For this reason, the system is configured to address the rocks that are visible first, and once these visible rocks are addressed, the partially hidden rocks will become completely visible by the system, so they can be correctly addressed at some later stage in the rock breaking process.

In case it is determined that a rock element does not pass through the grizzly, this rock element is considered for a target pose search. In this case, the corresponding points on the segmented cloud are grouped considering their position in the grid of sub-regions (215), in order to represent each group of points as a sub-region of the rock. An example of this aggrupation of points is illustrated in FIG. 8.

Preferably, all sub-regions with a low number of points are discarded as well as the sub-regions that are sharing points of different rock elements.

After identifying the rock elements that must be broken and generating corresponding sub-regions of the rock elements, the step of validating the feasibility to position the rock breaker on each sub-region is carried out, in which each subregion is analyzed to determine whether a rock breaking point can be found in the corresponding rock element, based on a set of rules. In this embodiment, a sub- region will be discarded if at least one rule is not fulfilled and preferably, the evaluation of the rules is carried out from the simplest to the most complex, in order to optimize the use of system resources. Preferably, three sub-region validation rules are considered: rule 1 : maintaining a vertical orientation of the rock breaker in relation to the surface of each sub-region. rule 2: the curvature of the surface of each sub-region is determined, and if said curvature does not exceed a predefined tolerance, hitting the rock on that surface. rule 3: if the size of the rock is greater than a predefined value, which depends on the size of the grizzly where the rock breaker is installed, evaluating and selecting sub-regions that are in a certain range of distance to the rock border, and if the distance between the position of each of said sub-regions and the edges of the rock is not greater than a predefined range, seeking break points on that sub-region.

Rule 1 seeks to ensure that the rock breaking process is always done in a vertical orientation, which allows that the impact force is in the opposite direction to the normal force produced by the rocks supported on the grizzly, allowing the rock to be held firmly when pressure is applied. To evaluate this rule, the method includes a comparison between the normal component of the surface within each sub-region and a desired vertical component (in this case, unit vector {0, 0, 1}), in order to obtain an angular error (E) between the normal vector and this component. The angular error can be defined as: g = cos^-1(n ■ {0,0, 1})

If this angular difference is greater than a predefined tolerance, the rule is not satisfied. FIGs. 9A, 9B, and 9C show an exemplary embodiment of the steps described above, by selecting sub-regions under rule 1 with different thresholds for the angular error. FIG. 9A shows the results using a threshold of 45 degrees, FIG. 9B shows the results using a threshold of 35 degrees, and FIG. 9C shows the results using a threshold of 20 degrees. Rule 2 seeks to ensure that the rock breaker will not slip when positioned on the rock and to ensure that the impact on the surface is regular in all points contacted by the tip of the rock breaker. Accordingly, the curvature ( ) of the surface of each sub-region is calculated and analyzed. If the determined curvature exceeds a predefined tolerance, the rule is not satisfied. To determine the curvature of a subregion, the curvature of a point and the normal component are previously generated, and a curvature value is obtained by calculating the eigenvalues To l and 2 coming from the covariance matrix of the nearby points. The curvature can be calculated as follows:

FIGs. 10A, 10B, and 10C show an exemplary embodiment in which rule 2 is applied to different rock elements, considering three different thresholds for a curvature tolerance. FIG. 10A shows the results using a threshold of 0.02, FIG. 10B shows the results using a threshold of 0.01 (value used in the preferred embodiment), and FIG. 10C shows the results using a threshold of 0.007.

Rule 3 seeks to ensure a proper fracturing process. Considering that the impact required to break a rock in the center is greater than that required to break it on its edges, in some cases is convenient to split “large” rocks from the edges to the center. If the size of the rock is greater than a predefined value, which is defined based on the size of the grizzly where the rock breaker is installed, they will be considered as “large” rocks. In this embodiment, in case a large rock is identified the method includes the step of selecting sub-regions that are not too far from the border. Notwithstanding the above, it would be also considered that breaking the rock too close to the border may be inefficient, because one of the resulting rocks could have a size similar to the original and the breaking process would need more iterations. To avoid this situation, the method considers an evaluation and selection of sub-regions that are in a certain range of distance to the rock border.

According to the above and as it is shown in FIG. 1 1 , in this embodiment the position of each sub-region (215) is evaluated in relation to the distance to the edges of the rock (220); if the distance is greater than a predefined range (for example 30 cm, 60 cm, etc.), the sub-region is discarded. In some cases, the rock elements are not large enough to discard the sub-regions in the center. To determine the distance of a sub-region to the edge, a distance analysis within the horizontal plane (210) is considered, wherein the distance between the center of the sub-region and the point associated to the nearest edge is measured. Preferably, to define the edges (220) the method may include a two-dimensional concave hull algorithm in the horizontal plane to find the enveloping points.

In some cases, a rock element may have narrow regions that should be considered as possible rock breaking points because it is easier to achieve rock fracture. Using the steps described above, rule 3 may discard the sub-regions in these narrow areas because the sub-regions could be considered as being too near to the border. For this reason, preferably the method further includes the step of detecting all narrow regions, for example, by using a lower tolerance on these areas to consider the sub-regions as valid.

Preferably, the detection of narrow regions is carried out by determining a border-to-border distance on the rock element, determining the eigenvector coordinates of the rock cloud distribution. Narrow regions are thus defined when the border-to-border distance is below a certain threshold. FIG. 12 discloses an exemplary implementation of these steps, in which dark lines are defining the border of a rock element, v1 and v2 correspond to the eigenvector coordinates, clear arrows (at the right side of the figure) correspond to border-to-border distance that are considered valid for finding narrow region, dark arrows (at the left of the figure) are border-to-border distances of regions that are not considered “narrow”, and the gray rectangle corresponds to the narrow area detected. The threshold defined to establish a narrow region is disclosed with letter T.

Generation and Hierarchization of Rock Breaking Target Poses

After defining the valid sub-regions, target pose candidates of the hydraulic rock breaker are defined, which is made by generating a rock breaking pose candidate for each valid sub-region and performing a hierarchical organization of the candidates according to predefined criteria. Preferably, the step of generating a target pose candidate includes finding the point of the sub-region that is in its center, in the horizontal plane. In this embodiment, the orientation should be always predefined as vertical.

For the hierarchical organization of the target pose candidates, different criteria may be considered for establishing a preference or priority, which may vary according to the state of operation of the breaking process, for example, starting the process by breaking the largest rock, and if it leaves residue when splitting, then selecting the most accessible residue. The criteria may also consider a desired mode of operation defined by the user, for example, in cases in which it is preferred to clear a specific region of the grizzly. In preferred embodiments, the hierarchical organization is carried out by applying three consecutive criteria: criterion 1 : selecting the most accessible pose, which is defined in terms of the less time required to reach the pose. criterion 2: breaking the largest rock, by identifying the rock element with the largest volume and identifying the target poses on that rock element. criterion 3: clearing a specific region, wherein all the poses detected corresponding to rock elements outside said specific region are not considered.

The purpose of criterion 1 is to reduce the displacement time of the rock breaker from its current pose to the rock breaking pose. To perform this action, the method preferably includes the following steps: determining a current pose of the rock breaker; carrying out an inverse kinematics model to predict how the rock breaker will move until it reaches the target pose; estimating the time required to move the rock breaker from the current pose to the target pose; and preparing a list of target poses, ordering each target pose according to the access time determined, from the shortest to the longest time.

FIGs. 13A and 13B show exemplary implementations of the previous steps, assigning different numbers to the target pose candidates according to the order of the list generated. The arrows represent the generated breaking poses, the white circle represents the tip of the chisel, the darker arrow is showing the first pose of the list, circled in black. FIG. 13A shows the result of the accessibility criterion with the tip of the chisel positioned near a first rock and FIG. 13B shows the result of the accessibility criterion with the tip of the chisel positioned near a second rock.

Criterion 2 is directed to starting the breaking process with the most critical case, since the rock with the largest dimension is the one that produces the greatest obstruction of the material on the grid. The object is to find the rock with the largest volume and identifying the target poses on that rock. In some embodiments, it is convenient to use this criterion in combination with criterion 1 in order to rank the breaking poses of this rock. An example of this combination is illustrated in FIG. 14, in which criterion 2 allows to select one of the rock elements detected and criterion 1 allows to establish a list of target pose candidates in the selected rock.

Criterion 3 refers to operational criteria, in cases in which clearing a specific region of the grizzly is preferred rather than clearing the complete zone. This may be useful, for example, in cases in which the process of clearing the grizzly could stop the operation of LHD vehicles, preventing them from unloading material, slowing the operation and production of the mine. In an exemplary embodiment disclosed in FIG. 15, the grizzly is divided into four regions, and all the poses detected corresponding to rock elements outside the selected region are removed from the model. This criterion allows limiting the use of the rock breaker to a certain region, which may be useful for reducing the use of the rock breaker and ensuring at the same time that the grid is not completely full.

In a second aspect of one embodiment of the invention a system is disclosed for determining and selecting rock breaking target poses for a rock breaker (100) in a pile of material (200), the system comprising: a plurality of sensors (300) configured to acquire images and range data of rock elements in a pile of material (200); a processor operatively connected to said plurality of sensors (300) to receive and process the sensor data, wherein the processor is configured to: generate a point cloud of the pile of material using the range data; identify rock elements in the pile of material performing a segmentation process, including: a first segmentation process of rock elements based on the point cloud of the pile of material, using a modified watershed method, for obtaining a segmented point cloud of rock elements; a second segmentation process of rock elements based on the image data of the pile of material, for obtaining image-based rock elements data; and a third segmentation process of rock elements by combining the segmented point cloud and the image-based rock elements data for obtaining a corrected segmented point cloud of rock elements; generate sub-regions in the rock elements from the corrected segmented point cloud of rock elements for obtaining rock breaking target pose candidates; validate said sub-regions according to predefined rules; and select rock breaking target poses for the rock breaker based on a hierarchical organization of the rock breaking target pose candidates, according to predefined criteria; wherein the selected pose determined by the processor is used by means for breaking rocks.

The means for breaking rocks may comprise a tool implemented in a user interface, which can help a user to visualize the determined target pose and target pose candidates in a virtual environment. In other embodiments, the means to control the movement of the rock breaker may be connected with actuation means, allowing to directly operate the movement of the rock breaker according to the target pose. Therefore, the means for breaking rocks may be implemented as a tool to assist a user in the operation of the rock breaker, or as controlling means for an automated, or semi-automated, operation of the rock breaker. In preferred embodiments, the sensors should be properly positioned in order to capture a specific percentage of the transfer grizzly and the rock material deposited on it, and preferably more than one sensor is used to generate the data. Preferably, the plurality of sensors includes at least one range sensor to capture point clouds, which are combined with the image data to generate the segmentation of the rock elements. Preferably, the measurements of the sensors are temporally integrated under a time window (for example, one second) to obtain a dense and complete point cloud of the material.

The range sensors used to generate the point cloud include any kind of sensor o sensor arrays capable of measuring distance, preferably including 2D LIDAR sensors, 3D LIDAR sensors binocular cameras, RADAR, TOF (Time of Flight) camera and 1 D laser scanners, among others. The image sensor includes the use of a camera to detect images, preferably selected from visual spectrum cameras, infrared spectrum cameras, thermal cameras, among others.

Several implementations were performed based on a simulation model, but using real data obtained from field operation. Additionally, exhaustive field tests are considered to be carried out in an industrial rock breaker operating in an underground copper mine located in Chile.

The simulation work was implemented in Gazebo 9, a robot simulator that includes among others a robust physics engine and high-quality graphical interfaces, which allowed to establish a first approach to reality. This simulation process allowed to simulate the characteristics of the rock breaker and the environmental conditions in order to test the different embodiments of the system and method described herein. FIG. 1 shows an exemplary implementation of the simulations of the working environment along with the rock breaker.

Implementing the simulation of LIDARs and visible spectrum cameras as sensors (300), considering a suitable location and orientation, allows simulating the data required to test the performance of the rock segmentation process. In the simulation, a computer software is used to generate a large number of rock elements with random positions on the grizzly in a realistic manner, for example considering the effect of gravity, allowing to generate different configurations of rock elements on the grizzly. With this information, data containing the point cloud of nonsegmented material and projected image detections are obtained and used to validate the segmentation process. The objective is to evaluate the number of well- detected rocks after running the point cloud segmentation, and after integrating the image detections to correct these detections.

In the tests, a correct detection is considered when the labeled point cloud of a rock element covers more than 90% of the material. The same simulation is used to qualitatively recognize the correspondence between the points and the rocks. If two or more rocks share the same labels, they are considered incorrectly segmented. Table 1 summarizes the results obtained with five different material configurations.

Table 1. Results of point cloud segmentation, image segmentation and segmentation correction by sensory integration.

From the results obtained, it can be seen that the first segmentation process allows recognizing the vast majority of rocks on the grizzly. In general, errors at this stage usually occur in smaller rocks or rocks that are partially occluded by being underneath another larger rock. From these results it can also be seen that the number of correct detections increases when combining the first and second segmentation processes, indicating that the image detections were able to correctly detect rocks that could not be detected using point cloud segmentation.

FIGs. 16A, 16B and 16C show three sequences of results of these tests, in which each figure shows different material distributions used in Table 1. In each figure, from left to right, the first image shows exemplary embodiments of rock elements used in the simulation, the second image shows the results of the separation between infrastructure and material, and additionally, the point cloud segmentation process of these three embodiments of rock elements, and the third image shows the results of the corresponding rock elements after the combination of the point cloud segmentation process with the image data processing.

In other implementations, field data was obtained from a specific sector of the mine, which was used to test the different embodiments of the system and method described herein. In this case, different sensors were used to capture real data from the operation. FIGs. 17A and 17B show an example of the results obtained with this data, in which FIG. 17A illustrates an integrated point cloud obtained when the rock breaker strikes a rock element, and FIG. 17B shows a visible image of the material on the grizzly.

Referring to FIG. 18, in this embodiment a detection process was carried out at a particular moment of the operation in which there were no rocks on the grizzly, allowing to train the model to recognize the infrastructure of the grizzly, without rock elements. Using the point cloud with this consideration, it is possible to use the environment model to separate the data corresponding to the rock elements from the grizzly. As a result, when new material is detected on the grizzly, the system is able to separate the points corresponding to the rock elements, as disclosed in FIGs. 19A and 19B. In the exemplary embodiment of FIG. 19A, the cloud of white points represents the environment, and the cloud of dark points represents the rock elements detected over the grizzly. FIG. 19B shows an image detected in the same moment, in which the same rock elements can be seen on the grizzly.

While the present invention has been described in terms of particular embodiments and applications, in both summarized and detailed forms, it is not intended that these descriptions in any way limit its scope to any such embodiments and applications, and it will be understood that many substitutions, changes and variations in the described embodiments, applications and details of the methods and system illustrated herein can be made by those skilled in the art without departing from the spirit of this invention.

Claims

CLAIMS A method for determining and selecting rock breaking target poses for a rock breaker, comprising the steps of: acquiring, by means of a plurality of sensors, images, and range data in a pile of material; generating, by a processor, a point cloud of the pile of material using the range data; identifying, by said processor, rock elements in the pile of material performing a segmentation process including: a first segmentation process of rock elements based on the point cloud of the pile of material, using a watershed method adapted for surfaces structured as point clouds, for obtaining a segmented point cloud of rock elements; a second segmentation process of rock elements based on the image data of the pile of material, using image data processing techniques, for obtaining image-based rock elements data; a third segmentation process of rock elements by combining the segmented point cloud and the image-based rock elements data for obtaining a corrected segmented point cloud of rock elements; generating, by said processor, sub-regions in the rock elements from the corrected segmented point cloud of rock elements for obtaining rock breaking target pose candidates; validating said sub-regions according to predefined rules; selecting, by said processor, rock breaking target poses for the rock breaker based on a hierarchical organization of the rock breaking target pose candidates, according to predefined criteria; wherein the selected pose is used for breaking rock.

2. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the first segmentation process of rock elements includes a first step of removing from the point cloud, by said processor, points that do not correspond to the points of the rock element.

3. The method for determining and selecting rock breaking target poses according to claim 1 or 2, wherein the first segmentation process of rock elements based on watershed method adapted for surfaces structured as point clouds comprises the steps of: an initial labeling of points of the point cloud for identifying points related to convex surfaces that are most similar to rock elements, including: obtaining a normal vector and a curvature of each point; determining a height value of each point using a first height function, which uses the normal vector and curvature previously obtained; assigning a label to all the points with a height value above a threshold; performing morphological operations of dilation and erosion to the labeled points, for connecting labeled points that are near according to a predefined proximity radius; grouping the connected points based on proximity, assigning to the points of each group a second label, which correspond to the seed points; applying a flooding process to the seed points, in order to propagate the second labels of each group; and labeling the points that were not processed previously by associating them to a label of the nearest point, thereby obtaining a completely segmented point cloud, with each point containing the normal vector and a label corresponding to each rock element.

4. The method for determining and selecting rock breaking target poses according to claim 3, wherein the label dilation comprises: analyzing each point of the point cloud by iteratively selecting a point (p) to be analyzed; selecting a set of points (Q) that are close to point (p) according to a distance radio (r); determining if point (p) has been previously labeled with a label value in the labeling process; in case that point (p) has been previously labeled with a label value, adding the set of points (Q) to a list of points (L); labeling the list of points (L) according to the label of point (p). The method for determining and selecting rock breaking target poses according to claim 3, wherein the label erosion comprises: analyzing each point of the point cloud by iteratively selecting a point (p) to be analyzed; selecting a set of points (Q) that are close to point (p) according to a distance radio (r); identifying if point (p) has been previously labeled with a label value in the labeling process; in case that point (p) has been previously labeled with a label value and the set of points (Q) contains any point with a label different from the label of point (p), adding points (p) to a list of points (L); labeling the list of points (L) according to the point found having the different label. The method for determining and selecting rock breaking target poses according to claim 3, wherein the flooding process comprises: determining a height value of each point of the point cloud based on a second height function; assigning a height level to each height value; performing a hierarchical ranking to the height values according to a hierarchical parameter, based on the height level; assigning the points to a hierarchical list of points, which organize the set of points according to their hierarchical parameter; and analyzing each point of the hierarchical list from higher to lower ranking to carry out the following steps: in case a point of the hierarchical list is labeled from the dilation and erosion process, performing a propagation to neighbor points in the inferior level of hierarchy; in case a point of the hierarchical list is not labeled from the dilation and erosion process, searching for the nearest point that is labeled and performing a propagation to neighbor points in the inferior level of hierarchy. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the second segmentation process includes processing the image data in order to represent the rock elements as ellipses. The method for determining and selecting rock breaking target poses according to claim 7, wherein the third segmentation process of rock elements includes determining the centers of the ellipses and projecting the centers on the segmented point cloud of rock elements, generating coordinates in the space of the point cloud in a camera reference system, including centroids (x_P,y_P,z_P), major axis a_P, minor axis b_P and orientation 0_P. The method for determining and selecting rock breaking target poses according to claim 8, wherein the third segmentation process further comprises a correction step directed to assure that each rock element contains only one centroid, which includes identifying if a rock element determined in the first segmentation process contains more than one centroid determined in the second segmentation process, and subdividing the rock element into smaller rock elements so that each rock element has a single centroid associated. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the third segmentation process comprises: identifying rock elements with an incorrect segmentation process; identifying an ellipse of the incorrectly segmented rock element, the ellipse comprising ellipse variables (x_P,yp,zp, a_P, bp, 0p') determining a normal vector of the rock element; correcting a depth of the centroid in order to position the centroid in the center of the rock element; calculating a Mahalanobis distance to the centroid, which includes using a mayor axis of the ellipse as the first diagonal component of the covariance matrix, and using a minor axis as the second diagonal component of the covariance matrix; once the Mahalanobis distance for all ellipses and points have been calculated, associating each point on the rock element to the ellipse having the smallest distance to the point; and clustering the points associated to the same ellipse. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the step of generating sub-regions comprises grouping each point of the corrected segmented point cloud according to its location within a predefined space on a projected horizontal plane parallel to the grizzly. The method for determining and selecting rock breaking target poses according to claim 11 , wherein the size of each sub-region is selected considering the size of a chisel of the rock breaker. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the predefined rules to validate the sub-regions are: rule 1 : maintaining a vertical orientation of the rock breaker in relation to a surface of each sub-region. rule 2: determining the curvature of the surface of each sub-region, and if said curvature does not exceed a predefined tolerance, hitting the rock on that surface; and rule 3: if the size of the rock is greater than a predefined value, evaluating and selecting sub-regions that are in a range of distance to the rock border. The method for determining and selecting rock breaking target poses according to claim 13, wherein a sub-region will be discarded if at least one rule is not fulfilled. The method for determining and selecting rock breaking target poses according to claim 1 , wherein the predefined criteria to select the candidates rock breaking target poses for the rock breaker are: criterion 1 : selecting the most accessible pose, which is defined in terms of the less time required to reach the pose; criterion 2: breaking the largest rock, by identifying the rock element with the largest volume and identifying the target poses on that rock element; and criterion 3: clearing a specific region, wherein all the poses detected corresponding to rock elements outside said specific region are not considered. The method for determining and selecting rock breaking target poses according to claim 15, wherein criterion 1 includes: determining a current pose of the rock breaker; carrying out an inverse kinematics model to predict how the rock breaker will move until it reaches the target pose; estimating the time required to move the rock breaker from the current pose to the target pose; and preparing a list of target poses, ordering each target pose according to the access time determined, from the shortest to the longest time. A system for determining and selecting rock breaking target poses for a rock breaker, the system comprising: a plurality of sensors configured to acquire images and range data of rock elements in a pile of material; a processor operatively connected to said plurality of sensors to receive and process the sensor data, wherein the processor is configured to: generate a point cloud of the pile of material using the range data; identify rock elements in the pile of material performing a segmentation process, including: a first segmentation process of rock elements based on the point cloud of the pile of material, using a modified watershed method, for obtaining a segmented point cloud of rock elements; a second segmentation process of rock elements based on the image data of the pile of material, for obtaining image-based rock elements data; and a third segmentation process of rock elements by combining the segmented point cloud and the image-based rock elements data for obtaining a corrected segmented point cloud of rock elements; generate sub-regions in the rock elements from the corrected segmented point cloud of rock elements for obtaining rock breaking target pose candidates; validate said sub-regions according to predefined rules; and select rock breaking target poses for the rock breaker based on a hierarchical organization of the rock breaking target pose candidates, according to predefined criteria; wherein the selected pose determined by the processor is used by means for breaking rocks. The system for determining and selecting rock breaking target poses according to claim 17, wherein the plurality of sensors includes at least one range sensor to capture point clouds, including a sensor or sensor arrays capable of measuring distance selected from the group consisting of 2D LIDAR sensor, 3D LIDAR sensor, binocular camera, RADAR, TOF (Time of

Flight) camera, and 1 D laser scanner; and a camera to detect images selected from the group consisting of visual spectrum cameras, infrared spectrum cameras and thermal cameras.