CN114202631A - Method for determining rock working face and working point in secondary rock crushing operation - Google Patents
Method for determining rock working face and working point in secondary rock crushing operation Download PDFInfo
- Publication number
- CN114202631A CN114202631A CN202111280387.7A CN202111280387A CN114202631A CN 114202631 A CN114202631 A CN 114202631A CN 202111280387 A CN202111280387 A CN 202111280387A CN 114202631 A CN114202631 A CN 114202631A
- Authority
- CN
- China
- Prior art keywords
- rock
- working
- point
- target
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000011435 rock Substances 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 claims abstract description 25
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 6
- 230000008030 elimination Effects 0.000 claims description 3
- 238000003379 elimination reaction Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000005065 mining Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000010845 search algorithm Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005422 blasting Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005553 drilling Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the field of intelligent mines, in particular to a method for determining a rock working face and a rock working point in secondary rock crushing operation, and aims to solve the problem of intelligently selecting the working face and the rock working point in the secondary rock crushing operation. The invention comprises the following steps: performing multi-sensor fusion on a color camera and three-dimensional point cloud acquisition equipment; acquiring image information of a target area; inputting the target area image information into a rock detection network to detect a target rock; performing point cloud extraction on the target rock to obtain a target rock point cloud; performing plane segmentation on the target rock point cloud to obtain a fitting plane; removing outliers from the fitting plane to obtain a candidate operation surface; estimating the surface area of the candidate working face; judging whether the candidate working surface meets working conditions or not and determining the working surface; and determining a working point according to the working surface. The method provided by the invention realizes high-precision and intelligent determination of the secondary rock crushing operation surface and the operation point.
Description
Technical Field
The invention relates to the field of intelligent mines, in particular to a method for determining a rock working face and a rock working point in secondary rock crushing operation.
Background
Mines have long been regarded as high-risk, high-pollution, labor-intensive production enterprises, and improving the labor conditions of mines through technical progress and equipment level improvement is always a key and pursuit target of interest in the mining industry. Blasting-borehole mining is currently a common mining technique used in open-pit mining. However, the results of drilling and blasting are often far from ideal, the resulting rock fragments may be larger than expected, they cannot be placed in a primary crusher for primary crushing, and mechanical equipment such as a breaking hammer is required for secondary crushing of the rock to further reduce the size of the rock. In the operation scene of engineering operation of a surface mine, aiming at the engineering operation requirement of secondary rock crushing, no related technology is applied to the task of secondary rock crushing operation executed by engineering machinery at present so as to realize intelligent auxiliary operation or unmanned automatic operation, and no related technology is applied to the determination of the secondary rock crushing operation surface and the operation point in the surface mine environment.
Disclosure of Invention
The invention provides an intelligent determination method for a rock working face and a rock working point in secondary rock crushing operation, which realizes intelligent determination of the rock crushing working face and the rock working point and solves the problem of intelligent selection of the rock working face and the rock working point in intelligent secondary rock crushing operation.
According to a first aspect of a method for determining a rock face and a working point in a rock secondary crushing operation, there is provided a method for determining a rock face and a working point in a rock secondary crushing operation, the method comprising:
step S1, performing multi-sensor fusion on the color camera and the three-dimensional point cloud acquisition equipment;
step S2, acquiring image information of the target area;
step S3, inputting the target area image information into a rock detection network to detect a target rock;
step S4, carrying out point cloud extraction on the target rock to obtain a target rock point cloud;
step S5, performing plane segmentation on the target rock point cloud to obtain a fitting plane;
step S6, performing outlier elimination on the fitting plane to obtain a candidate operation surface;
step S7, estimating a surface area of the candidate work surface;
step S8, judging whether the candidate working surface meets the working condition and determining the working surface;
and step S9, determining a rock breaking operation point according to the operation surface.
In the step S1, the color camera, the laser radar, and other three-dimensional point cloud obtaining devices are fixedly mounted on a movable arm of a hydraulic breaking hammer, and other mechanical devices, and the color camera and the three-dimensional point cloud obtaining devices are subjected to multi-sensor fusion, and the three-dimensional point cloud obtaining devices are laser radars or depth cameras.
In the step S2, the image information includes a color image and a point cloud image of the target region, where the target region is a rock region to be crushed in a visual field range captured by a single frame of the color camera and the three-dimensional point cloud obtaining device.
The rock detection neural network in step S3 is a neural network trained and cured by a rock breaking target detection data set, the input of the neural network is the target area color map in step S2, the output of the neural network is the position information and the category information of the target rock which is the region of interest, and the rock breaking target detection data set includes various rock images under different environments and images correspondingly labeled with the position information and the category information of the breaking target.
Performing point cloud extraction on the target in step S4, specifically, registering the color image of the target rock obtained in step S3 and the point cloud obtained in step S2 to extract the point cloud of the target rock.
In step S5, performing plane segmentation on the target rock point cloud to obtain a fitting plane, where the plane segmentation specifically is a process of finding out all points belonging to the same plane model through a point cloud plane segmentation algorithm to obtain the fitting plane.
In step S6, the outliers are some edge points or noise points existing after the point cloud plane is segmented.
Step S7, performing surface area estimation on the candidate working surface, specifically: firstly, performing surface reconstruction, namely triangulation, on the candidate working surface to obtain a three-dimensional triangular mesh structure, and then calculating the surface area of the triangular mesh, namely the sum of single triangular surfaces to determine the real area of the target working surface.
In step S8, the working condition is specifically that if the candidate working surface satisfies the working condition, the candidate working surface is determined to be a rock crushing working surface, and if the candidate working surface does not satisfy the working condition, the candidate working surface is discarded.
In step S9, determining a rock breaking operation point according to the operation surface, specifically:
(1) projecting the candidate working surface obtained in the step S6 to a fitting two-dimensional plane;
(2) traversing the point cloud on the fitting plane to find four points including a minimum value point of an abscissa, a maximum value point of the abscissa, a minimum value point of an ordinate and a maximum value point of the ordinate, and establishing a rectangular frame externally connected with the fitting plane by using the four points;
(3) giving interval length, carrying out grid division on the rectangular frame, and obtaining the coordinate of the center point of each grid;
(4) giving a radius, and searching and inquiring all neighboring points of the central point in each grid to obtain the number of the neighboring points of the central point of each grid;
(5) and taking the central point of the grid with the maximum number of neighboring points as a rock crushing operation point.
The invention has the following advantages and beneficial effects:
(1) the method is low in cost, the whole working process is automated, the manual participation is obviously reduced, and necessary conditions are provided for intellectualization of secondary rock crushing.
(2) The method can realize the category identification of the target rock and the positioning in the camera space under the mine operation scene by adopting the target detection algorithm based on the convolutional neural network, and can meet the real-time and high-accuracy positioning of the rock in the intelligent mine.
(3) The method determines the rock crushing operation surface and the operation points by combining the color images and the point cloud images, can effectively avoid the influence of the mine construction environment on the determination of the rock crushing operation surface and the operation points, can effectively remove the noise error of image information and the influence caused by the accumulated error of the method by carrying out outlier removal on the point cloud data after the three-dimensional point cloud plane segmentation of the target rock, and realizes the accurate selection of the rock crushing operation surface and the operation points.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become more readily apparent from the following description.
Drawings
The above and other features, properties and advantages of the present invention will become more apparent from the following description of the embodiments with reference to the accompanying drawings, in which like reference numerals refer to like features throughout, and which are not to be construed as limiting the invention. Wherein:
FIG. 1 is a block flow diagram of a method for determining a rock face and a working point in a secondary rock crushing operation according to the present invention;
FIG. 2 is a rock detection network structure model according to an embodiment of the invention;
FIG. 3 is a block diagram of a random sampling consistency algorithm employed in the plane segmentation according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a plane segmentation effect according to an embodiment of the present invention;
FIG. 5 is a block diagram of a statistical filter algorithm for outlier removal according to an embodiment of the present invention;
FIG. 6 is a graph of the effect of a rock breaking face determined by an embodiment of the present invention;
fig. 7 is a diagram illustrating the effect of the rock breaking operation point determined by the embodiment of the invention.
Detailed Description
Embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Descriptions of well-known functions and constructions are omitted for clarity and conciseness.
Referring to fig. 1, a flow chart of a method for determining a rock working face and a working point in a secondary rock crushing operation includes the following steps:
step S1, performing multi-sensor fusion on the color camera and the three-dimensional point cloud acquisition equipment;
step S2, acquiring image information of the target area;
step S3, inputting the target area image information into a rock detection network to detect a target rock;
step S4, carrying out point cloud extraction on the target rock to obtain a target rock point cloud;
step S5, performing plane segmentation on the target rock point cloud to obtain a fitting plane;
step S6, performing outlier elimination on the fitting plane to obtain a candidate operation surface;
step S7, estimating a surface area of the candidate work surface;
step S8, judging whether the candidate working surface meets the working condition and determining the working surface;
and step S9, determining a rock breaking operation point according to the operation surface.
In step S1, for the three-dimensional point cloud obtaining device, a new lidar based on a non-repetitive scanning mode, i.e., Livox 40 medium lidar, is selected in this embodiment. In order to obtain scene data containing a target rock to be crushed, it is necessary to mount a color camera and a laser radar on a boom of a hydraulic crushing hammer or the like so as to cover more rock. The embodiment performs multi-sensor fusion on the monocular camera and the laser radar so as to register the color image and the point cloud.
The two-dimensional rock detection in step S3 is one of the key steps of the method, and the purpose of this step is to identify the position information and the category information of the target rock in the two-dimensional image by single frame capture of a color camera. The YOLO-V3 network based on convolutional neural network regards the rock detection task as a regression task, and through once estimation of a single frame color image captured by the color camera, all target rocks in the color image can be predicted once, for secondary crushing of rock, the detection task is a single target detection task, and considering the real-time requirement of equipment tasks such as breaking hammer, the present embodiment selects a single-stage method based on YOLO-V3 neural network to perform two-dimensional rock detection, as shown in fig. 2, where the input X is an image obtained by image preprocessing of the color image obtained in step S2, Darknet53 with FC layer represents a Darknet53 neural network without a full connection layer, CBL is the minimum component in the YOLO-V3 neural network structure, Res unit is a neural network formed by using the residual structure in Res network, ResX is a network structure composed of 1 CBL and X residual components, the method is characterized in that the method is a component in the YOLO-V3 neural network, Concat refers to tensor splicing, add refers to tensor addition, BN refers to a regularization layer, Leaky relu is an activation function in the neural network, output refers to the prediction of three bounding boxes for each grid unit, targets with different sizes are detected by a multi-scale method, and three feature maps with different scales are output.
In this embodiment, a series of image preprocessing operations including graying, gaussian filtering and Sobel operator convolution are performed on the color image acquired by the color camera, and the obtained gradient image is used as the input of the rock detection network. In order to minimize the effect of noise on the final detection result, the noise must be filtered to prevent false detection due to noise. To smooth the image, the image is convolved with a gaussian filter. The gaussian kernel is the key to the overall solution, which is computed using a two-dimensional gaussian function. A discrete gaussian convolution kernel H: (2k +1) × (2k +1) can be calculated by the formula (1).
Where σ is the variance, k is the dimension of the kernel matrix, and i, j represents the (i, j) -th element.
The gradient strength and direction of each pixel in the image is then calculated. In the embodiment, a Sobel operator is selected to calculate the difference between Gx and Gy in the horizontal and vertical directions, and then the modulus and direction of the gradient are obtained.
θ＝atan2(Gx,Gy) (3)
Where A is the image matrix and Gx and Gy are the gradients in the horizontal and vertical directions.
And finally, taking a single-channel image subjected to graying, Gaussian filtering and gradient calculation as the input of the whole rock detection neural network.
And (4) clustering by using a K-Means algorithm according to the label of the training data set to obtain 9 prior frames. The input image is divided into s-by-s grid cells. When the target center falls within the grid, the grid cell is responsible for detecting it. Logistic regression is used to predict each bounding box. Coordinate b of the center point of the bounding boxxCoordinate byWidth bwHeight bhAnd confidence calculation formula as follows:
bx＝σ(tx)+cx (5)
by＝σ(ty)+cy (6)
confidence＝pr(object)*IOU(b,object)＝σ(t0) (9)
wherein, tx，ty，tw，th，t0Is the predicted output of the model; c. Cx,cyCoordinates of the upper left corner of the network cell relative to the whole image; p is a radical ofw、phRespectively representing the width and height, p, of the current bounding boxr(object) is the probability of whether the target rock is contained in the current bounding box; IOU (b, object) the intersection ratio of the predicted bounding box to the actual boundary when the current bounding box contains the target rock.
In step S4, the embodiment performs registration based on the two-dimensional region of interest obtained by rock detection on the color image and the point cloud map obtained by the laser radar, and then obtains a point cloud of the target rock. And selecting the point cloud subsets acquired by the laser radar by using the bounding box of the image after target detection, and fusing the target detection result of the two-dimensional image with the point cloud to obtain a plurality of point cloud subsets.
In the step S5, in the target rock point cloud plane segmentation, the random sample consensus algorithm ransac (random sample consensus) is used to segment the surface point cloud, so as to obtain a plurality of candidate working surfaces of the broken target. The random sample consensus algorithm is an iterative method for calculating mathematical model parameters from a series of data containing outliers, and the flow diagram is shown in fig. 3. The RANSAC algorithm essentially consists of two steps, continuously looping:
(1) randomly selecting the minimum number of elements which can form a mathematical model from input data, and calculating parameters of the corresponding model by using the elements;
(2) and checking which elements in all data can accord with the model obtained in the first step, wherein the elements exceeding the error threshold are regarded as outliers, and the elements smaller than the error threshold are regarded as interior points.
The steps are repeated for a plurality of times, and the model containing the most points is selected to obtain the final result.
The method for obtaining a plurality of broken target candidate operation surfaces by dividing the surface point cloud by using a random sample consensus (RANSAC) algorithm comprises the following specific steps:
(1) firstly, randomly selecting three points from the target rock point cloud in the step S4, and then calculating plane models A, B, C and D according to a formula (10), wherein x, y and z respectively represent unit vectors of three coordinate axes;
A·x+B·y+C·z+D＝0 (10)
(2) using the remaining cloud points of the target rock points to test the plane model in the step (1), calculating a result error, comparing the result error with a set threshold, determining the point as an interior point if the result error is smaller than the set threshold, counting the number of the interior points under the parameter model and recording the number;
(3) continuing to carry out the step (1) and the step (2), if the number of the interior points of the current model is larger than the stored maximum number of the interior points, updating the model parameters, and keeping the model parameters which are the model parameters with the maximum number of the interior points all the time;
(4) repeating the above steps, continuously iterating until an iteration threshold is reached, finding the model parameter with the largest number of interior points, and finally estimating the model parameter again by using the interior points, so as to obtain the final model parameter, namely the candidate operation surface of a plurality of targets, wherein fig. 4 shows a cloud image of the candidate operation surface point of one target.
After the target rock point cloud extraction and the target rock point cloud plane segmentation, there are some outer edge points and noise points. In order to better obtain the target working surface, it is necessary to filter out these redundant points, i.e., to perform the operation described in step S6. In the embodiment, the statistical filter is adopted to remove outliers in the candidate target plane, the outliers are distributed sparsely in the space, and in consideration of the characteristics of the outliers, a certain point cloud can be defined to be less than a certain density, namely, an invalid point cloud. Performing statistical analysis on the neighborhood of each point, calculating the average distance from each point to the nearest k points, wherein the distances of all the points in the point cloud should form a Gaussian distribution, and defining the points with the average distance outside the standard range (defined by the mean and variance of the global distances) as outliers and removing the outliers from the data, and the flow chart is shown in FIG. 5.
The specific calculation process of removing outliers in the point cloud of the candidate target plane by using the statistical filter in the embodiment in step S6 is as follows:
and (3) performing statistical analysis on the neighborhood of each point, and assuming that the distances of all points in the target rock point cloud form Gaussian distribution, wherein the shape of the Gaussian distribution is determined by the mean value mu and the standard deviation sigma. Setting the nth point coordinate in the point cloud as Pn(Xn,Yn,Zn) From the point to an arbitrary point Pm(Xm,Ym,Zm) The distance of (a) is:
the average formula for calculating the distance between each point traversed to any point is:
the standard deviation is:
and (4) setting the multiple of the standard deviation as std, inputting two thresholds of k and std in the algorithm implementation process, and keeping a point when the average distance of the point adjacent to k points is within a standard range (mu-sigma std, mu + sigma std), and defining the point not within the range as outlier deletion.
Since three-dimensional distance information cannot be obtained by two-dimensional target detection of rocks, whether the detected target rocks satisfy the crushing condition cannot be calculated. Therefore, as shown in step S7, the present embodiment uses the obtained plane point cloud to estimate the real area of the target working surface, and then determines whether the working surface meets the requirement of rock secondary crushing. Specifically, firstly, establishing a target operation surface point cloud, performing surface reconstruction, namely triangulation to obtain a three-dimensional triangular mesh structure, and then estimating the surface area of the candidate operation surface point cloud. In the process of triangulation, an alpha-shape algorithm is used, and in a three-dimensional layer, the alpha-shape algorithm can consider that a ball rolls in a pile of points in a centralized manner, three points meeting conditions form a triangle until the whole candidate operation surface forms a three-dimensional triangular mesh structure, the ball does not contain other points except three base points, and the alpha value is the radius of the ball. The surface area of the triangular mesh, i.e. the sum of the individual triangular surfaces, is then calculated to determine the true area of the target work surface.
In step S8, different requirements are imposed on the target rock to be broken because of different working scenarios. Therefore, it is necessary to set an area threshold to deal with the screening of the rock breaking target working surface under different working requirements, and fig. 6 is a graph showing the effect of the rock breaking target working surface. And if the surface area of the candidate working face meets the working requirement, determining the candidate working face as a rock crushing working face, and otherwise, abandoning the candidate working face.
And (4) judging that the candidate working face meets the working requirements and is determined to be a rock crushing working face by the judgment of the step S8, and then selecting the crushing working point in the step S9 by using the point cloud obtained in the step S6, wherein an effect graph of the rock crushing working point is shown in FIG. 7. The method for determining the operation point adopted by the embodiment comprises the following steps:
(1) projecting the point cloud obtained in the step S6 to a fitting two-dimensional plane;
(2) traversing the point cloud on the fitting plane to find four points including a minimum value point of an abscissa, a maximum value point of the abscissa, a minimum value point of an ordinate and a maximum value point of the ordinate, and establishing a rectangular frame externally connected with the fitting plane by using the four points;
(3) carrying out grid division on the rectangular frame by taking the length d as an interval, and obtaining the coordinate of the center point of each grid;
(4) given radius
Searching and inquiring all the neighbor points of the central point in each grid by using a radius neighbor search algorithm (RNN search algorithm) to obtain the number of the neighbor points of the central point of each grid;
(5) and setting the central point of the grid with the maximum number of neighboring points as a rock crushing operation point.
In the step S9, the point cloud is projected onto the fitting two-dimensional plane, and the algorithm used in this embodiment is summarized as follows:
a general equation for a given three-dimensional spatial plane is shown in equation (10), assuming that the three-dimensional spatial coordinates not on the plane are (x)0,y0,z0) The coordinate of the projection point on the plane is (x)p,yp,zp) Because the projection point is perpendicular to the current point and the plane, y can be known according to the vertical constraint conditionpAnd zpThe following conditions are satisfied:
by substituting expressions (14) and (15) into expression (10), the following can be solved:
substituting equation (16) into equations (14) and (15) can obtain:
from this, the projection coordinates (x) of the three-dimensional points in space to the plane can be solvedp,yp,zp) And repeating the steps through the point cloud obtained in the step S6, and projecting the three-dimensional point cloud to a fitting two-dimensional plane.
The specific setting method and implementation of the embodiment of the invention are described above. The above-described embodiments should not be construed as limiting the scope of the invention. It will be apparent to those of ordinary skill in the art that various modifications, combinations, sub-combinations, and substitutions can be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for determining a rock working face and a working point in secondary rock crushing operation is characterized by comprising the following steps:
step S1, performing multi-sensor fusion on the color camera and the three-dimensional point cloud acquisition equipment;
step S2, acquiring image information of the target area;
step S3, inputting the target area image information into a rock detection network to detect a target rock;
step S4, carrying out point cloud extraction on the target rock to obtain a target rock point cloud;
step S5, performing plane segmentation on the target rock point cloud to obtain a fitting plane;
step S6, performing outlier elimination on the fitting plane to obtain a candidate operation surface;
step S7, estimating a surface area of the candidate work surface;
step S8, judging whether the candidate working surface meets the working condition and determining the working surface;
and step S9, determining a rock breaking operation point according to the operation surface.
2. The method for determining the working face and the working point of the rock in the secondary rock crushing operation as claimed in claim 1, wherein the color camera and the three-dimensional point cloud obtaining device are fixedly installed on a movable arm of a mechanical device such as a hydraulic crushing hammer in the step S1, and the color camera and the three-dimensional point cloud obtaining device are subjected to multi-sensor fusion, and the three-dimensional point cloud obtaining device is a laser radar or a depth camera.
3. The method as claimed in claim 1, wherein the image information in step S2 includes color map and point cloud map of the target area, and the target area is a rock area to be crushed in the visual field range captured by a single frame of the color camera and the three-dimensional point cloud acquiring device.
4. The method as claimed in claim 1, wherein the rock detection neural network in step S3 is a hardened neural network trained by a rock target detection data set, the input of the hardened neural network is the target area color map in step S2, the output of the hardened neural network is the region of interest, the region of interest is the position information and the category information of the target rock, and the rock target detection data set comprises images of various rocks under different environments and the corresponding images marked with the position information and the category information of the target rock.
5. The method as claimed in claim 1, wherein the point cloud extraction is performed on the target rock in step S4, specifically, the color map of the target rock obtained in step S3 and the point cloud map obtained in step S2 are registered to extract the point cloud of the target rock.
6. The method of claim 1, wherein the plane segmentation in step S5 is a process of finding all points belonging to the same plane model by a point cloud plane segmentation algorithm to obtain a fitting plane.
7. The method of claim 1, wherein the outliers of step S6 are some edge points or noise points existing after the point cloud plane segmentation.
8. The method according to claim 1, wherein step S7 performs surface area estimation on the candidate worksurface, specifically:
(1) firstly, carrying out surface reconstruction, namely triangulation, on the candidate working surface to obtain a three-dimensional triangular mesh structure;
(2) the surface area of the triangular mesh, i.e., the sum of the individual triangular surfaces, is calculated to determine the true area of the candidate worksurface.
9. The method according to claim 1, wherein whether the surface area of the candidate working surface satisfies the working condition in step S8 is specifically: and if the candidate working face meets the working conditions, determining the candidate working face to be a rock crushing working face, and if the candidate working face does not meet the working conditions, abandoning the candidate working face.
10. The method according to claim 1, characterized in that in step S9 the rock breaking operation point is determined from the working plane, in particular:
(1) projecting the candidate working surface obtained in the step S6 to a fitting two-dimensional plane;
(2) traversing the point cloud on the fitting plane to find four points including a minimum value point of an abscissa, a maximum value point of the abscissa, a minimum value point of an ordinate and a maximum value point of the ordinate, and establishing a rectangular frame externally connected with the fitting plane by using the four points;
(3) carrying out grid division on the rectangular frame by taking the length d as an interval, and solving the coordinate of the central point of each grid, wherein the length d is determined according to different operation requirements;
(4) given radius
d, carrying out grid division on the rectangular frame for interval length, and searching and querying all neighbor points of the central point in each grid to obtain the number of the neighbor points of the central point of each grid;
(5) and taking the central point of the grid with the maximum number of neighboring points as a rock crushing operation point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111280387.7A CN114202631A (en) | 2021-11-01 | 2021-11-01 | Method for determining rock working face and working point in secondary rock crushing operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111280387.7A CN114202631A (en) | 2021-11-01 | 2021-11-01 | Method for determining rock working face and working point in secondary rock crushing operation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114202631A true CN114202631A (en) | 2022-03-18 |
Family
ID=80646780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111280387.7A Pending CN114202631A (en) | 2021-11-01 | 2021-11-01 | Method for determining rock working face and working point in secondary rock crushing operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114202631A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115110598A (en) * | 2022-08-10 | 2022-09-27 | 安徽建工集团股份有限公司总承包分公司 | Three-dimensional fitting site excavation crushing device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104183017A (en) * | 2014-07-29 | 2014-12-03 | 浙江大学 | Ground three-dimensional laser point cloud based method for realizing automatic extraction of geologic body occurrence |
| CN110264416A (en) * | 2019-05-28 | 2019-09-20 | 深圳大学 | Sparse point cloud segmentation method and device |
| CN111583337A (en) * | 2020-04-25 | 2020-08-25 | 华南理工大学 | Omnibearing obstacle detection method based on multi-sensor fusion |
| CN112396650A (en) * | 2020-03-30 | 2021-02-23 | 青岛慧拓智能机器有限公司 | Target ranging system and method based on fusion of image and laser radar |
| CN113298781A (en) * | 2021-05-24 | 2021-08-24 | 南京邮电大学 | Mars surface three-dimensional terrain detection method based on image and point cloud fusion |
-
2021
- 2021-11-01 CN CN202111280387.7A patent/CN114202631A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104183017A (en) * | 2014-07-29 | 2014-12-03 | 浙江大学 | Ground three-dimensional laser point cloud based method for realizing automatic extraction of geologic body occurrence |
| CN110264416A (en) * | 2019-05-28 | 2019-09-20 | 深圳大学 | Sparse point cloud segmentation method and device |
| CN112396650A (en) * | 2020-03-30 | 2021-02-23 | 青岛慧拓智能机器有限公司 | Target ranging system and method based on fusion of image and laser radar |
| CN111583337A (en) * | 2020-04-25 | 2020-08-25 | 华南理工大学 | Omnibearing obstacle detection method based on multi-sensor fusion |
| CN113298781A (en) * | 2021-05-24 | 2021-08-24 | 南京邮电大学 | Mars surface three-dimensional terrain detection method based on image and point cloud fusion |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115110598A (en) * | 2022-08-10 | 2022-09-27 | 安徽建工集团股份有限公司总承包分公司 | Three-dimensional fitting site excavation crushing device |
| CN115110598B (en) * | 2022-08-10 | 2023-11-28 | 安徽建工集团股份有限公司总承包分公司 | Three-dimensional fitting field excavating and crushing device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Computational methods of acquisition and processing of 3D point cloud data for construction applications | |
| Guo et al. | Dense construction vehicle detection based on orientation-aware feature fusion convolutional neural network | |
| Mirzaei et al. | 3D point cloud data processing with machine learning for construction and infrastructure applications: A comprehensive review | |
| Wang et al. | Automated estimation of reinforced precast concrete rebar positions using colored laser scan data | |
| KR102660837B1 (en) | Method for landslide crack detection based deep learning and Method for landslide monitoring therewith and Apparatus thereof | |
| CN110472553B (en) | Target tracking method, computing device and medium for fusion of image and laser point cloud | |
| Teizer et al. | Personnel tracking on construction sites using video cameras | |
| Wang et al. | Modeling indoor spaces using decomposition and reconstruction of structural elements | |
| CN110065790B (en) | Method for detecting blockage of coal mine belt transfer machine head based on visual algorithm | |
| US20160133008A1 (en) | Crack data collection method and crack data collection program | |
| US20140169639A1 (en) | Image Detection Method and Device | |
| Guo et al. | Evaluation-oriented façade defects detection using rule-based deep learning method | |
| US20160133007A1 (en) | Crack data collection apparatus and server apparatus to collect crack data | |
| Vetrivel et al. | Potential of multi-temporal oblique airborne imagery for structural damage assessment | |
| Dhiman et al. | A multi-frame stereo vision-based road profiling technique for distress analysis | |
| Truong-Hong et al. | Laser scanning for bridge inspection | |
| CN114022760B (en) | Railway tunnel barrier monitoring and early warning method, system, equipment and storage medium | |
| Zhang et al. | Concrete crack quantification using voxel-based reconstruction and Bayesian data fusion | |
| CN114202631A (en) | Method for determining rock working face and working point in secondary rock crushing operation | |
| CN113870326A (en) | Structural damage mapping, quantifying and visualizing method based on image and three-dimensional point cloud registration | |
| JP7348575B2 (en) | Deterioration detection device, deterioration detection system, deterioration detection method, and program | |
| Gehrung et al. | A fast voxel-based indicator for change detection using low resolution octrees | |
| CN117367404A (en) | Visual positioning mapping method and system based on SLAM (sequential localization and mapping) in dynamic scene | |
| Hamledari | Inpro: Automated indoor construction progress monitoring using unmanned aerial vehicles | |
| Benz et al. | Multi-View 3D Instance Segmentation of Structural Anomalies for Enhanced Structural Inspection of Concrete Bridges |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |