CN112330607A - Coal and gangue identification method, device and system based on image identification technology - Google Patents
Coal and gangue identification method, device and system based on image identification technology Download PDFInfo
- Publication number
- CN112330607A CN112330607A CN202011126884.7A CN202011126884A CN112330607A CN 112330607 A CN112330607 A CN 112330607A CN 202011126884 A CN202011126884 A CN 202011126884A CN 112330607 A CN112330607 A CN 112330607A
- Authority
- CN
- China
- Prior art keywords
- image
- coal
- gangue
- segmentation
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003245 coal Substances 0.000 title claims abstract description 259
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000005516 engineering process Methods 0.000 title claims abstract description 30
- 230000011218 segmentation Effects 0.000 claims abstract description 114
- 238000003709 image segmentation Methods 0.000 claims abstract description 28
- 239000000203 mixture Substances 0.000 claims abstract description 16
- 238000001514 detection method Methods 0.000 claims description 33
- 238000012545 processing Methods 0.000 claims description 14
- 238000010801 machine learning Methods 0.000 claims description 13
- 238000000926 separation method Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000000877 morphologic effect Effects 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 6
- 230000010339 dilation Effects 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 description 15
- 230000008569 process Effects 0.000 description 13
- 238000003860 storage Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000005303 weighing Methods 0.000 description 5
- 238000010187 selection method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 239000010878 waste rock Substances 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000704 physical effect Effects 0.000 description 2
- 241000270295 Serpentes Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000007885 magnetic separation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000035699 permeability Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 239000002341 toxic gas Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Abstract
The invention provides a coal and gangue identification method, a device and a system based on an image identification technology, wherein the method comprises the following steps: acquiring an image of a mixture of coal and gangue, and performing image segmentation on the image to obtain a segmentation result; and performing coal and gangue identification according to image characteristics corresponding to the segmentation result, wherein the image characteristics comprise at least one of the following items: color features of the image, texture features of the image, pixel variance features of the image. The method carries out the coal and gangue identification based on the image identification technology, does not need to set complex hardware equipment, is easy to deploy the hardware equipment, has no pollution to the environment, and has relatively low cost and high accuracy.
Description
Technical Field
The invention relates to the technical field of coal and gangue separation, in particular to a coal and gangue identification method, device and system based on an image identification technology.
Background
In the coal production process, raw coal directly mined often contains a certain proportion (generally between 10% and 30%) of gangue. The gangue is generally high in density and low in calorific value, has a large influence on the quality of raw coal, and needs to be separated from the coal.
At present, a plurality of coal-gangue separation technologies are provided, and gangue separation is mainly carried out according to the difference of the physical properties of coal and gangue. These physical properties include: density, hardness, gray scale, texture, magnetic permeability, emissivity, coefficient of friction, and the like. The traditional selection method comprises a dry selection method and a wet selection method, wherein the dry selection method mainly comprises the following steps: manual sorting, a selective crushing method, a wind power coal separation method, a ray perspective method, a magnetic separation method and the like; the wet separation method mainly comprises the following steps: heavy medium coal dressing method.
The traditional gangue selecting method has defects in the aspects of production cost, production environment and the like, for example, the manual sorting method has the problems of occupational hazards such as high noise, thick dust, poor environment, high labor intensity and the like; the wind coal separation method has very high requirements on the volume of coal; radioscopy is a great harm to the environment and human body.
Disclosure of Invention
The invention solves the problem that the traditional gangue selecting method has defects in the aspects of production cost, production environment and the like.
In order to solve the problems, the invention provides a coal and gangue identification method based on an image identification technology, which comprises the following steps: acquiring an image of a mixture of coal and gangue, and performing image segmentation on the image to obtain a segmentation result; performing coal and gangue identification according to the image characteristics corresponding to the segmentation result; wherein the image features comprise at least one of: color features of the image, texture features of the image, pixel variance features of the image.
Optionally, the image segmenting the image to obtain a segmentation result includes: performing image segmentation on the image according to a preset segmentation algorithm to obtain a segmentation object; the segmentation object comprises a contour and a coal gangue category; the coal and gangue identification according to the image characteristics corresponding to the segmentation result comprises the following steps: and identifying coal and gangue according to the image characteristics corresponding to the segmentation object so as to verify whether the coal and gangue category of the segmentation object is correct.
Optionally, obtaining the weight and the volume of a target corresponding to the segmentation result, and calculating the density of the target according to the weight and the volume; and verifying whether the coal and gangue identification result is correct or not according to the density of the target, the density of the coal and the density of the gangue.
Optionally, the performing coal and gangue identification according to the image feature corresponding to the segmentation result, where the image feature is a color feature of an image, includes: determining a target pixel point of the pixel value in the segmentation result within a preset pixel value range; and determining that the segmentation result corresponds to coal or gangue according to the size relation between the area of the target pixel point and a preset area threshold value.
Optionally, the image feature is a texture feature of an image, and performing coal and gangue identification according to the image feature corresponding to the segmentation result includes: extracting texture features of the segmentation result through a gray level co-occurrence matrix; if the numerical value corresponding to the texture feature is in a first texture range, determining that the segmentation result corresponds to the coal; the first texture range is a numerical range corresponding to the predetermined texture features of the coal image; if the numerical value corresponding to the texture feature is in a second texture range, determining that the segmentation result corresponds to the gangue; the second texture range is a numerical range corresponding to the texture features of the predetermined gangue image.
Optionally, the image feature is a pixel variance feature of an image, and performing coal and gangue identification according to the image feature corresponding to the segmentation result includes: performing morphological dilation processing on the segmentation result to obtain a dilated image; calculating a variance of pixel values of the dilated image; determining that the segmentation result corresponds to coal if the variance is within a first variance range; the first variance range is a predetermined variance range of the coal image; if the variance is within a second variance range, determining that the segmentation result corresponds to gangue; the second variance range is a predetermined variance range of the gangue image.
Optionally, after the step of performing morphological dilation processing on the segmentation result to obtain a dilated image, the method further includes: and performing pooling processing on the expanded image, and calculating the variance of the pixel values of the pooled image.
Optionally, the performing coal and gangue identification according to the image features corresponding to the segmentation result includes: classifying the segmentation result through a pre-trained machine learning model, and determining that the segmentation result corresponds to coal or gangue; the machine learning model is obtained by training a sample set of coal and gangue images, wherein samples of the sample set comprise a coal image and a gangue image, color features, texture features and pixel variance features of the coal image and the gangue image, and category labels of the coal image and the gangue image.
The embodiment of the invention provides a coal and gangue identification device based on an image identification technology, which comprises: the segmentation module is used for acquiring an image of a mixture of coal and gangue and performing image segmentation on the image to obtain a segmentation result; the coal and gangue identification module is used for identifying coal and gangue according to the image characteristics corresponding to the segmentation result; wherein the image features comprise at least one of: color features of the image, texture features of the image, pixel variance features of the image.
The embodiment of the invention provides a coal and gangue identification system based on an image identification technology, which comprises: the camera device is used for acquiring an image of the mixture of the coal and the gangue; the server is used for executing the coal and gangue identification method based on the image identification technology; the density detection device is used for detecting the density of the coal and/or the gangue; and the separation execution device is used for separating the coal and the gangue according to the coal and gangue identification result.
The embodiment can perform image segmentation on the image of the mixture of coal and gangue, and then perform coal and gangue identification according to the image characteristics corresponding to the segmentation result, wherein the image characteristics comprise color characteristics, texture characteristics and pixel variance characteristics.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic flow chart of a coal and gangue identification method based on image identification technology in one embodiment of the invention;
FIG. 2 is a schematic diagram illustrating the operation of an intelligent gangue sorting system according to one embodiment of the present invention;
FIG. 3 is a graph illustrating the effect of an image segmentation algorithm output according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a coal and gangue identification system based on image identification technology according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a coal gangue identification device based on an image identification technology in an embodiment of the invention.
Description of reference numerals:
401-a first camera; 402-a second camera; 403-a server; 404-a belt conveyor; 405-a load cell; 406-a robotic arm; 501-a segmentation module; 502-gangue identification module.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the coal production process, if the gangue is not reasonably treated, not only the space is occupied, but also the toxic gas emitted by the gangue can cause serious pollution to water sources, soil and air. In addition, the gangue is easy to self-ignite, thus causing fire hazard and bringing threat to the life and property safety of people. Therefore, a green and miniaturized coal and gangue identification technology is researched, so that the discovery of the gangue is close to the source as much as possible, links such as transportation, disposal and the like are reduced, and the method has great social benefit and economic benefit.
In the embodiment, the coal and gangue are identified by adopting the coal and gangue identification method based on the image identification technology, the adopted equipment is easy to deploy, has no pollution to the environment, has relatively low cost and has the accuracy rate equivalent to the most mature gangue selection technology at present. Furthermore, the adopted artificial intelligence image recognition algorithm also has very good adaptability and huge optimization space, and is beneficial to further upgrading and improving the recognition accuracy.
Fig. 1 is a schematic flow chart of a coal and gangue identification method based on an image identification technology in an embodiment of the invention, which includes:
and S102, acquiring an image of the mixture of the coal and the gangue, and performing image segmentation on the image to obtain a segmentation result.
In the coal mining process, the mixture of the mined coal and the gangue can be conveyed by a belt conveyor, and an image of the mixture of the coal and the gangue is acquired by arranging a camera above the belt conveyor. The server may then segment the image based on an image segmentation algorithm to obtain a segmentation result that includes contours of objects in the image.
In order to improve the overall accuracy of the coal and gangue identification method, the classification of each target in the image may be identified when the image is segmented, and optionally, S102 may include the following steps: and carrying out image segmentation on the image according to a preset segmentation algorithm to obtain a segmentation object, wherein the segmentation object can comprise the contour of each target and the coal and gangue category of each target. The preset segmentation algorithm may be, for example, an example segmentation algorithm, and performs image segmentation on the coal and the gangue to obtain categories and pixel coordinates of the coal and the gangue.
It should be noted that, during image segmentation, first coal and gangue identification is performed, and then, in subsequent steps, second coal and gangue identification is performed based on image features, so as to verify whether the coal and gangue type of the segmentation object is correct. The coal and gangue identification method based on image segmentation is characterized in that a subsequent coal and gangue identification method based on image features is combined according to contour shape features of coal and gangue, identification is respectively carried out from different angles based on surface optical features of the coal and gangue, and after combination, identification result verification is carried out, so that the final coal and gangue identification result can be improved.
And S104, identifying the coal and gangue according to the image characteristics corresponding to the segmentation result. Wherein the image features may include: color features of an image, texture features of an image, pixel variance features of an image, and the like.
After the image of the mixture of coal and gangue is segmented, segmented images of a plurality of targets are obtained, and the coal or gangue in each segmented image needs to be further identified. In this embodiment, the image features of the segmented image are used as a basis for image recognition, because the coal and the gangue have different characteristics in the surface optical features, the color features, the texture features and the pixel variance features of the image of the coal and the gangue have different characteristics, and the server can effectively recognize the coal and the gangue based on the different characteristics of the image features of the coal and the gangue, so that an accurate coal and gangue recognition result can be obtained.
The color feature, texture feature, and pixel variance feature of an image may be recognized as a recognition element alone or in combination.
The coal and gangue identification method based on the image identification technology provided by the embodiment can be used for carrying out image segmentation on the image of the mixture of coal and gangue and then carrying out coal and gangue identification according to the image characteristics corresponding to the segmentation result, wherein the image characteristics comprise color characteristics, texture characteristics and pixel variance characteristics.
After the gangue category is determined based on the profile shape feature accurate segmentation and identified based on the surface optical feature, a third-time gangue identification method is provided in the embodiment, that is, gangue identification is performed based on coal and gangue density estimation, so as to continuously verify whether the gangue category identified based on the surface optical feature is correct. The above method may further comprise the steps of: acquiring the weight and the volume of a target corresponding to the segmentation result, and calculating the density of the target according to the weight and the volume; according to the aboveAnd verifying whether the result of coal and gangue identification is correct or not according to the target density, the coal density and the gangue density. Wherein the density of the coal and the density of the gangue can be obtained by looking up the data, and the density of the coal is about 1370-1538 kg/m3The density of the gangue is about 1818-2128 kg/m3In the meantime.
Therefore, the embodiment proposes a gangue separation method of three detections by using an image recognition technology based on the physical and optical characteristics of coal and gangue, and the gangue separation method comprises the following steps: the method comprises the steps of accurate segmentation identification of coal and gangue profiles, identification of optical characteristics of coal and gangue surfaces and estimation identification of coal and gangue density. The three detection methods respectively identify the contour shape characteristics of the coal and gangue, the surface optical characteristics of the coal and gangue and the physical density characteristics of the coal and gangue from different angles, and then verify the identification result, so that the accuracy of coal and gangue identification can be better improved.
The following describes a detection process based on the optical feature recognition of the coal and gangue surfaces, and the method for extracting the image features in this embodiment includes color features, texture features, and variance features of the image. By combining the surface optical characteristics of the coal and gangue and the image characteristic identification method, the image identification method for effectively identifying the coal and gangue is provided.
Taking one image as an example for explanation, it is assumed that 10 targets are detected by the example segmentation algorithm in the image, and the category is coal or gangue, but there may be some false detection in the detection of the category, such as false detection of coal as gangue or false detection of gangue as coal. Because the gangue identification method considers the target with the diameter of 100mm to 400mm, the mechanical arm is used for grabbing the coal in the next detection process, so that the coal is mainly considered to be detected in the current detection process, namely, when an example segmentation algorithm detects the image of a certain frame, when the example segmentation algorithm detects the type of the coal, the image of the coal obtained by segmentation is firstly sent to an image optical characteristic detection program. If the detection result is coal, the category and the coordinate information of the target are sent to a program of a mechanical arm, and the mechanical arm grabs the target; in contrast, if the detection result of the optical feature detection program of the image is gangue, it means that the optical feature detection program considers the image of the coal as a false detected image, the true category of which should be gangue, and this category and coordinate information are not sent to the robot arm program. Therefore, the detection result in the last detection process is corrected, and the accuracy of the overall detection and identification is improved.
The detection method based on the image characteristics combines the surface optical characteristics actually possessed by the coal gangue and the image characteristic identification method, can effectively identify the coal gangue, and the identification results obtained by identifying the coal gangue through the three detection methods from different angles are sequentially verified so as to improve the accuracy of coal gangue identification. The method specifically comprises the following steps:
in the first method, the identification is performed based on color characteristics of the image, and the S104 may include: determining a target pixel point of which the pixel value is within a preset pixel value range in the segmentation result; and determining that the segmentation result corresponds to coal or gangue according to the size relation between the area of the target pixel point and a preset area threshold value.
For images of coal and gangue, the white bright spots on the surface of the coal are relatively more, and the gangue is relatively darker and has no obvious white bright spots. Firstly, determining a preset pixel value range by obtaining a pixel range where a white bright point is located; then, the area of the pixel having the pixel value within the range in the image corresponding to each division result is counted.
The counted white bright spot pixel area of the coal is inevitably larger than that of the gangue, so that the coal and the gangue are distinguished by setting a reasonable area threshold, wherein the category of the image larger than the area threshold is the coal, and the category of the image smaller than the area threshold is the gangue. When the segmentation image of which the category is marked as coal or gangue and output by the example segmentation algorithm is detected, the false detection picture can be picked out, and the accuracy of the overall identification is improved.
In the second method, the identification is performed based on the texture features of the image, and the step S104 may include: extracting texture features of the segmentation result through a gray level co-occurrence matrix, wherein the texture features can include: energy, entropy, contrast, uniformity, or correlation; if the numerical value corresponding to the texture feature is in the first texture range, determining that the segmentation result corresponds to the coal; the first texture range is a numerical range corresponding to the predetermined texture features of the coal image; if the numerical value corresponding to the texture feature is in the second texture range, determining that the segmentation result corresponds to the gangue; the second texture range is a numerical range corresponding to the texture features of the predetermined gangue image.
For the surface texture of coal and gangue, the surface texture of coal is coarser than that of gangue, and the texture features can be extracted through a gray level co-occurrence matrix. The gray level co-occurrence matrix comprises various statistics, mainly including energy, entropy, contrast, uniformity, correlation and the like. The texture features of the image can be measured through the various statistics. For example, by calculating the contrast of the images of coal and gangue, which are respectively within different ranges, using the contrast as a statistic, the images of coal and gangue can be distinguished by setting the threshold range of texture.
Method three, performing identification based on pixel variance of the image, and the step S104 may include: performing morphological expansion processing on the segmentation result to obtain an expanded image; the variance of the pixel values of the dilated image is calculated. If the variance is within a first variance range, determining that the segmentation result corresponds to coal; the first variance range is a predetermined variance range of the coal image; if the variance is within a second variance range, determining that the segmentation result corresponds to the gangue; the second variance range is a predetermined variance range of the gangue image.
Before the pixel variance of the image is calculated, morphological operation is required to be carried out on the image, namely, the image is subjected to expansion processing, the boundary points of the image can be expanded outwards by the expansion processing, because the surface of the coal has white bright spots, the bright spots are amplified after the expansion processing is carried out, and then the pixel value variance of the image is calculated, wherein the variance of the coal is generally larger than the variance of the gangue image.
Optionally, the method may further include: pooling the dilated image and calculating the variance of the pixel values of the pooled image. After the image is subjected to the expansion processing, since the image boundary points of the gangue are also expanded, in order to achieve a balance, the image can be subjected to the pooling processing after the image is subjected to the expansion processing, and a smoothing effect, such as maximum pooling, average pooling and the like, can be achieved.
After the image is preprocessed, the variance of the pixel values is calculated, and the coal and the gangue can be distinguished by the variance in different ranges.
The fourth method, performing recognition based on a machine learning method, where the step S104 may include: and classifying the segmentation result through a pre-trained machine learning model, and determining that the segmentation result corresponds to coal or gangue.
The machine learning model is obtained by training a sample set of the coal and gangue images, wherein samples of the sample set comprise the coal images and the gangue images, color features, texture features and pixel variance features of the coal images and the gangue images, and class labels of the coal images and the gangue images.
For the two classification problems of coal and gangue, firstly, the attribute of each class needs to be obtained, namely, feature extraction, the features considered in the embodiment are the color features, the contrast and the variance mentioned in the three methods, then, the three features and the class labels of the corresponding coal and gangue are used for making a training set, a machine learning model is trained to obtain the trained machine learning model, and then, the model is used for classifying the images, so that a better classification effect can be obtained. The machine learning model may be, for example, xgboost or the like.
After the coal and gangue are detected by using the segmentation recognition method and the image feature recognition method, the coal can be grabbed by using a mechanical arm based on the recognition result. When the mechanical arm is used for grabbing, the coal or gangue is subjected to the third detection, optionally, a weighing sensor is arranged on the mechanical arm, and the weight of the grabbed target is detected through the weighing sensor.
Taking the mechanical arm to grab coal as an example, after the mechanical arm grabs a target, the weight m of the target can be obtained through a weighing sensor on the mechanical arm, meanwhile, the volume v of the target can be estimated according to the diameter of the target, and the density ρ of the target is obtained through the mass and the volume.
When the mechanical arm grabs the target, the density of the target is obtained through the weighing sensor and the estimated volume, whether the currently grabbed target is coal or gangue is judged by judging the density, and if the density of the currently grabbed target is within the density range of the coal, the target needs to be grabbed; and if the density of the waste rock is within the density range of the waste rock, the belt conveyor is put back, and the waste rock is not grabbed.
After the coal and the gangue are detected for three times, the coal and the gangue can be effectively classified. After the recognition system recognizes the coal, the coordinates of the coal are sent to the mechanical arm, the mechanical arm grabs the coal, and certainly, when the coal is grabbed, the mechanical arm detects the grabbed target for the third time according to the density, and then the coal which needs to be grabbed finally is obtained.
It should be noted that the robotic arm may also be considered to grasp the gangue, leaving the coal on the belt conveyor for transport to the subsequent coal storage facility. And (3) grabbing coal or gangue, mainly depending on the acceptance degree of one type of error and two types of errors in statistics, selecting to grab coal if clean coal is expected, and selecting to grab gangue if the quality requirement on the coal is not particularly high. Further, the quality of the finally obtained coal can be controlled by adjusting the threshold setting for distinguishing the coal from the gangue in the three detection processes.
Considering that the target of the mechanical arm grabbing is mainly a medium and large target, namely the diameter of coal or gangue is about 100mm to 400mm, the gangue occupies a large proportion in the diameter range, and the gangue can be left on a belt conveyor and conveyed to a gangue storage device in a coal grabbing manner in the mechanical arm grabbing link.
The embodiment provides an intelligent coal gangue sorting system, which is shown in an operation schematic diagram of the intelligent coal gangue sorting system shown in fig. 2, and the intelligent coal gangue sorting system may include the following modules:
1. image segmentation module
The image segmentation module inputs a video of the coal flow, a camera for shooting the coal flow in real time is arranged above the coal flow, and the camera is connected with the image segmentation module and transmits the shot coal flow video to the image segmentation module in real time. The image segmentation module may use an example segmentation algorithm, such as yolcat, Deep snake, etc., to obtain the object class and contour in each frame of image by segmenting and identifying the object in each frame of image in the video. Referring to the image segmentation algorithm output effect map shown in fig. 3, each object in the image is marked by a rectangular box and the class of each object is identified.
When an image is segmented by using an instance segmentation algorithm, an instance segmentation model needs to be trained by using a labeled data set. In this embodiment, a labelme tool may be used for labeling, so as to obtain a labeled picture and a corresponding label file, and obtain a manufactured training set. After the annotation is completed, each picture generates a json (JavaScript object notation)) file corresponding to the picture, that is, a label file of the picture, including the target category of the picture and the coordinates of the annotated boundary point.
2. Image feature detection module
When the image segmentation model segments the images in the video, the targets in the video frame are classified and positioned, segmented pictures, corresponding categories and coordinates can be finally obtained, and then the segmented images of the coal and the segmented images of the gangue are input into the image feature detection module.
The image feature detection module carries out secondary detection on the category of the image feature detection module, so that the rate of misidentification in the previous module can be reduced. In this module, the image feature-based recognition method mainly used includes: identification based on color features, identification based on gray level co-occurrence matrix, identification based on variance, identification based on machine learning method, and the like.
For each target in the coal flow, the category and the coordinate of the target are obtained, the category and the coordinate can be input into the mechanical arm, and the mechanical arm is controlled to grab the coal.
3. Density detection module
After the two modules divide and identify the coal flow video, the category and the coordinates of the target in the video frame can be obtained. After recognizing that the class of a certain target is coal, sending the coordinate information of the target to an idle mechanical arm, and then grabbing the target by the mechanical arm.
When the mechanical arm grabs the target, the weighing sensor on the mechanical arm is used for carrying out third detection on the target. The load cell will obtain the mass of the grabbed object, divide by the volume of the object to obtain the density of the object, and then compare the density of the object to the density range of the coal and gangue. If the target is coal, then grabbing; if the target is mine spoil, the belt conveyor is replaced.
In estimating the volume of the target grasped by the robot arm, the length of the target in the horizontal direction and the vertical direction may be considered, and then the volume of the target may be estimated by the volume formula of a rectangular parallelepiped. For the lengths in the horizontal direction and the vertical direction, the length of the target can be estimated by a monocular distance measurement algorithm, only one camera is needed, and the length of the target can be estimated by the length of a reference object and the principle of projective geometry; the second method is to measure the length of the target in the horizontal and vertical directions by a length measuring sensor. In addition to the above method of estimating the target volume by the rectangular parallelepiped volume, the target volume may be measured by some volume measuring tool, such as a volume measuring instrument, a coal traying instrument, and the like.
4. Split execution module
After three detections, the accurate target class and coordinates are obtained, and the next step is to separate the coal and the gangue. The module can be selected from an ejection device, a high-pressure blowing device and the like, and the separation executing device is a mechanical arm in the embodiment. After receiving the coordinates of the coal, the mechanical arm grabs the coal, grabs the coal into the clean coal bin, and the gangue is left on the belt conveyor and conveyed into the gangue bin.
The embodiment realizes the identification of coal and gangue by combining an image segmentation method, an image feature-based detection method and a density detection method, and then uses a mechanical arm to separate the coal and the gangue, thereby constructing a set of complete intelligent coal and gangue sorting system.
This embodiment also provides a coal gangue identification system based on image recognition technology, includes: the camera device is used for acquiring an image of the mixture of the coal and the gangue; the server is used for executing the coal and gangue identification method based on the image identification technology of the embodiment; the density detection device is used for detecting the density of the coal and/or the gangue; and the separation executing device is used for separating coal and gangue according to the coal and gangue identification result.
Fig. 4 is a schematic structural diagram of a coal gangue identification system based on an image identification technology in an embodiment of the invention. In fig. 4, the image pickup apparatus includes a first camera 401 and a second camera 402; the first camera 401 and the second camera 402 are in communication connection with the server 403 and are both arranged above the belt conveyor 404, wherein one camera is used for acquiring a real-time video of a coal flow conveyed by the belt conveyor so as to enable the server to execute the coal and rock recognition method based on image characteristics, and the other camera is used for acquiring an image of a target grabbed by a mechanical arm so as to enable the server to execute volume estimation; the density detection means includes a load cell 405; the separation actuator includes a robot arm 406, and the load cell 405 is disposed at an upper end of a gripper of the robot arm 406.
Fig. 5 is a schematic structural diagram of a coal gangue identification device based on an image identification technology in an embodiment of the invention, and the device comprises:
the segmentation module 501 is configured to obtain an image of a mixture of coal and gangue, and perform image segmentation on the image to obtain a segmentation result;
a coal and gangue identification module 502, configured to perform coal and gangue identification according to the image features corresponding to the segmentation results; wherein the image features comprise at least one of: color features of the image, texture features of the image, pixel variance features of the image.
The coal and gangue identification device based on the image identification technology provided by the embodiment can perform image segmentation on the image of the mixture of coal and gangue, and then perform coal and gangue identification according to the image characteristics corresponding to the segmentation result, wherein the image characteristics comprise color characteristics, texture characteristics and pixel variance characteristics.
Optionally, as an embodiment, the segmentation module 501 is specifically configured to: performing image segmentation on the image according to a preset segmentation algorithm to obtain a segmentation object; the segmentation object comprises a contour and a coal gangue category; the coal and gangue identification module 502 is specifically configured to: and identifying coal and gangue according to the image characteristics corresponding to the segmentation object so as to verify whether the coal and gangue category of the segmentation object is correct.
Optionally, as an embodiment, the apparatus further includes a density identification module, configured to: acquiring the weight and the volume of a target corresponding to the segmentation result, and calculating the density of the target according to the weight and the volume; and verifying whether the coal and gangue identification result is correct or not according to the density of the target, the density of the coal and the density of the gangue.
Optionally, as an embodiment, the image feature is a color feature of an image, and the coal gangue identification module 502 is specifically configured to: determining a target pixel point of the pixel value in the segmentation result within a preset pixel value range; and determining that the segmentation result corresponds to coal or gangue according to the size relation between the area of the target pixel point and a preset area threshold value.
Optionally, as an embodiment, the image feature is a texture feature of an image, and the coal gangue identification module 502 is specifically configured to: extracting texture features of the segmentation result through a gray level co-occurrence matrix; the texture features include: energy, entropy, contrast, uniformity, or correlation; if the numerical value corresponding to the texture feature is in a first texture range, determining that the segmentation result corresponds to the coal; the first texture range is a numerical range corresponding to the predetermined texture features of the coal image; if the numerical value corresponding to the texture feature is in a second texture range, determining that the segmentation result corresponds to the gangue; the second texture range is a numerical range corresponding to the texture features of the predetermined gangue image.
Optionally, as an embodiment, the image feature is a pixel variance feature of an image, and the coal gangue identification module 502 is specifically configured to: performing morphological dilation processing on the segmentation result to obtain a dilated image; calculating a variance of pixel values of the dilated image; determining that the segmentation result corresponds to coal if the variance is within a first variance range; the first variance range is a predetermined variance range of the coal image; if the variance is within a second variance range, determining that the segmentation result corresponds to gangue; the second variance range is a predetermined variance range of the gangue image.
Optionally, as an embodiment, the gangue identification module 502 is further configured to: and performing pooling processing on the expanded image, and calculating the variance of the pixel values of the pooled image.
Optionally, as an embodiment, the coal gangue identification module 502 is specifically configured to: classifying the segmentation result through a pre-trained machine learning model, and determining that the segmentation result corresponds to coal or gangue; the machine learning model is obtained by training a sample set of coal and gangue images, wherein samples of the sample set comprise a coal image and a gangue image, color features, texture features and pixel variance features of the coal image and the gangue image, and category labels of the coal image and the gangue image.
The coal and gangue identification device based on the image identification technology provided by the embodiment can realize each process in the embodiment of the coal and gangue identification method based on the image identification technology, and is not repeated here for avoiding repetition.
The embodiment of the invention also provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program realizes each process of the coal and gangue identification embodiment based on the image identification technology, can achieve the same technical effect, and is not repeated here to avoid repetition. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
Of course, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments may be implemented by instructing the control device to perform operations through a computer, and the programs may be stored in a computer-readable storage medium, and when executed, the programs may include the processes of the above method embodiments, where the storage medium may be a memory, a magnetic disk, an optical disk, and the like.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A coal and gangue identification method based on an image identification technology is characterized by comprising the following steps:
acquiring an image of a mixture of coal and gangue, and performing image segmentation on the image to obtain a segmentation result;
performing coal and gangue identification according to the image characteristics corresponding to the segmentation result; wherein the image features comprise at least one of: color features of the image, texture features of the image, pixel variance features of the image.
2. The method of claim 1, wherein the image segmentation of the image results in a segmentation result, comprising:
performing image segmentation on the image according to a preset segmentation algorithm to obtain a segmentation object; the segmentation object comprises a contour and a coal gangue category;
the coal and gangue identification according to the image characteristics corresponding to the segmentation result comprises the following steps:
and identifying coal and gangue according to the image characteristics corresponding to the segmentation object so as to verify whether the coal and gangue category of the segmentation object is correct.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
acquiring the weight and the volume of a target corresponding to the segmentation result, and calculating the density of the target according to the weight and the volume;
and verifying whether the coal and gangue identification result is correct or not according to the density of the target, the density of the coal and the density of the gangue.
4. The method according to claim 1, wherein the image features are color features of an image, and performing coal gangue identification according to the image features corresponding to the segmentation result comprises:
determining a target pixel point of the pixel value in the segmentation result within a preset pixel value range;
and determining that the segmentation result corresponds to coal or gangue according to the size relation between the area of the target pixel point and a preset area threshold value.
5. The method according to claim 1, wherein the image features are texture features of an image, and performing coal gangue identification according to the image features corresponding to the segmentation result comprises:
extracting texture features of the segmentation result through a gray level co-occurrence matrix;
if the numerical value corresponding to the texture feature is in a first texture range, determining that the segmentation result corresponds to the coal; the first texture range is a numerical range corresponding to the predetermined texture features of the coal image;
if the numerical value corresponding to the texture feature is in a second texture range, determining that the segmentation result corresponds to the gangue; the second texture range is a numerical range corresponding to the texture features of the predetermined gangue image.
6. The method according to claim 1, wherein the image feature is a pixel variance feature of an image, and performing coal gangue identification according to the image feature corresponding to the segmentation result comprises:
performing morphological dilation processing on the segmentation result to obtain a dilated image;
calculating a variance of pixel values of the dilated image;
determining that the segmentation result corresponds to coal if the variance is within a first variance range; the first variance range is a predetermined variance range of the coal image;
if the variance is within a second variance range, determining that the segmentation result corresponds to gangue; the second variance range is a predetermined variance range of the gangue image.
7. The method of claim 6, wherein after the step of performing morphological dilation on the segmentation result to obtain a dilated image, the method further comprises:
and performing pooling processing on the expanded image, and calculating the variance of the pixel values of the pooled image.
8. The method according to claim 1, wherein the performing coal gangue identification according to the image features corresponding to the segmentation result comprises:
classifying the segmentation result through a pre-trained machine learning model, and determining that the segmentation result corresponds to coal or gangue;
the machine learning model is obtained by training a sample set of coal and gangue images, wherein samples of the sample set comprise a coal image and a gangue image, color features, texture features and pixel variance features of the coal image and the gangue image, and category labels of the coal image and the gangue image.
9. A coal and gangue identification device based on an image identification technology is characterized by comprising:
the segmentation module is used for acquiring an image of a mixture of coal and gangue and performing image segmentation on the image to obtain a segmentation result;
the coal and gangue identification module is used for identifying coal and gangue according to the image characteristics corresponding to the segmentation result; wherein the image features comprise at least one of: color features of the image, texture features of the image, pixel variance features of the image.
10. The utility model provides a coal gangue identification system based on image recognition technique which characterized in that includes:
the camera device is used for acquiring an image of the mixture of the coal and the gangue;
a server for executing the coal gangue identification method based on the image identification technology according to any one of claims 1 to 8;
the density detection device is used for detecting the density of the coal and/or the gangue;
and the separation execution device is used for separating the coal and the gangue according to the coal and gangue identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011126884.7A CN112330607A (en) | 2020-10-20 | 2020-10-20 | Coal and gangue identification method, device and system based on image identification technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011126884.7A CN112330607A (en) | 2020-10-20 | 2020-10-20 | Coal and gangue identification method, device and system based on image identification technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112330607A true CN112330607A (en) | 2021-02-05 |
Family
ID=74312088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011126884.7A Pending CN112330607A (en) | 2020-10-20 | 2020-10-20 | Coal and gangue identification method, device and system based on image identification technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330607A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283391A (en) * | 2021-06-28 | 2021-08-20 | 西安科技大学 | Method for identifying coal and gangue under complex working conditions in fully mechanized caving mining caving process |
| CN114066861A (en) * | 2021-11-22 | 2022-02-18 | 安徽理工大学 | Coal and gangue identification method based on cross algorithm edge detection theory and visual features |
| CN114535063A (en) * | 2022-01-28 | 2022-05-27 | 太原理工大学 | Underground coal and gangue sorting integrated device based on artificial intelligence image recognition |
| CN115069597A (en) * | 2022-06-30 | 2022-09-20 | 洪平 | Coal block and gangue distinguishing method, system, computer and readable storage medium |
| CN117011302A (en) * | 2023-10-08 | 2023-11-07 | 山东济宁运河煤矿有限责任公司 | Intelligent dry separation system based on coal gangue identification |
| CN114066861B (en) * | 2021-11-22 | 2024-04-19 | 安徽理工大学 | Coal gangue identification method based on intersection algorithm edge detection theory and visual characteristics |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103752533A (en) * | 2014-01-23 | 2014-04-30 | 河北联合大学 | Method and device for sorting coal and gangue online through image method |
| US20140219557A1 (en) * | 2013-02-04 | 2014-08-07 | Wistron Corporation | Image identification method, electronic device, and computer program product |
| CN107145884A (en) * | 2017-04-26 | 2017-09-08 | 太原理工大学 | Gangue near-infrared image identification technology |
| CN109655466A (en) * | 2019-01-08 | 2019-04-19 | 中国矿业大学 | A kind of spoil coal carrying rate online test method and device based on machine vision |
| CN110441320A (en) * | 2019-08-05 | 2019-11-12 | 北京泰豪信息科技有限公司 | A kind of gangue detection method, apparatus and system |
| WO2019242329A1 (en) * | 2018-06-20 | 2019-12-26 | 北京七鑫易维信息技术有限公司 | Convolutional neural network training method and device |
| CN111709935A (en) * | 2020-06-17 | 2020-09-25 | 西安科技大学 | Real-time coal gangue positioning and identifying method for ground moving belt |
-
2020
- 2020-10-20 CN CN202011126884.7A patent/CN112330607A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140219557A1 (en) * | 2013-02-04 | 2014-08-07 | Wistron Corporation | Image identification method, electronic device, and computer program product |
| CN103752533A (en) * | 2014-01-23 | 2014-04-30 | 河北联合大学 | Method and device for sorting coal and gangue online through image method |
| CN107145884A (en) * | 2017-04-26 | 2017-09-08 | 太原理工大学 | Gangue near-infrared image identification technology |
| WO2019242329A1 (en) * | 2018-06-20 | 2019-12-26 | 北京七鑫易维信息技术有限公司 | Convolutional neural network training method and device |
| CN109655466A (en) * | 2019-01-08 | 2019-04-19 | 中国矿业大学 | A kind of spoil coal carrying rate online test method and device based on machine vision |
| CN110441320A (en) * | 2019-08-05 | 2019-11-12 | 北京泰豪信息科技有限公司 | A kind of gangue detection method, apparatus and system |
| CN111709935A (en) * | 2020-06-17 | 2020-09-25 | 西安科技大学 | Real-time coal gangue positioning and identifying method for ground moving belt |
Non-Patent Citations (5)
| Title |
|---|
| DEBI PRASAD TRIPATHY 等: "Novel Methods for Separation of Gangue from Limestone and Coal using Multispectral and Joint Color-Texture Features", JOURNAL OF THE INSTITUTION OF ENGINEERS (INDIA): SERIES D, 26 February 2016 (2016-02-26) * |
| 吴开兴;宋剑;: "基于灰度和纹理的煤与矸石特征提取研究", 煤炭技术, no. 11, 10 November 2015 (2015-11-10) * |
| 杨慧刚;乔志敏;高绘彦;刘宇;赵一丁;: "煤与矸石分选系统设计", 工矿自动化, no. 08, 20 July 2018 (2018-07-20) * |
| 谭春超: "基于图像处理技术的煤矸识别与分选技术研究", 中国优秀硕士学位论文全文数据库 基础科学辑, no. 01, 15 January 2018 (2018-01-15), pages 021 - 394 * |
| 霍平;曾翰林;霍柯言;: "基于图像处理的煤/矸密度识别系统的研究", 选煤技术, no. 02, 25 April 2015 (2015-04-25) * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283391A (en) * | 2021-06-28 | 2021-08-20 | 西安科技大学 | Method for identifying coal and gangue under complex working conditions in fully mechanized caving mining caving process |
| CN114066861A (en) * | 2021-11-22 | 2022-02-18 | 安徽理工大学 | Coal and gangue identification method based on cross algorithm edge detection theory and visual features |
| CN114066861B (en) * | 2021-11-22 | 2024-04-19 | 安徽理工大学 | Coal gangue identification method based on intersection algorithm edge detection theory and visual characteristics |
| CN114535063A (en) * | 2022-01-28 | 2022-05-27 | 太原理工大学 | Underground coal and gangue sorting integrated device based on artificial intelligence image recognition |
| CN115069597A (en) * | 2022-06-30 | 2022-09-20 | 洪平 | Coal block and gangue distinguishing method, system, computer and readable storage medium |
| CN117011302A (en) * | 2023-10-08 | 2023-11-07 | 山东济宁运河煤矿有限责任公司 | Intelligent dry separation system based on coal gangue identification |
| CN117011302B (en) * | 2023-10-08 | 2024-01-09 | 山东济宁运河煤矿有限责任公司 | Intelligent dry separation system based on coal gangue identification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112330607A (en) | Coal and gangue identification method, device and system based on image identification technology | |
| CN110390691B (en) | Ore dimension measuring method based on deep learning and application system | |
| CN109359666B (en) | Vehicle type recognition method based on multi-feature fusion neural network and processing terminal | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN107782733B (en) | Image recognition nondestructive detection device and method for metal surface defects | |
| CN111242108B (en) | Belt transfer point coal blockage identification method based on target detection | |
| CN107389701A (en) | A kind of PCB visual defects automatic checkout system and method based on image | |
| Jiang et al. | A machine vision-based realtime anomaly detection method for industrial products using deep learning | |
| CN105044122A (en) | Copper part surface defect visual inspection system and inspection method based on semi-supervised learning model | |
| CN103279765A (en) | Steel wire rope surface damage detection method based on image matching | |
| CN104198497A (en) | Surface defect detection method based on visual saliency map and support vector machine | |
| Ahmad et al. | Overhead view person detection using YOLO | |
| CN112001878A (en) | Deep learning ore scale measuring method based on binarization neural network and application system | |
| CN112893159B (en) | Coal gangue sorting method based on image recognition | |
| CN110942450A (en) | Multi-production-line real-time defect detection method based on deep learning | |
| CN111242899A (en) | Image-based flaw detection method and computer-readable storage medium | |
| CN112295949A (en) | Visual intelligent sorting method and system based on deep neural network | |
| CN112329587A (en) | Beverage bottle classification method and device and electronic equipment | |
| CN110910363A (en) | Insufficient solder joint detection method, system and medium based on machine vision and deep learning | |
| CN102708367A (en) | Image identification method based on target contour features | |
| CN114092478B (en) | Anomaly detection method | |
| CN111461010A (en) | Power equipment identification efficiency optimization method based on template tracking | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| KR102391501B1 (en) | Classification System and method for atypical recycled goods using Deep learning | |
| EP1290625B1 (en) | Finding objects in an image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |