CN112893159A - Coal gangue sorting method based on image recognition - Google Patents
Coal gangue sorting method based on image recognition Download PDFInfo
- Publication number
- CN112893159A CN112893159A CN202110048060.0A CN202110048060A CN112893159A CN 112893159 A CN112893159 A CN 112893159A CN 202110048060 A CN202110048060 A CN 202110048060A CN 112893159 A CN112893159 A CN 112893159A
- Authority
- CN
- China
- Prior art keywords
- image
- coal
- coal gangue
- deformable
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
- B07C5/362—Separating or distributor mechanisms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/38—Collecting or arranging articles in groups
Abstract
The invention discloses a coal gangue sorting method based on image recognition, which comprises the following steps: firstly, constructing a coal gangue sorting platform; secondly, constructing a double-path deformable CNN model; thirdly, training a two-way deformable CNN model; fourthly, selecting the coal gangue frames; fifthly, conveying the coal blocks; sixthly, sorting the coal gangue. According to the coal gangue differential sorting machine, the coal block images and the differential images are processed simultaneously by utilizing the two-way deformable CNN model, the coal gangue is identified, each convolution layer adopts a deformable convolution layer, the self-adaptive change can be generated according to the content of the image, the shape and the size of different coal gangue can be adapted, the characteristics of different scales can be extracted, the coal gangue is grabbed from the short shaft position of the coal gangue, the mechanical arm grabbing stability is ensured, the labor is saved, the safety is ensured, the coal gangue on the main belt conveyor can be sorted underground under the condition that no worker participates, the working strength of workers is reduced, the manpower resource is saved, the production efficiency is ensured, and the automation degree is high.
Description
Technical Field
The invention belongs to the technical field of coal gangue sorting, and particularly relates to a coal gangue sorting method based on image recognition.
Background
For the power coal, the mixed gangue can reduce the calorific value of the coal and greatly reduce the quality of the power coal, so that the separation of the coal and the gangue is an indispensable one-step process in the coal washing and selecting process; at present, in addition to the existing manual coal separation technology, methods such as overlapping medium coal separation, dry coal separation, jigging coal separation, floating coal separation, electronic instrument detection and the like are available in China. Although there are several methods for coal dressing, each method has drawbacks. In the traditional coal and coal gangue separation method, a crushing method, a jigging coal separation method, a dense medium coal separation method and the like are adopted, the methods are complex in process and high in production cost, the existing production requirements and environmental protection requirements cannot be met, and various dry separation technologies such as a winnowing method, a magnetic separation method and the like are developed aiming at the problem. In recent years, the research on sorting coal gangue by using X-ray has not been applied in large scale although the research has been successful. Most dry separation methods meet the requirements of water resource protection and environmental friendliness, but in terms of separation accuracy, the separation accuracy is low due to the influence of equipment factors, raw coal moisture and the like, and the actual application effect is not ideal. With the continuous improvement of the application of the information technology in the coal gangue separation field, the artificial intelligence and machine vision technology is introduced into the coal gangue separation according to the difference of the coal and the gangue in the gray value, the texture and other physical properties.
Disclosure of Invention
The invention aims to solve the technical problem that the defects in the prior art are overcome, and provides a coal gangue sorting method based on image recognition, wherein a double-path deformable CNN model is used for simultaneously processing a coal block image and a difference image to recognize coal gangue, each convolution layer adopts a deformable convolution layer, can generate self-adaptive change according to image content, adapts to the shapes and sizes of different coal gangue, can extract features of different scales, and can grasp the coal gangue from a short axis position of the coal gangue, so that the mechanical arm is stable in grasping, labor-saving and safe, coal gangue on a main belt conveyor can be sorted underground without the participation of workers, the working strength of workers is reduced, the manpower resources are saved, the production efficiency is ensured, the automation degree is high, and the method is convenient to popularize and use.
In order to solve the technical problems, the invention adopts the technical scheme that: a coal gangue sorting method based on image recognition is characterized by comprising the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside the coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor and a coal gangue conveyor matched with the main belt conveyor, the coal gangue sorting platform comprises a vibrating feeder arranged at a material conveying end of the main belt conveyor, a coal gangue conveyor arranged beside the main belt conveyor and a manipulator arranged at the tail end of the main belt conveyor and used for grabbing coal gangue on the main belt conveyor to the coal gangue conveyor, a limiting portal frame used for arranging and sequencing the coal gangue and the coal gangue is arranged on a frame of the main belt conveyor and positioned at a discharge end of the vibrating feeder, and an image acquisition mechanism used for identifying the coal gangue and communicating with an upper computer is arranged at the tail end of the frame of the main belt conveyor;
step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first active layer I, the first active layer II, the first active layer III, the second active layer I, the second active layer II and the second active layer III are ReLU activation functions;
step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, taking a training sample image in the image training data set, carrying out gray level processing on the image, sending the training sample image after the gray level processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image after the gray level processing and a standard coal block gray level image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, selecting the coal gangue frames: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, carries out gray level processing on the actual coal conveying images, sends the actual coal conveying images subjected to gray level processing into a trained two-way deformable CNN model, wherein one way of the actual coal conveying images subjected to gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images is subjected to differential processing with a standard coal block gray level image, sends the differential images into a second deformable CNN channel after binarization processing, identifies the positions of coal gangue corresponding pixels on an output image after passing through a first main full connecting layer, a second main full connecting layer and an output layer, and executes a fifth step when the two-way deformable CNN model does not identify the positions of coal gangue corresponding pixels on the output image; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
step five, conveying the coal briquettes: the manipulator does not work, the coal blocks fall onto the coal block conveyor from the main belt conveyor, and the coal block conveyor is used for conveying the coal blocks;
step six, coal gangue sorting: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binarization processing on the coal gangue region, namely, the white part is the coal gangue region, the black part is the background region, thereby obtaining the binarization image of the coal gangue, and obtaining the binarization image of the coal gangue according to a formula
Calculating barycentric coordinates (x, y) of the binarized coal gangue image, wherein m00 is 0-order moment of the binarized coal gangue image, m10 is first moment of the binarized coal gangue image region in the x direction, and m01 is first moment of the binarized coal gangue image region in the y direction;
the method comprises the steps of obtaining a minor axis of a coal gangue region in a binarization image of the coal gangue by taking barycentric coordinates as a reference, wherein the position of the minor axis in the coal gangue region provides a grabbing reference position for sorting the coal gangue by a manipulator;
the manipulator grabs the coal gangue to the coal gangue conveyor from the short shaft position of the coal gangue, and the coal gangue conveyor is utilized to separate the sorted coal gangue from the coal blocks, so that the sorting of the coal gangue is realized.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: in the fourth step, the practical coal conveying image after the gray level processing is sent into a trained double-path deformable CNN model, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the formula is firstly adopted
Performing illumination compensation processing to obtain a compensated coal transportation actual image O (x, y), wherein F (x, y) is the coal transportation actual image after gray level processing, gamma is an index value for brightness enhancement
I (x, y) is the illumination component, and m is the mean value of the luminance of the illumination component.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: the image acquisition mechanism comprises a camera and a controller connected with the output end of the camera, and the controller is connected with an upper computer through a communication module.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: the number of the images in the image training data set is 8-9 times of the number of the images in the image testing data set.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: in step 303, when the image is subjected to gray scale processing, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path of the practical coal conveying image and a standard coal block gray level image, the illumination compensation processing is carried out, the irrelevant pixel elimination processing is carried out, and finally the difference processing is carried out on the other path of the practical coal conveying image and the standard coal block gray level image.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the illumination compensation processing is carried out, and then the irrelevant pixel elimination processing is carried out by a Lucas-Kanade optical flow method.
Compared with the prior art, the invention has the following advantages:
1. according to the coal gangue sorting device, the coal gangue sorting platform is constructed, the coal blocks and the coal gangue are queued on the main belt conveyor by the limiting door frame, so that the coal flow sequentially passes through the camera for image recognition, the coal gangue is sorted out after the image recognition, the coal gangue is sorted onto the coal gangue conveyor by the manipulator, the operation effect is good, the coal blocks can automatically fall onto the coal gangue conveyor along with the operation of the main belt conveyor, and the coal gangue sorting device is convenient to popularize and use.
2. According to the invention, the input image is subjected to feature extraction through two mutually independent channels, each convolution layer of the two-channel adopts a deformation convolution layer, the position of a convolution kernel sampling point can be subjected to self-adaptive change according to the content of the image, so that the method is suitable for geometric deformation such as the shape, the size and the like of different coal gangue, features of different scales in the input image can be extracted, then feature fusion is carried out on a full connection layer, finally, the coal gangue is identified through output by an output layer, the problems that the extraction features of a single convolution kernel of a single-channel convolution neural network are insufficient and the accuracy of an identification result is deficient are solved, and the use effect is good.
3. The method of the invention has simple steps, the gray level image of the second deformable CNN channel of the input differential image data path is subjected to illumination compensation processing, the illumination difference caused in the process of acquiring the image data by the monitoring video is removed, if the illumination compensation processing is not carried out, the illumination difference can be used as interference, the input differential image data path is reliable and stable, the use effect is good, the pixel points of irrelevant foreground objects such as moving pedestrians and the like collected in the process of collecting the video are eliminated, the irrelevant foreground is used as interference data to influence the performance of the model, the pixel area of the motility irrelevant target is determined, the directionality removal of the motility irrelevant interference in the corresponding image data is realized, the coal block image and the differential image are simultaneously processed by utilizing the two-way deformable CNN model, the coal gangue is identified, each convolution layer adopts a deformable convolution layer, and can generate self-adaptive change according to the content of, adapt to the shape of different coal gangues, size, can extract the characteristic of different yards, snatch the coal gangue through the minor axis position from the coal gangue, guarantee that the manipulator snatchs stably, laborsaving safety makes can be in the pit under the condition that does not have staff's participation, to the sorting of main belt conveyor coal gangue, has not only reduced staff's working strength, has saved manpower resources, has ensured production efficiency moreover, degree of automation is high, convenient to popularize and use.
In conclusion, the coal gangue is identified by simultaneously processing the coal block image and the difference image by utilizing the two-way deformable CNN model, each convolution layer adopts a deformable convolution layer, can generate self-adaptive change according to the content of the image, adapts to the shape and the size of different coal gangue, can extract the characteristics of different scales, and ensures stable grabbing by a manipulator and labor-saving safety by grabbing the coal gangue from the short shaft position of the coal gangue, so that the coal gangue on a main belt conveyor can be sorted underground without the participation of workers, the working strength of the workers is reduced, the manpower resource is saved, the production efficiency is ensured, the automation degree is high, and the coal gangue sorting machine is convenient to popularize and use.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
Fig. 1 is a schematic block circuit diagram of a data transmission apparatus employed in the present invention.
Fig. 2 is a flow chart of a method of data transmission according to the present invention.
Description of reference numerals:
1-main belt conveyor; 2-vibrating feeder; 3, limiting the door frame;
4-coal briquette; 5-coal gangue; 6, a camera;
7-a controller; 8, an upper computer; 9-a manipulator;
10-a coal conveyor; 11-coal gangue conveyor.
Detailed Description
As shown in fig. 1 and 2, the coal gangue sorting method based on image recognition of the invention includes the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside a coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor 1 and a coal gangue conveyor 10 matched with the main belt conveyor 1, the coal gangue sorting platform comprises a vibrating feeder 2 arranged at the material conveying end of the main belt conveyor 1, a coal gangue conveyor 11 arranged beside the main belt conveyor 1 and a manipulator 9 arranged at the tail end of the main belt conveyor 1 and used for grabbing coal gangue 5 on the main belt conveyor 1 to the coal gangue conveyor 11, a limiting door frame 3 used for arranging and sequencing coal gangue 4 and coal gangue 5 is arranged on the frame of the main belt conveyor 1 and positioned at the discharge end of the vibrating feeder 2, and an image acquisition mechanism used for identifying the coal gangue and communicating with an upper computer 8 is arranged at the tail end of the frame of the main belt conveyor 1;
in this embodiment, the image acquisition mechanism includes camera 6 and with controller 7 that camera 6 output is connected, controller 7 passes through communication module and is connected with host computer 8.
It should be noted that, through constructing the coal gangue sorting platform, the coal gangue and the coal gangue are queued on the main belt conveyor by using the limiting portal frame, so that the coal flow sequentially passes through the camera for image recognition, the coal gangue is sorted out after the image recognition, the coal gangue is sorted to the coal gangue conveyor by using the manipulator, the operation effect is good, and the coal briquette can automatically fall to the coal gangue conveyor along with the operation of the main belt conveyor.
Step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first active layer I, the first active layer II, the first active layer III, the second active layer I, the second active layer II and the second active layer III are ReLU activation functions;
it should be noted that the input image is subjected to feature extraction through two mutually independent channels, each convolution layer of the two-channel adopts a deformation convolution layer, the position of a convolution kernel sampling point can be subjected to self-adaptive change according to the content of the image, so that the method is suitable for geometric deformation such as shapes and sizes of different foreign matters, features of different scales in the input image can be extracted, feature fusion is carried out on a full connection layer, and finally, the output layer is used for outputting, the foreign matters are identified, and the problems that the single convolution kernel of a single-channel convolution neural network is insufficient in feature extraction and is deficient in the accuracy of an identification result are solved.
Step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor 1 with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
in this embodiment, the number of images in the image training data set is 8 to 9 times of the number of images in the image test data set.
Step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, taking a training sample image in the image training data set, carrying out gray level processing on the image, sending the training sample image after the gray level processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image after the gray level processing and a standard coal block gray level image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
in this embodiment, in step 303, when performing gray scale processing on the image, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
Step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, selecting the coal gangue frames: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, carries out gray level processing on the actual coal conveying images, sends the actual coal conveying images subjected to gray level processing into a trained two-way deformable CNN model, wherein one way of the actual coal conveying images subjected to gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images is subjected to differential processing with a standard coal block gray level image, sends the differential images into a second deformable CNN channel after binarization processing, identifies the positions of coal gangue corresponding pixels on an output image after passing through a first main full connecting layer, a second main full connecting layer and an output layer, and executes a fifth step when the two-way deformable CNN model does not identify the positions of coal gangue corresponding pixels on the output image; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
in the fourth step, the practical coal transportation image after the gray processing is sent to a trained two-way deformable CNN model, wherein before the difference processing is performed on the other way and a standard coal block gray image, the illumination compensation processing is performed, the irrelevant pixel removal processing is performed, and finally the difference processing is performed on the other way and a standard coal block gray image.
In the fourth step, the practical coal transportation image after the gray processing is sent to a trained two-way deformable CNN model, wherein before the difference processing is performed on the other way and a standard coal block gray image, illumination compensation processing is performed, and then irrelevant pixel removal processing is performed by a Lucas-Kanade optical flow method.
In this embodiment, in the fourth step, the practical coal transportation image after the gray level processing is sent to the trained two-way deformable CNN model, where before the difference processing is performed on the other way and a standard coal block gray level image, the other way and the standard coal block gray level image are firstly processed according to a formula
Performing illumination compensation processing to obtain a compensated coal transportation actual image O (x, y), wherein F (x, y) is the coal transportation actual image after gray level processing, gamma is an index value for brightness enhancement
I (x, y) is the illumination component, and m is the mean value of the luminance of the illumination component.
Step five, conveying the coal briquettes: the manipulator 9 does not work, the coal briquette 4 falls from the main belt conveyor 1 to the coal briquette conveyor 10, and the coal briquette conveyor 10 is utilized to transport the coal briquette 4;
step six, coal gangue sorting: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binarization processing on the coal gangue region, namely, the white part is the coal gangue region, the black part is the background region, thereby obtaining the binarization image of the coal gangue, and obtaining the binarization image of the coal gangue according to a formula
Calculating barycentric coordinates (x, y) of the binarized coal gangue image, wherein m00 is 0-order moment of the binarized coal gangue image, m10 is first moment of the binarized coal gangue image region in the x direction, and m01 is first moment of the binarized coal gangue image region in the y direction;
the gravity center coordinates are used as a reference, the minor axis of the coal gangue region in the binarization image of the coal gangue is obtained, and the position of the minor axis in the coal gangue region provides a grabbing reference position for sorting the coal gangue by a manipulator 9;
the manipulator 9 grabs the coal gangue 5 from the short axis position of the coal gangue to the coal gangue conveyor 11, and the coal gangue conveyor 11 is utilized to separate the sorted coal gangue 5 from the coal blocks 4, so that the sorting of the coal gangue 5 is realized.
When the invention is used, taking an RGB original image monitored by a 480 × 640 pixel camera as an example, a first deformable CNN channel takes RGB original image monitored by the 480 × 640 pixel camera compressed into 48 × 64 pixel RGB image data as input, a second deformable CNN channel firstly carries out weight average graying under computer vision understanding on the compressed 48 × 64 pixel RGB image, then carries out difference processing, illumination compensation processing and irrelevant foreground elimination processing on the image, and finally obtains a processing result as the input of a difference image channel. The specific two-way convolution process is as follows: the 48 × 64 pixel images input by the first deformable CNN channel and the second deformable CNN channel are filtered by 64 filters of 5 × 5 size to obtain 64 feature maps of 48 × 64 size, which is a single deformable convolution layer. Then, pooling operations of 3 × 3 sizes were performed on the 64 feature maps, respectively, to obtain pooled layer feature maps of 24 × 32 sizes. Convolution operation is also carried out between the first pooling layer and the second warped convolutional layer, and 64 filters with the size of 5 × 5 are shared, so that the second warped convolutional layer has 64 feature maps with the size of 24 × 32. The second pooling layer has 64 12 × 16 pixel feature maps, which are obtained by pooling in the upper layer. The warped convolutional layer three is still convolution operation, the filter size of the warped convolutional layer three is still 5 × 5, and 64 filters are in total, so that the feature map of the pooling layer two is subjected to convolution operation to obtain 64 12 × 16 feature maps. The pooling operation of 3 × 3 sizes was performed on each of the 64 feature maps, and a pooling layer three feature map of 6 × 8 size was obtained. The first full connection layer has 100 neurons, the second full connection layer has 40 neurons, and the output layer outputs the result of network classification. Compared with a fully-connected neural network, the convolutional neural network can obtain a characteristic diagram through a filter, so that weight sharing is realized, parameters of the network are greatly reduced, and the training efficiency of the network is improved. The convolution operation may make the input have translational or rotational invariance. The pooling operation can improve the generalization capability of the network model and increase the receptive field of the network while preserving the main features. The fully-connected layer acts as a "classifier" in the overall convolutional neural network. If we say that operations such as convolutional layers, pooling layers, and activation function layers map raw data to hidden layer feature space, the fully-connected layer serves to map the learned "distributed feature representation" to the sample label space. This embodiment utilizes double-circuit flexible CNN model to handle coal cinder image and difference image simultaneously, the discernment gangue, every convolution layer adopts the deformation convolution layer, can take place self-adaptation's change according to image content, adapt to the shape of different gangue, the size, can extract the characteristic of different yardstick, snatch the gangue through the minor axis position from the gangue, guarantee that the manipulator snatchs stably, laborsaving safety, make can be under the condition that does not have staff's participation in the pit, coal gangue letter sorting on the main belt conveyor, staff's working strength has not only been reduced, manpower resources have been saved, and ensure production efficiency, the degree of automation is high.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and all simple modifications, changes and equivalent structural changes made to the above embodiment according to the technical spirit of the present invention still fall within the protection scope of the technical solution of the present invention.
Claims (7)
1. A coal gangue sorting method based on image recognition is characterized by comprising the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside the coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor (1) and a coal block conveyor (10) matched with the main belt conveyor (1), the coal gangue sorting platform comprises a vibrating feeder (2) arranged at the material conveying end of the main belt conveyor (1), a coal gangue conveyor (11) arranged beside the main belt conveyor (1), and a manipulator (9) arranged at the tail end of the main belt conveyor (1) and used for grabbing the coal gangue (5) on the main belt conveyor (1) to the coal gangue conveyor (11), a limiting portal frame (3) for arranging and sequencing the coal briquettes (4) and the coal gangue (5) is arranged on the frame of the main belt conveyor (1) and at the discharge end of the vibrating feeder (2), the tail end of the frame of the main belt conveyor (1) is provided with an image acquisition mechanism which is used for identifying coal gangue and is communicated with an upper computer (8);
step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first active layer I, the first active layer II, the first active layer III, the second active layer I, the second active layer II and the second active layer III are ReLU activation functions;
step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor (1) with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, taking a training sample image in the image training data set, carrying out gray level processing on the image, sending the training sample image after the gray level processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image after the gray level processing and a standard coal block gray level image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, selecting the coal gangue frames: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, carries out gray level processing on the actual coal conveying images, sends the actual coal conveying images subjected to gray level processing into a trained two-way deformable CNN model, wherein one way of the actual coal conveying images subjected to gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images is subjected to differential processing with a standard coal block gray level image, sends the differential images into a second deformable CNN channel after binarization processing, identifies the positions of coal gangue corresponding pixels on an output image after passing through a first main full connecting layer, a second main full connecting layer and an output layer, and executes a fifth step when the two-way deformable CNN model does not identify the positions of coal gangue corresponding pixels on the output image; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
step five, conveying the coal briquettes: the manipulator (9) does not work, the coal briquette (4) falls onto the coal briquette conveyor (10) from the main belt conveyor (1), and the coal briquette conveyor (10) is utilized to transport the coal briquette (4);
step six, coal gangue sorting: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binarization processing on the coal gangue region, namely, the white part is the coal gangue region, the black part is the background region, thereby obtaining the binarization image of the coal gangue, and obtaining the binarization image of the coal gangue according to a formula
Calculating barycentric coordinates (x, y) of the binarized coal gangue image, wherein m00 is 0-order moment of the binarized coal gangue image, m10 is first moment of the binarized coal gangue image region in the x direction, and m01 is first moment of the binarized coal gangue image region in the y direction;
the gravity center coordinates are used as a reference, the minor axis of a coal gangue region in the binarization image of the coal gangue is obtained, and the position of the minor axis in the coal gangue region provides a grabbing reference position for a manipulator (9) to sort the coal gangue;
the manipulator (9) grabs the coal gangue (5) from the short axis position of the coal gangue to the coal gangue conveyor (11), and the coal gangue conveyor (11) is utilized to separate the sorted coal gangue (5) from the coal blocks (4), so that the sorting of the coal gangue (5) is realized.
2. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: in the fourth step, the practical coal conveying image after the gray level processing is sent into a trained double-path deformable CNN model, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the formula is firstly adopted
Performing illumination compensation processing to obtain a compensated coal transportation actual image O (x, y), wherein F (x, y) is the coal transportation actual image after gray level processing, gamma is an index value for brightness enhancement
I (x, y) is the illumination component, and m is the mean value of the luminance of the illumination component.
3. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: the image acquisition mechanism comprises a camera (6) and a controller (7) connected with the output end of the camera (6), and the controller (7) is connected with an upper computer (8) through a communication module.
4. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: the number of the images in the image training data set is 8-9 times of the number of the images in the image testing data set.
5. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: in step 303, when the image is subjected to gray scale processing, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
6. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path of the practical coal conveying image and a standard coal block gray level image, the illumination compensation processing is carried out, the irrelevant pixel elimination processing is carried out, and finally the difference processing is carried out on the other path of the practical coal conveying image and the standard coal block gray level image.
7. The coal gangue sorting method based on image recognition as claimed in claim 6, wherein: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the illumination compensation processing is carried out, and then the irrelevant pixel elimination processing is carried out by a Lucas-Kanade optical flow method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110048060.0A CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110048060.0A CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112893159A true CN112893159A (en) | 2021-06-04 |
| CN112893159B CN112893159B (en) | 2023-01-06 |
Family
ID=76114217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110048060.0A Active CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112893159B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283391A (en) * | 2021-06-28 | 2021-08-20 | 西安科技大学 | Method for identifying coal and gangue under complex working conditions in fully mechanized caving mining caving process |
| CN114985289A (en) * | 2022-06-02 | 2022-09-02 | 北京科技大学 | Device and method for evaluating separation effect of coal gangue pre-selection system based on intelligent ore |
| CN115138591A (en) * | 2022-09-01 | 2022-10-04 | 山西戴德测控技术有限公司 | X-ray-based online detection device for foreign matters on conveyor belt |
| CN116152344A (en) * | 2023-04-18 | 2023-05-23 | 天津德通电气有限公司 | Coal dressing method and system based on shape database identification |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013060095A1 (en) * | 2011-10-27 | 2013-05-02 | 山东博润工业技术股份有限公司 | Process and system for dry coal preparation by coal gangue removal |
| CN106269576A (en) * | 2016-09-12 | 2017-01-04 | 中国矿业大学 | A kind of Automatic Selection System of Waste Rock from Coal Bulk and method |
| CN107292319A (en) * | 2017-08-04 | 2017-10-24 | 广东工业大学 | The method and device that a kind of characteristic image based on deformable convolutional layer is extracted |
| CN107392925A (en) * | 2017-08-01 | 2017-11-24 | 西安电子科技大学 | Remote sensing image terrain classification method based on super-pixel coding and convolutional neural networks |
| CN108564108A (en) * | 2018-03-21 | 2018-09-21 | 天津市协力自动化工程有限公司 | The recognition methods of coal and device |
| CN110135468A (en) * | 2019-04-24 | 2019-08-16 | 中国矿业大学(北京) | A kind of recognition methods of gangue |
| CN110321908A (en) * | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN110569782A (en) * | 2019-09-05 | 2019-12-13 | 辽宁科技大学 | Target detection method based on deep learning |
| CN110766053A (en) * | 2019-09-23 | 2020-02-07 | 中国农业大学 | Corn seed production ear screening method and device based on double-path convolutional neural network |
| CN111079845A (en) * | 2019-12-20 | 2020-04-28 | 西安交通大学 | Classification and identification method for coal gangue and coal in coal |
| CN111318461A (en) * | 2020-03-11 | 2020-06-23 | 河南理工大学 | Gangue letter sorting system based on machine vision |
-
2021
- 2021-01-14 CN CN202110048060.0A patent/CN112893159B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013060095A1 (en) * | 2011-10-27 | 2013-05-02 | 山东博润工业技术股份有限公司 | Process and system for dry coal preparation by coal gangue removal |
| CN106269576A (en) * | 2016-09-12 | 2017-01-04 | 中国矿业大学 | A kind of Automatic Selection System of Waste Rock from Coal Bulk and method |
| CN107392925A (en) * | 2017-08-01 | 2017-11-24 | 西安电子科技大学 | Remote sensing image terrain classification method based on super-pixel coding and convolutional neural networks |
| CN107292319A (en) * | 2017-08-04 | 2017-10-24 | 广东工业大学 | The method and device that a kind of characteristic image based on deformable convolutional layer is extracted |
| CN108564108A (en) * | 2018-03-21 | 2018-09-21 | 天津市协力自动化工程有限公司 | The recognition methods of coal and device |
| CN110321908A (en) * | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN110135468A (en) * | 2019-04-24 | 2019-08-16 | 中国矿业大学(北京) | A kind of recognition methods of gangue |
| CN110569782A (en) * | 2019-09-05 | 2019-12-13 | 辽宁科技大学 | Target detection method based on deep learning |
| CN110766053A (en) * | 2019-09-23 | 2020-02-07 | 中国农业大学 | Corn seed production ear screening method and device based on double-path convolutional neural network |
| CN111079845A (en) * | 2019-12-20 | 2020-04-28 | 西安交通大学 | Classification and identification method for coal gangue and coal in coal |
| CN111318461A (en) * | 2020-03-11 | 2020-06-23 | 河南理工大学 | Gangue letter sorting system based on machine vision |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283391A (en) * | 2021-06-28 | 2021-08-20 | 西安科技大学 | Method for identifying coal and gangue under complex working conditions in fully mechanized caving mining caving process |
| CN114985289A (en) * | 2022-06-02 | 2022-09-02 | 北京科技大学 | Device and method for evaluating separation effect of coal gangue pre-selection system based on intelligent ore |
| CN115138591A (en) * | 2022-09-01 | 2022-10-04 | 山西戴德测控技术有限公司 | X-ray-based online detection device for foreign matters on conveyor belt |
| CN115138591B (en) * | 2022-09-01 | 2023-01-03 | 山西潞安环保能源开发股份有限公司五阳煤矿 | X-ray-based online detection device for foreign matters on conveyor belt |
| CN116152344A (en) * | 2023-04-18 | 2023-05-23 | 天津德通电气有限公司 | Coal dressing method and system based on shape database identification |
| CN116152344B (en) * | 2023-04-18 | 2023-07-11 | 天津德通电气有限公司 | Coal dressing method and system based on shape database identification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112893159B (en) | 2023-01-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112893159B (en) | Coal gangue sorting method based on image recognition | |
| CN110314854B (en) | Workpiece detecting and sorting device and method based on visual robot | |
| CN110310261B (en) | Contact net dropper defect detection model training method and defect detection method | |
| CN107392232B (en) | Flotation working condition classification method and system | |
| CN108108679B (en) | Full-automatic tungsten ore concentrating machine | |
| CN108971190B (en) | Machine vision-based household garbage sorting method | |
| CN105388162B (en) | Raw material silicon chip surface scratch detection method based on machine vision | |
| CN109784204B (en) | Method for identifying and extracting main fruit stalks of stacked cluster fruits for parallel robot | |
| CN109859181A (en) | A kind of PCB welding point defect detection method | |
| CN112318485B (en) | Object sorting system and image processing method and device thereof | |
| CN107328787A (en) | A kind of metal plate and belt surface defects detection system based on depth convolutional neural networks | |
| CN114581782B (en) | Fine defect detection method based on coarse-to-fine detection strategy | |
| CN111266315A (en) | Ore material online sorting system and method based on visual analysis | |
| CN104850854A (en) | Talc ore product sorting processing method and talc ore product sorting system | |
| CN104056790A (en) | Intelligent potato sorting method and apparatus | |
| CN104952754B (en) | Silicon chip method for separating after plated film based on machine vision | |
| CN106248680A (en) | A kind of engine commutator quality detecting system based on machine vision and detection method | |
| CN111940339B (en) | Red date letter sorting system based on artificial intelligence | |
| CN112560941A (en) | Coal and gangue identification method based on image detection | |
| CN106872473A (en) | A kind of potato defects detection identifying system design based on machine vision | |
| CN116665011A (en) | Coal flow foreign matter identification method for coal mine belt conveyor based on machine vision | |
| CN108108678B (en) | Tungsten ore identification and separation method | |
| CN114029943A (en) | Target grabbing and positioning method and system based on image data processing | |
| CN110554052A (en) | artificial board surface defect detection method and system | |
| CN110586502B (en) | Green plum sorting device and sorting method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |