CN112893159B - Coal gangue sorting method based on image recognition - Google Patents
Coal gangue sorting method based on image recognition Download PDFInfo
- Publication number
- CN112893159B CN112893159B CN202110048060.0A CN202110048060A CN112893159B CN 112893159 B CN112893159 B CN 112893159B CN 202110048060 A CN202110048060 A CN 202110048060A CN 112893159 B CN112893159 B CN 112893159B
- Authority
- CN
- China
- Prior art keywords
- image
- coal
- deformable
- coal gangue
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
- B07C5/362—Separating or distributor mechanisms
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/38—Collecting or arranging articles in groups
Abstract
The invention discloses a coal gangue sorting method based on image recognition, which comprises the following steps: 1. constructing a coal gangue sorting platform; 2. constructing a double-path deformable CNN model; 3. training a double-path deformable CNN model; 4. selecting the coal gangue frames; 5. conveying the coal blocks; 6. and (5) sorting the coal gangue. According to the coal gangue differential sorting machine, the coal block images and the differential images are processed simultaneously by utilizing the two-way deformable CNN model, the coal gangue is identified, each convolution layer adopts a deformable convolution layer, the self-adaptive change can be generated according to the content of the image, the shape and the size of different coal gangue can be adapted, the characteristics of different scales can be extracted, the coal gangue is grabbed from the short shaft position of the coal gangue, the mechanical arm grabbing stability is ensured, the labor is saved, the safety is ensured, the coal gangue on the main belt conveyor can be sorted underground under the condition that no worker participates, the working strength of workers is reduced, the manpower resource is saved, the production efficiency is ensured, and the automation degree is high.
Description
Technical Field
The invention belongs to the technical field of coal gangue sorting, and particularly relates to a coal gangue sorting method based on image recognition.
Background
For the power coal, the mixed gangue can reduce the heat productivity of the coal and greatly reduce the quality of the power coal, so that the separation of the coal and the gangue is an indispensable one-step process in the coal washing and selecting process; at present, in addition to the existing manual coal separation technology, methods such as overlapping medium coal separation, dry coal separation, jigging coal separation, floating coal separation, electronic instrument detection and the like are also available in China. Although there are several methods for coal dressing, each method has drawbacks. In the traditional coal and coal gangue separation method, a crushing method, a jigging coal separation method, a dense medium coal separation method and the like are adopted, the methods are complex in process and high in production cost, the existing production requirements and environmental protection requirements cannot be met, and various dry separation technologies such as a winnowing method, a magnetic separation method and the like are developed aiming at the problem. In recent years, the research on sorting coal gangue by using X-ray has not been applied in large scale although the research has been successful. Most dry separation methods meet the requirements of water resource protection and environmental friendliness, but in terms of separation accuracy, the accuracy is low due to the influence of equipment factors, raw coal moisture and the like, and the actual application effect is not ideal. With the continuous improvement of the application of the information technology in the field of coal gangue separation, artificial intelligence and machine vision technologies are introduced into the coal gangue separation according to the difference of the coal and gangue in gray value, texture and other physical properties.
Disclosure of Invention
The invention aims to solve the technical problem that the defects in the prior art are overcome, and provides a coal gangue sorting method based on image recognition, wherein a double-path deformable CNN model is used for simultaneously processing a coal block image and a difference image to recognize coal gangue, each convolution layer adopts a deformable convolution layer, can generate self-adaptive change according to image content, adapts to the shapes and sizes of different coal gangue, can extract features of different scales, and can grasp the coal gangue from a short axis position of the coal gangue, so that the mechanical arm is stable in grasping, labor-saving and safe, coal gangue on a main belt conveyor can be sorted underground without the participation of workers, the working strength of workers is reduced, the manpower resources are saved, the production efficiency is ensured, the automation degree is high, and the method is convenient to popularize and use.
In order to solve the technical problems, the invention adopts the technical scheme that: a coal gangue sorting method based on image recognition is characterized by comprising the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside the coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor and a coal gangue conveyor matched with the main belt conveyor, the coal gangue sorting platform comprises a vibrating feeder arranged at a material conveying end of the main belt conveyor, a coal gangue conveyor arranged beside the main belt conveyor and a manipulator arranged at the tail end of the main belt conveyor and used for grabbing coal gangue on the main belt conveyor to the coal gangue conveyor, a limiting portal frame used for arranging and sequencing the coal gangue and the coal gangue is arranged on a frame of the main belt conveyor and positioned at a discharge end of the vibrating feeder, and an image acquisition mechanism used for identifying the coal gangue and communicating with an upper computer is arranged at the tail end of the frame of the main belt conveyor;
step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first activation layer I, the first activation layer II, the first activation layer III, the second activation layer I, the second activation layer II and the second activation layer III are ReLU activation functions;
step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, taking a training sample image in the image training data set, carrying out gray level processing on the image, sending the training sample image after the gray level processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image after the gray level processing and a standard coal block gray level image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, selecting the coal gangue frames: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, carries out gray level processing on the actual coal conveying images, sends the actual coal conveying images subjected to gray level processing into a trained two-way deformable CNN model, wherein one way of the actual coal conveying images subjected to gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images is subjected to differential processing with a standard coal block gray level image, sends the differential images into a second deformable CNN channel after binarization processing, identifies the positions of coal gangue corresponding pixels on an output image after passing through a first main full connecting layer, a second main full connecting layer and an output layer, and executes a fifth step when the two-way deformable CNN model does not identify the positions of coal gangue corresponding pixels on the output image; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
step five, conveying the coal briquettes: the manipulator does not work, the coal blocks fall onto the coal block conveyor from the main belt conveyor, and the coal block conveyor is used for conveying the coal blocks;
step six, coal gangue sorting: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binaryzation processing on the coal gangue region, namely, a white part is the coal gangue region, a black part is the background region, thereby obtaining a binaryzation image of the coal gangue, and obtaining the binaryzation image of the coal gangue according to a formula
Calculating barycentric coordinates (x, y) of the binarized coal gangue image, wherein m00 is 0-order moment of the binarized coal gangue image, m10 is first moment of the binarized coal gangue image in the x direction, and m01 is first moment of the binarized coal gangue image in the y direction;
the method comprises the steps of obtaining a minor axis of a coal gangue region in a binarization image of the coal gangue by taking barycentric coordinates as a reference, wherein the position of the minor axis in the coal gangue region provides a grabbing reference position for sorting the coal gangue by a manipulator;
the manipulator grabs the coal gangue to the coal gangue conveyor from the short shaft position of the coal gangue, and the coal gangue conveyor is utilized to separate the sorted coal gangue from the coal blocks, so that the sorting of the coal gangue is realized.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: in the fourth step, the practical coal conveying image after the gray level processing is sent into a trained double-path deformable CNN model, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the formula is firstly adopted
Performing illumination compensation processing to obtain a compensated coal transportation actual image O (x, y), wherein F (x, y) is the coal transportation actual image after gray level processing, gamma is an index value for brightness enhancement
I (x, y) is the illumination component, and m is the luminance average of the illumination component.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: the image acquisition mechanism comprises a camera and a controller connected with the output end of the camera, and the controller is connected with an upper computer through a communication module.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: the number of images in the image training data set is 8-9 times of the number of images in the image testing data set.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: in step 303, when the image is subjected to gray scale processing, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path of the practical coal conveying image and a standard coal block gray level image, the illumination compensation processing is carried out, the irrelevant pixel elimination processing is carried out, and finally the difference processing is carried out on the other path of the practical coal conveying image and the standard coal block gray level image.
The coal gangue sorting method based on image recognition is characterized by comprising the following steps: and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the illumination compensation processing is carried out, and then the irrelevant pixel elimination processing is carried out by a Lucas-Kanade optical flow method.
Compared with the prior art, the invention has the following advantages:
1. according to the coal gangue sorting device, the coal gangue sorting platform is constructed, the coal blocks and the coal gangue are queued on the main belt conveyor by the limiting door frame, so that the coal flow sequentially passes through the camera for image recognition, the coal gangue is sorted out after the image recognition, the coal gangue is sorted onto the coal gangue conveyor by the manipulator, the operation effect is good, the coal blocks can automatically fall onto the coal gangue conveyor along with the operation of the main belt conveyor, and the coal gangue sorting device is convenient to popularize and use.
2. According to the invention, the input image is subjected to feature extraction through two mutually independent channels, each convolution layer of the two-channel adopts a deformation convolution layer, the position of a convolution kernel sampling point can be subjected to self-adaptive change according to the content of the image, so that the method is suitable for geometric deformation such as the shape, the size and the like of different coal gangue, features of different scales in the input image can be extracted, then feature fusion is carried out on a full connection layer, finally, the coal gangue is identified through output by an output layer, the problems that the extraction features of a single convolution kernel of a single-channel convolution neural network are insufficient and the accuracy of an identification result is deficient are solved, and the use effect is good.
3. The method has simple steps, the gray level image of the second deformable CNN channel of the input differential image data path is subjected to illumination compensation processing, the illumination difference caused in the process of acquiring image data by monitoring video is removed, if illumination compensation processing is not carried out, the illumination difference can be used as interference, the input differential image data path is reliable and stable, the use effect is good, pixel points of irrelevant foreground objects such as moving pedestrians and the like collected during video collection are removed, the irrelevant foreground is used as interference data to influence the performance of the model, the pixel area of a motility irrelevant target is determined, the directionality removal of the motility irrelevant interference in the corresponding image data is realized, the coal block image and the differential image are simultaneously processed by utilizing the two-way deformable CNN model, coal gangue is identified, each convolutional layer adopts a deformable convolutional layer, the self-adaptive change can be carried out according to the content of the image, the shapes and the sizes of different coal gangue can be adapted, the characteristics of different scales can be extracted, the coal gangue is grabbed from the short axis position, the mechanical arm is ensured to be stable, labor-saving and safe, the underground coal gangue can be automatically sorted under the condition without the participation of workers, the working intensity of the coal gangue is reduced, and the coal gangue is convenient for production and the high-efficiency is ensured.
In conclusion, the coal gangue is identified by simultaneously processing the coal block image and the difference image by utilizing the two-way deformable CNN model, each convolution layer adopts a deformable convolution layer, can generate self-adaptive change according to the content of the image, adapts to the shape and the size of different coal gangue, can extract the characteristics of different scales, and ensures stable grabbing by a manipulator and labor-saving safety by grabbing the coal gangue from the short shaft position of the coal gangue, so that the coal gangue on a main belt conveyor can be sorted underground without the participation of workers, the working strength of the workers is reduced, the manpower resource is saved, the production efficiency is ensured, the automation degree is high, and the coal gangue sorting machine is convenient to popularize and use.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
Fig. 1 is a schematic block circuit diagram of a data transmission apparatus employed in the present invention.
Fig. 2 is a flow chart of a method of data transmission according to the present invention.
Description of the reference numerals:
1-main belt conveyor; 2-vibrating feeder; 3, limiting the door frame;
4-coal briquette; 5-coal gangue; 6, a camera;
7, a controller; 8, an upper computer; 9-a manipulator;
10-a coal conveyor; 11-coal gangue conveyor.
Detailed Description
As shown in fig. 1 and 2, the coal gangue sorting method based on image recognition of the invention includes the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside the coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor 1 and a coal gangue conveyor 10 matched with the main belt conveyor 1, the coal gangue sorting platform comprises a vibrating feeder 2 arranged at the material conveying end of the main belt conveyor 1, a coal gangue conveyor 11 arranged beside the main belt conveyor 1 and a manipulator 9 arranged at the tail end of the main belt conveyor 1 and used for grabbing coal gangue 5 on the main belt conveyor 1 to the coal gangue conveyor 11, a limiting door frame 3 used for arranging and sequencing coal gangue 4 and coal gangue 5 is arranged on the rack of the main belt conveyor 1 and positioned at the discharge end of the vibrating feeder 2, and an image acquisition mechanism used for identifying the coal gangue and communicated with an upper computer 8 is arranged at the tail end of the rack of the main belt conveyor 1;
in this embodiment, the image acquisition mechanism includes camera 6 and with controller 7 that camera 6 output is connected, controller 7 passes through communication module and is connected with host computer 8.
It needs to be explained that through constructing coal gangue letter sorting platform, utilize spacing portal to queue up coal cinder and coal gangue on main belt conveyor, make the coal stream carry out image recognition through the camera in proper order, sort out the coal gangue after image recognition, utilize the manipulator with the coal gangue letter sorting to the coal gangue conveyor on, operation effect is good, the coal cinder can drop to the coal cinder conveyor along with main belt conveyor's operation is automatic.
Step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first active layer I, the first active layer II, the first active layer III, the second active layer I, the second active layer II and the second active layer III are ReLU activation functions;
it should be noted that the input image is subjected to feature extraction through two mutually independent channels, each convolution layer of the two-channel adopts a deformation convolution layer, the position of a convolution kernel sampling point can be subjected to self-adaptive change according to the content of the image, so that the method is suitable for geometric deformation such as shapes and sizes of different foreign matters, features of different scales in the input image can be extracted, feature fusion is carried out on a full connection layer, and finally, the output layer is used for outputting, the foreign matters are identified, and the problems that the single convolution kernel of a single-channel convolution neural network is insufficient in feature extraction and is deficient in the accuracy of an identification result are solved.
Step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor 1 with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
in this embodiment, the number of images in the image training data set is 8 to 9 times the number of images in the image test data set.
Step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, calling a training sample image in the image training data set, carrying out gray processing on the training sample image, sending the training sample image subjected to gray processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image subjected to gray processing and a standard coal block gray image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
in this embodiment, in step 303, when performing gray scale processing on the image, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
Step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, coal gangue frame selection: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, carries out gray level processing on the actual coal conveying images, sends the actual coal conveying images subjected to gray level processing into a trained two-way deformable CNN model, wherein one way of the actual coal conveying images subjected to gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images is subjected to differential processing with a standard coal block gray level image, sends the differential images into a second deformable CNN channel after binarization processing, identifies the positions of coal gangue corresponding pixels on an output image after passing through a first main full connecting layer, a second main full connecting layer and an output layer, and executes a fifth step when the two-way deformable CNN model does not identify the positions of coal gangue corresponding pixels on the output image; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
in the fourth step, the practical coal transportation image after the gray processing is sent to a trained two-way deformable CNN model, wherein before the difference processing is performed on the other way and a standard coal block gray image, the illumination compensation processing is performed, the irrelevant pixel removal processing is performed, and finally the difference processing is performed on the other way and a standard coal block gray image.
In the fourth step of this embodiment, the actual images of the coal transportation after the gray processing are sent to a two-way deformable CNN model after the training, where before the difference processing is performed on the other way and one standard coal block gray image, the illumination compensation processing is performed first, and then the irrelevant pixel elimination processing is performed by using the Lucas-Kanade optical flow method.
In this embodiment, in the fourth step, the practical coal transportation image after the gray level processing is sent to the trained two-way deformable CNN model, where before the difference processing is performed on the other way and a standard coal block gray level image, the other way and the standard coal block gray level image are firstly processed according to a formula
Performing illumination compensation processing to obtain a compensated coal transportation actual image O (x, y), wherein F (x, y) isActual image of coal transportation after gray level processing, gamma is index value for brightness enhancement and
i (x, y) is the illumination component, and m is the luminance average of the illumination component.
Step five, conveying the coal briquettes: the manipulator 9 does not work, the coal briquette 4 falls from the main belt conveyor 1 to the coal briquette conveyor 10, and the coal briquette conveyor 10 is utilized to transport the coal briquette 4;
step six, coal gangue sorting: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binarization processing on the coal gangue region, namely, the white part is the coal gangue region, the black part is the background region, thereby obtaining the binarization image of the coal gangue, and obtaining the binarization image of the coal gangue according to a formula
Calculating barycentric coordinates (x, y) of the binarized coal gangue image, wherein m00 is 0-order moment of the binarized coal gangue image, m10 is first moment of the binarized coal gangue image in the x direction, and m01 is first moment of the binarized coal gangue image in the y direction;
the gravity center coordinates are used as a reference, the minor axis of the coal gangue region in the binarization image of the coal gangue is obtained, and the position of the minor axis in the coal gangue region provides a grabbing reference position for sorting the coal gangue by a manipulator 9;
the manipulator 9 grabs the coal gangue 5 from the short axis position of the coal gangue to the coal gangue conveyor 11, and the coal gangue conveyor 11 is utilized to separate the sorted coal gangue 5 from the coal blocks 4, so that the sorting of the coal gangue 5 is realized.
When the invention is used, taking an RGB original image monitored by a 480 × 640 pixel camera as an example, a first deformable CNN channel takes RGB original image monitored by the 480 × 640 pixel camera compressed into 48 × 64 pixel RGB image data as input, a second deformable CNN channel firstly carries out weight average graying under computer vision understanding on the compressed 48 × 64 pixel RGB image, then carries out difference processing, illumination compensation processing and irrelevant foreground elimination processing on the image, and finally obtains a processing result as the input of a difference image channel. The specific two-way convolution process is as follows: the 48 × 64 pixel images input by the first deformable CNN channel and the second deformable CNN channel are filtered by 64 filters of 5 × 5 size to obtain 64 feature maps of 48 × 64 size, which is a single deformable convolution layer. Then, pooling operations of 3 × 3 sizes were performed on the 64 feature maps, respectively, to obtain pooled layer feature maps of 24 × 32 sizes. Convolution operation is also carried out between the first pooling layer and the second warped convolutional layer, and 64 filters with the size of 5 × 5 are shared, so that the second warped convolutional layer has 64 feature maps with the size of 24 × 32. The second pooling layer has 64 feature maps of 12 × 16 pixels, which are pooled by the upper network. The warped convolutional layer three is still convolution operation, the filter size of the warped convolutional layer three is still 5 × 5, and 64 filters are in total, so that the feature map of the pooling layer two is subjected to convolution operation to obtain 64 12 × 16 feature maps. The pooling operation of 3 × 3 sizes was performed on each of the 64 feature maps, and a pooling layer three feature map of 6 × 8 size was obtained. The first full connection layer has 100 neurons, the second full connection layer has 40 neurons, and the output layer outputs the result of network classification. Compared with a fully-connected neural network, the convolutional neural network can obtain a characteristic diagram through a filter, so that weight sharing is realized, parameters of the network are greatly reduced, and the training efficiency of the network is improved. The convolution operation may make the input have translational or rotational invariance. The pooling operation can improve the generalization capability of the network model and increase the receptive field of the network while preserving the main features. The fully-connected layer acts as a "classifier" in the overall convolutional neural network. If we say that operations such as convolutional layers, pooling layers, and activation function layers map raw data to hidden layer feature space, the fully-connected layer serves to map the learned "distributed feature representation" to the sample label space. This embodiment utilizes double-circuit flexible CNN model to handle coal cinder image and difference image simultaneously, the discernment gangue, every convolution layer adopts the deformation convolution layer, can take place self-adaptation's change according to image content, adapt to the shape of different gangue, the size, can extract the characteristic of different yardstick, snatch the gangue through the minor axis position from the gangue, guarantee that the manipulator snatchs stably, laborsaving safety, make can be under the condition that does not have staff's participation in the pit, coal gangue letter sorting on the main belt conveyor, staff's working strength has not only been reduced, manpower resources have been saved, and ensure production efficiency, the degree of automation is high.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and all simple modifications, changes and equivalent structural changes made to the above embodiment according to the technical spirit of the present invention still fall within the protection scope of the technical solution of the present invention.
Claims (4)
1. A coal gangue sorting method based on image recognition is characterized by comprising the following steps:
step one, constructing a coal gangue sorting platform: a coal gangue sorting platform is arranged beside a coal conveying mechanism, the coal conveying mechanism comprises a main belt conveyor (1) and a coal gangue conveyor (10) matched with the main belt conveyor (1), the coal gangue sorting platform comprises a vibrating feeder (2) arranged at a material conveying end of the main belt conveyor (1), a coal gangue conveyor (11) arranged beside the main belt conveyor (1) and a mechanical arm (9) arranged at the tail end of the main belt conveyor (1) and used for grabbing coal gangue (5) on the main belt conveyor (1) to the coal gangue conveyor (11), a limiting door frame (3) used for arranging and sequencing coal blocks (4) and coal gangue (5) is arranged on a frame of the main belt conveyor (1) and positioned at a material discharging end of the vibrating feeder (2), and an image acquisition mechanism used for identifying the coal gangue and communicating with an upper computer (8) is arranged at the tail end of the frame of the main belt conveyor (1);
step two, constructing a double-path deformable CNN model: constructing a double-path deformable CNN model, wherein the double-path deformable CNN model comprises a double-path deformable CNN module, a first main full-connection layer, a second main full-connection layer and an output layer which are sequentially arranged, the double-path deformable CNN module comprises a first deformable CNN channel and a second deformable CNN channel which are arranged in parallel, and the main full-connection layer simultaneously receives the output of the first deformable CNN channel and the output of the second deformable CNN channel;
the first deformable CNN channel comprises a first deformable convolution layer I, a first activation layer I, a first pooling layer I, a first deformable convolution layer II, a first activation layer II, a first pooling layer II, a first deformable convolution layer III, a first activation layer III, a first pooling layer III, a first full-connection layer I and a first full-connection layer II which are arranged in sequence;
the second deformable CNN channel comprises a second deformable convolution layer I, a second activation layer I, a second pooling layer I, a second deformable convolution layer II, a second activation layer II, a second pooling layer II, a second deformable convolution layer III, a second activation layer III, a second pooling layer III, a second full-connection layer I and a second full-connection layer II which are arranged in sequence;
the convolution kernel size of the first deformed convolution layer one, the first deformed convolution layer two, the first deformed convolution layer three, the second deformed convolution layer one, the second deformed convolution layer two and the second deformed convolution layer three is 5 multiplied by 5, and the step length is 1; the size of the pooling windows of the first pooling layer I, the first pooling layer II, the first pooling layer III, the second pooling layer I, the second pooling layer II and the second pooling layer III is 3 multiplied by 3, and the step length is 2; the first active layer I, the first active layer II, the first active layer III, the second active layer I, the second active layer II and the second active layer III are ReLU activation functions;
step three, training a double-path deformable CNN model, and the process is as follows:
step 301, constructing an image database: collecting standard coal block gray images of a plurality of coal blocks and training sample images of not less than 1000 coal gangues on a main belt conveyor (1) with a fixed visual angle by using an image collecting mechanism, forming an image database by the plurality of training sample images, and randomly classifying the image database to obtain an image training data set and an image testing data set;
step 302, initializing pixel point weights w and pixel point offsets delta pn in a first deformed convolution layer, a second deformed convolution layer, a third deformed convolution layer, a first deformed convolution layer, a second deformed convolution layer and a third deformed convolution layer;
step 303, taking a training sample image in the image training data set, carrying out gray level processing on the image, sending the training sample image after the gray level processing into a first deformable CNN channel, simultaneously carrying out difference processing on the training sample image after the gray level processing and a standard coal block gray level image, carrying out binarization processing on the difference image, sending the difference image into a second deformable CNN channel, and carrying out primary training on a double-path deformable CNN model;
step 304, updating pixel point weight w and pixel point offset delta pn in the first deformed convolution layer I, the first deformed convolution layer II, the first deformed convolution layer III, the second deformed convolution layer I, the second deformed convolution layer II and the second deformed convolution layer III;
step 305, the step 303 to the step 304 are circulated until the image in the image training data set is called, the training process of the two-path deformable CNN model is completed, and the final pixel point weight w and the optimal pixel point offset delta pn in the first deformable convolution layer I, the first deformable convolution layer II, the first deformable convolution layer III, the second deformable convolution layer I, the second deformable convolution layer II and the second deformable convolution layer III are obtained;
when the offset delta pn of the preferred pixel is a non-integer, rounding the offset delta pn of the preferred pixel to obtain the final offset delta pn of the pixel;
when the offset delta pn of the preferred pixel is an integer, the offset delta pn of the preferred pixel is the final offset delta pn of the pixel;
step 306, calling a training sample image from the image test data set, testing the double-path deformable CNN model, and obtaining a double-path deformable CNN model after training and testing;
step four, coal gangue frame selection: the method comprises the steps that an image acquisition mechanism acquires actual coal conveying images on site, gray level processing is carried out on the actual coal conveying images, the actual coal conveying images after gray level processing are sent into a trained two-way deformable CNN model, one way of the actual coal conveying images after gray level processing enters a first deformable CNN channel, the other way of the actual coal conveying images after gray level processing and a standard coal block gray level image are subjected to difference processing, the difference image is subjected to binarization processing and then sent into a second deformable CNN channel, the positions of coal gangue corresponding pixels on an output image are identified after passing through a first main full connecting layer, a second main full connecting layer and an output layer, when the positions of coal gangue corresponding pixels on the output image are not identified by the two-way deformable CNN model, the actual coal conveying images do not contain coal gangue, and the fifth step is executed; when the position of the corresponding pixels of the coal gangue on the output image is identified by the double-path deformable CNN model, the coal gangue exists in the actual coal conveying image, the corresponding pixels of the coal gangue on the output image are subjected to frame selection by using a selection frame, and the step six is executed;
step five, conveying the coal briquette: the manipulator (9) does not work, the coal blocks (4) fall onto the coal block conveyor (10) from the main belt conveyor (1), and the coal block conveyor (10) is utilized to transport the coal blocks (4);
step six, sorting coal gangue: searching the pixel position of the selection frame on the coal transportation actual image, using the selection frame to frame-select the coal gangue on the coal transportation actual image, carrying out binarization processing on the coal gangue region, namely, the white part is the coal gangue region, the black part is the background region, thereby obtaining the binarization image of the coal gangue, and obtaining the binarization image of the coal gangue according to a formula
And calculating the barycentric coordinates of the binarized coal gangue image
Wherein, in the process,
is 0 moment of the binarized coal gangue image,
is the first moment of the coal gangue image area after binarization in the x direction,
is the first moment of the coal gangue image area after binaryzation in the y direction；
The gravity center coordinates are used as a reference, the minor axis of a coal gangue region in the binarization image of the coal gangue is obtained, and the position of the minor axis in the coal gangue region provides a grabbing reference position for a manipulator (9) to sort the coal gangue;
the manipulator (9) grabs the coal gangue (5) from the short axis position of the coal gangue to a coal gangue conveyor (11), and the coal gangue conveyor (11) is utilized to separate the sorted coal gangue (5) from the coal blocks (4) so as to realize sorting of the coal gangue (5);
in the fourth step, the practical image of the coal transportation after the gray level processing is sent into a two-way deformable CNN model after the training is finished, wherein, before the difference processing is carried out on the other way and a standard coal block gray level image, the other way and the standard coal block gray level image are firstly processed according to a formula
Performing illumination compensation processing to obtain compensated coal transportation actual image
Wherein, in the process,
is the actual image of the coal transportation after the gray level processing,
is an index value for brightness enhancement and
，
as the component of the illumination, there is,
is the mean value of the luminance of the illumination components;
sending the practical coal conveying image after gray level processing into a trained two-path deformable CNN model, wherein before the difference processing is carried out on the other path of the practical coal conveying image and a standard coal block gray level image, illumination compensation processing is carried out, irrelevant pixel removal processing is carried out, and finally difference processing is carried out on the other path of the practical coal conveying image and the standard coal block gray level image;
and in the fourth step, the practical coal conveying image after the gray level processing is sent into a two-path deformable CNN model after the training is completed, wherein before the difference processing is carried out on the other path and a standard coal block gray level image, the illumination compensation processing is carried out, and then the irrelevant pixel elimination processing is carried out by a Lucas-Kanade optical flow method.
2. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: the image acquisition mechanism comprises a camera (6) and a controller (7) connected with the output end of the camera (6), and the controller (7) is connected with an upper computer (8) through a communication module.
3. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: the number of images in the image training data set is 8-9 times of the number of images in the image testing data set.
4. The coal gangue sorting method based on image recognition as claimed in claim 1, wherein: in step 303, when the image is subjected to gray scale processing, the weight of the R channel in the RGB three channels is 0.3004, the weight of the G channel in the RGB three channels is 0.4285, and the weight of the B channel in the RGB three channels is 0.2711.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110048060.0A CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110048060.0A CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112893159A CN112893159A (en) | 2021-06-04 |
| CN112893159B true CN112893159B (en) | 2023-01-06 |
Family
ID=76114217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110048060.0A Active CN112893159B (en) | 2021-01-14 | 2021-01-14 | Coal gangue sorting method based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112893159B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283391B (en) * | 2021-06-28 | 2023-04-07 | 西安科技大学 | Method for identifying coal and gangue under complex working conditions in fully mechanized caving mining caving process |
| CN114985289B (en) * | 2022-06-02 | 2023-04-14 | 北京科技大学 | Device and method for evaluating separation effect of coal gangue pre-selection system based on intelligent ore |
| CN115138591B (en) * | 2022-09-01 | 2023-01-03 | 山西潞安环保能源开发股份有限公司五阳煤矿 | X-ray-based online detection device for foreign matters on conveyor belt |
| CN116152344B (en) * | 2023-04-18 | 2023-07-11 | 天津德通电气有限公司 | Coal dressing method and system based on shape database identification |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013060095A1 (en) * | 2011-10-27 | 2013-05-02 | 山东博润工业技术股份有限公司 | Process and system for dry coal preparation by coal gangue removal |
| CN106269576A (en) * | 2016-09-12 | 2017-01-04 | 中国矿业大学 | A kind of Automatic Selection System of Waste Rock from Coal Bulk and method |
| CN107292319A (en) * | 2017-08-04 | 2017-10-24 | 广东工业大学 | The method and device that a kind of characteristic image based on deformable convolutional layer is extracted |
| CN107392925A (en) * | 2017-08-01 | 2017-11-24 | 西安电子科技大学 | Remote sensing image terrain classification method based on super-pixel coding and convolutional neural networks |
| CN108564108A (en) * | 2018-03-21 | 2018-09-21 | 天津市协力自动化工程有限公司 | The recognition methods of coal and device |
| CN110135468A (en) * | 2019-04-24 | 2019-08-16 | 中国矿业大学(北京) | A kind of recognition methods of gangue |
| CN110321908A (en) * | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN110569782A (en) * | 2019-09-05 | 2019-12-13 | 辽宁科技大学 | Target detection method based on deep learning |
| CN110766053A (en) * | 2019-09-23 | 2020-02-07 | 中国农业大学 | Corn seed production ear screening method and device based on double-path convolutional neural network |
| CN111079845A (en) * | 2019-12-20 | 2020-04-28 | 西安交通大学 | Classification and identification method for coal gangue and coal in coal |
| CN111318461A (en) * | 2020-03-11 | 2020-06-23 | 河南理工大学 | Gangue letter sorting system based on machine vision |
-
2021
- 2021-01-14 CN CN202110048060.0A patent/CN112893159B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013060095A1 (en) * | 2011-10-27 | 2013-05-02 | 山东博润工业技术股份有限公司 | Process and system for dry coal preparation by coal gangue removal |
| CN106269576A (en) * | 2016-09-12 | 2017-01-04 | 中国矿业大学 | A kind of Automatic Selection System of Waste Rock from Coal Bulk and method |
| CN107392925A (en) * | 2017-08-01 | 2017-11-24 | 西安电子科技大学 | Remote sensing image terrain classification method based on super-pixel coding and convolutional neural networks |
| CN107292319A (en) * | 2017-08-04 | 2017-10-24 | 广东工业大学 | The method and device that a kind of characteristic image based on deformable convolutional layer is extracted |
| CN108564108A (en) * | 2018-03-21 | 2018-09-21 | 天津市协力自动化工程有限公司 | The recognition methods of coal and device |
| CN110321908A (en) * | 2018-03-29 | 2019-10-11 | 华为技术有限公司 | Image-recognizing method, terminal device and computer readable storage medium |
| CN110135468A (en) * | 2019-04-24 | 2019-08-16 | 中国矿业大学(北京) | A kind of recognition methods of gangue |
| CN110569782A (en) * | 2019-09-05 | 2019-12-13 | 辽宁科技大学 | Target detection method based on deep learning |
| CN110766053A (en) * | 2019-09-23 | 2020-02-07 | 中国农业大学 | Corn seed production ear screening method and device based on double-path convolutional neural network |
| CN111079845A (en) * | 2019-12-20 | 2020-04-28 | 西安交通大学 | Classification and identification method for coal gangue and coal in coal |
| CN111318461A (en) * | 2020-03-11 | 2020-06-23 | 河南理工大学 | Gangue letter sorting system based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112893159A (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112893159B (en) | Coal gangue sorting method based on image recognition | |
| CN110314854B (en) | Workpiece detecting and sorting device and method based on visual robot | |
| CN107392232B (en) | Flotation working condition classification method and system | |
| CN108108679B (en) | Full-automatic tungsten ore concentrating machine | |
| CN108971190B (en) | Machine vision-based household garbage sorting method | |
| CN105388162B (en) | Raw material silicon chip surface scratch detection method based on machine vision | |
| CN108287010B (en) | Crab multi-index grading device and method | |
| CN112318485B (en) | Object sorting system and image processing method and device thereof | |
| CN104483320B (en) | Digitized defect detection device and detection method of industrial denitration catalyst | |
| CN104056790A (en) | Intelligent potato sorting method and apparatus | |
| CN104952754B (en) | Silicon chip method for separating after plated film based on machine vision | |
| CN104850854A (en) | Talc ore product sorting processing method and talc ore product sorting system | |
| CN106248680A (en) | A kind of engine commutator quality detecting system based on machine vision and detection method | |
| CN111940339B (en) | Red date letter sorting system based on artificial intelligence | |
| CN112560941A (en) | Coal and gangue identification method based on image detection | |
| CN106872473A (en) | A kind of potato defects detection identifying system design based on machine vision | |
| CN110586502B (en) | Green plum sorting device and sorting method | |
| CN110554052A (en) | artificial board surface defect detection method and system | |
| CN114581782A (en) | Fine defect detection method based on coarse-to-fine detection strategy | |
| CN108108678B (en) | Tungsten ore identification and separation method | |
| CN114029943A (en) | Target grabbing and positioning method and system based on image data processing | |
| CN111968082A (en) | Product packaging defect detection and identification method based on machine vision | |
| CN113245222B (en) | Visual real-time detection and sorting system and sorting method for foreign matters in panax notoginseng | |
| CN112338898B (en) | Image processing method and device of object sorting system and object sorting system | |
| CN210730217U (en) | Green plum sorting device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |