CN114120175A - Method for identifying foreign matters on coal conveying belt based on computer vision - Google Patents
Method for identifying foreign matters on coal conveying belt based on computer vision Download PDFInfo
- Publication number
- CN114120175A CN114120175A CN202111327503.6A CN202111327503A CN114120175A CN 114120175 A CN114120175 A CN 114120175A CN 202111327503 A CN202111327503 A CN 202111327503A CN 114120175 A CN114120175 A CN 114120175A
- Authority
- CN
- China
- Prior art keywords
- image
- foreign matters
- neural network
- convolutional
- conveying belt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The disclosure relates to a method for identifying foreign matters on a coal conveying belt based on computer vision, which is characterized by comprising the following steps: s1, acquiring common foreign body image data and constructing a training set; s2, constructing a convolutional neural network algorithm model, and training the neural network algorithm model through the training set to obtain an image sorter, wherein the image sorter is configured to mark different foreign matters; s3, acquiring a video stream of a running coal conveying belt; s4, extracting a plurality of image frames with certain time intervals from the video stream; s5, inputting a plurality of image frames into the image sorter, judging whether foreign matters exist in the image frames or not, and if so, sending out warning information. The method for identifying the foreign matters on the coal conveying belt based on the computer vision can be based on a convolutional neural network algorithm model, is greatly convenient for video monitoring personnel to remotely monitor the foreign matters on the coal conveying belt, and reduces the labor input.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method for recognizing foreign matters on a coal conveying belt based on computer vision.
Background
The coal mine industry is the supporting industry of national economy in China and occupies an important proportion in national energy structure and primary energy consumption. The underground coal mine is an industry with variable and complex environment, more working links, more operating personnel and huge and concentrated equipment and large comprehensive risk coefficient, potential safety hazards can occur at every moment, and safety accidents are easy to happen. The coal mine belt plays a role in non-wear-out in coal mine operation, and has the characteristics of large transportation capacity, complex working environment, strong bearing capacity, longer transportation distance and the like. However, due to the complex working environment of the mine, the carelessness of workers and the like, other articles except for the coal mine can appear on the belt, and potential safety hazards are generated. In the prior art, the method of video investigation is only carried out manually, so that the labor investment is large and the omission is easy to generate.
Disclosure of Invention
In view of the above problems in the prior art, the present invention is directed to a method for identifying foreign matters in a coal belt based on computer vision, which can assist in manual inspection of foreign matters in the coal belt.
In order to achieve the above object, an aspect of the present invention provides a computer vision-based method for identifying a foreign object in a coal belt, comprising:
s1, acquiring common foreign body image data and constructing a training set;
s2, constructing a convolutional neural network algorithm model, and training the neural network algorithm model through the training set to obtain an image sorter, wherein the image sorter is configured to mark different foreign matters;
s3, acquiring a video stream of a running coal conveying belt;
s4, extracting a plurality of image frames with certain time intervals from the video stream;
s5, inputting a plurality of image frames into the image sorter, judging whether foreign matters exist in the image frames or not, and if so, sending out warning information.
Preferably, in S2, the convolutional neural network includes at least a first convolutional neural network, a second convolutional neural network, and a full link layer; the first convolutional neural network at least comprises a first convolutional layer, and the second convolutional neural network at least comprises a second convolutional layer.
Preferably, in step S2, at least one convolutional layer is extracted from the first convolutional layer and the second convolutional layer of the trained convolutional neural network, each convolutional layer includes at least one convolutional kernel matrix, and at least one convolutional filter is formed in an output channel of the convolutional kernel matrix, where the convolutional filter is an area suspected of being a foreign object; the plurality of convolution filters is configured as the image sorter.
Preferably, in step S5, before the image frames are input into the image sorter, two adjacent image frames are grayed to obtain a grayscale difference image according to the grayscale difference between the pixels at the corresponding positions;
and the image sorter analyzes the gray level difference image to obtain the distribution characteristics of the pixels, divides the image according to the distribution characteristics of the pixels, determines the area of the suspected foreign matters and marks the area.
Preferably, the method further comprises performing histogram equalization processing and/or median filtering on the grayed image frame to eliminate partial noise in the image.
Preferably, in step S3, when acquiring the video stream of the running coal belt, two independent video streams are formed, and the two video streams respectively form the left-eye image frame and the right-eye image frame when extracting the image frames.
Preferably, calculating gray difference values between pixel points at corresponding positions of the left-eye image frame and the right-eye image frame respectively to obtain corresponding gray difference images;
respectively averaging the gray values of all pixel points of each gray difference image in the belt detection area to obtain corresponding sub-detection quantities;
and averaging all the sub-detection quantities to obtain detection quantities, and determining and marking the area of the suspected foreign matters according to the detection quantities.
In still another aspect of the present invention, there is provided a computer vision-based foreign matter identification system for a coal belt, including:
the model building module is configured to obtain common foreign body image data and build a training set; constructing a convolutional neural network algorithm model, and training the neural network algorithm model through the training set to obtain an image sorter, wherein the image sorter is configured to mark different foreign matters;
a video acquisition unit configured to acquire a video stream on a coal conveyor belt;
and the image analysis unit is configured to judge whether foreign matters exist in the image frames according to the image sorter and send out warning information if the foreign matters exist in the image frames.
The method for identifying the foreign matters on the coal conveying belt based on the computer vision can train the constructed neural network algorithm model based on the convolutional neural network algorithm model according to the image data of various common foreign matters prestored in the remote control center, and further generate the image sorter capable of identifying the common foreign matters. The process can greatly facilitate the remote monitoring of foreign matters on the coal conveying belt by video monitoring personnel, and the labor input is reduced.
Drawings
FIG. 1 is a flow chart of a method for identifying foreign matters in a coal conveying belt based on computer vision.
Fig. 2 is a block diagram of a system to which the method for identifying foreign matters in a coal belt based on computer vision according to the present invention is applied.
FIG. 3 is a schematic diagram of a network topology of a single monitoring point in a system to which the method for identifying foreign matters in a coal conveying belt based on computer vision is applied.
Detailed Description
In order to make the technical solutions of the present invention better understood, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
Various aspects and features of the present invention are described herein with reference to the drawings.
These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment, given as a non-limiting example, with reference to the accompanying drawings.
It should also be understood that, although the invention has been described with reference to some specific examples, a person of skill in the art shall certainly be able to achieve many other equivalent forms of the invention, having the characteristics as set forth in the claims and hence all coming within the field of protection defined thereby.
The above and other aspects, features and advantages of the present invention will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
As shown in the system block diagrams shown in fig. 2 and fig. 3, the system applied by the method for identifying foreign matters in a coal belt based on computer vision provided by the present invention can be deployed in a network environment based on a C/S architecture, the system includes a remote control center 1, the remote control center 1 is in communication connection with a plurality of upper computers 4 through a gateway 3, wherein, a data server 2 is also connected to the network through the gateway 3, and the data server 2 stores image data of common foreign matters and includes a training set 21 constructed by the same, and further, a storage unit for storing an identification record 22 is further provided in the data server 2. In addition, in the present invention, the basic idea is to identify the foreign matter by the neural network algorithm model 23 of multilayer convolution based on computer vision, but considering that the coal belt line is relatively long in general, a plurality of monitoring points 5 may be required, and the present invention is not suitable for the image identification work providing a large calculation power for the special environment of the coal belt. Therefore, in the present invention, the trained neural network algorithm model 23 is not stored in the upper computer 4 corresponding to the monitoring point 5, but is preferably stored in the data server. Furthermore, in the upper computer 4, the communication interface 41 can be connected with the remote control center 1 or the data server 2 through the gateway 3 for communication, and the calling of the neural network algorithm model 23 in the data server 2 is realized through the image recognition API. Therefore, the hardware investment of no monitoring point 5 can be saved, and the subsequent maintenance cost is reduced. Fig. 1 is a flow chart illustrating a method for identifying foreign matters in a coal belt based on computer vision according to the present invention, which may include: s1, acquiring common foreign body image data and constructing a training set; s2, constructing a convolutional neural network algorithm model, and training the neural network algorithm model through the training set to obtain an image sorter, wherein the image sorter is configured to mark different foreign matters; s3, acquiring a video stream of a running coal conveying belt; s4, extracting a plurality of image frames with certain time intervals from the video stream; s5, inputting a plurality of image frames into the image sorter, judging whether foreign matters exist in the image frames or not, and if so, sending out warning information.
Specifically, in the present invention, in step S2, the convolutional neural network includes at least a first convolutional neural network, a second convolutional neural network, and a full link layer; the first convolutional neural network at least comprises a first convolutional layer, and the second convolutional neural network at least comprises a second convolutional layer. In step S2, extracting at least one convolutional layer from the first convolutional layer and the second convolutional layer of the trained convolutional neural network, where each convolutional layer includes at least one convolutional kernel matrix, and at least one convolutional filter is formed in an output channel of the convolutional kernel matrix, where the convolutional filter is an area suspected of being a foreign object; the plurality of convolution filters is configured as the image sorter. The structure of the convolutional neural network in the invention does not belong to the contribution of the application to the prior art, and particularly, the method of the invention is to apply the neural network algorithm model to generate an image sorter which can be used for identifying foreign matters in the coal conveying belt. Specifically, the image sorter sorts suspected foreign objects in the image according to the foreign object features contained in the image, so as to select the most appropriate convolution filter for analysis. In a traditional analysis process, image pre-classification is usually judged by relying on meta-information generated by an image acquisition device. Meanwhile, the image sorter can divide the image into different sub-images according to different states of the coal materials on the conveyor belt, and the sub-images are processed by different convolution filters. For example, different image analyzers are used for analysis in the unloaded and loaded states, respectively.
Furthermore, the image sorter of the present invention may specifically be implemented by using an image retrieval algorithm. Specifically, a group of characteristic values are generated for each sample image and the image to be processed, and the closest sample image is found by matching the characteristic values of the image to be processed and the characteristic values of the sample images, so that the purpose of classification is achieved. The extraction of the image feature values can use the conventional features such as HOG features, LBP features and Haar features, and can also use the previously mentioned deep convolution network based on multilayer convolution. The deep convolutional network can be obtained by training a supervised classification task or an unsupervised automatic coding machine. Meanwhile, in order to effectively reduce the feature dimension and improve the matching efficiency, a feature cooling algorithm can be adopted. Conventional algorithms include Principal Component Analysis (PCA), Independent Component Analysis (ICA), dictionary learning combined with sparse encryption algorithm (sparse encryption and sparse coding), and more advanced algorithms of machine learning include word band model (bag of words), word vector algorithm (word2vec), and so on.
Of course, in other embodiments of the present invention, an image sorting algorithm may also be employed to implement the image sorter. For example, the images may be classified according to different belt carrying states by training a deep convolutional network to classify the images. The deep convolutional network extracts image characteristic information through the combination of a plurality of convolutional layers, and then calculates the final probability value through a plurality of full-connected layers. The convolution kernel and the connection weight in the network are obtained through computer optimization.
In other embodiments, as a further improvement, in step S5, before the image frames are input into the image sorter, the two adjacent image frames are grayed, and the grayscale difference between the pixel points at the corresponding positions is obtained as a grayscale difference image; and the image sorter analyzes the gray level difference image to obtain the distribution characteristics of the pixels, divides the image according to the distribution characteristics of the pixels, determines the area of the suspected foreign matters and marks the area. The method further comprises the step of carrying out histogram equalization processing and/or median filtering on the grayed image frame to eliminate partial noise in the image.
In other modifications, in step S3, when acquiring the video stream of the running coal belt, two independent video streams are formed, and the two video streams respectively form the left-eye image frame and the right-eye image frame when extracting the image frames. Based on the improvement, the gray difference between pixel points at corresponding positions of the left-eye image frame and the right-eye image frame can be further calculated respectively to obtain corresponding gray difference images; respectively averaging the gray values of all pixel points of each gray difference image in the belt detection area to obtain corresponding sub-detection quantities; and then averaging all the sub-detection quantities to obtain detection quantities, and determining and marking the area of the suspected foreign matters according to the detection quantities. Compared with a single camera scheme, a more accurate prediction result can be obtained.
The above embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and the scope of the present invention is defined by the claims. Various modifications and equivalents may be made by those skilled in the art within the spirit and scope of the present invention, and such modifications and equivalents should also be considered as falling within the scope of the present invention.
Claims (7)
1. A computer vision-based foreign matter identification method for a coal conveying belt is characterized by comprising the following steps:
s1, acquiring common foreign body image data and constructing a training set;
s2, constructing a convolutional neural network algorithm model, and training the neural network algorithm model through the training set to obtain an image sorter, wherein the image sorter is configured to mark different foreign matters;
s3, acquiring a video stream of a running coal conveying belt;
s4, extracting a plurality of image frames with certain time intervals from the video stream;
s5, inputting a plurality of image frames into the image sorter, judging whether foreign matters exist in the image frames or not, and if so, sending out warning information.
2. The computer vision-based foreign matter identification method for a coal belt according to claim 1, wherein in the step S2, the convolutional neural network includes at least a first convolutional neural network, a second convolutional neural network, and a full connection layer; the first convolutional neural network at least comprises a first convolutional layer, and the second convolutional neural network at least comprises a second convolutional layer.
3. The computer vision-based foreign matter identification method for a coal belt according to claim 2, wherein in step S2, at least one convolutional layer is extracted from the first convolutional layer and the second convolutional layer of the trained convolutional neural network, each convolutional layer includes at least one convolutional kernel matrix, and at least one convolutional filter is formed in an output channel of the convolutional kernel matrix, the convolutional filter being an area suspected of foreign matter; the plurality of convolution filters is configured as the image sorter.
4. The method for identifying foreign matters in a coal conveyor belt according to claim 1, wherein in step S5, before a plurality of image frames are inputted into the image sorter, two adjacent image frames are grayed to obtain a grayscale difference image according to the grayscale difference between pixels at corresponding positions;
and the image sorter analyzes the gray level difference image to obtain the distribution characteristics of the pixels, divides the image according to the distribution characteristics of the pixels, determines the area of the suspected foreign matters and marks the area.
5. The method for identifying the foreign matters on the coal conveying belt based on the computer vision is characterized by further comprising the step of carrying out histogram equalization processing and/or median filtering on the grayed image frames to eliminate partial noise in the images.
6. The method for identifying foreign matters in a coal conveyor belt according to claim 1, wherein in step S3, when acquiring the video stream of the coal conveyor belt in operation, two independent video streams are formed, and the two video streams respectively form the left-eye image frame and the right-eye image frame when extracting the image frames.
7. The method for identifying the foreign matters on the coal conveying belt based on the computer vision of claim 1, wherein gray level differences between pixel points at corresponding positions of a left-eye image frame and a right-eye image frame are respectively calculated to obtain corresponding gray level difference images;
respectively averaging the gray values of all pixel points of each gray difference image in the belt detection area to obtain corresponding sub-detection quantities;
and averaging all the sub-detection quantities to obtain detection quantities, and determining and marking the area of the suspected foreign matters according to the detection quantities.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111327503.6A CN114120175A (en) | 2021-11-10 | 2021-11-10 | Method for identifying foreign matters on coal conveying belt based on computer vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111327503.6A CN114120175A (en) | 2021-11-10 | 2021-11-10 | Method for identifying foreign matters on coal conveying belt based on computer vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114120175A true CN114120175A (en) | 2022-03-01 |
Family
ID=80378147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111327503.6A Pending CN114120175A (en) | 2021-11-10 | 2021-11-10 | Method for identifying foreign matters on coal conveying belt based on computer vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114120175A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115147770A (en) * | 2022-08-30 | 2022-10-04 | 山东千颐科技有限公司 | Belt foreign matter vision recognition system based on image processing |
-
2021
- 2021-11-10 CN CN202111327503.6A patent/CN114120175A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115147770A (en) * | 2022-08-30 | 2022-10-04 | 山东千颐科技有限公司 | Belt foreign matter vision recognition system based on image processing |
| CN115147770B (en) * | 2022-08-30 | 2022-12-02 | 山东千颐科技有限公司 | Belt foreign matter vision recognition system based on image processing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110390691B (en) | Ore dimension measuring method based on deep learning and application system | |
| CN111310645B (en) | Method, device, equipment and storage medium for warning overflow bin of goods accumulation | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| US9390333B2 (en) | System and method for robust motion detection | |
| CN111260629A (en) | Pantograph structure abnormity detection algorithm based on image processing | |
| CN108664874A (en) | Underground coal flow rate testing methods based on image recognition | |
| CN110163852B (en) | Conveying belt real-time deviation detection method based on lightweight convolutional neural network | |
| CN110942450A (en) | Multi-production-line real-time defect detection method based on deep learning | |
| CN112001878A (en) | Deep learning ore scale measuring method based on binarization neural network and application system | |
| CN103970771A (en) | Search method and system for human body | |
| CN108664875A (en) | Underground belt-conveying monitoring method based on image recognition | |
| CN112893159A (en) | Coal gangue sorting method based on image recognition | |
| CN114298948A (en) | Ball machine monitoring abnormity detection method based on PSPNet-RCNN | |
| CN116665011A (en) | Coal flow foreign matter identification method for coal mine belt conveyor based on machine vision | |
| CN115512134A (en) | Express item stacking abnormity early warning method, device, equipment and storage medium | |
| CN112508911A (en) | Rail joint touch net suspension support component crack detection system based on inspection robot and detection method thereof | |
| CN114120175A (en) | Method for identifying foreign matters on coal conveying belt based on computer vision | |
| CN114092478A (en) | Anomaly detection method | |
| CN113591735A (en) | Pedestrian detection method and system based on deep learning | |
| CN117315593A (en) | Recognition method for foreign matter invasion of underground coal mine transportation system | |
| CN111968082A (en) | Product packaging defect detection and identification method based on machine vision | |
| CN113793327B (en) | Token-based high-speed rail foreign matter detection method | |
| CN112712055B (en) | Double-path deformable CNN coal mine gateway belt conveying foreign matter monitoring method | |
| CN113610831B (en) | Wood defect detection method based on computer image technology and transfer learning | |
| CN115880476A (en) | Belt deviation detection method based on visible light and infrared image fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |