CN112991438B - Coal gangue detection and identification system, intelligent coal discharging system and training method of model - Google Patents
Coal gangue detection and identification system, intelligent coal discharging system and training method of model Download PDFInfo
- Publication number
- CN112991438B CN112991438B CN202110391898.XA CN202110391898A CN112991438B CN 112991438 B CN112991438 B CN 112991438B CN 202110391898 A CN202110391898 A CN 202110391898A CN 112991438 B CN112991438 B CN 112991438B
- Authority
- CN
- China
- Prior art keywords
- coal
- gangue
- image
- module
- gangue detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30184—Infrastructure
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a coal gangue detection and identification system, an intelligent coal discharging system and a training method of a model, which comprises the following steps: the intelligent camera equipment is used for acquiring a coal-fired video, detecting and identifying coal gangue of the coal-fired image frames in the coal-fired video through the current coal gangue detection model to obtain gangue positions and coal mine positions in the coal-fired image frames, and determining unlabeled coal-fired image data corresponding to the corresponding positions according to the gangue positions and the coal mine positions respectively; the image learning server is used for acquiring target data, carrying out optimization training on the current gangue detection model by utilizing the target data to obtain an optimized gangue detection model, and deploying the optimized gangue detection model to the intelligent camera equipment so that the intelligent camera equipment can carry out next gangue position detection by utilizing the optimized gangue detection model. The system can detect and identify the coal gangue through the coal gangue detection model to the coal gangue image frame, and has good robustness.
Description
Technical Field
The invention relates to the technical field of coal gangue identification, in particular to a coal gangue detection and identification system, an intelligent coal discharging system and a training method of a model.
Background
The main objective of the fully-mechanized coal mining and caving process is to collect the coal seam which is not mined by a coal mining machine in a manual or electrohydraulic control mode, and the thickness of the coal seam which is not mined can not be completely detected, so that whether the coal seam which falls down is collected or not needs to be judged, namely, gangue outside the coal seam appears, and the coal caving process needs on-site manual observation and operation due to the lack of a coal rock identification technical means, so that automatic coal caving can not be realized. Because the dust of the working face is larger when fully mechanized coal mining is carried out than that of fully mechanized coal mining, the coal mining operation environment condition is bad, the coal gangue falling process is observed on the production site by manual visual inspection and hearing, the gangue is wanted to be accurately identified, and the difficulty is great. The key technology and equipment for intelligent coal discharge control do not break through, so that the coal discharge intelligent degree is low, and the resource recovery rate and the coal quality are difficult to ensure. The accurate control of the coal caving mechanism is a key link of the intellectualization of fully mechanized caving coal mining, and is a technical problem for restricting the fully mechanized caving work to be intelligent and unmanned. The current automation degree of the top coal caving process can only achieve memory coal caving and timing coal caving, most coal caving operations also completely depend on manpower, and the automation degree is low. In order to reduce manual intervention and realize automatic coal discharge, the fallen gangue and the gangue ratio need to be accurately identified.
At present, the method for identifying the gangue mainly comprises the steps of identifying the gangue based on gray convolution, and carrying out gray distribution, gray average value and other characteristic extraction on collected coal-rock image information to identify the shape of the coal rock during coal discharging. The underground coal discharging process can generate a large amount of dust, the simple gray convolution has poor robustness, and when dust exists, false recognition can be generated; in addition, only the gray level is single, the color information is lacked, and the gangue with color such as Huang Gan can not be identified. The other method is that the coal gangue identification in the coal discharging process is realized through an image processing means, and the image is classified by using a fixed algorithm model, so that the method has no function of coal gangue detection. In the method for classifying the images acquired in the coal caving process through the classification model, when the acquired images have coal mines and gangue, whether the acquired images are of the coal mine type or the gangue type cannot be determined, the robustness is poor, the sensitivity to local gangue is poor, and the gangue omission is easy to occur.
In conclusion, the existing coal gangue detection and identification has the technical problems of poor robustness and poor accuracy.
Disclosure of Invention
In view of the above, the invention aims to provide a coal gangue detection and identification system, an intelligent coal discharging system and a training method of a model, so as to solve the technical problems of poor robustness and poor accuracy of the existing coal gangue detection and identification.
In a first aspect, an embodiment of the present invention provides a coal gangue detection and identification system, including: an intelligent image pickup apparatus and an image learning server;
the intelligent camera equipment is arranged under a coal mine and used for collecting coal-discharging videos, and detecting and identifying coal gangue of coal-discharging image frames in the coal-discharging videos through a current coal gangue detection model to obtain gangue positions and coal mine positions in the coal-discharging image frames; determining unlabeled coal-releasing image data corresponding to the corresponding positions according to the gangue position and the coal mine position respectively;
the image learning server is configured to obtain target data, perform optimization training on the current gangue detection model by using the target data to obtain an optimized gangue detection model, and deploy the optimized gangue detection model to the intelligent image capturing device, so that the intelligent image capturing device performs next gangue position detection by using the optimized gangue detection model, where the target data includes: and the marked coal caving image data is obtained by marking the unmarked coal caving image data.
Further, the method further comprises the following steps: a data storage server and an intelligent decision server;
the intelligent camera equipment is also used for sending the unlabeled coal-discharging image data and the coal-discharging video to the data storage server;
the data storage server is used for independently storing the unlabeled coal caving image data, the coal caving video and the labeled coal caving image data sent by the intelligent camera equipment;
the intelligent decision server is used for acquiring the optimized training mode information and the target data storage information configured by the user and controlling the data storage server and the image learning server to execute corresponding tasks based on the optimized training mode information and the target data storage information;
the data storage server is further used for conveying the target data to the image learning server according to the target data storage information under the control of the intelligent decision server;
the image learning server is used for carrying out optimization training on the current coal gangue detection model by adopting the target data based on the optimization training mode information under the control of the intelligent decision server to obtain an optimized coal gangue detection model, and deploying the optimized coal gangue detection model to the intelligent camera equipment.
Further, the intelligent image pickup apparatus includes: the system comprises a shooting parameter configuration module, a video acquisition module, a coal gangue detection module, an image processing module, a video reading module, an alarm configuration module, a coal parking configuration module and a control module;
the image capturing parameter configuration module is configured to obtain image capturing parameters configured by a user, where the image capturing parameters at least include: exposure time, frame rate, and gain;
the video acquisition module is used for acquiring the coal discharging video under the shooting parameters;
the gangue detection module is used for detecting and identifying the gangue of the coal-caving image frames in the coal-caving video through the current gangue detection model to obtain gangue positions and coal mine positions in the coal-caving image frames;
the image processing module is used for carrying out image matting processing on the coal-caving image frame according to the gangue position and the coal mine position respectively to obtain unlabeled coal-caving image data, and sending the unlabeled coal-caving image data and the coal-caving video to the data storage server;
the video reading module is configured to perform video playing according to a function button triggered by a user, where the function button at least includes: video browsing and video playback;
The alarm configuration module is used for acquiring alarm triggering conditions configured by a user;
the coal parking configuration module is used for acquiring a coal parking stopping triggering condition configured by a user;
and the control module is used for controlling the alarm to alarm when the alarm triggering condition is reached and controlling the coal discharging mechanism to stop discharging coal when the coal discharging stopping triggering condition is reached.
Further, the intelligent decision server comprises: the system comprises an optimization training mode configuration module, a target data storage configuration module and a decision control module;
the optimal training mode configuration module is used for acquiring the optimal training mode information of the intelligent camera equipment configured by a user;
the target data storage configuration module is used for acquiring target data storage information of the intelligent camera equipment configured by a user;
and the decision control module is used for controlling the data storage server and the image learning server to execute corresponding tasks based on the optimized training mode information and the target data storage information.
Further, the image learning server includes: the system comprises a data dividing module, a training module, a model selecting module, a model quantifying module and a model deploying module;
The data dividing module is used for dividing the target data into a training set, a verification set and a test set;
the training module is used for training and verifying the current gangue detection model by adopting the training set and the verification set to obtain a plurality of verified gangue detection models;
the model selection module is used for testing the plurality of verified coal gangue detection models by adopting the test set, and determining the optimized coal gangue detection model from the plurality of verified coal gangue detection models according to a test result;
the model quantization module is used for quantizing the optimized coal gangue detection model to obtain a quantized optimized coal gangue detection model;
the model deployment module is used for deploying the quantized optimized gangue detection model to the intelligent camera equipment.
Further, the optimized training mode information at least includes: optimizing the training mode, the target data source and the iteration times, wherein when the optimizing training mode is an automatic optimizing training mode, the optimizing training mode information further comprises: automatically optimizing the training time interval and the confidence threshold; the target data storage information includes at least: the target data stores location information.
Further, when the optimization training mode is an automatic optimization training mode, the target data includes: the noted coal caving image data and the noted non-coal caving image data; when the optimized training mode is a manual optimized training mode, the target data is the marked coal-caving image data.
Further, the current gangue detection model is a variant network of an MTCNN network, including: p-net network, R-net network, O-net network and convolution layer.
In a second aspect, an embodiment of the present invention further provides an intelligent coal-discharging system, including: the coal gangue detection and identification system according to any one of the first aspect, further comprising: an alarm and a coal discharging mechanism;
the alarm is connected with the gangue detection and identification system and used for alarming under the control of the gangue detection and identification system;
the coal discharging mechanism is connected with the coal gangue detection and identification system and is used for executing coal parking action under the control of the coal gangue detection and identification system.
In a third aspect, an embodiment of the present invention further provides a training method of a gangue detection model, applying the image learning server according to any one of the first aspect, where the method includes:
Obtaining target data, and dividing the target data into a training set, a verification set and a test set, wherein when the optimized training mode is an automatic optimized training mode, the target data comprises: the method comprises the steps that marked coal-fired image data and unmarked coal-fired image data in a data storage server are marked, and when the optimized training mode is a manual optimized training mode, the target data are the marked coal-fired image data;
training the current gangue detection model by adopting the training set to obtain a plurality of trained gangue detection models;
verifying the plurality of trained gangue detection models by adopting the verification set to obtain a plurality of verified gangue detection models;
and testing the plurality of verified gangue detection models by adopting the test set, and determining an optimal gangue detection model from the plurality of verified gangue detection models according to a test result.
In an embodiment of the present invention, there is provided a coal gangue detection and identification system, including: an intelligent camera device and an intelligent decision server; the intelligent camera equipment is arranged under the coal mine and used for acquiring coal-caving videos, detecting and identifying coal gangue through a current coal gangue detection model on coal-caving image frames in the coal-caving videos to obtain gangue positions and coal mine positions in the coal-caving image frames, and determining unlabeled coal-caving image data corresponding to the corresponding positions according to the gangue positions and the coal mine positions; the image learning server is used for acquiring target data, carrying out optimization training on a current gangue detection model by utilizing the target data to obtain an optimized gangue detection model, and deploying the optimized gangue detection model to the intelligent camera equipment so that the intelligent camera equipment can carry out next gangue position detection by utilizing the optimized gangue detection model, wherein the target data comprises: the marked coal caving image data and/or the unmarked coal caving image data are obtained by marking the unmarked coal caving image data. According to the coal gangue detection and identification system disclosed by the invention, the coal gangue detection and identification can be carried out on the coal gangue image frames through the current coal gangue detection model, even if the coal gangue exists in the coal gangue image frames and the coal mine exists in the coal gangue image frames, the positions of the coal gangue and the coal mine can be identified through detection, and the robustness is good.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a coal gangue detection and identification system provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of another coal gangue detection and identification system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an intelligent camera device according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an intelligent decision server according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an image learning server according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of an intelligent coal-discharging system according to an embodiment of the present invention;
fig. 7 is a flowchart of a training method of a coal gangue detection model according to an embodiment of the present invention.
Icon: 1-a gangue detection and identification system; 2-an alarm; 3-a coal discharging mechanism; 11-an intelligent camera device; 12-an image learning server; 13-a data storage server; 14-an intelligent decision server; 111-a camera parameter configuration module; 112-a video acquisition module; 113-a gangue detection module; 114-an image processing module; 115-a video reading module; 116-an alarm configuration module; 117-park coal configuration module; 118-a control module; 141-optimizing a training mode configuration module; 142-a target data store configuration module; 143-a decision control module; 121-a data dividing module; 122-a training module; 123-a model selection module; 124-a model quantization module; 125-model deployment module.
Detailed Description
The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
At present, the method for identifying the gangue mainly comprises two methods of identifying the gangue based on gray convolution and classifying images acquired in the coal discharging process through a classification model, and the two methods have the technical problems of poor robustness and poor accuracy in the coal gangue identification process.
Based on the above, the embodiment provides a gangue detection and identification system, which can detect and identify gangue through a gangue detection model on a coal-discharge image frame, and can identify the gangue position and the coal mine position through detection even if gangue exists in the coal-discharge image frame and coal mine exists in the coal-discharge image frame.
Embodiments of the present invention are further described below with reference to the accompanying drawings.
Embodiment one:
the embodiment of the invention provides a coal gangue detection and identification system, and the coal gangue detection and identification system provided by the embodiment of the invention is specifically introduced below.
FIG. 1 is a schematic diagram of a gangue detection and identification system according to an embodiment of the present invention, as shown in FIG. 1, the system includes: an intelligent image pickup apparatus 11 and an image learning server 12;
the intelligent camera equipment 11 is arranged under a coal mine and is used for collecting coal-discharging videos, and coal gangue detection and identification are carried out on coal-discharging image frames in the coal-discharging videos through a current coal gangue detection model to obtain gangue positions and coal mine positions in the coal-discharging image frames; determining unlabeled coal-releasing image data corresponding to the corresponding positions according to the gangue position and the coal mine position respectively;
the image learning server 12 is configured to acquire target data, perform optimization training on a current gangue detection model by using the target data to obtain an optimized gangue detection model, and deploy the optimized gangue detection model to the intelligent image capturing device 11, so that the intelligent image capturing device 11 performs next gangue position detection by using the optimized gangue detection model, where the target data includes: the marked coal caving image data and/or the unmarked coal caving image data are obtained by marking the unmarked coal caving image data.
In the embodiment of the present invention, the intelligent camera configuration software is run in the intelligent camera device 11, and is used for configuring the camera parameters of the intelligent camera device 11.
In the traditional scheme, the coal-releasing images acquired in the coal releasing process are classified through the classification model, when coal mines and gangue exist in the acquired coal-releasing images, whether the acquired coal-releasing images are of the coal mine type or the gangue type cannot be determined, the robustness is poor, the sensitivity of the classification model to local gangue is poor, the gangue omission is easy to occur, the coal gangue detection and identification can be carried out on the coal-releasing image frames through the gangue detection model, and even if the gangue exists in the coal-releasing image frames and the coal mines exist, the position of the gangue and the position of the coal mine can be identified through detection, and the robustness is good.
In addition, the gangue detection model can be continuously optimized through the image learning server 12, so that the gangue position and the coal mine position detected by the gangue detection model are more accurate.
In an embodiment of the present invention, there is provided a coal gangue detection and identification system, including: an intelligent image pickup apparatus 11 and an intelligent decision server 14; the intelligent camera equipment 11 is arranged under a coal mine and used for acquiring coal-fired video, detecting and identifying coal gangue of coal-fired image frames in the coal-fired video through a current coal gangue detection model to obtain gangue positions and coal mine positions in the coal-fired image frames, and determining unlabeled coal-fired image data corresponding to the corresponding positions according to the gangue positions and the coal mine positions; the image learning server 12 is configured to acquire target data, perform optimization training on a current gangue detection model by using the target data to obtain an optimized gangue detection model, and deploy the optimized gangue detection model to the intelligent image capturing device 11, so that the intelligent image capturing device 11 performs next gangue position detection by using the optimized gangue detection model, where the target data includes: the marked coal caving image data and/or the unmarked coal caving image data are obtained by marking the unmarked coal caving image data. According to the coal gangue detection and identification system disclosed by the invention, the coal gangue detection and identification can be carried out on the coal gangue image frames through the current coal gangue detection model, even if the coal gangue exists in the coal gangue image frames and the coal mine exists in the coal gangue image frames, the positions of the coal gangue and the coal mine can be identified through detection, and the robustness is good.
The foregoing briefly describes the coal gangue detection and identification system of the present invention, and the detailed description thereof will be presented below.
In an alternative embodiment of the present invention, referring to fig. 2, the coal gangue detecting and identifying system further includes: a data storage server 13 and an intelligent decision server 14;
the intelligent camera device 11 is further used for sending unlabeled coal-caving image data and coal-caving video to the data storage server 13;
a data storage server 13 for independently storing the unlabeled coal-fired image data, the coal-fired video and the labeled coal-fired image data sent by the intelligent camera device 11;
the intelligent decision server 14 is used for acquiring the optimized training mode information and the target data storage information configured by the user, and controlling the data storage server 13 and the image learning server 12 to execute corresponding tasks based on the optimized training mode information and the target data storage information;
the data storage server 13 is further configured to, under control of the intelligent decision server 14, transmit the target data to the image learning server 12 according to the target data storage information;
the image learning server 12 is configured to perform optimization training on the current coal gangue detection model by using target data based on the optimization training mode information under the control of the intelligent decision server 14, obtain an optimized coal gangue detection model, and deploy the optimized coal gangue detection model to the intelligent camera device 11.
The intelligent decision server 14 runs intelligent decision and self-learning software for acquiring the optimized training mode information and the target data storage information configured by the user, and can control the image learning server 12 and the data storage server 13 simultaneously. The data storage server 13 runs intelligent data management software, can store and read data, and maintains the coal-caving video data, the marked coal-caving image data and the unmarked coal-caving image data.
The number of the intelligent image capturing apparatuses 11 may be one, two, or more, and when the number of the intelligent image capturing apparatuses 11 is more than one, the data storage server 13 stores the data transmitted from each intelligent image capturing apparatus 11 independently, for example, stores the data transmitted from the intelligent image capturing apparatus 11A in a folder named intelligent image capturing apparatus 11A, and stores the data transmitted from the intelligent image capturing apparatus 11B in a folder named intelligent image capturing apparatus 11B; when the number of intelligent image capturing apparatuses 11 is plural, the optimal training mode information and the target data storage information configured by the user on the intelligent decision server 14 are configured for each intelligent image capturing apparatus 11, for example, the optimal training mode in the optimal training mode information configured for the intelligent image capturing apparatus 11A is an automatic optimal training mode (may also be referred to as self-learning), the target data source is data collected by the intelligent image capturing apparatus 11A, the iteration number is 100, the time interval of the automatic optimal training is one day, the confidence threshold is 0.95, the target data storage information is the storage position of the data collected by the intelligent image capturing apparatus 11A, the optimal training mode in the optimal training mode information configured for the intelligent image capturing apparatus 11B is also an automatic optimal training mode, the target data source is the data collected by the intelligent image capturing apparatus 11A and the data collected by the intelligent image capturing apparatus 11B, the iteration number is 200, the time interval of the automatic optimal training is one month, the confidence threshold is 0.97, the target data storage information is the storage position of the data collected by the intelligent image capturing apparatus 11A and the intelligent image capturing apparatus 11B, and so on.
It should be noted that, when the data storage server 13 transmits the target data to the image learning server 12 according to the target data storage information under the control of the intelligent decision server 14, the intelligent decision server 14 may acquire the target data from the data storage server 13 according to the target data storage information, and then transmit the acquired target data to the image learning server 12; the image learning server 12 may also read the target data from the data storage server 13 according to the target data storage information; the data storage server 13 may also be configured to transmit the target data to the image learning server 12 according to the target data storage information under the control of the intelligent decision server 14, and the above-described process is not particularly limited in the embodiment of the present invention.
In an alternative embodiment of the present invention, referring to fig. 3, the smart camera apparatus 11 includes: the system comprises a shooting parameter configuration module 111, a video acquisition module 112, a coal gangue detection module 113, an image processing module 114, a video reading module 115, an alarm configuration module 116, a coal parking configuration module 117 and a control module 118;
the image capturing parameter configuration module 111 is configured to obtain image capturing parameters configured by a user, where the image capturing parameters at least include: exposure time, frame rate, and gain;
The video acquisition module 112 is used for carrying out coal caving video acquisition under the shooting parameters;
the gangue detection module 113 is used for performing gangue detection and identification on the coal-caving image frames in the coal-caving video through the current gangue detection model to obtain gangue positions and coal mine positions in the coal-caving image frames;
the image processing module 114 is configured to perform a matting process on the coal-caving image frame according to the gangue position and the coal mine position, obtain unlabeled coal-caving image data, and send the unlabeled coal-caving image data and the coal-caving video to the data storage server 13;
the video reading module 115 is configured to perform video playing according to a function button triggered by a user, where the function button at least includes: video browsing and video playback;
an alarm configuration module 116, configured to obtain an alarm trigger condition configured by a user;
a coal parking configuration module 117, configured to obtain a coal parking stopping trigger condition configured by a user;
the control module 118 is used for controlling the alarm to alarm when the alarm triggering condition is reached and controlling the coal discharging mechanism to stop discharging coal when the coal discharging stopping triggering condition is reached.
Specifically, whether the detected gangue and coal mine are accurate or not can be observed through the video reading module 115. In addition, the alarm triggering condition and the coal discharging stopping triggering condition can be that when the number of the gangue reaches a certain value, an alarm is triggered or coal discharging is stopped; or triggering an alarm or triggering stopping coal discharge after the proportion of the gangue to the coal mine reaches a certain value.
In an alternative embodiment of the present invention, referring to FIG. 4, the intelligent decision server 14 comprises: an optimization training mode configuration module 141, a target data storage configuration module 142, and a decision control module 143;
the optimal training mode configuration module 141 is configured to obtain optimal training mode information of the intelligent camera device 11 configured by the user;
a target data storage configuration module 142 for acquiring target data storage information of the intelligent image capturing apparatus 11 configured by the user;
the decision control module 143 is configured to control the data storage server 13 and the image learning server 12 to perform corresponding tasks based on the optimized training mode information and the target data storage information.
In addition, the intelligent decision server 14 can also configure the number of intelligent image capturing apparatuses 11 (for example, which intelligent image capturing apparatuses 11 are connected to and the communication mode with each intelligent image capturing apparatus 11), the status (for example, on-line status, off-line status, etc.), the intelligent image capturing apparatuses 11 to be upgraded (that is, which intelligent image capturing apparatuses 11 need to be optimized again for the gangue detection model), and the like.
In an alternative embodiment of the present invention, referring to fig. 5, the image learning server 12 includes: a data partitioning module 121, a training module 122, a model selection module 123, a model quantization module 124, and a model deployment module 125;
A data dividing module 121 for dividing the target data into a training set, a verification set and a test set;
the training module 122 is configured to train and verify the current gangue detection model by using a training set and a verification set, so as to obtain a plurality of verified gangue detection models;
the model selecting module 123 is configured to test the plurality of verified gangue detection models by using a test set, and determine an optimized gangue detection model from the plurality of verified gangue detection models according to a test result;
the model quantization module 124 is configured to quantize the optimized coal gangue detection model to obtain a quantized optimized coal gangue detection model;
the model deployment module 125 is configured to deploy the quantized optimized gangue detection model to the intelligent camera device 11. Specifically, when the intelligent image pickup apparatus 11 is in an idle state, the optimized gangue detection model is deployed to the intelligent image pickup apparatus 11.
Specifically, the image learning server 12 has a plurality of GPUs, which have a relatively strong parallel computing capability, and the plurality of GPUs simultaneously train the current gangue detection model and select and quantize the model.
Specifically, the learning mode information at least includes: the optimized training mode information at least comprises: optimizing the training mode, the target data source and the iteration times, wherein when the optimizing training mode is an automatic optimizing training mode, the optimizing training mode information further comprises: automatically optimizing the training time interval and the confidence threshold; the target data storage information includes at least: target data storage location information; when the optimization training mode is an automatic optimization training mode, the target data includes: marked coal-releasing image data and unmarked coal-releasing image data; when the optimized training mode is a manual optimized training mode, the target data is marked coal-caving image data.
The process of manual optimization training and automatic optimization training is described in detail below:
when the optimized training mode is a manual optimized training mode, a user can label the unlabeled coal-releasing image data, after the labeling is completed, the data amount of the labeled coal-releasing image data is increased, the labeled coal-releasing image data is the target data, and then the target data (labeled coal-releasing image data) is divided into a training set, a verification set and a test set, and the current coal gangue detection model is trained and optimized.
When the optimized training mode is an automatic optimized training mode, unlabeled coal-releasing image data is detected through a current coal gangue detection model, so that category information of the unlabeled coal-releasing image data and confidence of the category information are obtained, the coal-releasing image data with the confidence greater than a confidence threshold is added into target data (comprising original labeled coal-releasing image data and new added data), then the current coal gangue detection model is trained through the target data, after training is completed, the rest unlabeled coal-releasing image data with lower confidence is detected through the newly trained coal gangue detection model, a new detection result is obtained, the coal-releasing image data with the confidence greater than a confidence threshold in the new detection result is added into the target data, then the last trained coal gangue detection model is trained through the target data, and the circulation is performed until the rest unlabeled coal-releasing image data with the confidence greater than the confidence threshold does not exist in the detection result when the last obtained coal gangue detection model detects the rest unlabeled coal-releasing image data.
The user may set a time interval of the automatic optimization training in the automatic optimization training manner, for example, once a day, once a month, etc., and when the time of the optimization training is reached, the operation of the automatic optimization training is performed.
Under the automatic optimization training mode, the user can label the unlabeled coal-fired image data, so that the labeled coal-fired image data is expanded, and the model has higher accuracy.
In addition, a fixed model may also be used, and when the user selects to use the fixed model, the current gangue detection model of the intelligent image pickup apparatus 11 is kept unchanged by default.
In an alternative embodiment of the present invention, the current gangue detection model is a variant network of the MTCNN network, comprising: p-net network, R-net network, O-net network and convolution layer.
Specifically, the gangue detection model uses a cascade classification module to perform progressive investigation on the gangue, uses a variant network of an MTCNN network, and comprises the following steps: the P-net network, the R-net network, the O-net network and the two-layer convolution layer enable the P-net network, the R-net network, the O-net network and the two-layer convolution layer to have stronger classification capability. Although the gangue with different coals has some differences, the bottom layer characteristics have little difference, so the automatic optimization training mode achieves the purposes of model adjustment and self-learning only by updating the network parameters of the O-net.
The gangue detection and identification system has the following advantages:
(1) The underground environment is identified, the coal gangue identification capacity is higher, the coal discharging process can be better controlled, and the waste caused by the premature stopping of coal discharging and the quality degradation of coal caused by the overlong coal discharging time are prevented;
(2) The gangue detection model has better robustness and an automatic optimization training function, and can be optimized in continuous automatic optimization training, so that the gangue detection model can adapt to more coal environments;
(3) The method has a more convenient model updating mode, if the detection effect of the model is poor through the video reading module of the intelligent camera equipment, the model effect can be improved through manual labeling, and the model is convenient to improve and maintain.
The invention provides a coal gangue detection and identification system based on a semi-supervised neural network, which can accurately identify the gangue and the gangue ratio in the coal discharging process, and realize data batch reflux storage through coordination of software and hardware, thereby realizing self-learning and self-training modes, improving identification capacity, adapting to the change of coal quality and providing a data basis for accurate control of a coal discharging mechanism.
Embodiment two:
the embodiment of the invention provides an intelligent coal caving system, and the intelligent coal caving system provided by the embodiment of the invention is specifically introduced below.
FIG. 6 is a schematic diagram of an intelligent coal caving system according to an embodiment of the invention, as shown in FIG. 6, comprising: the gangue detection and identification system 1 in the first embodiment further includes: an alarm 2 and a coal discharging mechanism 3;
the alarm 2 is connected with the gangue detection and identification system 1 and is used for alarming under the control of the gangue detection and identification system 1;
the coal discharging mechanism 3 is connected with the coal gangue detecting and identifying system 1 and is used for executing coal parking action under the control of the coal gangue detecting and identifying system 1.
Embodiment III:
according to an embodiment of the present invention, there is provided an embodiment of a training method of a coal gangue detection model, it should be noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that herein.
FIG. 7 is a flowchart of a training method of a gangue detection model according to an embodiment of the present invention, as shown in FIG. 7, the method includes the following steps:
step S701, obtaining target data, and dividing the target data into a training set, a verification set and a test set, where when the optimized training mode is an automatic optimized training mode, the target data includes: the method comprises the steps that marked coal-fired image data and unmarked coal-fired image data in a data storage server are marked, and when an optimal training mode is a manual optimal training mode, target data are marked coal-fired image data;
Step S702, training a current gangue detection model by adopting a training set to obtain a plurality of trained gangue detection models;
step S703, verifying a plurality of trained gangue detection models by using a verification set to obtain a plurality of verified gangue detection models;
step S704, testing the plurality of verified gangue detection models by adopting a test set, and determining an optimal gangue detection model from the plurality of verified gangue detection models according to a test result.
Optionally, after determining an optimal gangue detection model from the plurality of validated gangue detection models according to the test result, the method further comprises: quantizing the optimal gangue detection model to obtain a quantized optimal gangue detection model; and deploying the quantized optimal gangue detection model to intelligent camera equipment.
The computer program product of the coal gangue detection and identification system, the intelligent coal caving system and the training method of the model provided by the embodiment of the invention comprises a computer readable storage medium storing program codes, wherein the instructions included in the program codes can be used for executing the method described in the method embodiment, and specific implementation can be seen in the method embodiment and is not repeated here.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.
In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (10)
1. A coal gangue detection and identification system, comprising: an intelligent image pickup apparatus and an image learning server;
the intelligent camera equipment is arranged under a coal mine and used for collecting coal-discharging videos, and detecting and identifying coal gangue of coal-discharging image frames in the coal-discharging videos through a current coal gangue detection model to obtain gangue positions and coal mine positions in the coal-discharging image frames; carrying out image matting processing on the coal caving image frame according to the gangue position and the coal mine position respectively to obtain unlabeled coal caving image data;
the image learning server is configured to obtain target data, perform optimization training on the current gangue detection model by using the target data to obtain an optimized gangue detection model, and deploy the optimized gangue detection model to the intelligent image capturing device, so that the intelligent image capturing device performs next gangue position detection by using the optimized gangue detection model, where the target data includes: and the marked coal caving image data is obtained by marking the unmarked coal caving image data.
2. The coal gangue detection and identification system of claim 1, further comprising: a data storage server and an intelligent decision server;
the intelligent camera equipment is also used for sending the unlabeled coal-discharging image data and the coal-discharging video to the data storage server;
the data storage server is used for independently storing the unlabeled coal caving image data, the coal caving video and the labeled coal caving image data sent by the intelligent camera equipment;
the intelligent decision server is used for acquiring the optimized training mode information and the target data storage information configured by the user and controlling the data storage server and the image learning server to execute corresponding tasks based on the optimized training mode information and the target data storage information;
the data storage server is further used for conveying the target data to the image learning server according to the target data storage information under the control of the intelligent decision server;
the image learning server is used for carrying out optimization training on the current coal gangue detection model by adopting the target data based on the optimization training mode information under the control of the intelligent decision server to obtain an optimized coal gangue detection model, and deploying the optimized coal gangue detection model to the intelligent camera equipment.
3. The coal gangue detection and identification system of claim 2, wherein the intelligent camera device comprises: the system comprises a shooting parameter configuration module, a video acquisition module, a coal gangue detection module, an image processing module, a video reading module, an alarm configuration module, a coal parking configuration module and a control module;
the image capturing parameter configuration module is configured to obtain image capturing parameters configured by a user, where the image capturing parameters at least include: exposure time, frame rate, and gain;
the video acquisition module is used for acquiring the coal discharging video under the shooting parameters;
the gangue detection module is used for detecting and identifying the gangue of the coal-caving image frames in the coal-caving video through the current gangue detection model to obtain gangue positions and coal mine positions in the coal-caving image frames;
the image processing module is used for carrying out image matting processing on the coal-caving image frame according to the gangue position and the coal mine position respectively to obtain unlabeled coal-caving image data, and sending the unlabeled coal-caving image data and the coal-caving video to the data storage server;
the video reading module is configured to perform video playing according to a function button triggered by a user, where the function button at least includes: video browsing and video playback;
The alarm configuration module is used for acquiring alarm triggering conditions configured by a user;
the coal parking configuration module is used for acquiring a coal parking stopping triggering condition configured by a user;
and the control module is used for controlling the alarm to alarm when the alarm triggering condition is reached and controlling the coal discharging mechanism to stop discharging coal when the coal discharging stopping triggering condition is reached.
4. The coal gangue detection and identification system of claim 2, wherein the intelligent decision server comprises: the system comprises an optimization training mode configuration module, a target data storage configuration module and a decision control module;
the optimal training mode configuration module is used for acquiring the optimal training mode information of the intelligent camera equipment configured by a user;
the target data storage configuration module is used for acquiring target data storage information of the intelligent camera equipment configured by a user;
and the decision control module is used for controlling the data storage server and the image learning server to execute corresponding tasks based on the optimized training mode information and the target data storage information.
5. The coal gangue detection and identification system according to claim 2, wherein the image learning server comprises: the system comprises a data dividing module, a training module, a model selecting module, a model quantifying module and a model deploying module;
The data dividing module is used for dividing the target data into a training set, a verification set and a test set;
the training module is used for training and verifying the current gangue detection model by adopting the training set and the verification set to obtain a plurality of verified gangue detection models;
the model selection module is used for testing the plurality of verified coal gangue detection models by adopting the test set, and determining the optimized coal gangue detection model from the plurality of verified coal gangue detection models according to a test result;
the model quantization module is used for quantizing the optimized coal gangue detection model to obtain a quantized optimized coal gangue detection model;
the model deployment module is used for deploying the quantized optimized gangue detection model to the intelligent camera equipment.
6. The coal gangue detection and identification system according to claim 1, wherein the optimized training pattern information at least comprises: optimizing the training mode, the target data source and the iteration times, wherein when the optimizing training mode is an automatic optimizing training mode, the optimizing training mode information further comprises: automatically optimizing the training time interval and the confidence threshold; the target data storage information includes at least: the target data stores location information.
7. The coal gangue detection and identification system of claim 1, wherein when the optimal training mode is an automatic optimal training mode, the target data comprises: the noted coal caving image data and the noted non-coal caving image data; when the optimized training mode is a manual optimized training mode, the target data is the marked coal-caving image data.
8. The coal gangue detection and identification system of claim 1, wherein the current coal gangue detection model is a variant network of an MTCNN network, comprising: p-net network, R-net network, O-net network and convolution layer.
9. An intelligent coal caving system, comprising: the coal gangue detection and identification system of any one of claims 1 to 8 further comprising: an alarm and a coal discharging mechanism;
the alarm is connected with the gangue detection and identification system and used for alarming under the control of the gangue detection and identification system;
the coal discharging mechanism is connected with the coal gangue detection and identification system and is used for executing coal parking action under the control of the coal gangue detection and identification system.
10. A training method of a coal gangue detection model, applied to the image learning server of any one of claims 1 to 8, the method comprising:
Obtaining target data, and dividing the target data into a training set, a verification set and a test set, wherein when the optimized training mode is an automatic optimized training mode, the target data comprises: the method comprises the steps that marked coal-fired image data and unmarked coal-fired image data in a data storage server are marked, and when the optimized training mode is a manual optimized training mode, the target data are the marked coal-fired image data;
training the current gangue detection model by adopting the training set to obtain a plurality of trained gangue detection models;
verifying the plurality of trained gangue detection models by adopting the verification set to obtain a plurality of verified gangue detection models;
and testing the plurality of verified gangue detection models by adopting the test set, and determining an optimal gangue detection model from the plurality of verified gangue detection models according to a test result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110391898.XA CN112991438B (en) | 2021-04-12 | 2021-04-12 | Coal gangue detection and identification system, intelligent coal discharging system and training method of model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110391898.XA CN112991438B (en) | 2021-04-12 | 2021-04-12 | Coal gangue detection and identification system, intelligent coal discharging system and training method of model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112991438A CN112991438A (en) | 2021-06-18 |
| CN112991438B true CN112991438B (en) | 2023-05-12 |
Family
ID=76338023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110391898.XA Active CN112991438B (en) | 2021-04-12 | 2021-04-12 | Coal gangue detection and identification system, intelligent coal discharging system and training method of model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112991438B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113680709A (en) * | 2021-08-27 | 2021-11-23 | 重庆科华安全设备有限责任公司 | Coal and gangue sorting system and identification method |
| CN115527154A (en) * | 2022-10-21 | 2022-12-27 | 西安科技大学 | Precise closing control method for top coal caving port based on dynamic identification of coal gangue |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110441320A (en) * | 2019-08-05 | 2019-11-12 | 北京泰豪信息科技有限公司 | A kind of gangue detection method, apparatus and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170277955A1 (en) * | 2016-03-23 | 2017-09-28 | Le Holdings (Beijing) Co., Ltd. | Video identification method and system |
| CN111709935B (en) * | 2020-06-17 | 2023-04-07 | 西安科技大学 | Real-time coal gangue positioning and identifying method for ground moving belt |
| CN111881909A (en) * | 2020-07-27 | 2020-11-03 | 精英数智科技股份有限公司 | Coal and gangue identification method and device, electronic equipment and storage medium |
| CN112558577B (en) * | 2021-02-19 | 2021-06-08 | 天津美腾科技股份有限公司 | Caving coal control method, device, electronic equipment and computer readable storage medium |
-
2021
- 2021-04-12 CN CN202110391898.XA patent/CN112991438B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110441320A (en) * | 2019-08-05 | 2019-11-12 | 北京泰豪信息科技有限公司 | A kind of gangue detection method, apparatus and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112991438A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112991438B (en) | Coal gangue detection and identification system, intelligent coal discharging system and training method of model | |
| CN110390691B (en) | Ore dimension measuring method based on deep learning and application system | |
| JP4744918B2 (en) | Face detection method, apparatus, and program | |
| US20160098636A1 (en) | Data processing apparatus, data processing method, and recording medium that stores computer program | |
| CN111626201B (en) | Commodity detection method, commodity detection device and readable storage medium | |
| CN110378869A (en) | A kind of rail fastening method for detecting abnormality of sample automatic marking | |
| CN104954741A (en) | Tramcar on-load and no-load state detecting method and system based on deep-level self-learning network | |
| CN110458794B (en) | Quality detection method and device for accessories of rail train | |
| TW201906038A (en) | Semiconductor wafer analyzing system and method thereof | |
| CN110567969A (en) | Image identification method and system for fabric defect detection | |
| CN105426894A (en) | Image detection method and device for railroad plug nails | |
| CN115408925A (en) | Rock mass parameter prediction method and device for tunnel construction | |
| CN103942576B (en) | A kind of method that multiple dimensioned random character in use spatial domain recognizes coal petrography | |
| CN111488939A (en) | Model training method, classification method, device and equipment | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| KR102230559B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| TWM550465U (en) | Semiconductor wafer analyzing system | |
| CN112990350B (en) | Target detection network training method and target detection network-based coal and gangue identification method | |
| CN111563886A (en) | Tunnel steel rail surface disease detection method and device based on unsupervised feature learning | |
| US20220343113A1 (en) | Automatic model reconstruction method and automatic model reconstruction system for component recognition model | |
| CN115690514A (en) | Image recognition method and related equipment | |
| KR102342495B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN114005089A (en) | Multi-scene construction safety helmet and reflective clothes detection method | |
| CN116630752B (en) | Construction site target object identification method and device based on AI algorithm | |
| CN113505691B (en) | Coal rock identification method and identification credibility indication method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230725 Address after: 300467 Binhai Tianjin Binhai Zhongxin Ecological City West of Zhongcheng Avenue, South of Zhongbin Avenue Ecological Construction Apartment Building 8, Room 137, Floor 1 Patentee after: TIANJIN ZHONGXIN ZHIGUAN INFORMATION TECHNOLOGY CO.,LTD. Address before: Room 137, 1 / F, building 8, ecological construction apartment, south of Zhongbin Avenue, Zhongxin ecological city, Binhai New Area, Tianjin 300450 Patentee before: Tianjin Meiteng Technology Co.,Ltd. |