CN113158829A - Deep learning ore size measuring method and early warning system based on EfficientDet network - Google Patents

Deep learning ore size measuring method and early warning system based on EfficientDet network Download PDF

Info

Publication number
CN113158829A
CN113158829A CN202110343840.8A CN202110343840A CN113158829A CN 113158829 A CN113158829 A CN 113158829A CN 202110343840 A CN202110343840 A CN 202110343840A CN 113158829 A CN113158829 A CN 113158829A
Authority
CN
China
Prior art keywords
ore
network
size
efficientdet
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110343840.8A
Other languages
Chinese (zh)
Inventor
段章领
周行云
盛一帆
朱明杰
徐岳
杨富超
胡倩凝
汪志敏
马腾
张馨雨
周明祎
熊天乐
潘悦靓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui University
Original Assignee
Anhui University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui University filed Critical Anhui University
Priority to CN202110343840.8A priority Critical patent/CN113158829A/en
Publication of CN113158829A publication Critical patent/CN113158829A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

The invention discloses a deep learning ore measuring method based on an EfficientDet network and an application system, wherein the method comprises the following steps: acquiring an ore image on a flowing belt through a high frame rate camera; preprocessing an ore image: marking all ores in the image, removing abnormal data, and performing data enhancement on the image; the labeled samples were weighed as 7: 2: 1, dividing the training set into a training set, a verification set and a test set; training by using an EfficientDet network to obtain a network model; positioning the ore discharging position by using a prediction frame obtained by an EfficientDet network model during testing, and calculating the size of the ore through the focal length of a camera and the size of an image pixel; and sending out a prompt when the ore size exceeds the system threshold value according to the preset threshold value of the system. The invention can efficiently detect the size of the ore, uses fewer parameters and has higher detection speed compared with other network models, thereby greatly reducing the dependence on manpower.

Description

Deep learning ore size measuring method and early warning system based on EfficientDet network
Technical Field
The invention relates to an image target identification method, in particular to a deep learning ore size measuring method and an early warning system based on an EfficientDet network.
Background
Mineral resources refer to aggregates which have industrial values for the contents of minerals or useful elements buried underground or exposed on the surface of the earth through geological mineralization. Mineral resources are important natural resources and important material bases for social production development, and the mineral resources cannot be separated from the production and the life of people in the modern society. Mineral resources are non-renewable resources, and their reserves are limited. The extent and depth of utilization of mineral resources are to be increased. According to their characteristics and uses, they are generally classified into three categories, namely metal minerals, non-metal minerals and energy minerals. The metal ore is mined in an explosion mode, and when uneven ore is conveyed to a primary crusher through a belt conveyor, if the size of the ore is too large, the crusher body is easy to damage. At present, the main manual work is used for monitoring and screening the large ores so as to reduce the damage to the crusher body. The manual screening problem mainly has two points: firstly, the safety problem is that the lifting well belt conveyor has a severe environment and serious dust and noise, which damages the body of workers for a long time, and secondly, the manual monitoring causes visual fatigue and easy omission condition when working for a long time, which causes damage to the crusher body. Some developed countries abroad have already applied the relevant knowledge of machine vision to the mining of ores to improve the efficiency, while the relevant knowledge of machine vision is used less at present in China. This patent adopts the relevant knowledge of degree of depth learning field machine vision, constructs the EfficientDet detection network, realizes the real-time detection of all ore blocks, fixes a position its, discerns its size to carry out real-time early warning when meetting the ore block that exceeds system threshold value size, inform the relevant condition of belt control system, make the belt in time stop. Meanwhile, the abnormal signal is transmitted to the actuating mechanism, and the actuating mechanism pulls out the large ore. The manual work use has been reduced in a large number, has improved detection efficiency when guaranteeing workman's safety, greatly reduces the condition of lou examining. The model efficiency is very important in computer vision, and compared with the prior deep learning network such as Mask R-CNN and the like, the EfficientDet network can achieve higher precision by using fewer parameters and lower computation amount so as to improve the detection efficiency.
In conclusion, the traditional ore size detection needs a large amount of manual work to carry out the manual operation problem, the detection parameter quantity of the model based on the neural network is too large, and the technical problem of strong calculation power is needed.
Disclosure of Invention
For the problems in the prior art, the method for measuring the ore size through deep learning based on the EfficientDet network and the early warning system are provided, and the technical problems that manual work is greatly relied on in the ore size detection, the efficiency is low, the accuracy is low, the model parameter quantity is overlarge, and strong calculation force is needed are solved. The method specifically comprises the following steps: capturing video streams of ore transported on a belt through a high frame rate camera, converting the video streams into pictures, manually marking the images, and removing abnormal ore pictures; and (3) pressing the obtained ore picture according to the following steps of 7: 2: 1, dividing the picture into a training picture, a verification picture and a test picture; the method comprises the steps of carrying out data enhancement on an ore picture to improve generalization capability, using an EfficientNet network as a backbone, creating an EfficientDet target detection network, using a training data set to obtain a network model, using the trained EfficientDet network model for testing, using an obtained prediction frame for positioning the position of ore discharge and calculating the size of the ore through the focal length of a camera and the size of image pixels. According to a preset threshold value of the system, when the ore size exceeds the threshold value of the system, an early warning is sent out, meanwhile, an abnormal signal is transmitted to an execution mechanism, and the execution mechanism dials out a large ore.
The invention adopts the following technical scheme to solve the technical problems: a deep learning ore size measuring method and an early warning system based on an EfficientDet network are used for detecting the size of ore in the mining industry and intelligently processing ore blocks exceeding a threshold size, and the method comprises the following specific steps:
s1, data acquisition stage: shooting ores on a flowing belt through a high-speed camera, and storing key frames of a video stream as pictures;
s2, data preprocessing stage: manually labeling the image, detecting and removing abnormal ore pictures; and (3) pressing the obtained ore picture according to the following steps of 7: 2: 1, dividing the picture into a training picture, a verification picture and a test picture; enhancing the data of the ore picture to improve the generalization ability;
s3, network creating and training stage: using an EffectientNet network as a backbone network, using a BiFPN (bi-directional feature pyramid network) as a feature extraction network, selecting one of EffectientD 0-D7 to create an EffectientDet target detection network, setting parameters such as learning rate, Batch size, training round number, optimizer and the like, and obtaining a network model by using an ore training data set;
s4, testing: testing by using the trained EfficientDet network model, using the obtained prediction frame for positioning the position of ore discharge and calculating the size of the ore through the focal length of the camera and the size of the image;
s5, reminding: according to a preset threshold value of the system, when the ore size exceeds the threshold value of the system, an early warning is sent out, meanwhile, an abnormal signal is transmitted to an execution mechanism, and the execution mechanism dials out a large ore.
The above scheme is further described:
the data acquisition in step S1 includes the following steps:
(1) acquiring ore picture video streams through a plurality of high frame rate cameras with different angles, which are arranged near a belt;
(2) intercepting key frames from the acquired picture video stream as image data;
the data preprocessing in the step S2 includes the following steps:
(i) marking the position and the size of the ore in the obtained picture data by using an image marking tool Lableme, wherein the size of the ore is irregular and needs to be marked in a polygonal manner;
(ii) detecting ore picture data which do not meet the requirements, and rejecting the ore picture data; the method mainly comprises the steps of detecting whether marked data exceed the abnormal condition of the boundary of a picture or not and the condition that the coordinate position is reversed;
(iii) performing data enhancement on the training picture: randomly carrying out horizontal and vertical reversal at different angles on the ore picture; zooming the ore picture; adjusting pixel values of the image so that the values thereof become uniformly distributed through histogram equalization; adding random noise; converting the image from an RGB color space to an HSV color space, adjusting the brightness of the image, normalizing the image, and processing noise; therefore, the generalization capability of the network is improved.
The step S3 of creating and training a network stage is composed of the following steps:
using an EfficientNet network as a backbone network, fusing network characteristics extracted by the EfficientNet by adopting a BiFPN network, and classifying and performing regression prediction on the extracted characteristics by the Head of the EfficientDet;
(II) the entire EfficientNet has multiple versions of B0-B7, wherein EfficientNet-B0 consists of 1 Conv (3 × 3), 1 MBConv1(3 × 3), 2 MBConv6(3 × 3), 2 MBConv6(5 × 5), 3 MBConv6(3 × 3), 3 MBConv6(5 × 5), 4 MBConv6(5 × 5), one MBConv6(3 × 3), one Conv (1 × 1), one Pooling layer, and one FC layer. Wherein MBConv contains residual structures. The dimension increasing operation is performed by using the convolution of 1 × 1, then the convolution of 3 × 3 or 5 × 5 is performed, then the attention mechanism about the channel is added, and the dimension reducing operation is performed by using the convolution of 1 × 1, and then the residual structure is stacked. The activation function of MBConv uses the Swish function and is normalized using Batch Normalization;
the definition of Swish function is
Figure 100002_DEST_PATH_IMAGE001
Wherein
Figure 100002_DEST_PATH_IMAGE002
Is a constant or trainable parameter;
Figure 365017DEST_PATH_IMAGE003
the function is expressed as follows:
Figure 100002_DEST_PATH_IMAGE004
meanwhile, the EfficientNet-B0 uses the coefficients of the width (depth), the depth (width) and the resolution (resolution) of the network according to the formula X (TODO)
Figure 100002_DEST_PATH_IMAGE005
And performing composite scaling. Where α, β, γ are constants that can be determined by grid search. In that
Figure 100002_DEST_PATH_IMAGE006
And
Figure 891945DEST_PATH_IMAGE007
under the constraint of (3), the optimum value of EfficientNet-B0 is
Figure 100002_DEST_PATH_IMAGE008
=1.2，
Figure 609365DEST_PATH_IMAGE002
=1.1，
Figure 196118DEST_PATH_IMAGE009
=1.15。
Figure 100002_DEST_PATH_IMAGE010
The step S3 of creating a BiFPN includes the following steps:
(A) the BiFPN network can learn the importance from different input features while repeatedly applying top-down and bottom-up multi-size feature fusion. The enhanced extraction network is composed of a plurality of BiFPNs, wherein the EfficientDet-D0-D7 are respectively composed of 3, 4, 5, 6, 7, 8 and 8 BiFPNs;
(B) the EfficientDet comprises an EfficientNet backbone extraction network and a BiFPN enhanced extraction network, and the EfficientHead converts the extracted features into a prediction result. Firstly, the EfficientNet continuously performs downsampling on an input picture, the downsampling frequency of the original EfficientNet is 5 times, and the input picture can be obtained through the EfficientNet
Figure 682594DEST_PATH_IMAGE011
、
Figure 100002_DEST_PATH_IMAGE012
、
Figure 442739DEST_PATH_IMAGE013
、
Figure 100002_DEST_PATH_IMAGE014
、
Figure 647456DEST_PATH_IMAGE015
，
Figure 315197DEST_PATH_IMAGE011
Is the result of compressing the input picture length and width once,
Figure 452918DEST_PATH_IMAGE012
is the result of two compressions of the input picture length and width,
Figure 649544DEST_PATH_IMAGE013
is the result of three compressions of the input picture length and width, and so on. Due to the fact that
Figure 872715DEST_PATH_IMAGE011
And
Figure 78568DEST_PATH_IMAGE012
does not have high semantic information, so the method is not used in the reinforced extraction network BiFPN
Figure 70795DEST_PATH_IMAGE011
、
Figure 969481DEST_PATH_IMAGE012
、
Figure 679948DEST_PATH_IMAGE013
、
Figure 158334DEST_PATH_IMAGE014
、
Figure 270646DEST_PATH_IMAGE015
Has high semantic information, and is used in the reinforced extraction network BiFPN as three of 5 effective feature layers. With two downsamplings of P5 to obtain information with higher semantics
Figure 100002_DEST_PATH_IMAGE016
、
Figure 543496DEST_PATH_IMAGE017
. Obtaining 5 effective characteristic layers
Figure 741259DEST_PATH_IMAGE013
、
Figure 288915DEST_PATH_IMAGE014
、
Figure 255734DEST_PATH_IMAGE015
、
Figure 496222DEST_PATH_IMAGE016
、
Figure 915702DEST_PATH_IMAGE017
；
(C) Transmitting the 5 feature layers into an enhanced extraction network BiFPN for further feature extraction, wherein
Figure 100002_DEST_PATH_IMAGE018
Representing resolution as an input image
Figure 470312DEST_PATH_IMAGE019
The feature level of (c). For example, if the input resolution is 640x640,
Figure DEST_PATH_IMAGE020
represents a characteristic level 3(640 ^) with a resolution of 80x80
Figure 760479DEST_PATH_IMAGE021
= 80), and
Figure 100002_DEST_PATH_IMAGE022
represents a 7-level feature level representing a resolution of 5x 5; the method comprises the following specific steps:
firstly, adjusting the number of channels to obtain
Figure 640710DEST_PATH_IMAGE023
，
Figure 100002_DEST_PATH_IMAGE024
，
Figure 16328DEST_PATH_IMAGE025
，
Figure 100002_DEST_PATH_IMAGE026
，
Figure 374628DEST_PATH_IMAGE027
If entering BiFPN for the first time, will
Figure 50460DEST_PATH_IMAGE024
By reducing the number of channels
Figure 632751DEST_PATH_IMAGE024
_1 and
Figure 26823DEST_PATH_IMAGE024
_2，
Figure 719973DEST_PATH_IMAGE025
by reducing the passage to
Figure 984732DEST_PATH_IMAGE025
_1 and
Figure 737924DEST_PATH_IMAGE025
_2；
is obtaining
Figure 619293DEST_PATH_IMAGE023
、
Figure 116133DEST_PATH_IMAGE024
_1、
Figure 766557DEST_PATH_IMAGE024
_2、
Figure 425072DEST_PATH_IMAGE025
_1、
Figure 793736DEST_PATH_IMAGE025
_2、
Figure 563109DEST_PATH_IMAGE026
、
Figure 599198DEST_PATH_IMAGE027
Then need to be paired
Figure 428614DEST_PATH_IMAGE027
Performing upsampling, and performing upsampling
Figure 18995DEST_PATH_IMAGE026
Using attention mechanism to judge whether it is more concerned
Figure 857638DEST_PATH_IMAGE027
Or also
Figure 217075DEST_PATH_IMAGE026
Then activated by swish function and convolved to obtain
Figure 100002_DEST_PATH_IMAGE028
；
③ pair
Figure 686234DEST_PATH_IMAGE028
Performing upsampling, and performing upsampling
Figure 763911DEST_PATH_IMAGE025
1 adopts an attention mechanism to judge whether the attention is more focused
Figure 406245DEST_PATH_IMAGE028
Or also
Figure 885768DEST_PATH_IMAGE025
1, then activated by swish function, and then convolved to obtain
Figure 322566DEST_PATH_IMAGE029
；
Fourthly, to
Figure 887539DEST_PATH_IMAGE029
Performing upsampling, and performing upsampling
Figure 67985DEST_PATH_IMAGE024
1 adopts an attention mechanism to judge whether the attention is more focused
Figure 402014DEST_PATH_IMAGE029
Or also
Figure 755853DEST_PATH_IMAGE024
1, then activated by swish function, and then convolved to obtain
Figure 100002_DEST_PATH_IMAGE030
；
Fifthly, to
Figure 276964DEST_PATH_IMAGE030
Performing upsampling, and performing upsampling
Figure 995521DEST_PATH_IMAGE023
Using attention mechanism to judge whether it is more concerned
Figure 449636DEST_PATH_IMAGE030
Or also
Figure 493816DEST_PATH_IMAGE023
Then activated by swish function and convolved to obtain
Figure 767802DEST_PATH_IMAGE031
；
Is obtained by
Figure 290050DEST_PATH_IMAGE031
、
Figure 598672DEST_PATH_IMAGE030
、
Figure 548173DEST_PATH_IMAGE024
_2、
Figure 43877DEST_PATH_IMAGE029
、
Figure 369816DEST_PATH_IMAGE025
_2、
Figure 532944DEST_PATH_IMAGE026
、
Figure 653347DEST_PATH_IMAGE028
、
Figure 167505DEST_PATH_IMAGE027
Then, need to make
Figure 31555DEST_PATH_IMAGE031
Down-sampling, after down-sampling and
Figure 49190DEST_PATH_IMAGE030
、
Figure 340494DEST_PATH_IMAGE024
2 adopts an attention mechanism to judge whether the attention is more focused
Figure 76369DEST_PATH_IMAGE031
、
Figure 744111DEST_PATH_IMAGE030
Or also
Figure 616252DEST_PATH_IMAGE024
2, then activated by swish function, and then convolved to obtain
Figure 100002_DEST_PATH_IMAGE032
；
Is to
Figure 547299DEST_PATH_IMAGE032
Down-sampling, after down-sampling and
Figure 36049DEST_PATH_IMAGE029
、
Figure 241902DEST_PATH_IMAGE025
2 adopts an attention mechanism to judge whether the attention is more focused
Figure 234129DEST_PATH_IMAGE032
、
Figure 867236DEST_PATH_IMAGE029
Or also
Figure 312123DEST_PATH_IMAGE025
2, then activated by swish function, and then convolved to obtain
Figure 321668DEST_PATH_IMAGE033
(ii) a Then to
Figure 433980DEST_PATH_IMAGE033
Down-sampling, after down-sampling and
Figure 237988DEST_PATH_IMAGE026
、
Figure 435751DEST_PATH_IMAGE028
using attention mechanism to judge whether it is more concerned
Figure 983407DEST_PATH_IMAGE033
、
Figure 950226DEST_PATH_IMAGE026
Or also
Figure 925135DEST_PATH_IMAGE028
Then activated by swish function and convolved to obtain
Figure 100002_DEST_PATH_IMAGE034
(ii) a Then to
Figure 79036DEST_PATH_IMAGE034
Down-sampling, after down-sampling and
Figure 430383DEST_PATH_IMAGE027
using attention mechanism to judge whether it is more concerned
Figure 251709DEST_PATH_IMAGE034
Whether or not
Figure 397519DEST_PATH_IMAGE027
Then activated by swish function and convolved to obtain
Figure 304295DEST_PATH_IMAGE035
；
Will obtain
Figure 193754DEST_PATH_IMAGE031
、
Figure 135165DEST_PATH_IMAGE032
、
Figure 451877DEST_PATH_IMAGE033
、
Figure 580370DEST_PATH_IMAGE034
、
Figure 273520DEST_PATH_IMAGE035
As
Figure 69437DEST_PATH_IMAGE023
、
Figure 822630DEST_PATH_IMAGE024
、
Figure 703998DEST_PATH_IMAGE025
、
Figure 935259DEST_PATH_IMAGE026
、
Figure 320104DEST_PATH_IMAGE027
The previous steps are repeated for stacking, and the EffiicientDet-B0 is repeated for 2 times, wherein the steps are repeated
Figure 244198DEST_PATH_IMAGE024
_1 and
Figure 612862DEST_PATH_IMAGE024
the _2need not be separated,
Figure 647814DEST_PATH_IMAGE025
_1 and
Figure 152745DEST_PATH_IMAGE025
nor does it need to be separated 2. The above fusion features can be briefly described at layer 6 as:
Figure DEST_PATH_IMAGE036
Figure 716581DEST_PATH_IMAGE037
wherein
Figure DEST_PATH_IMAGE038
Represents the first
Figure DEST_PATH_IMAGE039
The characteristics of the layer, wherein
Figure DEST_PATH_IMAGE040
Is an intermediate feature of the 6 th stage in the top-down path, and
Figure DEST_PATH_IMAGE041
is the output characteristic of stage 6 in the bottom-up path;
when fusing features of different resolutions, one common approach is to first adjust them to the same resolution and then sum them, the previous approach treats all input features identically;
since different input features have different resolutions, their contributions to the output features are usually unequal;
BiFPN adds extra weight to each input and lets the network know the importance of each input feature. BiFPN was fused using Fast normalized fusion:
Figure DEST_PATH_IMAGE042
Figure 639452DEST_PATH_IMAGE043
is a learnable weight that can be a scalar (per feature), vector (per channel), or multidimensional tensor (per pixel). In order to avoid the instability of the values,
Figure DEST_PATH_IMAGE044
is set to a smaller value
Figure 681357DEST_PATH_IMAGE045
=0.0001；
The width and depth of the BiFPN are scaled using the following formula:
Figure 100002_DEST_PATH_IMAGE046
Figure DEST_PATH_IMAGE047
wherein 1.35 is used as BiFPN width scaling factor,
Figure 100002_DEST_PATH_IMAGE048
is a complex coefficient that controls all other scale dimensions. The width of the prediction network is the same as BiFPN:
Figure 447319DEST_PATH_IMAGE049
the depth of the prediction network is linearly increased using the equation:
Figure 100002_DEST_PATH_IMAGE050
the step S3 is to construct a loss function, which includes the following steps:
(P1) calculating the difference between the network result and the true value using the following loss function:
Figure 182057DEST_PATH_IMAGE051
wherein
Figure 100002_DEST_PATH_IMAGE052
Is the loss of the class of the object,
Figure 462997DEST_PATH_IMAGE053
is the regression loss.
Figure 105330DEST_PATH_IMAGE053
A Smooth-L1 loss function was used. The Smooth-L1 loss function is as follows:
Figure 100002_DEST_PATH_IMAGE054
(P2) an image has a plurality of candidate boxes, wherein positive samples are included in the image and negative samples are not included in the image. If the probability that the sample I belongs to the class I is 0.9, and the probability that the sample II belongs to the class I is 0.6, the former is a sample which is easy to classify, and the latter is a sample which is difficult to classify. The class of loss in EfficientDet is Focal loss. The Focal length can control the weight of positive and negative samples, and can control the weight of samples which are easy to classify and difficult to classify;
(P3) Focal loss is derived from the cross-entropy loss function, the cross-entropy function loss for the two classes is:
Figure 53695DEST_PATH_IMAGE055
use of
Figure 100002_DEST_PATH_IMAGE056
Simplified cross entropy loss function:
Figure DEST_PATH_IMAGE057
Figure 100002_DEST_PATH_IMAGE058
the weight of positive and negative samples is controlled, and a coefficient can be added before the cross entropy loss function
Figure DEST_PATH_IMAGE059
：
Figure 100002_DEST_PATH_IMAGE060
Figure DEST_PATH_IMAGE061
Controlling the weight of easy-to-classify and difficult-to-classify samples;
Figure DEST_PATH_IMAGE062
Figure 506804DEST_PATH_IMAGE063
referred to as modulation factor. When gamma =0, focal length is the cross entropy loss function, which can be adjusted
Figure 71778DEST_PATH_IMAGE009
A change of the modulation factor is effected. Two kinds of weights are combined to obtain:
Figure 100002_DEST_PATH_IMAGE064
wherein
Figure 455486DEST_PATH_IMAGE008
= 0.25 and
Figure 55094DEST_PATH_IMAGE009
= 1.5。
the testing stage in the step S4 is composed of the following steps:
(Q1) in order to distinguish from the common feature layer, the feature layer extracted by BiFPN is called as an effective feature layer, and the five effective feature layers are transmitted into ClassNet (classification prediction network) and BoxNet (regression prediction network) to obtain a prediction result;
(Q2) for Efficientdet-B0, ClassNet takes 3 convolutions of the depth separable convolution of 64 channels and 1 convolution of the number of a priori boxes that the feature layer possesses multiplied by how many classes of objects the network has in common, the number of a priori boxes being by default 9. BoxNet takes 3 convolutions of 64 channels and 1 convolution of the number of prior frames of the feature layer multiplied by four, wherein four refers to the adjustment condition of the prior frames, and the central position and the width and the height are adjusted. The EfficientDet defaults to have 9 prior frames, and the length-width ratio of the prior frames can be adjusted according to actual conditions so as to be suitable for a detected target. The EfficientDet judges the objects in the prior frame and the types of the objects, adjusts the prior frame, screens out the frame which belongs to the same type in a certain area and has the maximum confidence coefficient by using non-maximum suppression (soft-NMS), and obtains a final prediction frame;
(Q3) transmitting a reserved ore test picture into an EfficientDet network for prediction to obtain a prediction frame of ore in the picture, calculating the size of the prediction frame, and calculating the size of the real ore according to the size of the prediction frame and an error;
(Q4) calculating the real size of the camera according to the installation angle of the camera, the distance from the camera to the belt, the size of the network model prediction frame and the pixels of the image:
Figure 100002_DEST_PATH_IMAGE065
Figure 100002_DEST_PATH_IMAGE066
is the included angle between the camera and the vertical line of the belt,
Figure DEST_PATH_IMAGE067
which represents the true size of the ore,
Figure 100002_DEST_PATH_IMAGE068
indicating the vertical distance between the camera and the belt,
Figure 7001DEST_PATH_IMAGE069
which represents the focal length of the camera head,
Figure 100002_DEST_PATH_IMAGE070
the size of the EfficientDet network model prediction box is shown.
The reminding stage in the step S5 comprises the following steps:
(T1) setting a system threshold value as the sum of the alarm threshold value and the error threshold value of the ore size, and comparing the calculated real size of the ore with the system threshold value;
(T2) when the ore size exceeds the system threshold value, an early warning is given out, meanwhile, an abnormal signal is transmitted to an execution mechanism, and the execution mechanism dials out the large ore.
The early warning system comprises the following parts:
(D1) the user module is responsible for registration, login and management of users, displays personal user information and provides management authority of super users;
(D2) the ore video stream acquisition module is used for acquiring ore video streams through cameras arranged at various angles near the ore flow belt and sending acquired images to the real-time early warning module;
(D3) and the real-time early warning module is used for receiving the ore image sent by the ore video stream acquisition module, transmitting the ore image into a trained EfficientDet network to obtain the position and size of the ore, and comparing the real size of the ore obtained by calculation with a system threshold value. If the real size of the abnormal information is larger than the system threshold value, displaying the abnormal information on a page, and writing the abnormal information into a database;
(D4) the setting module is used for setting system related parameters and log related parameters, such as the size of the alarm ore block and the like;
(D5) and the log module is used for receiving the abnormal record of the real-time early warning system, displaying the abnormal record and synchronizing the abnormal record to the cloud server.
The user module of the early warning system Web end comprises the following parts:
(U1) providing registration and login functions for new managers, operating and using the whole system, storing information of the whole system to a cloud server, and giving corresponding authority;
(U2) providing a super administrator with high authority with high-level authority to operate the system, and managing other users while operating and using the entire system.
The real-time early warning module of the early warning system Web end comprises the following parts:
(R1) adding, deleting, modifying corresponding industrial personal computer devices;
(R2) displaying a belt rotation screen of the added industrial personal computer device in real time; supporting the starting and stopping of displaying a belt rotation picture of the industrial personal computer equipment;
(R3) when the fact that the ore size of an industrial personal computer is larger than a system threshold value is detected, abnormal information is displayed, a user can select whether to stop the operation of the industrial personal computer or not according to the abnormal information, and information of the industrial personal computer recorded in an abnormal mode, abnormal time and images are stored in a database and written into a log module; and synchronizing the abnormal records in the database to the cloud server at intervals.
The setting module of the early warning system Web end comprises the following parts:
the related information of the early warning system is set, and the ore block size of the early warning of the system, the size of the system error, the focal length of the camera, the cloud server port and the like are mainly set.
The log module of the early warning system Web end comprises the following parts:
(Z1) receiving the abnormal record of the real-time alarm system, displaying the abnormal record on a page in real time, and writing the abnormal record into a database;
(Z2) selecting whether to synchronize the log to the cloud server;
(Z3) provides a search function, possibly based on the information of the industrial personal computer equipment, time, threshold value, etc., for alarm records.
The user module of the early warning system APP end comprises the following parts:
(M1) keeping synchronization with the database of the Web end, providing registration and login functions for new managers, operating and using the whole system, storing the information to the cloud server, and giving corresponding authority;
(M2) the super administrator with high authority provides high-level authority for operating the system and manages other users while operating and using the entire system.
Real-time early warning module of early warning system APP end comprises following part:
(X1) synchronizing the content of the industrial personal computer equipment in the database, and adding, deleting and modifying the corresponding industrial personal computer equipment;
(X2) displaying a belt rotation picture of the added industrial personal computer device in real time; supporting the starting and stopping of displaying a belt rotation picture of the industrial personal computer equipment;
(X3) when the fact that the ore size of an industrial personal computer is larger than a system threshold value is detected, abnormal information is displayed, a user can select whether to stop the operation of the industrial personal computer or not according to the abnormal information, and information of the industrial personal computer recorded in an abnormal mode, abnormal time and images are stored in a database and written into a log module; and synchronizing the abnormal records in the database to the cloud server at intervals.
The log module of the APP end of the early warning system comprises the following parts:
(Y1) real-time querying the database to display the abnormal information on the page;
(Y2) provides a search function, possibly for alarm records based on the information of the industrial personal computer equipment, time, threshold values, etc.
As described above, the invention provides an EfficientDet network-based deep learning ore size measuring method and an early warning system, and solves the problems of extreme dependence on manpower, low efficiency, low accuracy, overlarge model parameter and the like in ore size detection through an EfficientDet network model.
Drawings
FIG. 1 is a schematic diagram of the steps of the ore size measurement method of the present invention;
FIG. 2 is a schematic diagram of an EfficientDet framework network EfficientNet according to the present invention;
FIG. 3 is a schematic diagram of the architecture of the EfficientDet feature extraction network BiFPN of the present invention;
FIG. 4 is a schematic diagram illustrating the overall structure of the EfficientDet network according to the present invention;
FIG. 5 is a flowchart illustrating step S1 in FIG. 1 in one embodiment;
FIG. 6 is a flowchart illustrating a specific example of step S2 shown in FIG. 1;
FIG. 7 is a flowchart illustrating a specific example of step S3 shown in FIG. 1;
FIG. 8 is a flowchart illustrating a specific example of step S4 shown in FIG. 1;
FIG. 9 is a flowchart illustrating a specific example of step S5 shown in FIG. 1;
FIG. 10 is a schematic diagram of a deep learning ore size measurement method and an early warning system module based on the EfficientDet network according to the present invention;
FIG. 11 is a block diagram illustrating an embodiment of the user module M3 of FIG. 10;
FIG. 12 is a block diagram of the real-time warning module M2 of FIG. 10 in an embodiment;
FIG. 13 is a block diagram illustrating the setting module M4 of FIG. 10 in one embodiment;
FIG. 14 is a block diagram of the log module M5 of FIG. 10 in one embodiment;
reference numerals: m1, a video stream acquisition module; m2 real-time early warning module; an M21APP terminal; an M22Web end; an M3 user module; m4 setup module; an M5 log module; an M6 database module; S1-S5 are method steps; S11-S14 are method steps; S21-S25 are method steps; S31-S36 are method steps; S41-S43 are method steps; S51-S54 are the steps of the method.
Detailed Description
According to fig. 1, 2 and 3, a schematic diagram of the steps of the ore size measuring method, an architecture of an EfficientDet skeleton network EfficientNet and an architecture of a feature extraction network BiFPN of the invention are shown; the invention aims to solve the problems of extreme dependence on manpower, low efficiency, low accuracy, overlarge model parameter and the like in ore size detection through an EfficientDet network model. The EfficientDet has 8 versions from EfficientDet-D0 to EfficientDet-D7, and the model parameters and the operand are improved along with the improvement of the version number, and the precision is also improved. An appropriate network version is selected to adapt to different application scenarios, and the efficientDet-D0 is taken as an example. The method comprises the following steps:
s1, data acquisition stage: shooting ores on a flowing belt through a high-speed camera, and storing key frames of a video stream as pictures;
s2, data preprocessing stage: manually labeling the image, detecting and removing abnormal ore pictures; and (3) pressing the obtained ore picture according to the following steps of 7: 2: 1, dividing the picture into a training picture, a verification picture and a test picture; enhancing the data of the ore picture to improve the generalization ability;
s3, network creating and training stage: using an EfficientNet network as a backbone, creating an EfficientDet target detection network, and obtaining a network model by using a training data set;
s4, testing: testing by using the trained EfficientDet network model, using the obtained prediction frame for positioning the position of ore discharge, and calculating the size of the ore through the focal length of the camera, the distance between the camera and the belt and the size of the image;
s5, reminding: and sending out a prompt when the ore size exceeds the system threshold value according to the preset threshold value of the system.
Referring to fig. 5, it is a specific implementation step of S1, including the following steps:
s11, installing a camera at a position with a high vertical height L above the flowing belt, so that the shooting range of the camera can cover the transverse width of the flowing belt to obtain a complete ore video stream;
s12, setting camera parameter data including parameters such as resolution, frame rate, color space and the like to obtain a clearer ore video;
s13, selecting a proper storage format of video acquisition data, and storing the video into a video stream, wherein the storage format is an MJPEG format;
and S14, intercepting key frame data in the video stream to be used as an ore picture for next training, verification and testing.
Referring to fig. 6, it is a specific implementation step of S2, including the following steps:
and S21, marking the position and size of the ore in the acquired image data by using an image marking tool Lableme, wherein the size of the ore is not regular, and polygonal marking and a saved json file are required. The json file name is the same as the name of the mark, and the storage format of the picture is Base64 code;
s22, processing the ore pictures according to the marked data, detecting the pictures which do not meet the requirements, and deleting the pictures;
s23, pressing the obtained ore picture to be 7: 2: 1, dividing the picture into a training picture, a verification picture and a test picture;
s24, performing data enhancement on the training picture: horizontally and vertically reversing the ore picture, zooming the ore picture and changing the corresponding labeling position at the same time;
s25, adding random noise; the image is converted from the RGB color space to the HSV color space, the brightness of the image is adjusted to adapt to different illumination changes, the image is normalized, and the training effect is improved.
Referring to fig. 7, the specific implementation step of S3 includes the following steps:
s31, selecting the backbone type of the EfficientNet-B0 network, and setting the hyper-parameters of the training network, the learning rate, the Batch size, the number of rounds of training (Epochs), the optimizer (such as SGD and Adam), and the like. The EfficientNet-B0 consists of 1 Conv (3 × 3), 1 MBConv1(3 × 3), 2 MBConv6(3 × 3), 2 MBConv6(5 × 5), 3 MBConv6(3 × 3), 3 MBConv6(5 × 5), 4 MBConv6(5 × 5), one MBConv6(3 × 3), one Conv (1 × 1), one Pooling layer, and one FC layer. Wherein MBConv contains residual structures. The dimension increasing operation is performed by using the convolution of 1 × 1, then the convolution of 3 × 3 or 5 × 5 is performed, then the attention mechanism about the channel is added, and the dimension reducing operation is performed by using the convolution of 1 × 1, and then the residual structure is stacked. The activation function of MBConv uses the Swish function and is normalized using Batch Normalization;
the definition of the Swish function is:
Figure 528112DEST_PATH_IMAGE071
wherein
Figure 981090DEST_PATH_IMAGE002
Is a constant or trainable parameter;
Figure 435205DEST_PATH_IMAGE003
the function is expressed as follows:
Figure 100002_DEST_PATH_IMAGE072
performing network scaling, and determining that the optimal value of EfficientNet-B0 scaling is alpha =1.2, beta =1.1 and gamma = 1.15;
s32, the BiFPN network can learn the importance from different input features while repeatedly applying top-down and bottom-up multi-size feature fusion. The enhanced extraction network consists of a plurality of BiFPNs, wherein the EfficientDet-D0 consists of 3 BiFPNs;
according to fig. 4, EfficientDet includes an EfficientNet backbone extraction network and a BiFPN enhanced extraction network, and efficienthead converts the extracted features into a prediction result. First EfficiiThe entNet continuously performs downsampling on an input picture, the downsampling frequency of the original EfficientNet is 5, and the original EfficientNet can be obtained through the EfficientNet
Figure 682647DEST_PATH_IMAGE011
、
Figure 222213DEST_PATH_IMAGE012
、
Figure 478882DEST_PATH_IMAGE013
、
Figure 787503DEST_PATH_IMAGE014
、
Figure 737005DEST_PATH_IMAGE015
，
Figure 498287DEST_PATH_IMAGE011
Is the result of compressing the input picture length and width once,
Figure 558647DEST_PATH_IMAGE012
is the result of two compressions of the input picture length and width,
Figure 721775DEST_PATH_IMAGE013
is the result of three compressions of the input picture length and width, and so on. Due to the fact that
Figure 842178DEST_PATH_IMAGE011
And
Figure 356336DEST_PATH_IMAGE012
does not have high semantic information, so the method is not used in the reinforced extraction network BiFPN
Figure 220387DEST_PATH_IMAGE011
、
Figure 238021DEST_PATH_IMAGE012
、
Figure 529325DEST_PATH_IMAGE013
、
Figure 265200DEST_PATH_IMAGE014
、
Figure 667363DEST_PATH_IMAGE015
Has high semantic information, and is used in the reinforced extraction network BiFPN as three of 5 effective feature layers. With two downsamplings of P5 to obtain information with higher semantics
Figure 805083DEST_PATH_IMAGE016
、
Figure 532868DEST_PATH_IMAGE017
. Obtaining 5 effective characteristic layers
Figure 490459DEST_PATH_IMAGE013
、
Figure 961892DEST_PATH_IMAGE014
、
Figure 954119DEST_PATH_IMAGE015
、
Figure 587225DEST_PATH_IMAGE016
、
Figure 297692DEST_PATH_IMAGE017
；
Transmitting the 5 feature layers into an enhanced extraction network BiFPN for further feature extraction, wherein
Figure 318955DEST_PATH_IMAGE018
Representing resolution as an input image
Figure 900109DEST_PATH_IMAGE019
The feature level of (c). For example, if the input resolution is 640x640,
Figure 969697DEST_PATH_IMAGE020
represents a characteristic level 3(640 ^) with a resolution of 80x80
Figure 901881DEST_PATH_IMAGE021
= 80), and
Figure 715116DEST_PATH_IMAGE022
represents a 7-level feature level representing a resolution of 5x 5;
the two fused features shown in fig. 3 are in the case of layer 6:
Figure 681935DEST_PATH_IMAGE073
Figure 100002_DEST_PATH_IMAGE074
wherein
Figure 100002_DEST_PATH_IMAGE075
Represents the first
Figure 100002_DEST_PATH_IMAGE076
The characteristics of the layer, wherein
Figure DEST_PATH_IMAGE077
Is an intermediate feature of the 6 th stage in the top-down path, and
Figure DEST_PATH_IMAGE078
is the output characteristic of stage 6 in the bottom-up path;
when fusing features of different resolutions, one common approach is to first adjust them to the same resolution and then sum them, the previous approach treats all input features identically;
since different input features have different resolutions, their contributions to the output features are usually unequal;
BiFPN adds extra weight to each input and lets the network know the importance of each input feature. BiFPN uses Fast normalized fusion
Figure DEST_PATH_IMAGE079
Figure 469893DEST_PATH_IMAGE043
Is a learnable weight that can be a scalar (per feature), vector (per channel), or multidimensional tensor (per pixel). In order to avoid the instability of the values,
Figure 154953DEST_PATH_IMAGE044
is set to a smaller value
Figure 506299DEST_PATH_IMAGE045
=0.0001；
The width and depth of the BiFPN are scaled using the following formula:
Figure 100002_DEST_PATH_IMAGE080
Figure 796467DEST_PATH_IMAGE081
wherein 1.35 is used as BiFPN width scaling factor,
Figure 942277DEST_PATH_IMAGE005
is a complex coefficient that controls all other proportional dimensions;
the width of the prediction network is the same as BiFPN:
Figure 100002_DEST_PATH_IMAGE082
the depth of the prediction network is linearly increased using the equation:
Figure 317895DEST_PATH_IMAGE083
s33, calculating the difference between the network result and the true value using the following loss function:
Figure DEST_PATH_IMAGE084
wherein
Figure 676195DEST_PATH_IMAGE052
Is the loss of the class of the object,
Figure 352027DEST_PATH_IMAGE053
is the regression loss.
Figure 934318DEST_PATH_IMAGE053
A Smooth-L1 loss function was used. The Smooth-L1 loss function is as follows:
Figure 328390DEST_PATH_IMAGE085
Figure 21540DEST_PATH_IMAGE052
focal length was used, as shown below
Figure DEST_PATH_IMAGE086
Figure 20720DEST_PATH_IMAGE087
Referred to as modulation factor. When gamma =0, focal length is the cross entropy loss function, which can be adjusted
Figure 773912DEST_PATH_IMAGE009
A change of the modulation factor is effected. Two kinds of weights are combined to obtain:
Figure DEST_PATH_IMAGE088
wherein
Figure 124122DEST_PATH_IMAGE008
= 0.25 and
Figure 355383DEST_PATH_IMAGE009
= 1.5
s34, visualizing the whole training process, judging whether the network is under-fit or over-fit by judging the performance of the EfficientDet-D0 network in a training set and a verification set, and adjusting parameters in a targeted manner, such as adjusting the learning rate, increasing the number of training rounds, replacing an optimizer, adjusting the size of an anchor and the like;
s35, selecting whether to finish training in advance according to the loss function and the network performance;
s36, finishing the training to obtain the EfficientDet-D0 network model.
Referring to fig. 8, it is a specific implementation step of S4, including the following steps:
s41, predicting the pre-reserved test picture by using the previously trained EfficientDet-D0 network model;
s42, obtaining a prediction frame of the ore in the picture, and calculating the size of the prediction frame;
s43, calculating the real size of the belt according to the installation angle of the camera, the distance between the camera and the belt, the size of the network model prediction frame and the pixels of the image:
Figure 5807DEST_PATH_IMAGE089
Figure 929901DEST_PATH_IMAGE066
is the included angle between the camera and the vertical line of the belt,
Figure 298565DEST_PATH_IMAGE067
which represents the true size of the ore,
Figure 333518DEST_PATH_IMAGE068
indicating the vertical distance between the camera and the belt,
Figure 838448DEST_PATH_IMAGE069
which represents the focal length of the camera head,
Figure 667864DEST_PATH_IMAGE070
represents the size of the EfficientDet-D0 network model prediction box.
Referring to fig. 9, it is a specific implementation step of S5, including the following steps:
s51, setting an ore size alarm threshold and an error threshold at the Web end of the re-system, wherein the system threshold is the sum of the ore size alarm threshold and the error threshold;
s52, comparing the real size of the ore calculated by the calculation system with a system threshold value;
s53, displaying abnormal information when the fact that the ore size of a certain industrial personal computer is larger than a system threshold value is detected;
and S54, the user can select whether to stop the operation of the industrial personal computer according to the abnormal information, and the information of the industrial personal computer recorded in the abnormal state, the abnormal time and the abnormal picture are stored in the database.
According to fig. 10, it consists of the following modules:
the video stream acquisition module (M1) acquires the video stream in real time, sends key frames in the video stream to the real-time early warning module, and judges whether to send out early warning or not by implementing the mirror image of the early warning module; the real-time early warning module (M2) comprises a Web end and an App end, so that the use of a user is more convenient, and the Web end comprises a user module, a real-time early warning module, a setting module and a log module. The App comprises a user module, a real-time early warning module and a log module. The Web end is mainly realized based on a Django framework. The user module (M3) according to fig. 11 mainly comprises: 1. a registration function 2, a login function 3 and an authority management function; the real-time early warning module (M4) according to fig. 12 mainly includes: 1. adding, deleting, modifying and listing 2 industrial personal computer equipment, displaying a belt corresponding picture 3 of the industrial personal computer equipment in real time, starting or stopping displaying a picture 4 of the industrial personal computer, displaying and sending abnormal information to an executing mechanism 5, and storing records of the abnormal information into a database. The setup module according to fig. 13 mainly comprises: 1. the method comprises the steps of setting the size of an ore block 2 for early warning of a system, setting the error size 3 of the system, setting a cloud server port 4 to be synchronized, and setting parameter information of a camera. The logging module (M5) according to fig. 14 mainly comprises: 1. inquiring the log 2, exporting the log 3, displaying the recent abnormal record 4, and selecting whether to synchronize the log from the local machine to the cloud server. Database module (M6): the local data can be synchronized to the cloud database, and the data records in the cloud database can be downloaded to the local.

Claims (9)

1. A deep learning ore size measuring method and an early warning system based on an EfficientDet network are used for detecting ore size in the mining industry and intelligently processing ore blocks exceeding a threshold size, and are characterized by comprising the following specific steps:
s1, data acquisition stage: shooting ores on a flowing belt through a high-speed camera, and storing key frames of a video stream as pictures;
s2, data preprocessing stage: manually labeling the image, detecting and removing abnormal ore pictures; and (3) pressing the obtained ore picture according to the following steps of 7: 2: 1, dividing the picture into a training picture, a verification picture and a test picture; enhancing the data of the ore picture to improve the generalization ability;
s3, network creating and training stage: using an EfficientNet network as a backbone, creating an EfficientDet target detection network, and obtaining a network model by using a training data set;
s4, testing: testing by using the trained EfficientDet network model, using the obtained prediction frame for positioning the position of ore discharge, and calculating the size of the ore according to the focal length of the camera, the distance between the camera and the belt and the size of the image;
s5, reminding: and sending out a prompt when the ore size exceeds the system threshold value according to the preset threshold value of the system.
2. The method for measuring the ore size for deep learning based on the EfficientDet network and the early warning system according to claim 1, wherein the data acquisition stage specifically comprises:
(1) acquiring video data streams of ore pictures through a plurality of cameras arranged near the belt and at different angles;
(2) and intercepting the key frame from the acquired video data stream as ore picture data.
3. The method for measuring the ore size for deep learning based on the EfficientDet network and the early warning system according to claim 1, wherein the data preprocessing stage specifically comprises:
(i) marking the position and the size of the ore in the obtained picture data by using an image marking tool Lableme, wherein the scale of the ore is not regular and polygonal marking is needed;
(ii) detecting ore picture data which do not meet the requirements, and rejecting the ore picture data;
(iii) and performing data enhancement on the training picture: randomly carrying out horizontal and vertical reversal at different angles on the ore picture; zooming the ore picture; adjusting pixel values of the image so that the values thereof become uniformly distributed through histogram equalization; adding random noise; the method comprises the steps of converting an image from an RGB color space to an HSV color space, adjusting the brightness of the image, normalizing the image and processing noise.
4. The method for measuring the size of the deep learning ore based on the efficientDet network and the early warning system according to claim 1, wherein the network creating and training stage specifically comprises:
using an EfficientNet network as a backbone network, fusing network characteristics extracted by the EfficientNet by adopting a BiFPN network, and classifying and performing regression prediction on the extracted characteristics by the Head of the EfficientDet;
(ii) the entire EfficientNet has versions B0-B7, wherein EfficientNet-B0 consists of 1 Conv (3 × 3), 1 MBConv1(3 × 3), 2 MBConv6(3 × 3), 2 MBConv6(5 × 5), 3 MBConv6(3 × 3), 3 MBConv6(5 × 5), 4 MBConv6(5 × 5), one MBConv6(3 × 3), one Conv (1 × 1), one Pooling layer, one FC layer;
wherein MBConv contains a residual structure;
performing dimension increasing operation by using convolution of 1 × 1, performing convolution of 3 × 3 or 5 × 5, adding an attention mechanism about a channel, performing dimension reducing operation by using convolution of 1 × 1, and stacking with a residual error structure;
the activation function of MBConv uses the Swish function and is normalized using Batch Normalization;
the definition of the Swish function is:
Figure DEST_PATH_IMAGE001
wherein
Figure DEST_PATH_IMAGE002
Is a constant or trainable parameter;
Figure 608517DEST_PATH_IMAGE003
the function is expressed as follows:
Figure DEST_PATH_IMAGE004
at the same time, efficiency-B0 uses the width (depth), depth (width) and resolution (resolution) of the network as the coefficients according to formula (3)
Figure DEST_PATH_IMAGE005
Performing composite zooming;
wherein α, β, γ are constants that can be determined by grid search;
in that
Figure DEST_PATH_IMAGE006
And
Figure 994499DEST_PATH_IMAGE007
under the constraint of (3), the optimum value of EfficientNet-B0 is
Figure DEST_PATH_IMAGE008
=1.2，
Figure 603597DEST_PATH_IMAGE002
=1.1，
Figure DEST_PATH_IMAGE010
=1.15；
Figure 451336DEST_PATH_IMAGE011
。
5. The method for measuring the ore size for deep learning based on the EfficientDet network and the early warning system according to claim 1, wherein the BiFPN is constructed and specifically comprises the following steps:
(A) the importance of different input features can be learned by the learned weight of a BiFPN (bi-directional feature pyramid) network, and multi-scale feature fusion from top to bottom and from bottom to top is repeatedly applied;
the enhanced extraction network is composed of a plurality of BiFPNs, wherein EfficientDet has eight versions D0-D7, and the eight versions are respectively composed of 3, 4, 5, 6, 7, 8 and 8 BiFPNs;
(B) the EfficientDet comprises an EfficientNet backbone extraction network and a BiFPN reinforced extraction network, and the EfficientHead converts the extracted features into a prediction result;
firstly, the EfficientNet continuously performs downsampling on an input picture, the downsampling frequency of the original EfficientNet is 5 times, and the input picture can be obtained through the EfficientNet
Figure DEST_PATH_IMAGE012
、
Figure 468971DEST_PATH_IMAGE013
、
Figure DEST_PATH_IMAGE014
、
Figure 245428DEST_PATH_IMAGE015
、
Figure DEST_PATH_IMAGE016
，
Figure 512461DEST_PATH_IMAGE012
Is the result of compressing the input picture length and width once,
Figure 39258DEST_PATH_IMAGE013
is the result of two compressions of the input picture length and width,
Figure 708136DEST_PATH_IMAGE014
the result of compressing the length and width of the input picture three times, and so on;
due to the fact that
Figure 904762DEST_PATH_IMAGE012
And
Figure 659092DEST_PATH_IMAGE013
does not have high semantic information, so the method is not used in the reinforced extraction network BiFPN
Figure 350098DEST_PATH_IMAGE012
、
Figure 873484DEST_PATH_IMAGE013
、
Figure 303328DEST_PATH_IMAGE014
、
Figure 138429DEST_PATH_IMAGE015
、
Figure 413552DEST_PATH_IMAGE016
The method has high semantic information, and therefore, the method is used in the reinforced extraction network BiFPN as three of 5 effective feature layers;
with two downsamplings of P5 to obtain information with higher semantics
Figure 260286DEST_PATH_IMAGE017
、
Figure DEST_PATH_IMAGE018
；
Obtaining 5 effective characteristic layers
Figure 595452DEST_PATH_IMAGE014
、
Figure 278368DEST_PATH_IMAGE015
、
Figure 357183DEST_PATH_IMAGE016
、
Figure 855160DEST_PATH_IMAGE017
、
Figure 830069DEST_PATH_IMAGE018
；
Transmitting the 5 feature layers into an enhanced extraction network BiFPN for further feature extraction, wherein
Figure 780708DEST_PATH_IMAGE019
Representing resolution as an input image
Figure 991109DEST_PATH_IMAGE021
A feature level of;
for example, if the input resolution is 640x640,
Figure DEST_PATH_IMAGE022
represents a characteristic level 3(640 ^) with a resolution of 80x80
Figure 343593DEST_PATH_IMAGE023
= 80), and
Figure DEST_PATH_IMAGE024
represents a 7-level feature level representing a resolution of 5x 5;
the method comprises the following specific steps:
firstly, adjusting the number of channels to obtain
Figure 240136DEST_PATH_IMAGE025
，
Figure DEST_PATH_IMAGE026
，
Figure 881333DEST_PATH_IMAGE027
，
Figure DEST_PATH_IMAGE028
，
Figure 754480DEST_PATH_IMAGE029
If entering BiFPN for the first time, will
Figure 227050DEST_PATH_IMAGE026
By reducing the number of channels
Figure DEST_PATH_IMAGE030
_1 and
Figure 435439DEST_PATH_IMAGE031
，
Figure 360670DEST_PATH_IMAGE027
by reducing the passage to
Figure DEST_PATH_IMAGE032
1 and
Figure 788240DEST_PATH_IMAGE027
_2；
is obtaining
Figure 849737DEST_PATH_IMAGE025
、
Figure 586618DEST_PATH_IMAGE026
_1、
Figure 999145DEST_PATH_IMAGE026
_2、
Figure 856505DEST_PATH_IMAGE027
_1、
Figure 38087DEST_PATH_IMAGE027
_2、
Figure 227760DEST_PATH_IMAGE028
、
Figure 330845DEST_PATH_IMAGE029
Then need to be paired
Figure 896956DEST_PATH_IMAGE029
Performing upsampling, and performing upsampling
Figure 651154DEST_PATH_IMAGE028
Using attention mechanism to judge whether it is more concerned
Figure 11728DEST_PATH_IMAGE029
Or also
Figure 398847DEST_PATH_IMAGE028
Then activated by swish function and convolved to obtain；
③ pair
Figure 971911DEST_PATH_IMAGE033
Performing upsampling, and performing upsampling
Figure 862507DEST_PATH_IMAGE027
1 adopts an attention mechanism to judge whether the attention is more focused
Figure 488922DEST_PATH_IMAGE033
Or also
Figure 97758DEST_PATH_IMAGE027
1, then activated by swish function, and then convolved to obtain
Figure DEST_PATH_IMAGE034
；
Fourthly, to
Figure 723781DEST_PATH_IMAGE034
Performing upsampling, and performing upsampling
Figure 734462DEST_PATH_IMAGE026
1 adopts an attention mechanism to judge whether the attention is more focused
Figure 171260DEST_PATH_IMAGE034
Or also
Figure 267392DEST_PATH_IMAGE026
1, then activated by swish function, and then convolved to obtain
Figure 978996DEST_PATH_IMAGE035
；
Fifthly, to
Figure 673544DEST_PATH_IMAGE035
Perform upsamplingAfter upsampling and
Figure 812402DEST_PATH_IMAGE025
using attention mechanism to judge whether it is more concerned
Figure 395830DEST_PATH_IMAGE035
Or also
Figure 98075DEST_PATH_IMAGE025
Then activated by swish function and convolved to obtain
Figure 83349DEST_PATH_IMAGE037
；
Is obtained by
Figure 596370DEST_PATH_IMAGE037
、
Figure 667094DEST_PATH_IMAGE035
、
Figure 720501DEST_PATH_IMAGE026
_2、
Figure 490838DEST_PATH_IMAGE034
、
Figure 971498DEST_PATH_IMAGE027
_2、
Figure 591835DEST_PATH_IMAGE028
、
Figure 448933DEST_PATH_IMAGE033
、
Figure 143219DEST_PATH_IMAGE029
Then, need to make
Figure 998043DEST_PATH_IMAGE037
Down-sampling, after down-sampling and
Figure 43359DEST_PATH_IMAGE035
、
Figure 392563DEST_PATH_IMAGE026
2 adopts an attention mechanism to judge whether the attention is more focused
Figure 941356DEST_PATH_IMAGE037
、
Figure 29398DEST_PATH_IMAGE035
Or also
Figure 889906DEST_PATH_IMAGE026
2, then activated by swish function, and then convolved to obtain
Figure 88807DEST_PATH_IMAGE039
；
Is to
Figure 492106DEST_PATH_IMAGE039
Down-sampling, after down-sampling and
Figure 954311DEST_PATH_IMAGE034
、
Figure 708641DEST_PATH_IMAGE027
2 adopts an attention mechanism to judge whether the attention is more focused
Figure 399647DEST_PATH_IMAGE039
、
Figure 923033DEST_PATH_IMAGE034
Or also
Figure 352877DEST_PATH_IMAGE027
2, then activated by swish function, and then convolved to obtain
Figure 532186DEST_PATH_IMAGE041
(ii) a Then to
Figure 72888DEST_PATH_IMAGE041
Down-sampling, after down-sampling and
Figure 716359DEST_PATH_IMAGE028
、
Figure 51526DEST_PATH_IMAGE033
using attention mechanism to judge whether it is more concerned
Figure 905081DEST_PATH_IMAGE041
、
Figure 983896DEST_PATH_IMAGE028
Or also
Figure 481873DEST_PATH_IMAGE033
Then activated by swish function and convolved to obtain
Figure 987941DEST_PATH_IMAGE043
(ii) a Then to
Figure 830257DEST_PATH_IMAGE043
Down-sampling, after down-sampling and
Figure 712762DEST_PATH_IMAGE029
using attention mechanism to judge whether it is more concerned
Figure 65246DEST_PATH_IMAGE043
Whether or not
Figure 7794DEST_PATH_IMAGE029
Then activated by swish function and convolved to obtain
Figure 570363DEST_PATH_IMAGE045
；
Will obtain
Figure DEST_PATH_IMAGE046
、
Figure 990980DEST_PATH_IMAGE039
、
Figure 463550DEST_PATH_IMAGE041
、
Figure 937518DEST_PATH_IMAGE043
、
Figure 862749DEST_PATH_IMAGE045
As
Figure 87057DEST_PATH_IMAGE025
、
Figure 414133DEST_PATH_IMAGE026
、
Figure 823118DEST_PATH_IMAGE027
、
Figure 235645DEST_PATH_IMAGE028
、
Figure 998064DEST_PATH_IMAGE029
The previous steps are repeated for stacking, and the EffiicientDet-B0 is repeated for 2 times, wherein the steps are repeated
Figure 179647DEST_PATH_IMAGE026
_1 and
Figure 260998DEST_PATH_IMAGE026
the _2need not be separated,
Figure 160821DEST_PATH_IMAGE027
_1 and
Figure 726931DEST_PATH_IMAGE027
nor does it need to be separated;
for example, the above fusion features can be summarized at layer 6 as:
Figure 763020DEST_PATH_IMAGE047
Figure DEST_PATH_IMAGE048
wherein
Figure 513807DEST_PATH_IMAGE049
Represents the first
Figure DEST_PATH_IMAGE050
The characteristics of the layer, wherein
Figure 900926DEST_PATH_IMAGE051
Is an intermediate feature of the 6 th stage in the top-down path, and
Figure DEST_PATH_IMAGE052
is the output characteristic of stage 6 in the bottom-up path;
the width and depth of the BiFPN are scaled using the following formula:
Figure 631247DEST_PATH_IMAGE053
Figure DEST_PATH_IMAGE054
Figure 787422DEST_PATH_IMAGE055
wherein 1.35 is used as BiFPN width scaling factor,
Figure 709111DEST_PATH_IMAGE005
is the composite coefficient that controls all other proportional dimensions:
the width of the prediction network is the same as BiFPN:
Figure DEST_PATH_IMAGE056
Figure 317946DEST_PATH_IMAGE057
the depth of the predicted network increases linearly using equation (9):
Figure DEST_PATH_IMAGE058
Figure 117537DEST_PATH_IMAGE059
。
6. the method for measuring the size of the deep learning ore based on the EfficientDet network and the early warning system according to claim 1 are characterized by constructing a loss function, and specifically comprising the following steps:
(P1) calculating the difference between the network result and the true value using the following loss function:
Figure DEST_PATH_IMAGE060
)
wherein
Figure 128219DEST_PATH_IMAGE061
Is the loss of the class of the object,
Figure 96175DEST_PATH_IMAGE063
is the regression loss;
Figure 316941DEST_PATH_IMAGE063
the Smooth-L1 loss function is used;
the Smooth-L1 loss function is as follows:
Figure DEST_PATH_IMAGE064
)
(P2) an image has a plurality of candidate boxes, wherein positive samples are included in the image and negative samples are not included in the image;
if the probability that the first sample belongs to the first class is 0.9, the probability that the second sample belongs to the first class is 0.6, the former is a sample which is easy to classify, and the latter is a sample which is difficult to classify;
the class of the EfficientDet is Focal;
the Focal length can control the weight of positive and negative samples, and can control the weight of samples which are easy to classify and difficult to classify;
(P3) Focal loss is derived from the cross-entropy loss function, the cross-entropy function loss for the two classes is:
Figure DEST_PATH_IMAGE065
)
use of
Figure DEST_PATH_IMAGE066
Simplified cross entropy loss function:
Figure 389064DEST_PATH_IMAGE067
)
Figure DEST_PATH_IMAGE068
the weight of positive and negative samples is controlled, and a coefficient can be added before the cross entropy loss function
Figure 519831DEST_PATH_IMAGE069
：
Figure DEST_PATH_IMAGE070
Figure 783322DEST_PATH_IMAGE071
Controlling the weights of easy-to-classify and difficult-to-classify samples:
Figure DEST_PATH_IMAGE072
)
Figure 366750DEST_PATH_IMAGE073
called modulation factor (modulation factor);
when gamma =0, focal length is the cross entropy loss function, which can be adjusted
Figure 616466DEST_PATH_IMAGE010
Implementing a change in modulation factor;
two kinds of weights are combined to obtain:
Figure DEST_PATH_IMAGE074
)
wherein
Figure 227838DEST_PATH_IMAGE008
= 0.25 and
Figure 537597DEST_PATH_IMAGE010
= 1.5。
7. the method for measuring the ore size for deep learning based on the EfficientDet network and the early warning system according to claim 1, wherein the prediction result specifically comprises the following steps:
(Q1) in order to distinguish from the common feature layer, the feature layer extracted by BiFPN is called as an effective feature layer, and the five effective feature layers are transmitted into ClassNet (classification prediction network) and BoxNet (regression prediction network) to obtain a prediction result;
(Q2) for Efficientdet-B0, classsnet employs 3 convolutions of depth separable volume of 64 channels and 1 convolution of the number of prior boxes possessed by the feature layer times how many classes of objects the network has in common, the number of prior boxes being by default 9;
BoxNet adopts convolution of 3 times of 64 channels and convolution of 1 time of multiplying the prior frame number of the characteristic layer by four, wherein the four refers to the adjustment condition of the prior frame, and the adjustment of the central position and the width and the height are increased;
the EfficientDet defaults to have 9 prior frames, and the length-width ratio of the prior frames can be adjusted according to actual conditions so as to be suitable for a detected target;
and the EfficientDet judges the object in the prior frame and the type of the object, adjusts the prior frame, screens out the frame which belongs to the same type in a certain area and has the maximum confidence coefficient by using non-maximum suppression (soft-NMS), and obtains a final prediction frame.
8. The method for measuring the ore size for deep learning based on the EfficientDet network and the early warning system according to claim 1 are characterized by performing network model test, and specifically comprise the following steps:
(T1) transmitting a reserved ore test picture into an EfficientDet network for prediction to obtain a prediction frame of ore in the picture, calculating the size of the prediction frame, and calculating the size of the real ore according to the size of the prediction frame and an error;
(T2) calculating the true size of the belt from the mounting angle of the camera, the distance from the camera to the belt, the size of the prediction frame of the network model, and the pixels of the image:
Figure DEST_PATH_IMAGE075
Figure DEST_PATH_IMAGE076
is the included angle between the camera and the vertical line of the belt,
Figure 732955DEST_PATH_IMAGE077
which represents the true size of the ore,
Figure 786361DEST_PATH_IMAGE079
indicating the vertical distance between the camera and the belt,
Figure DEST_PATH_IMAGE080
which represents the focal length of the camera head,
Figure DEST_PATH_IMAGE082
the size of the EfficientDet network model prediction box is shown.
9. The method for measuring the ore size for deep learning and the early warning system based on the EfficientDet network as claimed in claim 1, wherein the method for measuring the ore size for deep learning and the early warning system based on the EfficientDet network are characterized in that the method for measuring the ore size for deep learning and the early warning system based on the EfficientDet network send out a prompt when the ore size exceeds a system threshold, and specifically comprise the following steps:
setting an ore size alarm threshold, setting the size of a system threshold as the sum of the alarm threshold and an error threshold, and comparing the actual size of the ore obtained through calculation with the system threshold;
if the real size is larger than the threshold value, generating abnormal information, sending the abnormal information to the real-time alarm module,
the alarm system according to the authority, characterized by comprising:
(D1) the user module is responsible for registration, login and management of users, displays personal user information and provides management authority of super users;
(D2) the ore video stream acquisition module is used for acquiring ore video streams through cameras arranged at various angles near the ore flow belt and sending acquired images to the real-time alarm module;
(D3) the real-time alarm module is used for receiving the ore image sent by the ore video stream acquisition module, transmitting the ore image into a trained EfficientDet network to obtain the position and the size of the ore, and comparing the actual size of the ore obtained by calculation with a system threshold;
if the real size of the abnormal information is larger than the system threshold value, displaying the abnormal information on a page, and writing the abnormal information into a database;
(D4) the setting module is used for setting system related parameters and log related parameters, such as the size of the alarm ore block and the like;
(D5) the log module is used for receiving the abnormal record of the real-time warning system, displaying the abnormal record and synchronizing the abnormal record to the cloud server;
the user module of the warning system specifically comprises:
(U1) providing registration and login functions for new managers, operating and using the whole system, storing information of the whole system to a cloud server, and giving corresponding authority;
(U2) providing a super administrator with high authority to operate the system and manage other users while operating and using the whole system;
the real-time warning module of the warning system specifically comprises:
(R1) industrial personal computer device management, including adding, deleting, modifying, listing corresponding industrial personal computer devices;
(R2) displaying a belt rotation screen of the added industrial personal computer device in real time; supporting the starting and stopping of displaying a belt rotation picture of the industrial personal computer equipment;
(R3) when the fact that the ore size of an industrial personal computer is larger than a system threshold value is detected, abnormal information is displayed, a user can select whether to stop the operation of the industrial personal computer or not according to the abnormal information, and information of the industrial personal computer recorded in an abnormal mode, abnormal time and images are stored in a database and written into a log module; synchronizing the abnormal records in the database to a cloud server at intervals;
the setting module of the warning system specifically comprises:
setting related information of an alarm system, wherein the related information mainly comprises the size of an ore block for setting early warning of the system, the size of a system error, the focal length of a camera, a cloud server port and the like;
the log module of the warning system specifically comprises:
(Z1) receiving the abnormal record of the real-time alarm system, displaying the abnormal record on a page in real time, and writing the abnormal record into a database;
(Z2) selecting whether to synchronize the log to the cloud server;
(Z3) provides a search function, possibly based on the information of the industrial personal computer equipment, time, threshold value, etc., for alarm records.
CN202110343840.8A 2021-03-31 2021-03-31 Deep learning ore size measuring method and early warning system based on EfficientDet network Pending CN113158829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110343840.8A CN113158829A (en) 2021-03-31 2021-03-31 Deep learning ore size measuring method and early warning system based on EfficientDet network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110343840.8A CN113158829A (en) 2021-03-31 2021-03-31 Deep learning ore size measuring method and early warning system based on EfficientDet network

Publications (1)

Publication Number Publication Date
CN113158829A true CN113158829A (en) 2021-07-23

Family

ID=76885508

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110343840.8A Pending CN113158829A (en) 2021-03-31 2021-03-31 Deep learning ore size measuring method and early warning system based on EfficientDet network

Country Status (1)

Country Link
CN (1) CN113158829A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113435425A (en) * 2021-08-26 2021-09-24 绵阳职业技术学院 Wild animal emergence and emergence detection method based on recursive multi-feature fusion
CN114462555A (en) * 2022-04-13 2022-05-10 国网江西省电力有限公司电力科学研究院 Multi-scale feature fusion power distribution network equipment identification method based on raspberry pi
CN116612311A (en) * 2023-03-13 2023-08-18 浙江大学 Sample imbalance-oriented unqualified immunohistochemical image recognition system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284780A (en) * 2018-09-10 2019-01-29 中山大学 Ore mineral image automatic identification and classification method
CN110390691A (en) * 2019-06-12 2019-10-29 合肥合工安驰智能科技有限公司 A kind of ore scale measurement method and application system based on deep learning
CN110852395A (en) * 2019-11-15 2020-02-28 鞍钢集团矿业有限公司 Ore granularity detection method and system based on autonomous learning and deep learning
CN111161292A (en) * 2019-11-21 2020-05-15 合肥合工安驰智能科技有限公司 Ore size measurement method and application system
CN111563494A (en) * 2020-07-16 2020-08-21 平安国际智慧城市科技股份有限公司 Behavior identification method and device based on target detection and computer equipment
CN112001878A (en) * 2020-05-21 2020-11-27 合肥合工安驰智能科技有限公司 Deep learning ore scale measuring method based on binarization neural network and application system
CN112348776A (en) * 2020-10-16 2021-02-09 上海布眼人工智能科技有限公司 Fabric flaw detection method based on EfficientDet
CN112419397A (en) * 2020-12-08 2021-02-26 鞍钢集团矿业有限公司 Ore particle size grading method and system based on image and deep neural network

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284780A (en) * 2018-09-10 2019-01-29 中山大学 Ore mineral image automatic identification and classification method
CN110390691A (en) * 2019-06-12 2019-10-29 合肥合工安驰智能科技有限公司 A kind of ore scale measurement method and application system based on deep learning
CN110852395A (en) * 2019-11-15 2020-02-28 鞍钢集团矿业有限公司 Ore granularity detection method and system based on autonomous learning and deep learning
CN111161292A (en) * 2019-11-21 2020-05-15 合肥合工安驰智能科技有限公司 Ore size measurement method and application system
CN112001878A (en) * 2020-05-21 2020-11-27 合肥合工安驰智能科技有限公司 Deep learning ore scale measuring method based on binarization neural network and application system
CN111563494A (en) * 2020-07-16 2020-08-21 平安国际智慧城市科技股份有限公司 Behavior identification method and device based on target detection and computer equipment
CN112348776A (en) * 2020-10-16 2021-02-09 上海布眼人工智能科技有限公司 Fabric flaw detection method based on EfficientDet
CN112419397A (en) * 2020-12-08 2021-02-26 鞍钢集团矿业有限公司 Ore particle size grading method and system based on image and deep neural network

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
MINGXING TAN ET AL: "EfficientDet: Scalable and Efficient Object Detection", 《ARXIV》 *
MINGXING TAN ET AL: "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks", 《ARXIV》 *
余阿祥 等: "多注意力机制的口罩检测网络", 《南京师范大学学报(工程技术版)》 *
董洪义: "《深度学习之PyTorch物体检测实战》", 31 January 2020, 《机械工业出版社》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113435425A (en) * 2021-08-26 2021-09-24 绵阳职业技术学院 Wild animal emergence and emergence detection method based on recursive multi-feature fusion
CN114462555A (en) * 2022-04-13 2022-05-10 国网江西省电力有限公司电力科学研究院 Multi-scale feature fusion power distribution network equipment identification method based on raspberry pi
US11631238B1 (en) 2022-04-13 2023-04-18 Iangxi Electric Power Research Institute Of State Grid Method for recognizing distribution network equipment based on raspberry pi multi-scale feature fusion
CN116612311A (en) * 2023-03-13 2023-08-18 浙江大学 Sample imbalance-oriented unqualified immunohistochemical image recognition system

Similar Documents

Publication Publication Date Title
CN110390691B (en) Ore dimension measuring method based on deep learning and application system
CN113158829A (en) Deep learning ore size measuring method and early warning system based on EfficientDet network
US9805451B2 (en) Building material classifications from imagery
CN111161292B (en) Ore scale measurement method and application system
CN109580656B (en) Mobile phone light guide plate defect detection method and system based on dynamic weight combination classifier
CN110751195B (en) Fine-grained image classification method based on improved YOLOv3
CN109409327B (en) RRU module object pose detection method based on end-to-end deep neural network
CN111209858B (en) Real-time license plate detection method based on deep convolutional neural network
CN112001878A (en) Deep learning ore scale measuring method based on binarization neural network and application system
CN104202547A (en) Method for extracting target object in projection picture, projection interaction method and system thereof
CN115331172A (en) Workshop dangerous behavior recognition alarm method and system based on monitoring video
CN115169855B (en) Unsafe state detection method based on digital twin workshop mixed data set
CN117011295B (en) UHPC prefabricated member quality detection method based on depth separable convolutional neural network
CN104318254A (en) Quick coal and rock recognition method based on DCT low-frequency component characteristics
CN111563408B (en) High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning
CN112785557A (en) Belt material flow detection method and device and belt material flow detection system
CN111507379A (en) Ore automatic identification and rough sorting system based on deep learning
CN116152226A (en) Method for detecting defects of image on inner side of commutator based on fusible feature pyramid
CN111507398A (en) Transformer substation metal instrument corrosion identification method based on target detection
CN113378825B (en) Sandstone sheet image identification method and system based on artificial intelligence
CN117152617A (en) Urban flood identification method and system
CN115272940B (en) Neural network-based intelligent rock debris particle metering method and system
CN117076983A (en) Transmission outer line resource identification detection method, device, equipment and storage medium
CN115497006B (en) Urban remote sensing image change depth monitoring method and system based on dynamic mixing strategy
CN111709991B (en) Railway tool detection method, system, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210723

WD01 Invention patent application deemed withdrawn after publication