CN116543303A - Bridge plate type rubber support disease identification and detection method based on deep learning - Google Patents
Bridge plate type rubber support disease identification and detection method based on deep learning Download PDFInfo
- Publication number
- CN116543303A CN116543303A CN202310488349.3A CN202310488349A CN116543303A CN 116543303 A CN116543303 A CN 116543303A CN 202310488349 A CN202310488349 A CN 202310488349A CN 116543303 A CN116543303 A CN 116543303A
- Authority
- CN
- China
- Prior art keywords
- detection
- rubber support
- type rubber
- bridge plate
- plate type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 149
- 201000010099 disease Diseases 0.000 title claims abstract description 117
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 117
- 238000013135 deep learning Methods 0.000 title claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 46
- 238000003062 neural network model Methods 0.000 claims abstract description 28
- 238000000034 method Methods 0.000 claims description 14
- 230000000694 effects Effects 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 239000011800 void material Substances 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000010008 shearing Methods 0.000 claims description 6
- 238000011282 treatment Methods 0.000 claims description 6
- 230000002068 genetic effect Effects 0.000 claims description 5
- 238000003064 k means clustering Methods 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 4
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 238000005336 cracking Methods 0.000 claims description 3
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000007689 inspection Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims 1
- 230000001629 suppression Effects 0.000 abstract description 3
- 238000013527 convolutional neural network Methods 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 238000009440 infrastructure construction Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of bridge disease detection, and relates to a bridge plate type rubber support disease identification detection method based on deep learning, which comprises the following steps: building a neural network model, training an initial detection model, constructing a disease data set, training a detection model, and detecting image diseases; according to the invention, the bridge plate type rubber support disease data set can be constructed by marking part of the image, manually correcting and inputting the data into the model for training for many times, so that the semi-automatic marking of the bridge plate type rubber support data is realized, the marking workload is saved, the marking errors and the uneven marking formats caused by manual marking are reduced, the recognition and detection of the bridge plate type rubber support disease are realized, the marking time is saved, and the Soft-NMS is used for replacing NMS non-maximum suppression to obtain a better target detection frame, and the detection efficiency and accuracy are improved.
Description
Technical Field
The invention belongs to the technical field of bridge disease detection, and relates to a bridge plate type rubber support disease identification detection method based on deep learning.
Background
With the continuous development of infrastructure construction in China in recent years, the number of road and bridge construction shows a rapid increase trend. The bridge rubber support is used as a key component of a bridge, has an important effect on ensuring the health and normal traffic of the bridge structure, and can influence the stress state of the whole bridge structure once the rubber support is damaged, thereby threatening the traffic safety of passing vehicles and pedestrians. Therefore, detection of bridge rubber supports and identification of rubber support diseases are indispensable in bridge maintenance work. The traditional bridge plate type rubber support detection method is manual in-situ detection, and is time-consuming, labor-consuming, high in cost, and large in manpower and material resources, and is not in line with the current increasing bridge maintenance requirements. At present, the identification of the bridge rubber support at home and abroad is realized by the transition of automatic extraction detected by the traditional manual field detection, the transition greatly improves the efficiency of detecting the bridge rubber support diseases, and provides data support for bridge maintenance management. However, the diseases of the traditional digital image recognition bridge plate type rubber support are easily affected by illumination, and recognition errors or leakage results are easily caused; the labeling workload of the data set is large, a large amount of manpower and time are consumed, the labeling errors are easily caused by the artificial subjective labeling data, and the labeling among different staff is easily caused by the non-uniformity of the labeling formats; the bridge rubber support crack has slender, irregular and discontinuous mathematical morphology, and the deep learning method based on regression frame prediction is difficult to obtain good effect on slender objects.
In the prior art, chinese patent CN108288269A discloses an automatic identification method for bridge plate type rubber support diseases based on unmanned aerial vehicle and convolutional neural network, which comprises the following steps: acquiring a photo of an automatic identification method of the bridge plate type rubber support diseases, and increasing the data volume for training a convolutional neural network by using an image processing method; dividing the obtained photos of the bridge plate type rubber support disease automatic identification method into a training set and a testing set; establishing a convolutional neural network, and iteratively training weights of all layers of the convolutional neural network through a gradient descent method and a counter propagation algorithm; obtaining a convolutional neural network model with the function of automatically identifying the bridge plate type rubber support diseases; the ground control system controls the unmanned aerial vehicle to cruise, and an image acquisition device carried by the unmanned aerial vehicle is used for acquiring a photo of the bridge plate type rubber support; data acquired by the unmanned aerial vehicle are transmitted into a cloud for data processing, and a trained convolutional neural network model is used for automatic recognition of the bridge plate type rubber support disease automatic recognition method; chinese patent CN110222701B discloses an automatic identifying method for bridge diseases, comprising the following steps: s1: constructing a bridge disease detection data set; s2: dividing the data set into a training set and a testing set; s3: constructing a target detection model of bridge diseases based on a convolutional neural network SSD; training the constructed target detection model of the bridge disease in the step S3, evaluating the trained target detection model of the bridge disease, and if the evaluation standard is passed, inputting the bridge image with the disease to be detected into the target detection model with updated parameters, and determining the final position, type and size of the disease through feature fusion; chinese patent CN115713647a discloses a three-stage recognition method and recognition system for bridge apparent information based on deep learning, which comprises classifying training data sets step by step to obtain training data sets of each bridge part, training data sets of each bridge member, and training data sets of each disease class; then, respectively training an improved VGG16 model by using the previous training data set to obtain a part recognition model of each bridge part, a component recognition model of each component and a disease recognition model of each disease category to form a three-stage recognition model; finally, inputting the bridge apparent image to be predicted into a part recognition model to recognize the bridge part corresponding to the image, and then inputting a component recognition model corresponding to the bridge part to recognize a component corresponding to the image; and finally, inputting a disease identification model of the corresponding member, and outputting the disease category of the image to be predicted, which contains the bridge apparent information. However, in the prior art, semi-automatic marking of bridge plate type rubber support data cannot be achieved by constructing a training set, so that a high-efficiency comprehensive disease detection model is trained, and the detection complexity is high.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and designs and provides a bridge plate type rubber support disease identification and detection method based on deep learning, which solves the problem that the semi-automatic marking of bridge plate type rubber support data can not be realized by constructing a training set in the prior art, so that a high-efficiency comprehensive disease detection model is trained.
In order to achieve the purpose, the bridge plate type rubber support disease identification and detection method based on deep learning provided by the invention specifically comprises the following steps:
s1: building a neural network model: building a bridge plate type rubber support disease detection neural network model, and finally outputting the network as predicted disease category and position information on an image;
s2: training an initial detection model: collecting bridge plate type rubber support images, marking the collected partial images to obtain initial data, and training the bridge plate type rubber support disease detection neural network model built in the step S1 by using the initial data to obtain an initial detection model suitable for a bridge plate type rubber support disease detection task;
s3: constructing a disease data set: detecting an image of an unlabeled part in the image acquired in the step S2 by using an initial detection model, manually correcting the detection result to obtain correction data, combining and processing the correction data and the initial data to obtain a bridge plate type rubber support disease data set;
s4: training a detection model: the bridge plate type rubber support disease detection neural network model built in the step S1 is trained by using the bridge plate type rubber support disease data set, so that a detection model suitable for a bridge plate type rubber support disease detection task is obtained;
s5: detecting image diseases: and (3) inputting the image to be detected into the detection model obtained in the step S4 for detection, deleting other repeated results, and retaining the detection result with the best effect.
The invention relates to a bridge plate type rubber support disease identification detection method based on deep learning, wherein the step S1 specifically comprises the following steps:
s11: building a bridge plate type rubber support disease detection neural network model based on the YOLOv4 neural network model; the system comprises an image preprocessing module, a detection module and a loss function module;
s12: an image preprocessing module is established and is used for uniformly adjusting the size of an input image to 608 x 608 or 416 x 416; 4 input images are acquired at one time by using a Moscaic data enhancement mode and spliced in a random scaling, random cutting and random arrangement mode so as to enrich the background of an object to be detected and improve the training speed of a model;
s13: the method comprises the steps of establishing a detection module, wherein the detection module is used for adding a CSPX module after each residual block of a DarkNet53 to form a main network CSPDarkNet53 so as to extract characteristic information of an input image, and the CSPX module is formed by connecting a convolution layer and X Resblock modules; adding a receptive field by using a space pooling pyramid structure, converting the extracted characteristic information into coordinates by using a path aggregation network PANet, carrying out characteristic fusion by using upsampling and downsampling, and outputting three characteristic graphs with the sizes of 77 x 255, 38 x 355 and 19 x 255; comparing the output result with a real label, and calculating a loss function, so as to predict the image characteristics, generate a boundary box and predict the category;
s14: a loss function module is established, and the loss function consists of three parts: location loss (LOSScoor), classification loss (losmclass), confidence loss (losdconf);
wherein the position loss is defined as follows:
the CIOU used for the location loss is defined as follows:
the classification loss is defined as follows:
confidence loss is defined as follows:
lambda in the above formula (1), formula (2), formula (3) and formula (4) coor Lambda is the positive sample weight coefficient noobj Is a negative sample weight coefficient and is used to determine the weight of the sample,judging whether the sample is a positive sample, wherein the value is not 0, namely 1; c is a category index, which represents a predicted category, and the value range of the category index is from 0 to C-1, wherein C is a category number, C epsilon class is a target category number to be detected, and c=4; k is the mesh size, i.e. the image is divided into K meshes, each of which is responsible for predicting a bounding box,/K->Indicating traversing all prediction frames, kxk=13×13+26×26+52×52, m=3, +.>For the sample value, C i Is a predicted value; b, b gt Representing the center points of the predicted and real frames, respectively, ρ representing twoThe Euclidean distance between the center points; c represents the diagonal distance of the minimum closure area capable of containing both predicted and real frames,/c>Is a parameter for balancing the proportion, v is used for measuring the proportion consistency between the anchor frame and the real frame; wi: prediction frame center point width, hi: the center point of the prediction frame is high; p≡ (c) is the probability that the j-th bounding box in the i-th grid predicts belonging to class c; αt is a weight coefficient, which represents different treatments of positive samples (the true class is c) and negative samples (the true class is not c), the value ranges from 0 to 1, and generally, αt is larger for the positive samples of the minority class and smaller for the negative samples of the majority class; gamma is an adjustment factor representing the different treatments of difficult samples, ranging from 0 to infinity, and generally, gamma amplifies their losses for difficult samples (low prediction probability) and reduces them for easy samples (high prediction probability).
The invention relates to a bridge plate type rubber support disease identification detection method based on deep learning, wherein the step S2 specifically comprises the following steps:
s21: in the manual inspection of the bridge, the images of the bridge plate-type rubber support are collected through the image collecting devices such as a mobile phone, an unmanned aerial vehicle and a camera, and the collected images are front views of the bridge plate-type rubber support, so that the bridge plate-type rubber support and surrounding structures thereof can be clearly reflected;
s22: screening the images acquired in the step S21, and picking out images capable of clearly reflecting support breakage, support void and support shearing deformation; selecting part of the images, and marking the obtained images of the support damage, support Void, support shear deformation and support by using LabelImg marking software in an xml format, wherein the images are respectively marked as a Crack damage, a Void, shear deformation and a Bearing to obtain initial data;
s23: and (3) obtaining a priori frame suitable for the data set by using a k-means clustering and genetic algorithm, training the bridge plate type rubber support disease detection neural network model constructed in the step (S1) by using the artificial annotation data obtained in the step (S22), and reserving model weight with the best effect to obtain an initial model.
The invention relates to a bridge plate type rubber support disease identification detection method based on deep learning, wherein the step S3 specifically comprises the following steps:
s31: inputting an image of an unlabeled part in the image acquired in the step S2 into an initial model for detection, taking a detection result as an unlabeled data label, examining a label result, and manually correcting the conditions of missing labels, label errors and label inaccuracy to obtain correction data;
s32: combining initial data and correction data, randomly adjusting brightness to 0.35-1.2 times of original brightness, randomly translating (the maximum translation amount is equal to the boundary of a marking frame), randomly rotating (the rotation angle range is-15 degrees to +15 degrees), horizontally mirror-turning and correcting coordinates of the marking frame, and amplifying all obtained marked sample images, wherein a data set is divided into a training set, a verification set and a test set according to a ratio of 6:2:2, so that a bridge plate type rubber support disease data set is obtained.
The invention relates to a bridge plate type rubber support disease identification detection method based on deep learning, wherein the step S4 specifically comprises the following steps:
s41: inputting a training set in the bridge plate type rubber support disease data set into the bridge plate type rubber support disease detection neural network model constructed in the step S1 to train the bridge plate type rubber support disease detection neural network model;
s42: judging whether the model is fitted or not by checking the change relation of the loss values of the training set and the verification set along with the epoch, if so, stopping training in time, adjusting the model structure and the super parameters according to the situation, determining the performance gap possessed by the super parameters by the performance on the verification set, and obtaining the detection model suitable for the bridge plate type rubber support disease detection task by verifying on the verification set, self-adaptively adjusting the super parameters of the network model and reserving the model weight with the best effect.
The invention relates to a bridge plate type rubber support disease identification detection method based on deep learning, wherein the step S5 specifically comprises the following steps:
s51: firstly, dividing an input rubber support picture into 76 x 76 grids after 8 times downsampling, into 38 x 38 grids after 16 times downsampling, and into 19 x 19 grids after 32 times downsampling; each grid is responsible for the target of the grid to which the prediction center falls, 3 prediction frames are calculated, and each prediction frame corresponds to 5+4 values; wherein 4 represents the total number of categories in the data set, namely rubber support, cracking and breakage, void and shearing deformation, and 5 represents the center point coordinates (x, y) of the prediction boundary frame, the width and height dimensions (w, h) of the frame and confidence parameter information; the input image size is 608 x 3, the model finally outputs 76 x 27, 38 x 27, 19 x 27 tensors (wherein 27= (4+5) x 3,4 is the total number of categories on the bridge plate type rubber support disease data set), and the three tensors are analyzed, and a Soft-NMS is used for screening out a prediction frame with higher confidence value as a target detection frame, so that the positioning of the target is realized, and the category, the confidence and the coordinate value of the detection frame are output;
s52: the detection frame obtained in step S51 is drawn on the input image, and the category and confidence information are annotated as output images, and the category, the detection frame coordinate value and the confidence are output to the txt file according to the image name.
Compared with the prior art, the invention has the following advantages: (1) According to the method, the disease data set of the bridge plate-type rubber support is constructed by marking part of the images, manually correcting and inputting the data into the model for training for many times, and the semi-automatic marking method of the plate-type bridge rubber support is adopted from the data marking layer, so that the marking workload is saved, and the problems of marking errors and non-uniform marking formats caused by manual marking are reduced; (2) The k-means clustering and genetic algorithm are adopted on the model to obtain the priori frame suitable for the data set, so that the aspect ratio of the priori frame is more in line with the disease distribution characteristics of the plate-type bridge rubber support, and the Soft-NMS is used for replacing NMS non-maximum suppression to obtain a better target detection frame, so that the detection accuracy and efficiency are improved.
Drawings
Fig. 1 is a process flow block diagram of a bridge plate type rubber support disease identification and detection method based on deep learning.
Detailed Description
The technical scheme of the invention is further described below with reference to the accompanying drawings and specific embodiments.
Example 1:
the embodiment relates to a bridge plate type rubber support disease identification and detection method based on deep learning, which comprises the following steps:
s1: building a neural network model: building a bridge plate type rubber support disease detection neural network model, and finally outputting the network as predicted disease category and position information on an image;
s2: training an initial detection model: collecting bridge plate type rubber support images, marking the collected partial images to obtain initial data, and training the bridge plate type rubber support disease detection neural network model built in the step S1 by using the initial data to obtain an initial detection model suitable for a bridge plate type rubber support disease detection task;
s3: constructing a disease data set: detecting an image of an unlabeled part in the image acquired in the step S2 by using an initial detection model, manually correcting the detection result to obtain correction data, combining and processing the correction data and the initial data to obtain a bridge plate type rubber support disease data set;
s4: training a detection model: the bridge plate type rubber support disease detection neural network model built in the step S1 is trained by using the bridge plate type rubber support disease data set, so that a detection model suitable for a bridge plate type rubber support disease detection task is obtained;
s5: detecting image diseases: and (3) inputting the image to be detected into the detection model obtained in the step S4 for detection, deleting other repeated results, and retaining the detection result with the best effect.
The bridge plate type rubber support disease identification and detection method based on deep learning, which is related to in the embodiment, specifically comprises the following steps of:
s11: building a bridge plate type rubber support disease detection neural network model based on the YOLOv4 neural network model; the system comprises an image preprocessing module, a detection module and a loss function module;
s12: an image preprocessing module is established and is used for uniformly adjusting the size of an input image to 608 x 608 or 416 x 416; 4 input images are acquired at one time by using a Moscaic data enhancement mode and spliced in a random scaling, random cutting and random arrangement mode so as to enrich the background of an object to be detected and improve the training speed of a model;
s13: the method comprises the steps of establishing a detection module, wherein the detection module is used for adding a CSPX module after each residual block of a DarkNet53 to form a main network CSPDarkNet53 so as to extract characteristic information of an input image, and the CSPX module is formed by connecting a convolution layer and X Resblock modules; adding a receptive field by using a space pooling pyramid structure, converting the extracted characteristic information into coordinates by using a path aggregation network PANet, carrying out characteristic fusion by using upsampling and downsampling, and outputting three characteristic graphs with the sizes of 77 x 255, 38 x 355 and 19 x 255; comparing the output result with a real label, and calculating a loss function, so as to predict the image characteristics, generate a boundary box and predict the category;
s14: a loss function module is established, and the loss function consists of three parts: location loss (LOSScoor), classification loss (losmclass), confidence loss (losdconf);
wherein the position loss is defined as follows:
the CIOU used for the location loss is defined as follows:
the classification loss is defined as follows:
confidence loss is defined as follows:
lambda in the above formula (1), formula (2), formula (3) and formula (4) coor Lambda is the positive sample weight coefficient noobj Is a negative sample weight coefficient and is used to determine the weight of the sample,judging whether the sample is a positive sample, wherein the value is not 0, namely 1; c is a category index, which represents a predicted category, and the value range of the category index is from 0 to C-1, wherein C is a category number, C epsilon class is a target category number to be detected, and c=4; k is the mesh size, i.e. the image is divided into K meshes, each of which is responsible for predicting a bounding box,/K->Indicating traversing all prediction frames, kxk=13×13+26×26+52×52, m=3, +.>For the sample value, C i Is a predicted value; b, b gt Representing the center points of the prediction frame and the real frame respectively, wherein ρ represents the Euclidean distance between the two center points; c represents the diagonal distance of the minimum closure area capable of containing both predicted and real frames,/c>Is a parameter for balancing the proportion, v is used for measuring the proportion consistency between the anchor frame and the real frame; wi: prediction frame center point width, hi: the center point of the prediction frame is high; p≡ (c) is the probability that the j-th bounding box in the i-th grid predicts belonging to class c; αt is a weight coefficient, which represents different treatments of positive samples (the true class is c) and negative samples (the true class is not c), the value ranges from 0 to 1, and generally, αt is larger for the positive samples of the minority class and smaller for the negative samples of the majority class; gamma is an adjustment factor representing the different treatments of difficult samples, ranging from 0 to infinity, and generally for difficult samples (low predictive probability), gamma amplifies themFor easy samples (high prediction probability), γ reduces their loss.
The bridge plate type rubber support disease identification and detection method based on deep learning, which is related to in the embodiment, specifically comprises the following steps of:
s21: in the manual inspection of the bridge, the images of the bridge plate-type rubber support are collected through the image collecting devices such as a mobile phone, an unmanned aerial vehicle and a camera, and the collected images are front views of the bridge plate-type rubber support, so that the bridge plate-type rubber support and surrounding structures thereof can be clearly reflected;
s22: screening the images acquired in the step S21, and picking out images capable of clearly reflecting support breakage, support void and support shearing deformation; selecting part of the images, and marking the obtained images of the support damage, support Void, support shear deformation and support by using LabelImg marking software in an xml format, wherein the images are respectively marked as a Crack damage, a Void, shear deformation and a Bearing to obtain initial data;
s23: and (3) obtaining a priori frame suitable for the data set by using a k-means clustering and genetic algorithm, training the bridge plate type rubber support disease detection neural network model constructed in the step (S1) by using the artificial annotation data obtained in the step (S22), and reserving model weight with the best effect to obtain an initial model.
The bridge plate type rubber support disease identification and detection method based on deep learning, which is related to in the embodiment, specifically comprises the following steps of:
s31: inputting an image of an unlabeled part in the image acquired in the step S2 into an initial model for detection, taking a detection result as an unlabeled data label, examining a label result, and manually correcting the conditions of missing labels, label errors and label inaccuracy to obtain correction data;
s32: combining initial data and correction data, randomly adjusting brightness to 0.35-1.2 times of original brightness, randomly translating (the maximum translation amount is equal to the boundary of a marking frame), randomly rotating (the rotation angle range is-15 degrees to +15 degrees), horizontally mirror-turning and correcting coordinates of the marking frame, and amplifying all obtained marked sample images, wherein a data set is divided into a training set, a verification set and a test set according to a ratio of 6:2:2, so that a bridge plate type rubber support disease data set is obtained.
The bridge plate type rubber support disease identification and detection method based on deep learning, which is related to in the embodiment, specifically comprises the following steps of:
s41: inputting a training set in the bridge plate type rubber support disease data set into the bridge plate type rubber support disease detection neural network model constructed in the step S1 to train the bridge plate type rubber support disease detection neural network model;
s42: judging whether the model is fitted or not by checking the change relation of the loss values of the training set and the verification set along with the epoch, if so, stopping training in time, adjusting the model structure and the super parameters according to the situation, determining the performance gap possessed by the super parameters by the performance on the verification set, and obtaining the detection model suitable for the bridge plate type rubber support disease detection task by verifying on the verification set, self-adaptively adjusting the super parameters of the network model and reserving the model weight with the best effect.
The bridge plate type rubber support disease identification and detection method based on deep learning, which is related to in the embodiment, specifically comprises the following steps of:
s51: firstly, dividing an input rubber support picture into 76 x 76 grids after 8 times downsampling, into 38 x 38 grids after 16 times downsampling, and into 19 x 19 grids after 32 times downsampling; each grid is responsible for the target of the grid to which the prediction center falls, 3 prediction frames are calculated, and each prediction frame corresponds to 5+4 values; wherein 4 represents the total number of categories in the data set, namely rubber support, cracking and breakage, void and shearing deformation, and 5 represents the center point coordinates (x, y) of the prediction boundary frame, the width and height dimensions (w, h) of the frame and confidence parameter information; the input image size is 608 x 3, the model finally outputs 76 x 27, 38 x 27, 19 x 27 tensors (wherein 27= (4+5) x 3,4 is the total number of categories on the bridge plate type rubber support disease data set), and the three tensors are analyzed, and a Soft-NMS is used for screening out a prediction frame with higher confidence value as a target detection frame, so that the positioning of the target is realized, and the category, the confidence and the coordinate value of the detection frame are output;
s52: the detection frame obtained in step S51 is drawn on the input image, and the category and confidence information are annotated as output images, and the category, the detection frame coordinate value and the confidence are output to the txt file according to the image name.
Example 2:
the implementation of this embodiment is to verify the actual detection effect of the bridge plate type rubber support disease identification detection method based on deep learning in embodiment 1, and mainly performs the following comparative experiment:
(1) Experimental facilities: the bridge plate type rubber support disease identification and detection method based on deep learning according to the embodiment 1 is used;
(2) The experimental method comprises the following steps: and inputting the image to be detected into a detection model suitable for the bridge plate type rubber support disease detection task, and carrying out detection evaluation according to the detection speed and the calculation speed.
(3) Experimental results:
table 1 teaching results table
| Name of the name | Actual results | Desired results |
| Accuracy rate of | 100％ | 100％ |
| Detection time | 10ms | <0.1s |
| Detection effect | Excellent in | Excellent in |
As can be seen from Table 1, the bridge plate-type rubber support disease recognition detection method based on deep learning provided by the invention constructs a bridge plate-type rubber support disease data set by marking part of images, manually correcting and inputting data into a model for training for many times, saves the marking workload, reduces marking errors, increases the detection accuracy after model training, obtains a priori frame suitable for the data set by adopting k-means clustering and genetic algorithm on the model, enables the aspect ratio of the priori frame to be more in line with the disease distribution characteristics of the plate-type bridge rubber support, and obtains a better target detection frame by replacing NMS non-maximum suppression with Soft-NMS, thereby further improving the detection accuracy and efficiency and having good application prospects.
Claims (10)
1. A bridge plate type rubber support disease identification and detection method based on deep learning is characterized by comprising the following steps of: the method comprises the following steps:
s1: building a neural network model: building a bridge plate type rubber support disease detection neural network model, and finally outputting the network as predicted disease category and position information on an image;
s2: training an initial detection model: collecting bridge plate type rubber support images, marking the collected partial images to obtain initial data, and training the bridge plate type rubber support disease detection neural network model built in the step S1 by using the initial data to obtain an initial detection model suitable for a bridge plate type rubber support disease detection task;
s3: constructing a disease data set: detecting an image of an unlabeled part in the image acquired in the step S2 by using an initial detection model, manually correcting the detection result to obtain correction data, combining and processing the correction data and the initial data to obtain a bridge plate type rubber support disease data set;
s4: training a detection model: the bridge plate type rubber support disease detection neural network model built in the step S1 is trained by using the bridge plate type rubber support disease data set, so that a detection model suitable for a bridge plate type rubber support disease detection task is obtained;
s5: detecting image diseases: and (3) inputting the image to be detected into the detection model obtained in the step S4 for detection, deleting other repeated results, and retaining the detection result with the best effect.
2. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 1, which is characterized in that: the step S1 specifically comprises the following steps:
s11: building a bridge plate type rubber support disease detection neural network model based on the YOLOv4 neural network model; the system comprises an image preprocessing module, a detection module and a loss function module;
s12: an image preprocessing module is established and is used for uniformly adjusting the size of an input image to 608 x 608 or 416 x 416; 4 input images are acquired at one time by using a Moscaic data enhancement mode and spliced in a random scaling, random cutting and random arrangement mode so as to enrich the background of an object to be detected and improve the training speed of a model;
s13: the method comprises the steps of establishing a detection module, wherein the detection module is used for adding a CSPX module after each residual block of a DarkNet53 to form a main network CSPDarkNet53 so as to extract characteristic information of an input image, and the CSPX module is formed by connecting a convolution layer and X Resblock modules; adding a receptive field by using a space pooling pyramid structure, converting the extracted characteristic information into coordinates by using a path aggregation network PANet, carrying out characteristic fusion by using upsampling and downsampling, and outputting three characteristic graphs with the sizes of 77 x 255, 38 x 355 and 19 x 255; comparing the output result with a real label, and calculating a loss function, so as to predict the image characteristics, generate a boundary box and predict the category;
s14: a loss function module is established, and the loss function consists of three parts: location loss (LOSScoor), classification loss (losvclass), confidence loss (losdconf).
3. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 2, which is characterized in that: the position loss is defined as follows:
the CIOU used for the position loss is defined as follows:
the classification loss is defined as follows:
the confidence loss is defined as follows:
lambda in the above formula (1), formula (2), formula (3) and formula (4) coor Lambda is the positive sample weight coefficient noobj Is a negative sample weight coefficient and is used to determine the weight of the sample,judging whether the sample is a positive sample, wherein the value is not 0, namely 1; c is a category index, which represents a predicted category, and the value range of the category index is from 0 to C-1, wherein C is a category number, C epsilon class is a target category number to be detected, and c=4; k is the mesh size, i.e. the image is divided into K meshes, each of which is responsible for predicting a bounding box,/K->Indicating traversing all prediction frames, kxk=13×13+26×26+52×52, m=3, +.>For the sample value, C i Is a predicted value; b, b gt Representing the center points of the prediction frame and the real frame respectively, wherein ρ represents the Euclidean distance between the two center points; c represents the diagonal distance of the minimum closure area capable of containing both predicted and real frames,/c>Is a parameter for balancing the proportion, v is used for measuring the proportion consistency between the anchor frame and the real frame; wi: prediction frame center point width, hi: the center point of the prediction frame is high; p≡ (c) is the probability that the j-th bounding box in the i-th grid predicts belonging to class c; αt is a weight coefficient representing different treatments for positive samples, i.e. samples with a true class c, and negative samples, i.e. samples with a true class other than c, ranging from 0 to 1; gamma is an adjustment coefficient, and represents different treatments of difficult samples, and the value range is from 0 to infinity.
4. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 1, which is characterized in that: the step S2 specifically comprises the following steps:
s21: in the manual inspection of the bridge, the images of the bridge plate-type rubber support are collected through the image collecting devices such as a mobile phone, an unmanned aerial vehicle and a camera, and the collected images are front views of the bridge plate-type rubber support, so that the bridge plate-type rubber support and surrounding structures thereof can be clearly reflected;
s22: screening the images acquired in the step S21, and picking out images capable of clearly reflecting support breakage, support void and support shearing deformation; selecting part of the images, and marking the obtained images of the support damage, support Void, support shear deformation and support by using LabelImg marking software in an xml format, wherein the images are respectively marked as a Crack damage, a Void, shear deformation and a Bearing to obtain initial data;
s23: and (3) obtaining a priori frame suitable for the data set by using a k-means clustering and genetic algorithm, training the bridge plate type rubber support disease detection neural network model constructed in the step (S1) by using the artificial annotation data obtained in the step (S22), and reserving model weight with the best effect to obtain an initial model.
5. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 1, which is characterized in that: the step S3 specifically comprises the following steps:
s31: inputting an image of an unlabeled part in the image acquired in the step S2 into an initial model for detection, taking a detection result as an unlabeled data label, examining a label result, and manually correcting the conditions of missing labels, label errors and label inaccuracy to obtain correction data;
s32: and combining the initial data with the correction data, and dividing the data set into a training set, a verification set and a test set according to a ratio of 6:2:2 by processing all the obtained marked sample images to obtain the bridge plate type rubber support disease data set.
6. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 5, which is characterized in that: the processing is specifically to randomly adjust the brightness to 0.35-1.2 times of the original brightness, randomly translate the maximum translation to the boundary of the labeling frame, randomly rotate the labeling frame in a rotation angle range of-15 degrees to +15 degrees, horizontally mirror-flip the labeling frame and correct the coordinates of the labeling frame, and amplify all the obtained labeled sample images.
7. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 1, which is characterized in that: the step S4 specifically comprises the following steps:
s41: inputting a training set in the bridge plate type rubber support disease data set into the bridge plate type rubber support disease detection neural network model constructed in the step S1 to train the bridge plate type rubber support disease detection neural network model;
s42: judging whether the model is fitted or not by checking the change relation of the loss values of the training set and the verification set along with the epoch, if so, stopping training in time, adjusting the model structure and the super parameters according to the situation, determining the performance gap possessed by the super parameters by the performance on the verification set, and obtaining the detection model suitable for the bridge plate type rubber support disease detection task by verifying on the verification set, self-adaptively adjusting the super parameters of the network model and reserving the model weight with the best effect.
8. The bridge plate type rubber support disease identification and detection method based on deep learning according to claim 1, which is characterized in that: the step S5 specifically comprises the following steps:
s51: firstly, dividing an input rubber support picture into a plurality of grids after three downsampling; each grid is responsible for the target of the grid to which the prediction center falls, 3 prediction frames are calculated, and each prediction frame corresponds to 5+4 values; wherein 4 represents the total number of categories in the data set, namely rubber support, cracking and breakage, void and shearing deformation, and 5 represents the center point coordinates (x, y) of the prediction boundary frame, the width and height dimensions (w, h) of the frame and confidence parameter information; according to the size of an input image, the model finally outputs tensors of a plurality of scales, and a predictive frame with a higher confidence coefficient value is screened out by using a Soft-NMS as a target detection frame through analysis of the tensors, so that the positioning of the target is realized, and the category, the confidence coefficient and the coordinate value of the detection frame are output;
s52: the detection frame obtained in step S51 is drawn on the input image, and the category and confidence information are annotated as output images, and the category, the detection frame coordinate value and the confidence are output to the txt file according to the image name.
9. The bridge plate type rubber support disease identification and detection method based on deep learning of claim 8 is characterized in that: the three downsampling grids are respectively and specifically divided into 76 grids by 8 times, 38 grids by 16 times and 19 grids by 32 times.
10. The bridge plate type rubber support disease identification and detection method based on deep learning of claim 8 is characterized in that: the input image size is 608 x 3, and the final output of the model is tensor of three scales of 76 x 27, 38 x 27 and 19 x 27.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310488349.3A CN116543303A (en) | 2023-05-04 | 2023-05-04 | Bridge plate type rubber support disease identification and detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310488349.3A CN116543303A (en) | 2023-05-04 | 2023-05-04 | Bridge plate type rubber support disease identification and detection method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116543303A true CN116543303A (en) | 2023-08-04 |
Family
ID=87449947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310488349.3A Pending CN116543303A (en) | 2023-05-04 | 2023-05-04 | Bridge plate type rubber support disease identification and detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116543303A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079256A (en) * | 2023-10-18 | 2023-11-17 | 南昌航空大学 | Fatigue driving detection algorithm based on target detection and key frame rapid positioning |
| CN117455902A (en) * | 2023-12-15 | 2024-01-26 | 北京交通大学 | Track defect detection method and device based on pre-training large model |
-
2023
- 2023-05-04 CN CN202310488349.3A patent/CN116543303A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079256A (en) * | 2023-10-18 | 2023-11-17 | 南昌航空大学 | Fatigue driving detection algorithm based on target detection and key frame rapid positioning |
| CN117079256B (en) * | 2023-10-18 | 2024-01-05 | 南昌航空大学 | Fatigue driving detection algorithm based on target detection and key frame rapid positioning |
| CN117455902A (en) * | 2023-12-15 | 2024-01-26 | 北京交通大学 | Track defect detection method and device based on pre-training large model |
| CN117455902B (en) * | 2023-12-15 | 2024-03-08 | 北京交通大学 | Track defect detection method and device based on pre-training large model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111754498B (en) | Conveyor belt carrier roller detection method based on YOLOv3 | |
| CN110660052B (en) | Hot-rolled strip steel surface defect detection method based on deep learning | |
| CN111444821B (en) | Automatic identification method for urban road signs | |
| US20210319561A1 (en) | Image segmentation method and system for pavement disease based on deep learning | |
| WO2022077605A1 (en) | Wind turbine blade image-based damage detection and localization method | |
| CN116543303A (en) | Bridge plate type rubber support disease identification and detection method based on deep learning | |
| Feng et al. | Pavement crack detection and segmentation method based on improved deep learning fusion model | |
| CN114092832B (en) | High-resolution remote sensing image classification method based on parallel hybrid convolutional network | |
| CN108711148B (en) | Tire defect intelligent detection method based on deep learning | |
| CN108257114A (en) | A kind of transmission facility defect inspection method based on deep learning | |
| CN111724355B (en) | Image measuring method for abalone body type parameters | |
| CN116612120B (en) | Two-stage road defect detection method for data unbalance | |
| CN116863274A (en) | Semi-supervised learning-based steel plate surface defect detection method and system | |
| CN116664846B (en) | Method and system for realizing 3D printing bridge deck construction quality monitoring based on semantic segmentation | |
| CN112613569A (en) | Image recognition method, and training method and device of image classification model | |
| CN103679214A (en) | Vehicle detection method based on online area estimation and multi-feature decision fusion | |
| CN115131747A (en) | Knowledge distillation-based power transmission channel engineering vehicle target detection method and system | |
| CN114898327A (en) | Vehicle detection method based on lightweight deep learning network | |
| CN116109616A (en) | Pavement crack detection and small-surface element fitting detection method based on YOLOv5 | |
| CN114359702A (en) | Method and system for identifying building violation of remote sensing image of homestead based on Transformer | |
| CN115171045A (en) | YOLO-based power grid operation field violation identification method and terminal | |
| CN117197085A (en) | Road rapid-inspection image pavement disease detection method based on improved YOLOv8 network | |
| CN112132088A (en) | Inspection point location missing inspection identification method | |
| CN112084941A (en) | Target detection and identification method based on remote sensing image | |
| CN116433959A (en) | Ground object classification method based on visible light and infrared satellite image fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |