CN113724219A - Building surface disease detection method and system based on convolutional neural network - Google Patents

Building surface disease detection method and system based on convolutional neural network Download PDF

Info

Publication number
CN113724219A
CN113724219A CN202110993730.6A CN202110993730A CN113724219A CN 113724219 A CN113724219 A CN 113724219A CN 202110993730 A CN202110993730 A CN 202110993730A CN 113724219 A CN113724219 A CN 113724219A
Authority
CN
China
Prior art keywords
model
deep learning
building surface
network model
iterative training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110993730.6A
Other languages
Chinese (zh)
Inventor
李佳阳
赵林畅
尚赵伟
何静媛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CN202110993730.6A priority Critical patent/CN113724219A/en
Publication of CN113724219A publication Critical patent/CN113724219A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a building surface disease detection method and system based on a deep learning network model. The method comprises the following steps: acquiring a building surface image as a data set; inputting the data set into a deep learning network model for learning, wherein the deep learning network model detects and fuses the multi-scale feature map of the feature extraction network in the learning process; performing primary iterative training on the fusion characteristic diagram in the deep learning network model, performing secondary training at the cosine annealing learning rate within a set range after the primary training is finished, storing the parameters of the model iterated each time in the secondary training, and solving the median of all models to obtain a new model; and then identifying the surface diseases of the building based on the trained deep learning network model. The method can identify smaller disease features, greatly improves the accuracy of the model AP and the accuracy of target positioning and classification, and enables identification of the disease features to be more accurate.

Description

Building surface disease detection method and system based on convolutional neural network
Technical Field
The invention relates to the field of deep learning, in particular to a building surface disease detection method and system based on a convolutional neural network.
Background
The reliability, safety and integrity of the building are vital to social welfare, so that the detection of the surface disease condition of the building is very important. By taking a bridge as an example, the bridge surface damage condition is detected, so that bridge abrasion can be effectively prevented, bridge maintenance is promoted, and the service life of the bridge is prolonged.
However, currently in the art of identifying and monitoring bridge nondestructive lesions, manual visual inspection is the primary means, resulting in inefficiencies, time and labor consuming, and subjective assessments. Under the background, a detection technology based on computer vision is applied and developed, a wall-climbing robot or an unmanned aerial vehicle is used for acquiring a bridge image, and a machine learning algorithm is used for analyzing a target image. For example, Prasanna et al propose a bridge crack automatic detection algorithm (stum) based on machine learning for the problem of bridge surface diseases, and although the performance of the method is superior to that of the traditional image recognition algorithm, the image processing efficiency and robustness of the method still need to be improved.
In recent years, with continuous innovation of a target detection algorithm based on deep learning, an automatic detection and recognition technology has a good effect in the fields of face recognition, target detection, image segmentation and the like, but research on bridge appearance disease detection is less. Currently, target detection algorithms Based on Anchor-Based are generally divided into two types, one type is a region-Based two-stage target detection algorithm, namely fast-RCNN [1] and Mask-RCNN [2], and the like, although the algorithms have high precision, the model speed is slow to operate and the real-time performance is poor due to the fact that the algorithms integrate defect feature extraction, region suggestion networks, boundary frame regression and the like. For example, Cha et al [3] use fast R-CNN to detect and quantify five surface damages in reinforced concrete bridges, although good results are obtained, the detection speed is not ideal. The other type is a single-stage algorithm SSD [4], a YOLO series and the like which directly marks the position and the category of the image of the target by utilizing a regression idea, the algorithm makes up the defects of a region-based two-stage target detection algorithm, the speed is greatly improved, the precision is slightly reduced, and particularly the SSD algorithm cannot fully utilize a shallow high-resolution feature map to ensure that the identification precision is not ideal.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a building surface disease detection method and system based on a convolutional neural network.
In order to achieve the above object, the present invention provides a building surface disease detection method based on a deep learning network model, comprising the following steps:
acquiring a building surface image as a data set;
inputting the data set into a deep learning network model for learning, wherein the deep learning network model detects and fuses the multi-scale feature map of the feature extraction network in the learning process;
performing iterative training on the fusion characteristic diagram in the deep learning network model, wherein the training process is divided into primary iterative training and secondary iterative training, storing the parameters of the model iterated each time in the secondary iterative training, and solving the median of all models to obtain a new model;
and identifying the surface diseases of the building based on the obtained new model.
According to the building surface disease detection method, the detection and fusion of the feature extraction network multi-scale feature map can identify smaller disease features, and meanwhile, the model AP accuracy and the target positioning and classification accuracy are greatly improved by adopting a median to perform iterative training in the training process under the condition that the number of parameters is not increased, so that the identification of the disease features is more accurate.
The preferable scheme of the building surface disease detection method is as follows: the deep learning network model is used for learning based on a Yolov5 network, the Yolov5 network uses PANET as a feature extraction backbone network, and 4-time, 8-time, 16-time and 32-time downsampling feature maps of the feature extraction network are output and fused.
Yolov5 uses the feature extraction backbone network of PANet, which not only extracts and learns the feature map effectively, but also fuses the learned feature maps. The structure of the PANet is improved, namely 4 times of down-sampling feature maps of the feature extraction network are output and fused, so that the detection effect of the network model on the defect with a small area is further improved.
The preferable scheme of the building surface disease detection method is as follows: the method comprises the steps that the 2 nd layer of the Yolov5 network is BottleneckCSP multiplied by 3, meanwhile, the connection between the 16 th layer and the 17 th layer is removed, the characteristic diagram obtained from the 16 th layer is subjected to BottleneckCSP operation to extract characteristics, the dimensionality is reduced through 1 multiplied by 1 convolution, the characteristic diagram is spliced with the characteristic diagram of the 2 nd layer through upsampling, then the first output is obtained through the BottleneckCSP multiplied by 3 operation, meanwhile, the characteristic diagram is subjected to 3 multiplied by 3 convolution to reduce the dimensionality to obtain a new characteristic diagram, the new characteristic diagram is subjected to characteristic fusion downwards after splicing operation, and a characteristic fusion network containing four outputs is finally formed. The detection capability of the network model to small targets is enhanced. The model is beneficial to detecting the defects of the tiny objects.
The preferable scheme of the building surface disease detection method is as follows: and performing secondary iterative training under the model finally obtained after the primary iterative training by using the cosine annealing learning rate in the set range in the secondary iterative training, storing the model parameters obtained by each iteration in the secondary iterative training, and solving the median of all the model parameters in the secondary iterative training to obtain a new model.
The reason for adopting the second iterative training is that the fluctuation of the cosine annealing learning can lead the stabilized model to explore more peripheral areas, so that the model parameters can jump out of the current local optimal solution to search more optimal solutions. The median is obtained from all model parameters, so that the results explored by the cosine annealing learning rate can be better integrated, and the effect is better than that of the average according to the experimental median.
The preferable scheme of the building surface disease detection method is as follows: when a disease area of a building surface disease is extracted, sorting prediction frames inferred by the model according to the confidence coefficient from high to low, finding out a prediction frame with the highest confidence coefficient, calculating the IOU values of the prediction frame with the highest confidence coefficient and other prediction frames, and reducing the confidence coefficient of the prediction frame by using the following formula for the prediction frame larger than the threshold IOU:
Figure BDA0003233182310000041
wherein s isiIs a pending prediction box biThe degree of confidence of (a) is,
Figure BDA0003233182310000042
the prediction box with the highest confidence level is selected,
Figure BDA0003233182310000043
the prediction box with the highest confidence coefficient and the check box biThe IOU value of (a), is a hyperparameter. The method improves the detection effect under the dense condition, and has obvious defect detection effect under the condition of improving the multi-dense condition.
The invention also provides a computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute the operation corresponding to the building surface disease detection method.
The invention further provides a building surface disease detection system, which comprises a processor and a memory, wherein the processor is in communication connection with the memory, the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the building surface disease detection method.
The invention has the beneficial effects that: the invention can identify the minor diseases on the surface of the building, has high identification precision and is particularly suitable for identifying and detecting the diseases on the surface of the bridge.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic diagram of a modified Yolov5 model;
FIG. 2 is a diagram of function convergence information on a training set and a validation set;
FIG. 3 is a diagram of detection performance on a validation set;
FIG. 4 is a schematic diagram of the PR curve of modified Yolov 5;
fig. 5 is a diagram showing the results of the performance test.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.
The invention provides a building surface disease detection method based on a deep learning network model, which comprises the following steps:
an image of a building surface is acquired as a data set.
And inputting the data set into a deep learning network model for learning, and detecting and fusing the multi-scale feature map of the feature extraction network by the deep learning network model in the learning process.
The deep learning network model in the embodiment learns based on the Yolov5 network, the Yolov5 network uses the PANet as a feature extraction backbone network, the Yolov5 backbone network is deepened, the structure of the PANet is improved, and 4-time, 8-time, 16-time and 32-time downsampling feature maps of the feature extraction network are output and fused.
Specifically, as shown in fig. 1, a layer 2 of the Yolov5 network in the scheme is a bottleeck csp × 3, so as to better extract defect features, meanwhile, the connection between the layers 16 and 17 is removed, a feature map obtained at the layer 16 is subjected to a bottleeck csp operation to extract features, dimensionality reduction is performed through 1 × 1 convolution, upsampling is performed to splice with the feature map at the layer 2, then, a first output for detecting small object defects is obtained through a bottleeck csp × 3 operation, meanwhile, a new feature map is obtained after dimensionality reduction is performed through 3 × 3 convolution on the feature map, features of the new feature map are fused downwards after splicing operation, and a feature fusion network finally containing four outputs is formed, so as to enhance the detection capability of the network model on small objects.
And after the feature maps are output and fused, performing iterative training on the fused feature maps in the deep learning network model, wherein the training process comprises primary iterative training and secondary iterative training, storing parameters of the model iterated each time in the secondary iterative training, and solving the median of all models to obtain a new model.
Specifically, in the embodiment, the first iterative training is to perform 300 times of iterative training by using the improved model, the second iterative training is to perform 24 times of iterative training again under the final model obtained by the first iterative training with a cosine annealing learning rate within a set range, each model parameter obtained by 24 times of iteration is stored, and the median of the 24 model parameters is obtained to obtain a new model. The stability of the model performance is improved by adopting the more robust median rule, the stability of the model performance of YOLOv5 is enhanced, and the accuracy of the model AP and the accuracy of the target positioning and classification are greatly improved under the condition that the number of parameters is not increased.
And then identifying the surface diseases of the building based on the trained deep learning network model.
In particular, for buildingsWhen the surface disease of the object is extracted from the disease area, a soft non-maximum inhibition method can be adopted to replace the original non-maximum inhibition method: sorting the prediction boxes inferred by the model according to the confidence coefficient from high to low, finding out the box with the highest confidence coefficient, calculating the IOU value of the prediction box with the highest confidence coefficient and other prediction boxes, and reducing the confidence coefficient of the prediction box by using the formula for the prediction box which is larger than a threshold IOU (has higher overlapping degree):
Figure BDA0003233182310000061
rather than removing them as coarsely as soon as they are above the threshold in the original NMS, where siIs a pending prediction box biThe degree of confidence of (a) is,
Figure BDA0003233182310000071
the prediction box with the highest confidence level is selected,
Figure BDA0003233182310000072
the prediction box with the highest confidence coefficient and the check box biThe IOU value of (a), is a hyperparameter.
The following takes bridge surface defects as an example:
experimental data
The experimental data mainly come from various bridge disease photos collected in 2015-2020 years, the bridge image data containing defects are labeled by using Labelimg software through manual screening, and a bridge defect data set used for the experiment is sorted out. The disease-resistant and anti-aging coating comprises six types of diseases such as cracks, peeling, honeycombs, holes, exposed ribs, water seepage and the like, the total number of the diseases is 3828 pictures, and detailed data set information is shown in table 1. In the experimental process, 3461 pictures are randomly selected as a training set, and the rest pictures are taken as a testing set.
TABLE 1 Experimental data details
Figure BDA0003233182310000073
Evaluation indexes are as follows:
the example mainly uses precision, recall, average accuracy and average accuracy mean to evaluate the target detection performance of the experimental method.
1. Precision (P) and Recall (R) are calculated from TP (true positives), FP (false positives), FN (false negatives), where TP represents the number of correctly divided positive samples, FP represents the number of incorrectly divided positive samples, and FN represents the number of divided negative samples but actually negative samples. It is calculated as shown below:
Figure BDA0003233182310000081
2. average Accuracy (AP) and mean Average accuracy (mAP), AP represents a certain class of accuracy, mAP represents the Average of all classes of APs, and AP and mAP are calculated as follows:
Figure BDA0003233182310000082
where N is the number of target categories, usually the precision ratio is increased with a decrease in the recall ratio. The mAP is the sum average of all classes of APs, and mAP50 and mAP0.5:0.95 are used in the experiment to measure the performance of the detection algorithm. mAP50 sets the IOU threshold to 0.5, i.e. when the IOU of the prediction box and the real box is greater than 0.5, it is considered as a positive sample (TP); a negative sample (FP) is considered when the prediction and true frame IOU thresholds are less than 0.5. mAP0.5:0.95 is the mAP value calculated every 0.05 when the IOU threshold value is between 0.5 and 0.95, so that 10 mAP values are total, and the average value of the 10 mAP values is the mAP0.5:0.95 index value.
Results and analysis of the experiments
The experiment is carried out through a deep learning library of the pytorch and a dependency package thereof under the environments of i9-10900K CPU, 2080Ti GPU, 64GB memory and Windows 10. During the experiment, the blocksize was 8, the initial learning rate was 0.01, and the weight decay was 0.0005. For fair experiments, all experimental methods take the mean value of repeated training. After 200 epoch iterative training, the information of the convergence of the loss function when the proposed model achieves the best effect is shown in fig. 4.
From fig. 2, it can be seen that after 300 epoch training, the Box, Objectness, and Classification loss function information in the training set and the verification set all have good convergence lower limits. Meanwhile, fig. 5 shows precision ratio, recall ratio, average accuracy ratio and average accuracy ratio mean value of the proposed method on the verification set.
From fig. 3, it can be seen that the precision ratio, the recall ratio, the average accuracy rate and the average accuracy rate mean value of the verification set all obtained ideal results after 300 epoch training. And figure 4 shows the best performing mAP50 and map0.5:0.95 for the validation set.
From fig. 4, it can be seen that the optimal values of the mapp 50 and the map0.5:0.95 of the proposed method are 0.608 and 0.305, respectively, then the model under the performance is saved, and the saved model is retrained for 24 times at a cosine annealing learning rate between 0.001 and 0.00001, then the median of the parameters of the 24 models is calculated to determine the final version network model, and finally the performance detection is performed on the verification set. Table 2 reports the results of comparative experiments of the original Yolov5 algorithm on mAP50 and mAP0.5:0.95 indexes in the bridge defect data set.
TABLE 2 mAP comparison on bridge Defect dataset
Figure BDA0003233182310000091
From table 2, it can be seen that in terms of the mAP50 index value, neither the addition of SWA algorithm nor the STAM algorithm improves the model performance compared to the original Yolov5 algorithm. However, in the aspect of mAP0.5:0.95 indexes, the performance of the introduced SWA algorithm is improved by 0.9% compared with that of the original Yolov5 algorithm, and the performance of the added STAM algorithm is improved by 1.5% compared with that of the original Yolov5 algorithm. The STAM random median training algorithm designed by the method is more effective. Table 3 reports the comparison results of different experimental methods on the bridge defect data set.
TABLE 3 comparison of the different methods
Figure BDA0003233182310000092
Figure BDA0003233182310000101
From Table 3, it can be seen that in terms of mAP50 index, the method of Our Yolov5x + STAM + Soft-NMS provided herein is improved by 2.2% compared with the original Yolov5x, and is improved by 8.1% compared with the poorest yov 3, and the performance value (0.622) is optimal compared with other improved methods. In terms of mAP0.5:0.95 index, the provided method, Our Yolov5x + STAM + Soft-NMS, continuously maintains performance advantage, is improved by 2.1% compared with Yolov5x, is improved by 6.9% compared with the poorest yov 3, and is optimal in performance value (0.32) compared with other improved methods. FIG. 5 shows the results of the detection of the method presented herein on the validation set.
From fig. 5, even if the defect types of the bridge image set are numerous, the method can accurately identify the defect types, accurately position the defect area, automatically match the defect types, and accurately and efficiently identify the defect target.
The application also provides an embodiment of a computer storage medium, wherein the storage medium stores at least one executable instruction, and the executable instruction causes a processor to execute the operation corresponding to the building surface disease detection method.
The application also provides a building surface disease detection system, which comprises a processor and a memory, wherein the processor is in communication connection with the memory, and the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the building surface disease detection method.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (8)

1. A building surface disease detection method based on a deep learning network model is characterized by comprising the following steps:
acquiring a building surface image as a data set;
inputting the data set into a deep learning network model for learning, wherein the deep learning network model detects and fuses the multi-scale feature map of the feature extraction network in the learning process;
performing iterative training on the fusion characteristic diagram in the deep learning network model, wherein the training process is divided into primary iterative training and secondary iterative training, storing the parameters of the model iterated each time in the secondary iterative training, and solving the median of all models to obtain a new model;
and identifying the surface diseases of the building based on the obtained new model.
2. The method for detecting the building surface diseases based on the deep learning network model according to claim 1,
the deep learning network model is used for learning based on a Yolov5 network, the Yolov5 network uses PANET as a feature extraction backbone network, and 4-time, 8-time, 16-time and 32-time downsampling feature maps of the feature extraction network are output and fused.
3. The building surface disease detection method based on the deep learning network model according to claim 2, characterized in that the 2 nd layer of the Yolov5 network is a bottleckcsp × 3, the connection between the 16 th layer and the 17 th layer is removed, the characteristic diagram obtained from the 16 th layer is subjected to a bottleckcsp operation to extract characteristics, the dimensionality is reduced through 1 × 1 convolution, the upsampling and the characteristic diagram of the 2 nd layer are spliced, then the bottleckcsp × 3 operation is performed to obtain the first output, the characteristic diagram is subjected to a 3 × 3 convolution to reduce the dimensionality to obtain a new characteristic diagram, the new characteristic diagram is subjected to splicing operation and then subjected to characteristic downward fusion, and a characteristic fusion network containing four outputs is formed finally.
4. The building surface disease detection method based on the deep learning network model according to any one of claims 1 to 3, characterized in that the second iterative training is performed with a cosine annealing learning rate within a set range under a model finally obtained after the first iterative training, model parameters obtained by each iteration in the second iterative training are saved, and the median of all the model parameters in the second iterative training is solved to obtain a new model.
5. The building surface disease detection method based on the deep learning network model according to claim 4, wherein in the second iterative training, the model finally obtained after the first iterative training is iteratively trained at a cosine annealing learning rate of 0.001-0.00001.
6. The building surface disease detection method based on the deep learning network model according to claim 1, wherein when the building surface disease is extracted, the prediction frames inferred from the model are ranked according to the confidence coefficients from high to low, the prediction frame with the highest confidence coefficient is found, the IOU values of the prediction frame with the highest confidence coefficient and other prediction frames are calculated, and the confidence coefficient of the prediction frame is reduced by using the following formula for the prediction frame with the confidence coefficient larger than the threshold IOU:
Figure FDA0003233182300000021
wherein s isiIs a pending prediction box biThe degree of confidence of (a) is,
Figure FDA0003233182300000022
the prediction box with the highest confidence level is selected,
Figure FDA0003233182300000023
the prediction box with the highest confidence coefficient and the check box biThe IOU value of (a), is a hyperparameter.
7. A computer storage medium having stored therein at least one executable instruction for causing a processor to perform operations corresponding to the method for detecting building surface defects according to any one of claims 1 to 6.
8. A building surface disease detection system comprising a processor and a memory, the processor and the memory being communicatively coupled, the memory being configured to store at least one executable instruction that causes the processor to perform operations corresponding to the building surface disease detection method of any one of claims 1-6.
CN202110993730.6A 2021-08-27 2021-08-27 Building surface disease detection method and system based on convolutional neural network Pending CN113724219A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110993730.6A CN113724219A (en) 2021-08-27 2021-08-27 Building surface disease detection method and system based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110993730.6A CN113724219A (en) 2021-08-27 2021-08-27 Building surface disease detection method and system based on convolutional neural network

Publications (1)

Publication Number Publication Date
CN113724219A true CN113724219A (en) 2021-11-30

Family

ID=78678395

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110993730.6A Pending CN113724219A (en) 2021-08-27 2021-08-27 Building surface disease detection method and system based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN113724219A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092467A (en) * 2021-12-01 2022-02-25 重庆大学 Scratch detection method and system based on lightweight convolutional neural network
CN114913212A (en) * 2022-06-24 2022-08-16 成都云擎科技有限公司 DeepSORT target tracking method based on feature sharing
CN116645371A (en) * 2023-07-27 2023-08-25 中铁十二局集团铁路养护工程有限公司 Rail surface defect detection method and system based on feature search
CN117541922A (en) * 2023-11-09 2024-02-09 国网宁夏电力有限公司建设分公司 SF-YOLOv 5-based power station roofing engineering defect detection method

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170228639A1 (en) * 2016-02-05 2017-08-10 International Business Machines Corporation Efficient determination of optimized learning settings of neural networks
US20190130250A1 (en) * 2017-10-30 2019-05-02 Samsung Electronics Co., Ltd. Method and apparatus with neural network performing convolution
US20200053559A1 (en) * 2016-10-24 2020-02-13 Lg Electronics Inc. Deep learning neural network based security system and control method therefor
CN112163520A (en) * 2020-09-29 2021-01-01 广西科技大学 MDSSD face detection method based on improved loss function
US20210056412A1 (en) * 2019-08-20 2021-02-25 Lg Electronics Inc. Generating training and validation data for machine learning
CN112580439A (en) * 2020-12-01 2021-03-30 中国船舶重工集团公司第七0九研究所 Method and system for detecting large-format remote sensing image ship target under small sample condition
CN112604186A (en) * 2020-12-30 2021-04-06 佛山科学技术学院 Respiratory motion prediction method
CN112926584A (en) * 2021-05-11 2021-06-08 武汉珈鹰智能科技有限公司 Crack detection method and device, computer equipment and storage medium
US20210174258A1 (en) * 2019-12-10 2021-06-10 Arthur AI, Inc. Machine learning monitoring systems and methods
CN113052334A (en) * 2021-04-14 2021-06-29 中南大学 Method and system for realizing federated learning, terminal equipment and readable storage medium
CN113158956A (en) * 2021-04-30 2021-07-23 杭州电子科技大学 Garbage detection and identification method based on improved yolov5 network
CN113160209A (en) * 2021-05-10 2021-07-23 上海市建筑科学研究院有限公司 Target marking method and target identification method for building facade damage detection

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170228639A1 (en) * 2016-02-05 2017-08-10 International Business Machines Corporation Efficient determination of optimized learning settings of neural networks
US20200053559A1 (en) * 2016-10-24 2020-02-13 Lg Electronics Inc. Deep learning neural network based security system and control method therefor
US20190130250A1 (en) * 2017-10-30 2019-05-02 Samsung Electronics Co., Ltd. Method and apparatus with neural network performing convolution
US20210056412A1 (en) * 2019-08-20 2021-02-25 Lg Electronics Inc. Generating training and validation data for machine learning
US20210174258A1 (en) * 2019-12-10 2021-06-10 Arthur AI, Inc. Machine learning monitoring systems and methods
CN112163520A (en) * 2020-09-29 2021-01-01 广西科技大学 MDSSD face detection method based on improved loss function
CN112580439A (en) * 2020-12-01 2021-03-30 中国船舶重工集团公司第七0九研究所 Method and system for detecting large-format remote sensing image ship target under small sample condition
CN112604186A (en) * 2020-12-30 2021-04-06 佛山科学技术学院 Respiratory motion prediction method
CN113052334A (en) * 2021-04-14 2021-06-29 中南大学 Method and system for realizing federated learning, terminal equipment and readable storage medium
CN113158956A (en) * 2021-04-30 2021-07-23 杭州电子科技大学 Garbage detection and identification method based on improved yolov5 network
CN113160209A (en) * 2021-05-10 2021-07-23 上海市建筑科学研究院有限公司 Target marking method and target identification method for building facade damage detection
CN112926584A (en) * 2021-05-11 2021-06-08 武汉珈鹰智能科技有限公司 Crack detection method and device, computer equipment and storage medium

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
HAOYANG ZHANG 等: "SWA Object Detection", 《ARXIV:2012.12645V3》, pages 1 - 9 *
XINGKUI ZHU 等: "TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios", 《ARXIV:2108.11539V1》, pages 1 - 11 *
李佳阳: "基于YOLOv5改进的目标检测算法研究及应用", 《中国知网》, pages 1 - 62 *
梦坠凡尘(AICV与前沿): "YOLOv4的Tricks解读三--目标检测后处理（Soft-NMS/DIoU-NMS）", pages 2, Retrieved from the Internet <URL:《https://blog.csdn.net/c2250645962/article/details/106210819》> *
樊荣荣 等: "迭代模型重建技术参数设置对肝脏低剂量增强CT扫描图像质量的影响", 《中国医学影像技术》, vol. 33, no. 11, pages 1711 - 1715 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114092467A (en) * 2021-12-01 2022-02-25 重庆大学 Scratch detection method and system based on lightweight convolutional neural network
CN114913212A (en) * 2022-06-24 2022-08-16 成都云擎科技有限公司 DeepSORT target tracking method based on feature sharing
CN116645371A (en) * 2023-07-27 2023-08-25 中铁十二局集团铁路养护工程有限公司 Rail surface defect detection method and system based on feature search
CN116645371B (en) * 2023-07-27 2023-10-17 中铁十二局集团铁路养护工程有限公司 Rail surface defect detection method and system based on feature search
CN117541922A (en) * 2023-11-09 2024-02-09 国网宁夏电力有限公司建设分公司 SF-YOLOv 5-based power station roofing engineering defect detection method

Similar Documents

Publication Publication Date Title
CN111553387B (en) Personnel target detection method based on Yolov3
CN113724219A (en) Building surface disease detection method and system based on convolutional neural network
Li et al. Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network
CN109658387B (en) Method for detecting defects of pantograph carbon slide plate of electric train
CN113240623B (en) Pavement disease detection method and device
CN115995056A (en) Automatic bridge disease identification method based on deep learning
CN113066047A (en) Method for detecting impurity defects of tire X-ray image
CN113763364B (en) Image defect detection method based on convolutional neural network
CN115294033A (en) Tire belt layer difference level and misalignment defect detection method based on semantic segmentation network
CN117011260A (en) Automatic chip appearance defect detection method, electronic equipment and storage medium
CN117152746A (en) Method for acquiring cervical cell classification parameters based on YOLOV5 network
CN109543498B (en) Lane line detection method based on multitask network
CN113313107A (en) Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge
CN112396580A (en) Circular part defect detection method
KR102494829B1 (en) Structure damage evaluation method for using the convolutional neural network, and computing apparatus for performing the method
CN117237911A (en) Image-based dynamic obstacle rapid detection method and system
CN117058459A (en) Rapid pavement disease detection method and system based on YOLOV7 algorithm
CN116577345A (en) Method and system for detecting number of tabs of lithium battery
CN115830302A (en) Multi-scale feature extraction and fusion power distribution network equipment positioning identification method
CN115082650A (en) Implementation method of automatic pipeline defect labeling tool based on convolutional neural network
CN115953387A (en) Radiographic image weld defect detection method based on deep learning
KR20230063742A (en) Method for detecting defect of product using hierarchical CNN in smart factory, and recording medium thereof
Yazid et al. Automated system form concrete damage classification identification using pretrained deep learning model
Cui et al. Road crack classification based on improved vgg convolutional neural network
CN111091150A (en) Railway wagon cross rod cover plate fracture detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20211130