CN116485729B - Multistage bridge defect detection method based on transformer - Google Patents
Multistage bridge defect detection method based on transformer Download PDFInfo
- Publication number
- CN116485729B CN116485729B CN202310350506.4A CN202310350506A CN116485729B CN 116485729 B CN116485729 B CN 116485729B CN 202310350506 A CN202310350506 A CN 202310350506A CN 116485729 B CN116485729 B CN 116485729B
- Authority
- CN
- China
- Prior art keywords
- bridge
- layer
- image data
- convolution
- feature map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 90
- 238000001514 detection method Methods 0.000 title claims abstract description 21
- 230000007246 mechanism Effects 0.000 claims abstract description 45
- 201000010099 disease Diseases 0.000 claims abstract description 24
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000013135 deep learning Methods 0.000 claims abstract description 12
- 238000011176 pooling Methods 0.000 claims description 29
- 238000010586 diagram Methods 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 230000002950 deficient Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 3
- 230000009466 transformation Effects 0.000 abstract description 2
- 238000004519 manufacturing process Methods 0.000 abstract 1
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30184—Infrastructure
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a transformation-based multistage bridge defect detection method, which comprises the following steps of: s1, acquiring a bridge bottom image, and establishing a bridge image data set; s2, manufacturing a bridge disease data set according to the bridge image data set; s3, inputting the bridge disease data set into a two-class network to obtain image data of a defect class; s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes. According to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.
Description
Technical Field
The invention belongs to the technical field of bridge defect detection, and particularly relates to a transformation-based multistage bridge defect detection method.
Background
I build highway bridges vigorously in our country, so that some highway bridges are older by even more than 40 years. According to the road investigation results in recent years, more and more highway bridges are obviously aged and damaged, which results in more and more dangerous bridges. They all commonly have the problems of bridge deck narrowness, broken cracks, exposed ribs and the like.
If the road bridge diseases can be detected in early stage and treated in time, the maintenance cost of the road bridge can be reduced to a great extent, and meanwhile, the driving safety of the road bridge can be guaranteed to a certain extent. Under the condition of not influencing transportation, the efficient and rapid checking and detecting of the cracks becomes a difficulty which needs to be solved urgently at present. At present, most of damage detection of highway bridges in China still depends on manual detection and investigation. The main disadvantages of the traditional artificial bridge defect detection are as follows: (1) labor cost is too high: the surface of the highway bridge is inspected and detected by human eyes, so that professional workers are inevitably required to survey the site of the highway bridge, however, for extremely long highway bridges of roads, the work intensity of inspecting the road bridge one by using human eyes is definitely huge, and the labor and the number of the professional workers are correspondingly high. (2) time consuming: the road bridge is inspected by naked eyes of workers, and a great amount of time is required for inspecting the road bridge by naked eyes because bridge surface defects such as cracks and hollows are in the centimeter level. (3) low accuracy: the human eyes can clearly distinguish the contrast of the light and shade, but are easily influenced by different colors to influence the judgment, and accordingly, the precision is reduced. (4) high cost: the detection can be completed only by inputting a great deal of manpower, material resources and time, and the cost is too high.
Disclosure of Invention
Aiming at the defects in the prior art, the multi-stage bridge defect detection method based on the transformer solves the problems of high labor cost, time consumption, low accuracy and high cost of the traditional artificial bridge defect detection.
In order to achieve the aim of the invention, the invention adopts the following technical scheme: a method for detecting defects of a multistage bridge based on a transducer comprises the following steps:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes.
Further: the step S1 specifically comprises the following steps:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
Further: the step S2 specifically comprises the following steps:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
Further: the step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
The beneficial effects of the above-mentioned further scheme are: the classification network continuously rewards normal classes and penalizes defective classes in the classification process, so that the identification accuracy of the normal classes is very high, and all the pictures of the normal classes can be detected and identified in the step.
Further: the step S31 specifically includes:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
Further: the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map.
Further: in the step S4, the deep learning network model of the transducer attention mechanism comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer which are sequentially connected;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units;
the second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step size of the fifth convolution layer is 8, and the expansion coefficient is 4.
The beneficial effects of the above-mentioned further scheme are: the 3*3 cavity convolution kernel is adopted, so that a model can obtain a larger receptive field, and the effect of identifying smaller defects can be improved.
Further: the transducer-based attention mechanism subunit comprises a query_conv convolution, a key_conv convolution and a value_conv convolution which are connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
Further: in SA1, the query_conv convolution sets the size of the received image data to b×c×w×h, where B is the size of the training batch, C is the number of channels, W is the image width, and H is the image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
The beneficial effects of the invention are as follows:
(1) According to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.
(2) The invention introduces a cavity convolution and a transducer attention mechanism, so that the traditional network model has better performance in the aspect of bridge detection, and is more suitable for bridge defect classification detection.
(3) The invention greatly saves manpower and material resources and improves the accuracy of bridge defect detection.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a schematic diagram of a deep learning network model of the transducer attention mechanism of the present invention.
FIG. 3 is a schematic diagram of a transducer-based attention mechanism subunit of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
Example 1:
as shown in fig. 1, in one embodiment of the present invention, a method for detecting a defect of a multi-stage bridge based on a transducer includes the steps of:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes.
The step S1 specifically comprises the following steps:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
The step S2 specifically comprises the following steps:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
In this embodiment, each type of collected image is about 10000, and the total of bridge disease images is about 60000 images.
The step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
The step S31 specifically includes:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
In this embodiment, the classification network continuously rewards the normal class during the classification process, penalizes the defect class, so that the accuracy of identifying the normal class is very high, and all the pictures of the normal class can be detected and identified in this step.
The learning rate set by the two-class network is 0.0001, the optimization function is Adam algorithm, and the relevant parameters of the convolution kernel and the pooling layer of the VGG layer are continuously adjusted in the training process of the two-class network. And in the classifying process, normal classes are continuously rewarded, defect classes are punished, so that the identification accuracy of the normal classes is very high, all pictures of the normal classes are detected and identified as far as possible, the classification of the existence of diseases of bridge disease images is preliminarily realized, and the image data of the defect classes are obtained through screening.
In this embodiment, the deep learning network model of the transducer attention mechanism is shown in fig. 2; the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map (FeatureMap) with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map.
In the step S4, the deep learning network model of the transducer attention mechanism includes a first convolution layer (stage 1), a second convolution layer (stage 2), a third convolution layer (stage 3), a fourth convolution layer (stage 4), a fifth convolution layer (stage 5) and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer (Maxpool) which are connected in sequence;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units.
The second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step size of the fifth convolution layer is 8, and the expansion coefficient is 4.
The first max pooling layer is used for downsampling, when out and the channel number of the original input x are the same, downsamples are None, out and the original input x can be directly overlapped, when out and the channel number of the original input x are different, the channel number is increased through downsampling and then overlapped with out, and n is determined by the value in the list transmitted before.
In this embodiment, the cavity convolution kernel is set to 3*3, so that the model can obtain a larger receptive field, and the effect of identifying smaller defects can be improved. It will be apparent that the use of hole convolution instead of downsampling/upsampling can preserve the spatial features of the image well without losing image information.
As shown in fig. 3, the transducer-based attention mechanism subunit includes a query_conv convolution, a key_conv convolution, and a value_conv convolution connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
In SA1, the query_conv convolution sets the size of the received image data to b×c×w×h, where B is the size of the training batch, C is the number of channels, W is the image width, and H is the image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
The beneficial effects of the invention are as follows: according to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.
The invention introduces a cavity convolution and a transducer attention mechanism, so that the traditional network model has better performance in the aspect of bridge detection, and is more suitable for bridge defect classification detection.
The invention greatly saves manpower and material resources and improves the accuracy of bridge defect detection.
In the description of the present invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," "radial," and the like indicate or are based on the orientation or positional relationship shown in the drawings, merely to facilitate description of the present invention and to simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be configured and operated in a particular orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be interpreted as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defined as "first," "second," "third," or the like, may explicitly or implicitly include one or more such feature.
Claims (6)
1. A method for detecting defects of a multistage bridge based on a transducer is characterized by comprising the following steps of:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes;
the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map;
in the step S4, the deep learning network model of the transducer attention mechanism comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer which are sequentially connected;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units;
the second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step length of the fifth convolution layer is 8, and the expansion coefficient is 4;
the transducer-based attention mechanism subunit comprises a query_conv convolution, a key_conv convolution and a value_conv convolution which are connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
2. The method for detecting defects of a multistage bridge based on transformers according to claim 1, wherein the step S1 is specifically:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
3. The method for detecting defects of a multi-stage bridge based on a transducer according to claim 1, wherein the step S2 is specifically:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
4. The method for detecting defects in a multi-stage bridge based on a transformer according to claim 3, wherein the step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
5. The method for detecting defects of a multi-stage bridge based on a transducer according to claim 4, wherein the step S31 specifically comprises:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
6. The method for detecting defects of a multistage bridge based on a transducer according to claim 1, wherein in SA1, a query_conv convolution sets a received image data size of b×c×w×h, wherein B is a training lot size, C is a channel number, W is an image width, and H is an image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310350506.4A CN116485729B (en) | 2023-04-03 | 2023-04-03 | Multistage bridge defect detection method based on transformer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310350506.4A CN116485729B (en) | 2023-04-03 | 2023-04-03 | Multistage bridge defect detection method based on transformer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116485729A CN116485729A (en) | 2023-07-25 |
| CN116485729B true CN116485729B (en) | 2024-01-12 |
Family
ID=87216980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310350506.4A Active CN116485729B (en) | 2023-04-03 | 2023-04-03 | Multistage bridge defect detection method based on transformer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116485729B (en) |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191714A (en) * | 2019-12-28 | 2020-05-22 | 浙江大学 | Intelligent identification method for bridge appearance damage diseases |
| WO2021135477A1 (en) * | 2020-07-31 | 2021-07-08 | 平安科技（深圳）有限公司 | Probabilistic graphical model-based text attribute extraction method and apparatus, computer device and storage medium |
| CN113139470A (en) * | 2021-04-25 | 2021-07-20 | 安徽工业大学 | Glass identification method based on Transformer |
| CN113397572A (en) * | 2021-07-23 | 2021-09-17 | 中国科学技术大学 | Surface electromyographic signal classification method and system based on Transformer model |
| CN113591770A (en) * | 2021-08-10 | 2021-11-02 | 中国科学院深圳先进技术研究院 | Multimode fusion obstacle detection method and device based on artificial intelligence blind guiding |
| CN114841320A (en) * | 2022-05-07 | 2022-08-02 | 西安邮电大学 | Organ automatic segmentation method based on laryngoscope medical image |
| CN114937285A (en) * | 2022-05-25 | 2022-08-23 | 兰州大学 | Dynamic gesture recognition method, device, equipment and storage medium |
| CN114943694A (en) * | 2022-05-09 | 2022-08-26 | 浙江科技学院 | Defect detection method based on confrontation generation network and attention |
| CN115063351A (en) * | 2022-05-24 | 2022-09-16 | 中国科学院深圳先进技术研究院 | Deep learning-based fetal MRI brain tissue segmentation method and device |
| CN115205972A (en) * | 2022-07-13 | 2022-10-18 | 安徽大学 | Six-step hand washing evaluation method and device based on deep learning |
| CN115205230A (en) * | 2022-06-21 | 2022-10-18 | 福州大学 | Concrete bridge apparent crack identification method based on novel attention mechanism |
| CN115240195A (en) * | 2022-08-04 | 2022-10-25 | 成都泓睿科技有限责任公司 | Medicine bottle defect automatic detection method and device and storage medium |
| CN115359550A (en) * | 2022-07-06 | 2022-11-18 | 华南师范大学 | Gait emotion recognition method and device based on Transformer, electronic device and storage medium |
| CN115359306A (en) * | 2022-10-24 | 2022-11-18 | 中铁科学技术开发有限公司 | Intelligent identification method and system for high-definition images of railway freight inspection |
| CN115482491A (en) * | 2022-09-23 | 2022-12-16 | 湖南大学 | Bridge defect identification method and system based on transformer |
| CN115601355A (en) * | 2022-11-10 | 2023-01-13 | 四川启睿克科技有限公司（Cn） | Method and device for detecting and classifying product surface defects and storage medium |
| CN115619726A (en) * | 2022-09-29 | 2023-01-17 | 西安理工大学 | Lightweight defect detection method for detecting defects of automobile body paint surface |
| CN115690715A (en) * | 2022-11-09 | 2023-02-03 | 中汽创智科技有限公司 | Lane line detection method, electronic device and storage medium |
| CN115861305A (en) * | 2023-02-20 | 2023-03-28 | 佛山市南海区广工大数控装备协同创新研究院 | Flexible circuit board detection method and device, computer equipment and storage medium |
-
2023
- 2023-04-03 CN CN202310350506.4A patent/CN116485729B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191714A (en) * | 2019-12-28 | 2020-05-22 | 浙江大学 | Intelligent identification method for bridge appearance damage diseases |
| WO2021135477A1 (en) * | 2020-07-31 | 2021-07-08 | 平安科技（深圳）有限公司 | Probabilistic graphical model-based text attribute extraction method and apparatus, computer device and storage medium |
| CN113139470A (en) * | 2021-04-25 | 2021-07-20 | 安徽工业大学 | Glass identification method based on Transformer |
| CN113397572A (en) * | 2021-07-23 | 2021-09-17 | 中国科学技术大学 | Surface electromyographic signal classification method and system based on Transformer model |
| CN113591770A (en) * | 2021-08-10 | 2021-11-02 | 中国科学院深圳先进技术研究院 | Multimode fusion obstacle detection method and device based on artificial intelligence blind guiding |
| CN114841320A (en) * | 2022-05-07 | 2022-08-02 | 西安邮电大学 | Organ automatic segmentation method based on laryngoscope medical image |
| CN114943694A (en) * | 2022-05-09 | 2022-08-26 | 浙江科技学院 | Defect detection method based on confrontation generation network and attention |
| CN115063351A (en) * | 2022-05-24 | 2022-09-16 | 中国科学院深圳先进技术研究院 | Deep learning-based fetal MRI brain tissue segmentation method and device |
| CN114937285A (en) * | 2022-05-25 | 2022-08-23 | 兰州大学 | Dynamic gesture recognition method, device, equipment and storage medium |
| CN115205230A (en) * | 2022-06-21 | 2022-10-18 | 福州大学 | Concrete bridge apparent crack identification method based on novel attention mechanism |
| CN115359550A (en) * | 2022-07-06 | 2022-11-18 | 华南师范大学 | Gait emotion recognition method and device based on Transformer, electronic device and storage medium |
| CN115205972A (en) * | 2022-07-13 | 2022-10-18 | 安徽大学 | Six-step hand washing evaluation method and device based on deep learning |
| CN115240195A (en) * | 2022-08-04 | 2022-10-25 | 成都泓睿科技有限责任公司 | Medicine bottle defect automatic detection method and device and storage medium |
| CN115482491A (en) * | 2022-09-23 | 2022-12-16 | 湖南大学 | Bridge defect identification method and system based on transformer |
| CN115619726A (en) * | 2022-09-29 | 2023-01-17 | 西安理工大学 | Lightweight defect detection method for detecting defects of automobile body paint surface |
| CN115359306A (en) * | 2022-10-24 | 2022-11-18 | 中铁科学技术开发有限公司 | Intelligent identification method and system for high-definition images of railway freight inspection |
| CN115690715A (en) * | 2022-11-09 | 2023-02-03 | 中汽创智科技有限公司 | Lane line detection method, electronic device and storage medium |
| CN115601355A (en) * | 2022-11-10 | 2023-01-13 | 四川启睿克科技有限公司（Cn） | Method and device for detecting and classifying product surface defects and storage medium |
| CN115861305A (en) * | 2023-02-20 | 2023-03-28 | 佛山市南海区广工大数控装备协同创新研究院 | Flexible circuit board detection method and device, computer equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| Context-aware Method for Small Object Segmentation in Road Scenes;H. Wang et al;《2022 International Conference on Advanced Robotics and Mechatronics》;238-243 * |
| 基于多级transformer的超大倍率重建网络：参考图像超分辨率;陈彤等;《计算机与现代化》(第8期);121-126 * |
| 基于残差空洞卷积神经网络的网络安全实体识别方法;谢博等;《网络与信息安全学报》;第6卷(第5期);126-138 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116485729A (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108364280B (en) | Method and equipment for automatically describing structural crack and accurately measuring width | |
| CN113362326B (en) | Method and device for detecting defects of welding spots of battery | |
| KR102121958B1 (en) | Method, system and computer program for providing defect analysis service of concrete structure | |
| CN112598672A (en) | Pavement disease image segmentation method and system based on deep learning | |
| CN111667455A (en) | AI detection method for various defects of brush | |
| CN109215009B (en) | Continuous casting billet surface image defect detection method based on deep convolution neural network | |
| CN110569730B (en) | Road surface crack automatic identification method based on U-net neural network model | |
| CN110119677B (en) | Image classification network-based carbon fiber composite core cable damage detection method | |
| TW202004776A (en) | Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method | |
| CN110838117A (en) | Rock face porosity identification method based on hole wall image | |
| CN106934829A (en) | The detection method and system of a kind of surface crack | |
| CN110555831A (en) | Drainage pipeline defect segmentation method based on deep learning | |
| CN114705689A (en) | Unmanned aerial vehicle-based method and system for detecting cracks of outer vertical face of building | |
| CN111724358A (en) | Concrete quality detection method and system based on image and convolutional neural network | |
| CN111951289B (en) | Underwater sonar image data segmentation method based on BA-Unet | |
| CN111754463A (en) | Ballastless track CA mortar layer defect detection method based on convolutional neural network | |
| CN114066808A (en) | Pavement defect detection method and system based on deep learning | |
| CN111105389A (en) | Detection method for pavement crack by fusing Gabor filter and convolutional neural network | |
| CN115223054A (en) | Remote sensing image change detection method based on partition clustering and convolution | |
| CN112927237A (en) | Honeycomb lung focus segmentation method based on improved SCB-Unet network | |
| CN116485729B (en) | Multistage bridge defect detection method based on transformer | |
| CN111199539A (en) | Crack detection method based on integrated neural network | |
| CN111325724B (en) | Tunnel crack region detection method and device | |
| CN110321944A (en) | A kind of construction method of the deep neural network model based on contact net image quality evaluation | |
| CN115438547A (en) | Overall evaluation method and system based on pavement service state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |