CN116485729B - Multistage bridge defect detection method based on transformer - Google Patents

Multistage bridge defect detection method based on transformer Download PDF

Info

Publication number
CN116485729B
CN116485729B CN202310350506.4A CN202310350506A CN116485729B CN 116485729 B CN116485729 B CN 116485729B CN 202310350506 A CN202310350506 A CN 202310350506A CN 116485729 B CN116485729 B CN 116485729B
Authority
CN
China
Prior art keywords
bridge
layer
image data
convolution
feature map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310350506.4A
Other languages
Chinese (zh)
Other versions
CN116485729A (en
Inventor
路永钢
陈丝璐
王玉锟
吕锦辉
赵礼刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lanzhou University
Gansu Road and Bridge Construction Group Co Ltd
Original Assignee
Lanzhou University
Gansu Road and Bridge Construction Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lanzhou University, Gansu Road and Bridge Construction Group Co Ltd filed Critical Lanzhou University
Priority to CN202310350506.4A priority Critical patent/CN116485729B/en
Publication of CN116485729A publication Critical patent/CN116485729A/en
Application granted granted Critical
Publication of CN116485729B publication Critical patent/CN116485729B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30184Infrastructure
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04SSYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a transformation-based multistage bridge defect detection method, which comprises the following steps of: s1, acquiring a bridge bottom image, and establishing a bridge image data set; s2, manufacturing a bridge disease data set according to the bridge image data set; s3, inputting the bridge disease data set into a two-class network to obtain image data of a defect class; s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes. According to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.

Description

Multistage bridge defect detection method based on transformer
Technical Field
The invention belongs to the technical field of bridge defect detection, and particularly relates to a transformation-based multistage bridge defect detection method.
Background
I build highway bridges vigorously in our country, so that some highway bridges are older by even more than 40 years. According to the road investigation results in recent years, more and more highway bridges are obviously aged and damaged, which results in more and more dangerous bridges. They all commonly have the problems of bridge deck narrowness, broken cracks, exposed ribs and the like.
If the road bridge diseases can be detected in early stage and treated in time, the maintenance cost of the road bridge can be reduced to a great extent, and meanwhile, the driving safety of the road bridge can be guaranteed to a certain extent. Under the condition of not influencing transportation, the efficient and rapid checking and detecting of the cracks becomes a difficulty which needs to be solved urgently at present. At present, most of damage detection of highway bridges in China still depends on manual detection and investigation. The main disadvantages of the traditional artificial bridge defect detection are as follows: (1) labor cost is too high: the surface of the highway bridge is inspected and detected by human eyes, so that professional workers are inevitably required to survey the site of the highway bridge, however, for extremely long highway bridges of roads, the work intensity of inspecting the road bridge one by using human eyes is definitely huge, and the labor and the number of the professional workers are correspondingly high. (2) time consuming: the road bridge is inspected by naked eyes of workers, and a great amount of time is required for inspecting the road bridge by naked eyes because bridge surface defects such as cracks and hollows are in the centimeter level. (3) low accuracy: the human eyes can clearly distinguish the contrast of the light and shade, but are easily influenced by different colors to influence the judgment, and accordingly, the precision is reduced. (4) high cost: the detection can be completed only by inputting a great deal of manpower, material resources and time, and the cost is too high.
Disclosure of Invention
Aiming at the defects in the prior art, the multi-stage bridge defect detection method based on the transformer solves the problems of high labor cost, time consumption, low accuracy and high cost of the traditional artificial bridge defect detection.
In order to achieve the aim of the invention, the invention adopts the following technical scheme: a method for detecting defects of a multistage bridge based on a transducer comprises the following steps:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes.
Further: the step S1 specifically comprises the following steps:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
Further: the step S2 specifically comprises the following steps:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
Further: the step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
The beneficial effects of the above-mentioned further scheme are: the classification network continuously rewards normal classes and penalizes defective classes in the classification process, so that the identification accuracy of the normal classes is very high, and all the pictures of the normal classes can be detected and identified in the step.
Further: the step S31 specifically includes:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
Further: the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map.
Further: in the step S4, the deep learning network model of the transducer attention mechanism comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer which are sequentially connected;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units;
the second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step size of the fifth convolution layer is 8, and the expansion coefficient is 4.
The beneficial effects of the above-mentioned further scheme are: the 3*3 cavity convolution kernel is adopted, so that a model can obtain a larger receptive field, and the effect of identifying smaller defects can be improved.
Further: the transducer-based attention mechanism subunit comprises a query_conv convolution, a key_conv convolution and a value_conv convolution which are connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
Further: in SA1, the query_conv convolution sets the size of the received image data to b×c×w×h, where B is the size of the training batch, C is the number of channels, W is the image width, and H is the image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
The beneficial effects of the invention are as follows:
(1) According to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.
(2) The invention introduces a cavity convolution and a transducer attention mechanism, so that the traditional network model has better performance in the aspect of bridge detection, and is more suitable for bridge defect classification detection.
(3) The invention greatly saves manpower and material resources and improves the accuracy of bridge defect detection.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a schematic diagram of a deep learning network model of the transducer attention mechanism of the present invention.
FIG. 3 is a schematic diagram of a transducer-based attention mechanism subunit of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
Example 1:
as shown in fig. 1, in one embodiment of the present invention, a method for detecting a defect of a multi-stage bridge based on a transducer includes the steps of:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting the image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes.
The step S1 specifically comprises the following steps:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
The step S2 specifically comprises the following steps:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
In this embodiment, each type of collected image is about 10000, and the total of bridge disease images is about 60000 images.
The step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
The step S31 specifically includes:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
In this embodiment, the classification network continuously rewards the normal class during the classification process, penalizes the defect class, so that the accuracy of identifying the normal class is very high, and all the pictures of the normal class can be detected and identified in this step.
The learning rate set by the two-class network is 0.0001, the optimization function is Adam algorithm, and the relevant parameters of the convolution kernel and the pooling layer of the VGG layer are continuously adjusted in the training process of the two-class network. And in the classifying process, normal classes are continuously rewarded, defect classes are punished, so that the identification accuracy of the normal classes is very high, all pictures of the normal classes are detected and identified as far as possible, the classification of the existence of diseases of bridge disease images is preliminarily realized, and the image data of the defect classes are obtained through screening.
In this embodiment, the deep learning network model of the transducer attention mechanism is shown in fig. 2; the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map (FeatureMap) with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map.
In the step S4, the deep learning network model of the transducer attention mechanism includes a first convolution layer (stage 1), a second convolution layer (stage 2), a third convolution layer (stage 3), a fourth convolution layer (stage 4), a fifth convolution layer (stage 5) and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer (Maxpool) which are connected in sequence;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units.
The second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step size of the fifth convolution layer is 8, and the expansion coefficient is 4.
The first max pooling layer is used for downsampling, when out and the channel number of the original input x are the same, downsamples are None, out and the original input x can be directly overlapped, when out and the channel number of the original input x are different, the channel number is increased through downsampling and then overlapped with out, and n is determined by the value in the list transmitted before.
In this embodiment, the cavity convolution kernel is set to 3*3, so that the model can obtain a larger receptive field, and the effect of identifying smaller defects can be improved. It will be apparent that the use of hole convolution instead of downsampling/upsampling can preserve the spatial features of the image well without losing image information.
As shown in fig. 3, the transducer-based attention mechanism subunit includes a query_conv convolution, a key_conv convolution, and a value_conv convolution connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
In SA1, the query_conv convolution sets the size of the received image data to b×c×w×h, where B is the size of the training batch, C is the number of channels, W is the image width, and H is the image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
The beneficial effects of the invention are as follows: according to the method, the normal class and the five defect classes are identified through the two-class network, and the result of the two-class network is used as the input of the network model based on the transducer attention mechanism, so that the network model based on the transducer attention mechanism can better identify each defect class, and the identification accuracy is further improved.
The invention introduces a cavity convolution and a transducer attention mechanism, so that the traditional network model has better performance in the aspect of bridge detection, and is more suitable for bridge defect classification detection.
The invention greatly saves manpower and material resources and improves the accuracy of bridge defect detection.
In the description of the present invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," "radial," and the like indicate or are based on the orientation or positional relationship shown in the drawings, merely to facilitate description of the present invention and to simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be configured and operated in a particular orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be interpreted as indicating or implying a relative importance or number of technical features indicated. Thus, a feature defined as "first," "second," "third," or the like, may explicitly or implicitly include one or more such feature.

Claims (6)

1. A method for detecting defects of a multistage bridge based on a transducer is characterized by comprising the following steps of:
s1, acquiring a bridge bottom image, and establishing a bridge image data set;
s2, performing data cleaning, noise reduction and enhancement treatment on the bridge image data set to obtain a bridge disease data set;
s3, inputting the bridge disease data set into a two-class classification network, and removing normal image data to obtain defective image data;
s4, inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain detection results of different defect classes;
the step S4 specifically comprises the following steps:
s41, sequentially inputting image data of the defect class into a deep learning network model of a transducer attention mechanism to obtain a defect class feature map with a weighted relation;
s42, sequentially inputting the defect class feature map into a first average pooling layer, a full-connection layer and a first softmax layer to obtain a global dependency relationship of the defect class feature map;
s43, obtaining an image data defect classification result of the defect class corresponding to the defect class feature map according to the global dependency relationship in the defect class feature map;
in the step S4, the deep learning network model of the transducer attention mechanism comprises a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer and a second average pooling layer which are sequentially connected;
the first convolution layer comprises a first cavity convolution layer, a batch normalization layer, a first ReLU activation function and a first maximum pooling layer which are sequentially connected;
the second convolution layer, the third convolution layer, the fourth convolution layer and the fifth convolution layer all comprise a plurality of block units which are connected in sequence, wherein the second convolution layer comprises 3 block units, the third convolution layer comprises 4 block units, the fourth convolution layer comprises 6 block units, and the fifth convolution layer comprises 3 block units;
the second average pooling layer includes a global tie pooling layer and a second softmax layer that are interconnected;
each block unit comprises a second 3*3 hole convolution layer, a transducer-based attention mechanism subunit, a third 3*3 hole convolution layer and a first pooling layer which are connected in sequence; the transducer-based attention mechanism subunit is configured to weight received image data;
the step length of the first convolution layer is 2, and the expansion coefficient is 2; the step length of the second convolution layer is 2, and the expansion coefficient is 2; the step length of the third convolution layer is 2, and the expansion coefficient is 2; the step length of the fourth convolution layer is 8, and the expansion coefficient is 2; the step length of the fifth convolution layer is 8, and the expansion coefficient is 4;
the transducer-based attention mechanism subunit comprises a query_conv convolution, a key_conv convolution and a value_conv convolution which are connected in sequence;
the method for weighting the received image data based on the transducer attention mechanism subunit specifically comprises the following steps:
SA1, inputting image data received by a transducer attention mechanism subunit into query_conv for convolution to obtain a first feature map;
SA2, inputting the first feature map into a key_conv convolution to obtain a second feature map;
SA3, performing matrix multiplication on the first feature map and the second feature map, and performing softmax normalization on the result of matrix multiplication to obtain a third feature map;
SA4, inputting the image data received by the transducer attention mechanism subunit into value_conv convolution to obtain a fourth feature map, and performing matrix multiplication on the third feature map and the fourth feature map to obtain a fifth feature map;
SA5, weighting the fifth feature map, and superposing the weighted fifth feature map with the image data received by the transducer-based attention mechanism subunit to obtain a sixth feature map;
SA6, taking the sixth characteristic diagram as an output result of the transducer attention mechanism subunit, and completing weighting of the received image data.
2. The method for detecting defects of a multistage bridge based on transformers according to claim 1, wherein the step S1 is specifically:
shooting by a highway bridge bottom data acquisition cradle head to acquire bridge bottom image data, and removing an unclear image to obtain a bridge bottom image; and processing the bridge bottom image through a random operator to generate a plurality of bridge image data with the size of 256 x 256, and taking the bridge image data as a bridge image data set.
3. The method for detecting defects of a multi-stage bridge based on a transducer according to claim 1, wherein the step S2 is specifically:
performing data cleaning and noise reduction processing of a matching model on bridge image data in the bridge image data set, performing data enhancement on the processed bridge image data, performing random translation, rotation, overturning and mirroring on the bridge image data subjected to data enhancement, generating a plurality of bridge defect images, and taking the bridge defect images as a bridge defect data set;
the image categories of the bridge disease images in the bridge disease data set comprise normal bridge floors, honeycomb pitted surfaces, cracks, broken exposed ribs, water seepage and repair.
4. The method for detecting defects in a multi-stage bridge based on a transformer according to claim 3, wherein the step S3 comprises the following sub-steps:
s31, setting a label of a bridge disease image in the bridge disease data set;
s32, inputting the bridge defect image and the label of the bridge defect image into a two-class network, and removing the image data of the normal class to obtain the image data of the defect class.
5. The method for detecting defects of a multi-stage bridge based on a transducer according to claim 4, wherein the step S31 specifically comprises:
setting the label of the bridge defect image with the image category of a normal bridge deck to 0, and setting the labels of the bridge defect images with the rest image categories to 1;
in the step S32, the two classification networks include a first VGG layer, a second VGG layer, a third VGG layer, a fourth VGG layer, and a fifth VGG layer that are sequentially connected; the first VGG layer, the second VGG layer, the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise a convolution layer sub-network and a pooling layer which are connected with each other; wherein the pooling layer is specifically a2×2 pooling window, and the stride is 2;
the convolution layer sub-networks in the first VGG layer and the second VGG layer comprise two interconnected convolution layers, the convolution kernel of the convolution layers is 3 multiplied by 3, and the convolution kernel is filled with 1;
the convolutional layer sub-networks in the third VGG layer, the fourth VGG layer and the fifth VGG layer all comprise three convolutional layers which are connected in sequence, and the convolutional core of the convolutional layers is 3 multiplied by 3, and the filling is 1.
6. The method for detecting defects of a multistage bridge based on a transducer according to claim 1, wherein in SA1, a query_conv convolution sets a received image data size of b×c×w×h, wherein B is a training lot size, C is a channel number, W is an image width, and H is an image height;
in SA2, the first feature map has a size of BXC/8 (WXH) and the second feature map has a size of BXC/8 (WXH);
the SA3 specifically comprises:
and (3) carrying out matrix multiplication on the first characteristic diagram and the second characteristic diagram to generate an intermediate characteristic diagram with the size of B× (W×H) × (W×H), and carrying out softmax normalization on the intermediate characteristic diagram to obtain a third characteristic diagram.
CN202310350506.4A 2023-04-03 2023-04-03 Multistage bridge defect detection method based on transformer Active CN116485729B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310350506.4A CN116485729B (en) 2023-04-03 2023-04-03 Multistage bridge defect detection method based on transformer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310350506.4A CN116485729B (en) 2023-04-03 2023-04-03 Multistage bridge defect detection method based on transformer

Publications (2)

Publication Number Publication Date
CN116485729A CN116485729A (en) 2023-07-25
CN116485729B true CN116485729B (en) 2024-01-12

Family

ID=87216980

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310350506.4A Active CN116485729B (en) 2023-04-03 2023-04-03 Multistage bridge defect detection method based on transformer

Country Status (1)

Country Link
CN (1) CN116485729B (en)

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111191714A (en) * 2019-12-28 2020-05-22 浙江大学 Intelligent identification method for bridge appearance damage diseases
WO2021135477A1 (en) * 2020-07-31 2021-07-08 平安科技(深圳)有限公司 Probabilistic graphical model-based text attribute extraction method and apparatus, computer device and storage medium
CN113139470A (en) * 2021-04-25 2021-07-20 安徽工业大学 Glass identification method based on Transformer
CN113397572A (en) * 2021-07-23 2021-09-17 中国科学技术大学 Surface electromyographic signal classification method and system based on Transformer model
CN113591770A (en) * 2021-08-10 2021-11-02 中国科学院深圳先进技术研究院 Multimode fusion obstacle detection method and device based on artificial intelligence blind guiding
CN114841320A (en) * 2022-05-07 2022-08-02 西安邮电大学 Organ automatic segmentation method based on laryngoscope medical image
CN114937285A (en) * 2022-05-25 2022-08-23 兰州大学 Dynamic gesture recognition method, device, equipment and storage medium
CN114943694A (en) * 2022-05-09 2022-08-26 浙江科技学院 Defect detection method based on confrontation generation network and attention
CN115063351A (en) * 2022-05-24 2022-09-16 中国科学院深圳先进技术研究院 Deep learning-based fetal MRI brain tissue segmentation method and device
CN115205972A (en) * 2022-07-13 2022-10-18 安徽大学 Six-step hand washing evaluation method and device based on deep learning
CN115205230A (en) * 2022-06-21 2022-10-18 福州大学 Concrete bridge apparent crack identification method based on novel attention mechanism
CN115240195A (en) * 2022-08-04 2022-10-25 成都泓睿科技有限责任公司 Medicine bottle defect automatic detection method and device and storage medium
CN115359550A (en) * 2022-07-06 2022-11-18 华南师范大学 Gait emotion recognition method and device based on Transformer, electronic device and storage medium
CN115359306A (en) * 2022-10-24 2022-11-18 中铁科学技术开发有限公司 Intelligent identification method and system for high-definition images of railway freight inspection
CN115482491A (en) * 2022-09-23 2022-12-16 湖南大学 Bridge defect identification method and system based on transformer
CN115601355A (en) * 2022-11-10 2023-01-13 四川启睿克科技有限公司(Cn) Method and device for detecting and classifying product surface defects and storage medium
CN115619726A (en) * 2022-09-29 2023-01-17 西安理工大学 Lightweight defect detection method for detecting defects of automobile body paint surface
CN115690715A (en) * 2022-11-09 2023-02-03 中汽创智科技有限公司 Lane line detection method, electronic device and storage medium
CN115861305A (en) * 2023-02-20 2023-03-28 佛山市南海区广工大数控装备协同创新研究院 Flexible circuit board detection method and device, computer equipment and storage medium

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111191714A (en) * 2019-12-28 2020-05-22 浙江大学 Intelligent identification method for bridge appearance damage diseases
WO2021135477A1 (en) * 2020-07-31 2021-07-08 平安科技(深圳)有限公司 Probabilistic graphical model-based text attribute extraction method and apparatus, computer device and storage medium
CN113139470A (en) * 2021-04-25 2021-07-20 安徽工业大学 Glass identification method based on Transformer
CN113397572A (en) * 2021-07-23 2021-09-17 中国科学技术大学 Surface electromyographic signal classification method and system based on Transformer model
CN113591770A (en) * 2021-08-10 2021-11-02 中国科学院深圳先进技术研究院 Multimode fusion obstacle detection method and device based on artificial intelligence blind guiding
CN114841320A (en) * 2022-05-07 2022-08-02 西安邮电大学 Organ automatic segmentation method based on laryngoscope medical image
CN114943694A (en) * 2022-05-09 2022-08-26 浙江科技学院 Defect detection method based on confrontation generation network and attention
CN115063351A (en) * 2022-05-24 2022-09-16 中国科学院深圳先进技术研究院 Deep learning-based fetal MRI brain tissue segmentation method and device
CN114937285A (en) * 2022-05-25 2022-08-23 兰州大学 Dynamic gesture recognition method, device, equipment and storage medium
CN115205230A (en) * 2022-06-21 2022-10-18 福州大学 Concrete bridge apparent crack identification method based on novel attention mechanism
CN115359550A (en) * 2022-07-06 2022-11-18 华南师范大学 Gait emotion recognition method and device based on Transformer, electronic device and storage medium
CN115205972A (en) * 2022-07-13 2022-10-18 安徽大学 Six-step hand washing evaluation method and device based on deep learning
CN115240195A (en) * 2022-08-04 2022-10-25 成都泓睿科技有限责任公司 Medicine bottle defect automatic detection method and device and storage medium
CN115482491A (en) * 2022-09-23 2022-12-16 湖南大学 Bridge defect identification method and system based on transformer
CN115619726A (en) * 2022-09-29 2023-01-17 西安理工大学 Lightweight defect detection method for detecting defects of automobile body paint surface
CN115359306A (en) * 2022-10-24 2022-11-18 中铁科学技术开发有限公司 Intelligent identification method and system for high-definition images of railway freight inspection
CN115690715A (en) * 2022-11-09 2023-02-03 中汽创智科技有限公司 Lane line detection method, electronic device and storage medium
CN115601355A (en) * 2022-11-10 2023-01-13 四川启睿克科技有限公司(Cn) Method and device for detecting and classifying product surface defects and storage medium
CN115861305A (en) * 2023-02-20 2023-03-28 佛山市南海区广工大数控装备协同创新研究院 Flexible circuit board detection method and device, computer equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Context-aware Method for Small Object Segmentation in Road Scenes;H. Wang et al;《2022 International Conference on Advanced Robotics and Mechatronics》;238-243 *
基于多级transformer的超大倍率重建网络:参考图像超分辨率;陈彤等;《计算机与现代化》(第8期);121-126 *
基于残差空洞卷积神经网络的网络安全实体识别方法;谢博等;《网络与信息安全学报》;第6卷(第5期);126-138 *

Also Published As

Publication number Publication date
CN116485729A (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN108364280B (en) Method and equipment for automatically describing structural crack and accurately measuring width
CN113362326B (en) Method and device for detecting defects of welding spots of battery
KR102121958B1 (en) Method, system and computer program for providing defect analysis service of concrete structure
CN112598672A (en) Pavement disease image segmentation method and system based on deep learning
CN111667455A (en) AI detection method for various defects of brush
CN109215009B (en) Continuous casting billet surface image defect detection method based on deep convolution neural network
CN110569730B (en) Road surface crack automatic identification method based on U-net neural network model
CN110119677B (en) Image classification network-based carbon fiber composite core cable damage detection method
TW202004776A (en) Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method
CN110838117A (en) Rock face porosity identification method based on hole wall image
CN106934829A (en) The detection method and system of a kind of surface crack
CN110555831A (en) Drainage pipeline defect segmentation method based on deep learning
CN114705689A (en) Unmanned aerial vehicle-based method and system for detecting cracks of outer vertical face of building
CN111724358A (en) Concrete quality detection method and system based on image and convolutional neural network
CN111951289B (en) Underwater sonar image data segmentation method based on BA-Unet
CN111754463A (en) Ballastless track CA mortar layer defect detection method based on convolutional neural network
CN114066808A (en) Pavement defect detection method and system based on deep learning
CN111105389A (en) Detection method for pavement crack by fusing Gabor filter and convolutional neural network
CN115223054A (en) Remote sensing image change detection method based on partition clustering and convolution
CN112927237A (en) Honeycomb lung focus segmentation method based on improved SCB-Unet network
CN116485729B (en) Multistage bridge defect detection method based on transformer
CN111199539A (en) Crack detection method based on integrated neural network
CN111325724B (en) Tunnel crack region detection method and device
CN110321944A (en) A kind of construction method of the deep neural network model based on contact net image quality evaluation
CN115438547A (en) Overall evaluation method and system based on pavement service state

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant