CN112699736B - Bridge bearing disease identification method based on spatial attention - Google Patents
Bridge bearing disease identification method based on spatial attention Download PDFInfo
- Publication number
- CN112699736B CN112699736B CN202011442501.7A CN202011442501A CN112699736B CN 112699736 B CN112699736 B CN 112699736B CN 202011442501 A CN202011442501 A CN 202011442501A CN 112699736 B CN112699736 B CN 112699736B
- Authority
- CN
- China
- Prior art keywords
- image
- sample image
- bridge
- grid point
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 201000010099 disease Diseases 0.000 title claims abstract description 29
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 29
- 238000013528 artificial neural network Methods 0.000 claims abstract description 19
- 230000007246 mechanism Effects 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims abstract description 10
- 238000012512 characterization method Methods 0.000 claims description 3
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 210000001503 joint Anatomy 0.000 claims 1
- 230000006870 function Effects 0.000 abstract description 9
- 238000013527 convolutional neural network Methods 0.000 abstract description 4
- 238000003062 neural network model Methods 0.000 abstract 2
- 238000002372 labelling Methods 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 5
- 238000005336 cracking Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000010008 shearing Methods 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009440 infrastructure construction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000035882 stress Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a bridge bearing disease identification method based on spatial attention, which comprises the following steps: and acquiring bridge support image data, and endowing a label with the label by a manual labeling method, wherein the label comprises a normal support and various support diseases possibly occurring in the bridge service process. Constructing a neural network model with a spatial attention mechanism, wherein the spatial attention mechanism generates 4 attention coordinate values through a small neural network, screens out valuable areas in an image according to the 4 coordinate values, and scales to a specified size through a grid point generating function and a bilinear interpolation method; and training the output of the spatial attention mechanism as the input of the convolutional neural network to obtain a neural network model with the function of predicting the support diseases. The attention model can enable the network model to automatically extract valuable areas in the bridge support image for learning, and compared with the traditional convolutional neural network model, the recognition accuracy of the support diseases can be effectively improved.
Description
Technical Field
The invention relates to a bridge bearing disease identification method based on spatial attention, and belongs to the technical field of civil engineering and artificial intelligence interaction.
Background
Along with the rapid development of infrastructure construction, the development of the civil industry is rapid, a large number of bridges are built, and bridge supports are important components for bridge stress, and can generate diseases such as aging, cracking and the like in the long-term service process, so that the normal use functions of the bridge supports are affected. People have relied on daily and periodic checks, samplings, and temporary checks to obtain information about the structure. However, the existing bridge appearance detection mainly depends on manual detection, the method has low efficiency, long time consumption and high cost, the manual detection method is greatly influenced by factors such as environment, professional technical literacy of detection personnel and the like, and the detection result has uncertainty.
Image processing technology based on deep learning is rapidly developed and widely applied in various industries, however, a deep learning model often needs a large amount of annotation data to train to reach higher precision. In actual engineering, some disease image data are difficult to acquire, and the support is used as a connecting member of a bridge, and other members and some background information are contained in the image acquisition engineering, so that the recognition result of the model is affected. Therefore, a method for improving the recognition accuracy of the model under the condition of limited data sets and complex scenes is urgently needed.
Disclosure of Invention
The invention aims to solve the technical problem of providing a bridge support disease identification method based on spatial attention, which is used for improving the training efficiency and accuracy of a neural network for identifying bridge support diseases.
The invention adopts the following technical scheme for solving the technical problems: the invention designs a bridge support disease identification method based on spatial attention, which is based on a plurality of sample images respectively containing only one bridge support, and comprises the following steps of A to E, training and obtaining a target neural network based on a spatial attention mechanism for identifying bridge support diseases; the target neural network comprises a space coordinate generation network and a classification network; then, aiming at a target image to be detected, sequentially applying a space coordinate generation network and a classification network, so as to obtain a classification result of the bridge support contained in the target image;
Step A, respectively associating each sample image with a preset classification label type of the bridge support contained in each sample image to construct a bridge support sample image database;
Defining a minimum rectangular area where a bridge support is located in a sample image as a space attention area, and constructing a space coordinate generation network taking the sample image as input and four space coordinates of the space attention area in the sample image as output based on a bridge support sample image database, wherein the four space coordinates of the space attention area are x min,ymin,xmax,ymax, x min,ymin is the abscissa and the ordinate of the upper left corner of the space attention area, and x max,ymax is the abscissa and the ordinate of the lower right corner of the space attention area;
step C, aiming at each sample image, applying a grid point generating function according to four space coordinates of a space attention area in the sample image to obtain coordinates of each grid point in the space attention area in the sample image under the grid division corresponding to the preset proportion, and then entering the step D;
Step D, obtaining pixel values of all grid points in a spatial attention area in each sample image through a preset interpolation method, and obtaining processed pictures based on an attention mechanism, corresponding to each sample image;
and E, constructing a classification network which takes the attention-based picture of each sample image as input and the category of the support image in the corresponding sample image as output based on the label in the bridge support sample image database and the attention-based picture of each sample image, and generating a network and the classification network through end-to-end iterative training space coordinates according to the output category of the classification network and the difference between the classification network and the labeled category in the step A as a network optimization target to obtain the target neural network.
In the step A, the constructed bridge support sample image database contains different states of the bridge support, namely normal support states and states of various support diseases preset in the bridge service process.
In the step A, the resolution of the image in the constructed bridge support sample image database is more than 800 multiplied by 600.
As a preferred technical solution of the present invention, the space coordinate generating network in step B includes a convolution layer, a pooling layer and a full connection layer.
In step C, as a preferred technical solution of the present invention, the grid point generating function generates the image grid points according to the proportional relationship between the 4 image coordinates generated by the attention mechanism and the original image, and the description of the grid point generating function is as follows:
Wherein x 'ij is the abscissa of the grid point i, j corresponding to the original image, y' ij is the ordinate of the grid point i, j corresponding to the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, x ij is the abscissa of the grid point i, j in the preset grid point image, and y ij is the ordinate of the grid point i, j in the preset grid point image.
In the step D, the preset interpolation method is a differentiable interpolation method to meet the requirement of error back propagation.
In step E, the classification network is a neural network that satisfies the preset specific feature characterizing capability.
As a preferable technical solution of the present invention, the neural network with the preset specific feature characterizing capability is VGG or ResNet.
As a preferable technical scheme of the invention, in the step E, the iterative training of the target neural network adopts a gradient descent method.
Compared with the prior art, the bridge bearing disease identification method based on spatial attention has the following technical effects:
According to the invention, a target neural network consisting of a network and a classification network can be generated through joint training of space coordinates under a limited data set, a neural network based on a space attention mechanism is obtained, aiming at a bridge support image to be detected by a target, firstly, an interested region in the bridge support image is identified, and then whether diseases and types of diseases of the bridge support are obtained through the classification network.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of a spatial coordinate generation network according to the present invention;
FIG. 3 is a flow chart of the attention area acquisition in the present invention;
FIG. 4 is a schematic diagram of a VGG-16 classification network used in the present invention.
Detailed Description
The following describes the embodiments of the present invention in further detail with reference to the drawings.
The present embodiment trains the hardware conditions of the convolutional neural network: the Amazon AWS cloud computing service is used for configuring Amazon EC2P2. Xlage examples, 1 GPU,4 vCPU and 61GB random access memory are configured, a ubuntu system is adopted by the system, python is adopted by a programming language, and pytorch is adopted by a deep learning platform.
As shown in the flowchart of fig. 1, in the method for identifying bridge bearing diseases based on spatial attention according to the embodiment, 5000 sample images with resolution of more than 800×600 and containing only one bridge bearing are selected, wherein each sample image contains different states of the bridge bearing, including a normal bearing state and states of various bearing diseases preset in the bridge service process, and the bridge bearing states in the sample images in the embodiment include three types of normal bearing, cracking bearing and shearing deformation bearing.
Based on the sample images, training and acquiring a target neural network based on a spatial attention mechanism for identifying bridge bearing diseases by applying the following steps A to E; the target neural network comprises a space coordinate generation network and a classification network.
And A, respectively associating each sample image with a preset classification label type of the bridge support contained in each sample image to construct a bridge support sample image database, wherein the bridge support contained in the classification label in the embodiment is in a normal support type, a cracking support type and a shearing deformation support type.
And B, defining a minimum rectangular area where the bridge support is located in the sample image as a space attention area, constructing a space coordinate generation network taking the sample image as an input and four space coordinates of the space attention area in the sample image as an output based on a bridge support sample image database, wherein the space coordinate generation network comprises a convolution layer, a pooling layer and a full connection layer, the four space coordinates of the space attention area are x min,ymin,xmax,ymax, x min,ymin is the abscissa and the ordinate of the upper left corner of the space attention area, and x max,ymax is the abscissa and the ordinate of the lower right corner of the space attention area.
And C, as shown in fig. 3, respectively aiming at each sample image, applying the following grid point generating function according to four space coordinates of a space attention area in the sample image, and obtaining the coordinates of each grid point in the space attention area in the sample image under the grid division corresponding to the preset proportion.
The grid point generating function generates image grid points according to the proportional relation between 4 image coordinates generated by an attention mechanism and an original image, and the grid point generating function is described as follows:
Wherein x 'ij is the abscissa of the grid point i, j corresponding to the original image, y' ij is the ordinate of the grid point i, j corresponding to the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, x ij is the abscissa of the grid point i, j in the preset grid point image, and y ij is the ordinate of the grid point i, j in the preset grid point image.
And D, aiming at each sample image, obtaining pixel values of each grid point in a spatial attention area in the sample image by a differentiable interpolation method meeting the requirement of error back propagation, and obtaining the processed picture based on the attention mechanism, corresponding to each sample image.
And E, as shown in fig. 4, constructing a classification network which is based on labels in a bridge support sample image database and pictures based on attention mechanisms of each sample image, takes the pictures based on attention mechanisms of each sample image as input, takes the category of support images in the corresponding sample images as output and meets the preset appointed characteristic characterization capability, wherein a VGG network in a classical convolutional neural network is specifically selected in the embodiment, and then, according to the output category of the classification network, the difference between the classification network and the labeled category in the step A is used as a network optimization target, and the gradient descent method is utilized to perform joint training on a space coordinate generation network and the classification network to obtain a target neural network.
Then, a space coordinate generation network and a classification network are sequentially applied to the target image to be detected, so that a classification result of the bridge support contained in the target image is obtained.
The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present invention.
Claims (8)
1. A bridge support disease identification method based on spatial attention is characterized in that based on a plurality of sample images respectively containing only one bridge support, the following steps A to E are applied to train and acquire a target neural network based on a spatial attention mechanism for identifying the bridge support disease; the target neural network comprises a space coordinate generation network and a classification network; then, aiming at a target image to be detected, sequentially applying a space coordinate generation network and a classification network, so as to obtain a classification result of the bridge support contained in the target image;
Step A, respectively associating each sample image with a preset classification label type of the bridge support contained in each sample image to construct a bridge support sample image database;
Defining a minimum rectangular area where a bridge support is located in a sample image as a space attention area, and constructing a space coordinate generation network taking the sample image as input and four space coordinates of the space attention area in the sample image as output based on a bridge support sample image database, wherein the four space coordinates of the space attention area are x min,ymin,xmax,ymax, x min,ymin is the abscissa and the ordinate of the upper left corner of the space attention area, and x max,ymax is the abscissa and the ordinate of the lower right corner of the space attention area;
step C, aiming at each sample image, applying a grid point generating function according to four space coordinates of a space attention area in the sample image to obtain coordinates of each grid point in the space attention area in the sample image under the grid division corresponding to the preset proportion, and then entering the step D;
Step D, obtaining pixel values of all grid points in a spatial attention area in each sample image through a preset interpolation method, and obtaining processed pictures based on an attention mechanism, corresponding to each sample image;
E, constructing a classification network which takes the attention-based picture of each sample image as input and the category of the support image in the corresponding sample image as output based on the label in the bridge support sample image database and the attention-based picture of each sample image, and generating a network and the classification network through end-to-end iterative training space coordinates according to the output category of the classification network and the difference between the classification network and the labeled category in the step A as a network optimization target to obtain a target neural network;
In step C, the grid point generating function generates image grid points according to the proportional relation between 4 image coordinates generated by the attention mechanism and the original image, and the description of the grid point generating function is as follows:
Wherein x 'ij is the abscissa of the grid point i, j corresponding to the original image, y' ij is the ordinate of the grid point i, j corresponding to the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, x ij is the abscissa of the grid point i, j in the preset grid point image, and y ij is the ordinate of the grid point i, j in the preset grid point image.
2. The method for identifying bridge bearing diseases based on spatial attention according to claim 1, wherein in the step A, the constructed bridge bearing sample image database contains different states of bridge bearings, including normal bearing states and states of various bearing diseases preset in the bridge service process.
3. The method for identifying the bridge beam bearing diseases based on the spatial attention as set forth in claim 1, wherein in the step A, the resolution of the image in the constructed bridge beam bearing sample image database is more than 800×600.
4. The method for identifying bridge beam joint damage based on spatial attention according to claim 1, wherein the spatial coordinate generating network in the step B comprises a convolution layer, a pooling layer and a full connection layer.
5. The method for identifying bridge beam bearing diseases based on spatial attention according to claim 1, wherein in the step D, the preset interpolation method is a differentiable interpolation method so as to meet the requirement of error back propagation.
6. The method for identifying bridge beam supports diseases based on spatial attention according to claim 1, wherein in the step E, the classification network is a neural network satisfying the characteristic characterization capability of preset specified characteristics.
7. The method for identifying bridge beam supports diseases based on spatial attention according to claim 6, wherein the neural network with preset appointed characteristic characterization capability is VGG or ResNet.
8. The method for identifying bridge beam supports diseases based on spatial attention according to claim 1, wherein in the step E, a gradient descent method is selected for iterative training of the target neural network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011442501.7A CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011442501.7A CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112699736A CN112699736A (en) | 2021-04-23 |
| CN112699736B true CN112699736B (en) | 2024-06-07 |
Family
ID=75507773
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011442501.7A Active CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699736B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297959B (en) * | 2021-05-24 | 2024-07-09 | 南京邮电大学 | Target tracking method and system based on corner point attention twin network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108288269A (en) * | 2018-01-24 | 2018-07-17 | 东南大学 | Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks |
| CN108363961A (en) * | 2018-01-24 | 2018-08-03 | 东南大学 | Bridge pad disease recognition method based on transfer learning between convolutional neural networks |
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN110070008A (en) * | 2019-04-04 | 2019-07-30 | 中设设计集团股份有限公司 | Bridge disease identification method adopting unmanned aerial vehicle image |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN111562220A (en) * | 2020-06-02 | 2020-08-21 | 吉林大学 | Rapid and intelligent detection method for bridge diseases |
| WO2020199538A1 (en) * | 2019-04-04 | 2020-10-08 | 中设设计集团股份有限公司 | Bridge key component disease early-warning system and method based on image monitoring data |
-
2020
- 2020-12-08 CN CN202011442501.7A patent/CN112699736B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN108288269A (en) * | 2018-01-24 | 2018-07-17 | 东南大学 | Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks |
| CN108363961A (en) * | 2018-01-24 | 2018-08-03 | 东南大学 | Bridge pad disease recognition method based on transfer learning between convolutional neural networks |
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN110070008A (en) * | 2019-04-04 | 2019-07-30 | 中设设计集团股份有限公司 | Bridge disease identification method adopting unmanned aerial vehicle image |
| WO2020199538A1 (en) * | 2019-04-04 | 2020-10-08 | 中设设计集团股份有限公司 | Bridge key component disease early-warning system and method based on image monitoring data |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN111562220A (en) * | 2020-06-02 | 2020-08-21 | 吉林大学 | Rapid and intelligent detection method for bridge diseases |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112699736A (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | A2-FPN for semantic segmentation of fine-resolution remotely sensed images | |
| Vega Díaz et al. | Solar panel detection within complex backgrounds using thermal images acquired by UAVs | |
| Awrangjeb et al. | An effective data-driven method for 3-d building roof reconstruction and robust change detection | |
| Zhao et al. | Recognition of building group patterns using graph convolutional network | |
| Albano | Investigation on roof segmentation for 3D building reconstruction from aerial LIDAR point clouds | |
| Zhang et al. | Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model | |
| Farella et al. | 4D building reconstruction with machine learning and historical maps | |
| Kim et al. | Automated classification of thermal defects in the building envelope using thermal and visible images | |
| Chai et al. | Automatic as-built modeling for concurrent progress tracking of plant construction based on laser scanning | |
| Lotte et al. | 3D façade labeling over complex scenarios: A case study using convolutional neural network and structure-from-motion | |
| CN112699736B (en) | Bridge bearing disease identification method based on spatial attention | |
| Kang et al. | HSV color-space-based automated object localization for robot grasping without prior knowledge | |
| CN112488990B (en) | Bridge bearing disease identification method based on attention regularization mechanism | |
| Krapf et al. | Rid—roof information dataset for computer vision-based photovoltaic potential assessment | |
| Urbieta et al. | Generating BIM model from structural and architectural plans using Artificial Intelligence | |
| Daranagama et al. | Automatic building detection with polygonizing and attribute extraction from high-resolution images | |
| Pu et al. | Autonomous concrete crack semantic segmentation using deep fully convolutional encoder–decoder network in concrete structures inspection | |
| Lee et al. | Implementing a digital twin of an underground utility tunnel for geospatial feature extraction using a multimodal image sensor | |
| Hou et al. | A novel building temperature simulation approach driven by expanding semantic segmentation training datasets with synthetic aerial thermal images | |
| Siountri et al. | The classification of cultural heritage buildings in athens using deep learning techniques | |
| Qu et al. | Optimization algorithm for steel surface defect detection based on PP-YOLOE | |
| Shinozaki et al. | Detection of deterioration of furnace walls using large-scale point-clouds | |
| Huang et al. | Deep residual network training for reinforced concrete defects intelligent classifier | |
| Cheng et al. | An Algorithm for Crack Detection, Segmentation, and Fractal Dimension Estimation in Low-Light Environments by Fusing FFT and Convolutional Neural Network | |
| Shi et al. | Development of large-scale synthetic 3D point cloud datasets for vision-based bridge structural condition assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province Applicant after: Jiangxi Academy of Transportation Sciences Co.,Ltd. Applicant after: SOUTHEAST University Address before: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province Applicant before: Jiangxi Transportation Research Institute Applicant before: SOUTHEAST University |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant |