CN114758206B - Steel truss structure abnormity detection method and device - Google Patents
Steel truss structure abnormity detection method and device Download PDFInfo
- Publication number
- CN114758206B CN114758206B CN202210662669.1A CN202210662669A CN114758206B CN 114758206 B CN114758206 B CN 114758206B CN 202210662669 A CN202210662669 A CN 202210662669A CN 114758206 B CN114758206 B CN 114758206B
- Authority
- CN
- China
- Prior art keywords
- attention
- channel
- spatial
- feature map
- remodeling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 211
- 239000010959 steel Substances 0.000 title claims abstract description 211
- 238000001514 detection method Methods 0.000 title claims abstract description 92
- 230000004927 fusion Effects 0.000 claims abstract description 57
- 230000005856 abnormality Effects 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000007634 remodeling Methods 0.000 claims description 135
- 238000010586 diagram Methods 0.000 claims description 61
- 238000010606 normalization Methods 0.000 claims description 18
- 230000003190 augmentative effect Effects 0.000 claims description 16
- 150000001875 compounds Chemical class 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 14
- 230000003416 augmentation Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 238000002372 labelling Methods 0.000 claims description 4
- 230000001788 irregular Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 17
- 238000010276 construction Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 235000021167 banquet Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009182 swimming Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a steel truss structure abnormity detection method and a device, wherein the method comprises the following steps: acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module; acquiring a structural image of the steel truss to be detected; performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map; and determining the detection result of the abnormality of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map. According to the invention, the attention module is guided to improve the feature extraction and learning capacity of the complex irregular grid structure, and the accuracy of the steel grid structure abnormity detection result can be improved.
Description
Technical Field
The invention relates to the technical field of steel mesh frame structure detection, in particular to a steel mesh frame structure abnormity detection method and device.
Background
With the rapid soaring of society and economy, large and medium public buildings such as swimming pools, stadiums, airport terminals, industrial factory buildings, scientific exhibition halls, banquet halls and the like are highly valued in all countries in the world. The steel grid structure has the advantages of light weight, large space rigidity, good earthquake resistance, economic material, convenient construction, beautiful appearance and the like, and is widely applied. However, in the development of a large-span building, the steel net rack is increasingly large in scale, and the structural form of the net rack is also increasingly complex, so that the construction difficulty of the net rack is increased. Meanwhile, in order to pursue rapid and low-cost construction, the probability of quality safety problems of the steel truss structure in the construction process is increased. In addition, after the steel mesh frame structure is built, the steel mesh frame structure is corroded by the external environment for a long time, the effective section of the member is reduced, the stress of the rod is larger, the strength and the rigidity of the steel frame are reduced, the steel frame enters the yield limit of the material prematurely, the structure generates larger deformation and internal force redistribution, and the safe use of the structure is influenced, and even the steel mesh frame structure collapses suddenly. The durability and safety of steel lattice framed structures are increasingly problematic. The steel net frame structure abnormity detection and identification are very necessary in time, and the steel net frame structure abnormity detection and identification method has important significance for safety accident protection.
The traditional method is mainly based on visual inspection of the steel framework structure condition on a manual site, and the mode belongs to a labor-intensive task. Because the steel grid structure has large span and high height, the common visual inspection is difficult to completely relate to the whole structural details, and extremely heavy manual labor is inevitably brought. And because of individual difference, subjective judgment errors are inevitably introduced by manual inspection. More seriously, potential safety accidents can be brought by the high-altitude climbing operation. This makes it necessary to develop an intelligent anomaly detection method for a steel grid structure that is safe, convenient, and accurate. Therefore, the intelligent detection of the steel mesh frame structure abnormity through the deep learning neural network is provided in the prior art, but the steel mesh frame structure is complex and is often irregular, and the accuracy of the deep learning neural network in the prior art on the steel mesh frame structure abnormity detection is low.
Disclosure of Invention
In view of the above, it is necessary to provide a method and an apparatus for detecting an abnormality of a steel grid structure, so as to solve the technical problem in the prior art that the accuracy of detecting an abnormality of a steel grid structure is low.
In one aspect, the invention provides a steel truss structure anomaly detection method, which comprises the following steps:
acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of the steel truss to be detected;
performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map;
and determining a steel grid structure abnormity detection result based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
In some possible implementations, the attention directing module includes a first tensor stitching submodule, a spatial attention submodule, a channel attention submodule, a first tensor summing submodule, a first codec, a first dot product operation submodule, a second tensor stitching submodule, a second tensor summing submodule, a second codec, and a third dot product operation submodule;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
the spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel features in the first spliced feature map to obtain a first channel attention feature map;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coder-decoder is used for coding and decoding the plurality of feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion characteristic diagram to obtain a second spliced characteristic diagram;
the spatial attention submodule is further used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention sub-module is further used for extracting channel features in the second spliced feature map to obtain a second channel attention feature map;
the second tensor summation submodule is used for carrying out tensor summation on the second spatial attention feature map and the second channel attention feature map to obtain a second attention feature map;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
and the third dot product operation submodule is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result.
In some possible implementations, the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total guide loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs;
is the guidance loss in the nth characteristic diagram;
a reconstruction constraint penalty for the nth signature;
is a first encoding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; | | non-woven hair 2 Is a two-norm.
In some possible implementations, the spatial attention submodule includes a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer, and a spatial attention tensor concatenation layer;
the first spatial attention convolutional layer, the second spatial attention convolutional layer and the third spatial attention convolutional layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
the first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively performing spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map to obtain a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map correspondingly;
the first spatial attention dot product operation layer is used for performing dot product operation on the first spatial remodeling graph and the second spatial remodeling graph to obtain spatial correlation of each pixel position in the first spatial remodeling graph and the second spatial remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on the spatial correlation to obtain a spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product diagram to obtain a fourth spatial remodeling diagram;
the space attention tensor splicing layer is used for carrying out tensor splicing on the fourth space remodeling image and the first splicing characteristic image to obtain the first space attention characteristic image.
In some possible implementations, the channel attention submodule includes a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer, and a channel attention tensor splicing layer;
the first channel attention convolutional layer, the second channel attention convolutional layer and the third channel attention convolutional layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and correspondingly obtaining a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature diagram, the second channel sub-feature diagram and the third channel sub-feature diagram to correspondingly obtain a first channel remodeling diagram, a second channel remodeling diagram and a third channel remodeling diagram;
the first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
the fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product diagram to obtain a fourth channel remodeling diagram;
the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling image and the first splicing characteristic image to obtain the first channel attention characteristic image.
In some possible implementation manners, the obtaining a well-trained target steel mesh structure abnormality detection model includes:
constructing an initial steel truss structure abnormity detection model;
constructing a steel mesh frame structure sample set;
and training the initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model with complete training.
In some possible implementations, the total loss function is:
in the formula (I), the compound is shown in the specification,
is a total loss function;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is obtained;
the occurrence frequency of the c-type pixel points in the steel grid structure sample set is set;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focusing coefficient.
In some possible implementations, the constructing a steel lattice structure sample set includes:
acquiring a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
performing first type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set;
and/or the presence of a gas in the gas,
and carrying out second type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set.
In some possible implementations, the performing a second type of augmentation process on the initial set of steel grid structure image samples to obtain the set of steel grid structure samples includes:
randomly selecting a first initial steel grid structure image and a second initial steel grid structure image from the initial steel grid structure image sample set;
determining a first label image and a second label image of the first initial steel lattice structure image and the second initial steel lattice structure image, respectively;
randomly zooming the first initial steel grid structure image and the first label image to obtain a first zoomed steel grid structure image and a first random label image;
randomly determining a target type, and obtaining a binary mask file according to the target type and the first random label image;
mixing the first zooming steel grid structure image, the second initial steel grid structure image, the first random label image and the second label image according to the binaryzation mask file to obtain an augmented steel grid structure image and an augmented label image;
and obtaining the steel grid structure sample set according to the initial steel grid structure image sample set, the augmented steel grid structure image and the augmented label image.
On the other hand, the invention also provides a steel framework structure abnormity detection device, which comprises:
the target detection model acquisition unit is used for acquiring a target steel grid structure abnormity detection model which is trained completely, and the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
the to-be-detected image acquisition unit is used for acquiring a to-be-detected steel truss structure image;
the multi-scale feature fusion unit is used for performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and the abnormity detection unit is used for determining the abnormity detection result of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
The beneficial effects of adopting the above embodiment are: according to the steel grid structure abnormity detection method provided by the invention, the target steel grid structure abnormity detection model comprises the multi-scale fusion module and the attention guiding module, and the attention guiding module is used for improving the feature extraction and learning capacity of the complex irregular grid structure, so that the accuracy of the steel grid structure abnormity detection result can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of an embodiment of a steel framework structure anomaly detection method provided by the present invention;
FIG. 2 is a schematic structural diagram of an embodiment of a multi-scale fusion module provided in the present invention;
FIG. 3 is a schematic structural diagram of an attention module according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an embodiment of a spatial attention submodule provided in the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of a channel attention submodule provided in the present invention;
FIG. 6 is a schematic flow chart of one embodiment of S101 of FIG. 1;
FIG. 7 is a flowchart illustrating an embodiment of S602 of FIG. 6 according to the present invention;
FIG. 8 is a flowchart illustrating an embodiment of S703 of FIG. 7 according to the present invention;
FIG. 9 is a schematic structural view of an embodiment of the steel grid structure abnormality detection apparatus provided in the present invention;
fig. 10 is a schematic structural diagram of an embodiment of an electronic device provided in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It should be apparent that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
It should be understood that the schematic drawings are not necessarily to scale. The flowcharts used in this disclosure illustrate operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flow diagrams may be performed out of order, and that steps without logical context may be performed in reverse order or concurrently. One skilled in the art, under the direction of this summary, may add one or more other operations to, or remove one or more operations from, the flowchart.
In the description of the embodiment of the present invention, "and/or" describes an association relationship of an association object, which means that three relationships may exist, for example: a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor systems and/or microcontroller systems.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The embodiment of the invention provides a method and a device for detecting structural abnormality of a steel truss, which are respectively explained below.
Fig. 1 is a schematic flow diagram of an embodiment of a steel grid structure abnormality detection method provided by the present invention, and as shown in fig. 1, the steel grid structure abnormality detection method includes:
s101, acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
s102, acquiring a structural image of the steel truss to be detected;
s103, performing multi-scale feature extraction on the steel space truss structure image to be detected based on a multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and S104, determining the steel grid structure abnormity detection result based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
Compared with the prior art, the steel grid structure abnormity detection method provided by the embodiment of the invention has the advantages that the target steel grid structure abnormity detection model comprises the multi-scale fusion module and the attention guiding module, and the attention guiding module is used for improving the feature extraction and learning capacity of the complex irregular grid structure, so that the accuracy of the steel grid structure abnormity detection result can be improved.
In a specific embodiment of the present invention, the multi-scale fusion module is a network structure left after the ResNet50 goes out of the connection layer, specifically: as shown in fig. 2, the multi-scale fusion module includes 4 residual error units, 4 upsampling layers corresponding to the 4 residual error units one to one, a multi-scale tensor splicing layer, and a multi-scale convolution layer, each residual error unit is used for performing feature extraction of one scale, extracting features of 4 scales, and obtaining four feature maps, which are respectively F after passing through the upsampling layers 1 ’、F 2 ’、F 3 ' and F 4 ’。F 1 ’、F 2 ’、F 3 ' and F 4 After passing through a multi-scale tensor splicing layer and a multi-scale convolution layer, a multi-scale fusion eigen map F is generated MS 。
In some embodiments of the present invention, as shown in fig. 3, the attentiveness directing module includes a first tensor concatenation sub-module, a spatial attention sub-module, a channel attention sub-module, a first tensor summation sub-module, a first codec, a first dot product operation sub-module, a second tensor concatenation sub-module, a second tensor summation sub-module, a second codec, and a third dot product operation sub-module;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
the spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel characteristics in the first spliced characteristic diagram to obtain a first channel attention characteristic diagram;
the first tensor summation submodule is used for carrying out tensor summation on the first space attention characteristic diagram and the first channel attention characteristic diagram to obtain a first attention characteristic diagram;
the first coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion characteristic graph to obtain a second spliced characteristic graph;
the spatial attention submodule is also used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention sub-module is also used for extracting channel characteristics in the second spliced characteristic diagram to obtain a second channel attention characteristic diagram;
the second tensor summation submodule is used for carrying out tensor summation on the second space attention characteristic diagram and the second channel attention characteristic diagram to obtain a second attention characteristic diagram;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
and the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result.
It should be noted that: the structures of the first codec and the second codec in the embodiments of the present invention are UNet network structures, which are not described in detail herein.
In a specific embodiment of the present invention, the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total boot loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs;
is the guidance loss in the nth characteristic diagram;
a reconstruction constraint penalty for the nth signature;
is a first coding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; | | non-woven hair 2 Is a two-norm.
According to the embodiment of the invention, the results of the attention loss function of the attention guiding module for constraining the space attention and the channel attention are consistent as much as possible before and after passing through different encoders and decoders, so that the accuracy of detecting the abnormal steel truss structure can be further improved.
In some embodiments of the present invention, as shown in fig. 4, the spatial attention submodule includes a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation (softmax) layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer, and a spatial attention tensor concatenation layer;
the first spatial attention convolution layer, the second spatial attention convolution layer and the third spatial attention convolution layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
the first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively performing spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map to correspondingly obtain a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map;
the first space attention dot product operation layer is used for performing dot product operation on the first space remodeling graph and the second space remodeling graph to obtain the space correlation of each pixel position in the first space remodeling graph and the second space remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on the spatial correlation to obtain a spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product diagram to obtain a fourth spatial remodeling diagram;
and the spatial attention tensor splicing layer is used for carrying out tensor splicing on the fourth spatial remodeling image and the first splicing characteristic image to obtain a first spatial attention characteristic image.
In a specific embodiment of the present invention, the spatial correlation of each pixel position in the first spatial remodeling map and the second spatial remodeling map is:
in the formula (I), the compound is shown in the specification,
a spatial correlation for each pixel position in the first and second spatial remodelling maps;
the ith pixel position in the first space reshaping map is obtained;
the jth pixel position in the second spatial remodeling map is obtained; w, H are the width and height of the first and second spatial remodelling maps, respectively.
First spatial attention feature map
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
is a spatial attention weight;
a third spatial remodeling map;
a spatial correlation matrix for the first and second spatial reshaping maps;
is a first stitched feature map.
It should be understood that: the spatial attention module is further configured to obtain a second spatial attention feature map according to the second stitching feature map, and a process of the spatial attention module is the same as the process of obtaining the first spatial attention feature map according to the first stitching feature map, which is not described in detail herein.
In some embodiments of the present invention, as shown in fig. 5, the channel attention submodule includes a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation (softmax) layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer, and a channel attention tensor splicing layer;
the first channel attention convolutional layer, the second channel attention convolutional layer and the third channel attention convolutional layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and correspondingly obtaining a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature diagram, the second channel sub-feature diagram and the third channel sub-feature diagram to correspondingly obtain a first channel remodeling diagram, a second channel remodeling diagram and a third channel remodeling diagram;
the first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
the fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product diagram to obtain a fourth channel remodeling diagram;
and the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling graph and the first splicing characteristic graph to obtain the first channel attention characteristic graph.
In a specific embodiment of the present invention, the channel correlation for each pixel position in the first spatial remodeling map and the second spatial remodeling map is:
in the formula (I), the compound is shown in the specification,
correlating the channels in the first channel remodeling map and the second channel remodeling map;
remodeling the ith channel in the map for the first channel;
remodeling a jth channel in the graph for the second channel; and c is the total number of channels in the first channel remodeling map and the second channel remodeling map.
First channel attention feature map
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
is the channel attention weight;
remodeling the map for the third channel;
a channel correlation matrix for the first channel remodeling map and the second channel remodeling map;
is a first stitching signature.
It should be understood that: the channel attention module is further configured to obtain a second channel attention feature map according to the second stitching feature map, and a process of the channel attention module is the same as the process of obtaining the first channel attention feature map according to the first stitching feature map, which is not described in detail herein.
In some embodiments of the present invention, as shown in fig. 6, step S101 includes:
s601, constructing an initial steel truss structure abnormity detection model;
s602, constructing a steel framework structure sample set;
s603, training an initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model which is well trained.
Because in the actual scene, relative to background and normal steel lattice structure, the condition that unusual steel lattice structure appears is rare, namely: in order to improve the detection accuracy of the unbalanced samples, in some embodiments of the present invention, the total loss function is:
in the formula (I), the compound is shown in the specification,
as a function of total loss;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is set;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight;
is a weighting coefficient;
is the focus factor.
Initial weight
Is determined by a weighting coefficient
The control is carried out by controlling the temperature of the air conditioner,
the larger the initial weight of the class with the smaller frequency of occurrence is. The dynamic correction quantity of the weight is controlled by the focusing coefficient, and the larger the dynamic correction quantity of the weight is, the stronger the function of the model for relieving the category unbalance phenomenon by adjusting the weight is. Therefore, the class training loss with less sample size and the sample training loss with larger sample size reach a balanced state, and the problem caused by unbalanced class of the training set is solved. Therefore, the detection accuracy of the steel truss structure abnormity detection can be improved.
Further, in order to improve the training speed of the initial steel grid structure abnormality detection model, find the optimal solution to obtain the target steel grid structure abnormality detection model which is completely trained, in some embodiments of the present invention, the learning rate dynamic adjustment strategy in the training process is a Poly strategy, specifically, the Poly strategy is:
in the formula (I), the compound is shown in the specification,
is a new learning rate;
a reference learning rate;
the current iteration number is;
is the maximum iteration number;
to control the form factor of the shape of the curve.
According to the embodiment of the invention, the learning rate dynamic adjustment strategy in the training process is set to the Poly strategy, so that the training speed of the initial steel mesh frame structure abnormity detection model can be increased.
In some embodiments of the present invention, as shown in fig. 7, step S602 includes:
s701, obtaining a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
s702, performing first-class augmentation treatment on the initial steel mesh frame structure image sample set to obtain a steel mesh frame structure sample set;
and/or the presence of a gas in the atmosphere,
and S703, performing second type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set.
According to the embodiment of the invention, the number of the formed images of the steel grid structure sample set can be increased by performing the first type of augmentation treatment and/or the second type of augmentation treatment on the initial steel grid structure image sample set, so that the generalization capability of the formed target steel grid structure abnormality detection model can be improved.
In an embodiment of the present invention, the slicing processing in step S701 specifically includes: the steel grid structure image was sliced, each slice having a size of 512 × 3. The labeling in step S701 is specifically: and digitally expressing the object types in the slices, wherein the background type, the normal steel net rack type and the abnormal steel net rack type are respectively represented as 0,1,2.
In an embodiment of the present invention, the first type of augmentation process in step S702 is: and (4) turning, rotating, zooming, cutting, adjusting the color and the like on each slice in the initial steel mesh frame structure image sample set.
In some embodiments of the present invention, as shown in fig. 8, step S703 includes:
s801, randomly selecting a first initial steel grid structure image from the initial steel grid structure image sample set
And a second initial steel lattice structure image
；
S802, respectively determining a first initial steel grid structure image I A And a second initial steel lattice structure image I B First label image of
And a second label image
；
S803, forming the first initial steel mesh frame structure image
And a first label image
Carrying out random zooming to obtain a first zoomed steel grid structure image
And a first random label image
;
S804, randomly determining a target class c, and according to the target class c and the first random label image
Obtaining a binary mask file
；
S805, according to the binary mask file
The first zooming steel mesh frame structure image
Second initial steel grid structure image
A first random label image
And a second label image
Mixing to obtain an augmented steel grid structure image X and an augmented label image Y;
s806, obtaining a steel framework structure sample set according to the initial steel framework structure image sample set, the augmented steel framework structure image X and the augmented label image Y.
Specifically, a binary mask file
Comprises the following steps:
the image X of the augmented steel grid structure and the image Y of the augmented label are respectively as follows:
in the formula (I), the compound is shown in the specification,
is a Hadamard product operation.
According to the invention, the augmented steel mesh structure image and the augmented label image are obtained by the category scaling and mixing method, so that the sample number of the steel mesh structure sample set can be further increased, and the reliability of the target steel mesh structure abnormity detection model is ensured.
In order to better implement the steel truss structure abnormality detection method in the embodiment of the present invention, on the basis of the steel truss structure abnormality detection method, correspondingly, the embodiment of the present invention further provides a steel truss structure abnormality detection apparatus, as shown in fig. 9, the steel truss structure abnormality detection apparatus 900 includes:
a target detection model obtaining unit 901, configured to obtain a target steel grid structure abnormality detection model which is completely trained, where the target steel grid structure abnormality detection model includes a multi-scale fusion module and a guidance attention module;
the to-be-detected image acquisition unit 902 is used for acquiring a to-be-detected steel truss structure image;
a multi-scale feature fusion unit 903, configured to perform multi-scale feature extraction on the steel space truss structure image to be detected based on the multi-scale fusion module to obtain multiple feature maps, and fuse the multiple feature maps to obtain a multi-scale fusion feature map;
and an anomaly detection unit 904, configured to determine a steel grid structure anomaly detection result based on the guidance attention module and the plurality of feature maps and the multi-scale fusion feature map.
The steel truss structure abnormality detection apparatus 900 provided in the above-mentioned embodiment can implement the technical solutions described in the above-mentioned steel truss structure abnormality detection method embodiments, and the specific implementation principles of the above-mentioned modules or units can refer to the corresponding contents in the above-mentioned steel truss structure abnormality detection method embodiments, and are not described here again.
As shown in fig. 10, the present invention further provides an electronic device 1000 accordingly. The electronic device 1000 includes a processor 1001, a memory 1002, and a display 1003. Fig. 10 shows only some of the components of the electronic device 1000, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The processor 1001 may be a Central Processing Unit (CPU), a microprocessor or other data Processing chip in some embodiments, and is used for running program codes stored in the memory 1002 or Processing data, for example, the steel grid structure abnormality detection method in the present invention.
In some embodiments, processor 1001 may be a single server or a group of servers. The server groups may be centralized or distributed. In some embodiments, the processor 1001 may be local or remote. In some embodiments, the processor 1001 may be implemented in a cloud platform. In an embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an intra-site, a multi-cloud, and the like, or any combination thereof.
The storage 1002 may be an internal storage unit of the electronic device 1000 in some embodiments, such as a hard disk or a memory of the electronic device 1000. The memory 1002 may also be an external storage device of the electronic device 1000 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device 1000.
Further, the memory 1002 may also include both internal storage units and external storage devices of the electronic device 1000. The memory 1002 is used for storing application software and various data for installing the electronic device 1000.
The display 1003 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like in some embodiments. The display 1003 is used to display information at the electronic device 1000 and to display a visual user interface. The components 1001-1003 of the electronic device 1000 communicate with each other via a system bus.
In one embodiment, when the processor 1001 executes the steel grid structure abnormality detection program in the memory 1002, the following steps may be implemented:
acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of the steel truss to be detected;
performing multi-scale feature extraction on the steel space truss structure image to be detected based on a multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and determining the steel grid structure abnormity detection result based on the guide attention module and the plurality of feature maps and the multi-scale fusion feature map.
It should be understood that: when the processor 1001 executes the steel grid structure abnormality detection program in the memory 1002, other functions may be implemented in addition to the above functions, which may be specifically referred to the description of the corresponding method embodiment above.
Further, the type of the mentioned electronic device 1000 is not specifically limited in the embodiment of the present invention, and the electronic device 1000 may be a portable electronic device such as a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a wearable device, and a laptop computer (laptop). Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices that carry an IOS, android, microsoft, or other operating system. The portable electronic device may also be other portable electronic devices such as laptop computers (laptop) with touch sensitive surfaces (e.g., touch panels), etc. It should also be understood that in other embodiments of the present invention, the electronic device 1000 may not be a portable electronic device, but may be a desktop computer having a touch-sensitive surface (e.g., a touch panel).
Accordingly, the present application also provides a computer-readable storage medium, which is used for storing a computer-readable program or instruction, and when the program or instruction is executed by a processor, the steps or functions in the steel grid structure abnormality detection method provided by the above-mentioned method embodiments can be implemented.
Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by instructing relevant hardware (such as a processor, a controller, etc.) by a computer program, and the computer program may be stored in a computer readable storage medium. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The steel truss structure abnormality detection method and device provided by the invention are described in detail above, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the invention; meanwhile, for those skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (6)
1. A steel truss structure abnormality detection method is characterized by comprising the following steps:
acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of the steel truss to be detected;
performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map;
determining a steel framework structure abnormity detection result based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map;
the attention guiding module comprises a first tensor splicing submodule, a space attention submodule, a channel attention submodule, a first tensor adding submodule, a first coder-decoder, a first dot product operation submodule, a second tensor splicing submodule, a second tensor adding submodule, a second coder-decoder and a third dot product operation submodule;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
the spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel features in the first spliced feature map to obtain a first channel attention feature map;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion characteristic diagram to obtain a second spliced characteristic diagram;
the spatial attention submodule is further used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention sub-module is further used for extracting channel features in the second spliced feature map to obtain a second channel attention feature map;
the second tensor summation submodule is used for carrying out tensor summation on the second spatial attention feature map and the second channel attention feature map to obtain a second attention feature map;
the second coder-decoder is used for coding and decoding the plurality of feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result;
the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total boot loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs;
is as followsGuidance loss in the n feature maps;
a reconstruction constraint penalty for the nth signature;
is a first coding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; i O 2 Is a two-norm;
the method for acquiring the target steel mesh structure abnormity detection model with complete training comprises the following steps:
constructing an initial steel truss structure abnormity detection model;
constructing a steel mesh frame structure sample set;
training the initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model which is completely trained;
the total loss function is:
in the formula (I), the compound is shown in the specification,
as a function of total loss;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focus factor.
2. The method for detecting the anomaly in the steel truss structure according to the claim 1, wherein the spatial attention submodule comprises a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer and a spatial attention tensor splicing layer;
the first spatial attention convolution layer, the second spatial attention convolution layer and the third spatial attention convolution layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
the first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively performing spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map to obtain a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map correspondingly;
the first spatial attention dot product operation layer is used for performing dot product operation on the first spatial remodeling graph and the second spatial remodeling graph to obtain spatial correlation of each pixel position in the first spatial remodeling graph and the second spatial remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on the spatial correlation to obtain a spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product diagram to obtain a fourth spatial remodeling diagram;
the space attention tensor splicing layer is used for carrying out tensor splicing on the fourth space remodeling image and the first splicing characteristic image to obtain the first space attention characteristic image.
3. The method for detecting the abnormality of the steel truss structure according to claim 1, wherein the channel attention submodule comprises a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer and a channel attention tensor splicing layer;
the first channel attention convolutional layer, the second channel attention convolutional layer and the third channel attention convolutional layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and correspondingly obtaining a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature diagram, the second channel sub-feature diagram and the third channel sub-feature diagram to correspondingly obtain a first channel remodeling diagram, a second channel remodeling diagram and a third channel remodeling diagram;
the first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
the fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product map to obtain a fourth channel remodeling map;
the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling image and the first splicing characteristic image to obtain the first channel attention characteristic image.
4. The method for detecting the abnormality of the steel lattice structure according to claim 1, wherein the constructing a steel lattice structure sample set includes:
acquiring a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
performing first type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set;
and/or the presence of a gas in the gas,
and carrying out second type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set.
5. The method for detecting the abnormality of the steel lattice structure according to claim 4, wherein the performing of the second kind of augmentation process on the initial steel lattice structure image sample set to obtain the steel lattice structure sample set includes:
randomly selecting a first initial steel grid structure image and a second initial steel grid structure image from the initial steel grid structure image sample set;
determining a first label image and a second label image of the first initial steel lattice structure image and the second initial steel lattice structure image, respectively;
randomly zooming the first initial steel grid structure image and the first label image to obtain a first zoomed steel grid structure image and a first random label image;
randomly determining a target type, and obtaining a binary mask file according to the target type and the first random label image;
mixing the first scaled steel grid structure image, the second initial steel grid structure image, the first random label image and the second label image according to the binaryzation mask file to obtain an augmented steel grid structure image and an augmented label image;
and obtaining the steel grid structure sample set according to the initial steel grid structure image sample set, the augmented steel grid structure image and the augmented label image.
6. A steel truss structure abnormality detection apparatus, comprising:
the target detection model acquisition unit is used for acquiring a target steel grid structure abnormity detection model which is trained completely, and the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
the to-be-detected image acquisition unit is used for acquiring a to-be-detected steel truss structure image;
the multi-scale feature fusion unit is used for performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map;
an anomaly detection unit, configured to determine a steel framework structure anomaly detection result based on the attention-directing module, the plurality of feature maps and the multi-scale fusion feature map;
the attention guiding module comprises a first tensor splicing submodule, a space attention submodule, a channel attention submodule, a first tensor adding submodule, a first codec, a first dot product operation submodule, a second tensor splicing submodule, a second tensor adding submodule, a second codec and a third dot product operation submodule;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
the spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel features in the first spliced feature map to obtain a first channel attention feature map;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion feature map to obtain a second spliced feature map;
the spatial attention submodule is further used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention sub-module is further used for extracting channel features in the second spliced feature map to obtain a second channel attention feature map;
the second tensor summation submodule is used for carrying out tensor summation on the second spatial attention feature map and the second channel attention feature map to obtain a second attention feature map;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result;
the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total guide loss;
constraint losses for the total weight; n is the total number of the plurality of characteristic graphs;
is the guidance loss in the nth characteristic diagram;
a reconstruction constraint penalty for the nth signature;
is a first encoding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; i O 2 Is a two-norm;
the method for acquiring the target steel mesh structure abnormity detection model with complete training comprises the following steps:
constructing an initial steel truss structure abnormity detection model;
constructing a steel mesh frame structure sample set;
training the initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model which is completely trained;
the total loss function is:
in the formula (I), the compound is shown in the specification,
as a function of total loss;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focus factor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662669.1A CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662669.1A CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114758206A CN114758206A (en) | 2022-07-15 |
| CN114758206B true CN114758206B (en) | 2022-10-28 |
Family
ID=82336676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210662669.1A Active CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114758206B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861306B (en) * | 2023-02-20 | 2023-05-05 | 武汉纺织大学 | Industrial product abnormality detection method based on self-supervision jigsaw module |
| CN117409327A (en) * | 2023-12-13 | 2024-01-16 | 中国地质调查局武汉地质调查中心(中南地质科技创新中心) | Dynamic quantitative remote sensing monitoring method for surface mine |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN110209713A (en) * | 2019-06-26 | 2019-09-06 | 国家电网有限公司 | Abnormal grid structure recognition methods and device |
| CN112784685A (en) * | 2020-12-28 | 2021-05-11 | 山东师范大学 | Crowd counting method and system based on multi-scale guiding attention mechanism network |
| CN113435269A (en) * | 2021-06-10 | 2021-09-24 | 华东师范大学 | Improved water surface floating object detection and identification method and system based on YOLOv3 |
| CN114332007A (en) * | 2021-12-28 | 2022-04-12 | 福州大学 | Transformer-based industrial defect detection and identification method |
| WO2022090483A1 (en) * | 2020-11-02 | 2022-05-05 | Tomtom Global Content B.V. | Neural network models for semantic image segmentation |
| CN114529839A (en) * | 2022-02-09 | 2022-05-24 | 国网河南省电力公司电力科学研究院 | Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system |
-
2022
- 2022-06-13 CN CN202210662669.1A patent/CN114758206B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN110209713A (en) * | 2019-06-26 | 2019-09-06 | 国家电网有限公司 | Abnormal grid structure recognition methods and device |
| WO2022090483A1 (en) * | 2020-11-02 | 2022-05-05 | Tomtom Global Content B.V. | Neural network models for semantic image segmentation |
| CN112784685A (en) * | 2020-12-28 | 2021-05-11 | 山东师范大学 | Crowd counting method and system based on multi-scale guiding attention mechanism network |
| CN113435269A (en) * | 2021-06-10 | 2021-09-24 | 华东师范大学 | Improved water surface floating object detection and identification method and system based on YOLOv3 |
| CN114332007A (en) * | 2021-12-28 | 2022-04-12 | 福州大学 | Transformer-based industrial defect detection and identification method |
| CN114529839A (en) * | 2022-02-09 | 2022-05-24 | 国网河南省电力公司电力科学研究院 | Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system |
Non-Patent Citations (3)
| Title |
|---|
| Multi-Scale Self-Guided Attention for Medical Image Segmentation;Ashish Sinha 等;《IEEE Journal of Biomedical and Health Informatics》;20200414;第121-130页 * |
| 结合注意力机制的乳腺双模态超声分类网络;赵绪 等;《中国图象图形学报》;20220331;全文 * |
| 结合深度学习去噪和超分辨的SAR检测识别;王俊等;《空天防御》;20200915(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114758206A (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114758206B (en) | Steel truss structure abnormity detection method and device | |
| Zhang et al. | LAANet: lightweight attention-guided asymmetric network for real-time semantic segmentation | |
| CN107481309B (en) | Three-dimensional reflection rendering method and device | |
| CN112132032A (en) | Traffic sign detection method and device, electronic equipment and storage medium | |
| Zhu et al. | HMFCA-Net: Hierarchical multi-frequency based Channel attention net for mobile phone surface defect detection | |
| CN112070040A (en) | Text line detection method for video subtitles | |
| CN113901909A (en) | Video-based target detection method and device, electronic equipment and storage medium | |
| Guo et al. | Adaln: a vision transformer for multidomain learning and predisaster building information extraction from images | |
| Hu et al. | Automating building damage reconnaissance to optimize drone mission planning for disaster response | |
| WO2022174517A1 (en) | Crowd counting method and apparatus, computer device and storage medium | |
| Shi et al. | RAOD: refined oriented detector with augmented feature in remote sensing images object detection | |
| CN110490165B (en) | Dynamic gesture tracking method based on convolutional neural network | |
| Gao et al. | TSVT: Token Sparsification Vision Transformer for robust RGB-D salient object detection | |
| WO2023006011A1 (en) | Wallpaper generation method and apparatus and electronic device | |
| CN116597260A (en) | Image processing method, electronic device, storage medium, and computer program product | |
| CN112132037B (en) | Pavement detection method, device, equipment and medium based on artificial intelligence | |
| CN115937993A (en) | Living body detection model training method, living body detection device and electronic equipment | |
| CN115578358A (en) | RecTransformer-based surface defect detection model construction method | |
| Wang et al. | Pyramid self-attention mechanism-based change detection in hyperspectral imagery | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| Wang et al. | CAW: A Remote‐Sensing Scene Classification Network Aided by Local Window Attention | |
| CN112749707A (en) | Method, apparatus, and medium for object segmentation using neural networks | |
| Zhao et al. | OASL: Orientation-aware adaptive sampling learning for arbitrary oriented object detection | |
| Bi et al. | Refined regression detector for multiclass-oriented target in optical remote sensing images | |
| Feng et al. | Real-time object detection method based on YOLOv5 and efficient mobile network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Mingpeng Inventor after: Gao Jian Inventor before: Li Mingpeng Inventor before: Gao Jian Inventor before: Hu Xudong |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |