CN117274789B - Underwater crack semantic segmentation method for hydraulic concrete structure - Google Patents
Underwater crack semantic segmentation method for hydraulic concrete structure Download PDFInfo
- Publication number
- CN117274789B CN117274789B CN202311549825.4A CN202311549825A CN117274789B CN 117274789 B CN117274789 B CN 117274789B CN 202311549825 A CN202311549825 A CN 202311549825A CN 117274789 B CN117274789 B CN 117274789B
- Authority
- CN
- China
- Prior art keywords
- underwater
- concrete structure
- underwater crack
- crack
- hydraulic concrete
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 16
- 238000003384 imaging method Methods 0.000 claims abstract description 7
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 6
- 238000012360 testing method Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 11
- 101100295091 Arabidopsis thaliana NUDT14 gene Proteins 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 6
- 239000002131 composite material Substances 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 2
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 6
- 238000013139 quantization Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 14
- 230000000694 effects Effects 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000009991 scouring Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/05—Underwater scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/16—Image acquisition using multiple overlapping images; Image stitching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a semantic segmentation method of underwater cracks of a hydraulic concrete structure, which comprises the steps of extracting underwater crack images from videos shot by an underwater robot on a reservoir engineering site frame by frame, processing the underwater crack images and constructing an underwater crack data set; constructing a convolutional neural network model and performing iterative training, and adjusting model parameters by calculating a loss function to obtain a hydraulic concrete structure underwater crack semantic segmentation model; performing binarization processing on the segmented image, and performing pixel-level quantization processing on a predicted result after the binarization processing; and converting a pixel-level quantized result of the underwater crack of the hydraulic concrete structure into an actual physical quantized result by combining a camera imaging principle and parameters of a camera to obtain a segmentation result of the underwater crack of the hydraulic concrete structure. The invention realizes quantitative identification of the underwater crack of the hydraulic concrete structure and improves the accuracy of the underwater crack detection of the hydraulic concrete structure.
Description
Technical Field
The invention belongs to the technical field of underwater crack identification of hydraulic concrete structures, and particularly relates to a semantic segmentation method for underwater cracks of hydraulic concrete structures.
Background
The hydraulic concrete structure is an important component of hydraulic and hydroelectric engineering, is commonly used for building hydraulic buildings such as dams, spillways, water delivery tunnels, ship lock gates and the like, plays a remarkable role in flood control, power generation, shipping, water resource allocation and the like, and is a hot spot of current research how to carry out scientific and effective safety diagnosis. However, the hydraulic concrete structure has complex working conditions, is subjected to the influence of water flow scouring, internal stress state and chemical reaction for a long time in the operation process, and has poor construction quality or perfect foundation treatment, so that cracks are inevitably generated, and the safe operation of the hydraulic concrete structure is seriously influenced. Compared with the underwater crack, the underwater crack is more difficult to find under the influence of the underwater complex environment, so that the images shot by the underwater camera are fewer, and the problems of low contrast, blurred imaging, serious noise interference and the like exist.
The existing method for detecting the underwater cracks of the hydraulic concrete structure mainly comprises manual inspection, and the operation is very difficult to collect and record the underwater cracks and has weak operability. The method based on deep learning can effectively reduce manual operation links and improve the automatic process of underwater crack detection.
The current deep learning method has a lot of recognition on the water cracks and shows excellent performance, but the following defects exist in the aspect of recognition on the underwater cracks of the hydraulic concrete structure: (1) The excellent performance of the existing deep learning method is built on a large number of data sets, and the number of underwater crack data sets which are disclosed and meet the requirements is small, so that the detection precision of the method for identifying the underwater crack is low, and the actual identification requirement is not met; (2) There is a lack of adequate resolution of the problem of class imbalance in underwater fracture dataset.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a semantic segmentation method for underwater cracks of a hydraulic concrete structure. The invention realizes quantitative identification of the underwater crack of the hydraulic concrete structure and improves the accuracy of the underwater crack detection of the hydraulic concrete structure.
In order to solve the technical problems, the invention is realized by the following technical scheme:
a semantic segmentation method for underwater cracks of a hydraulic concrete structure comprises the following steps:
s1: collecting data on a reservoir engineering site through an underwater robot, preprocessing a video shot by the underwater robot, and extracting clear underwater crack images which can be resolved by human eyes from the underwater crack images frame by frame;
s2: expanding the underwater crack data volume of the preprocessed image by using an image expansion technology to obtain an underwater crack data set;
s3: carrying out pixel-level labeling on the underwater crack data set by using pixel-level labeling software, and dividing the underwater crack data set into a training set, a verification set and a test set according to a proportion;
s4: constructing a convolutional neural network model, performing iterative training on a training set and a verification set, and adjusting parameters of the segmentation model by calculating a loss function between a segmentation result and a real label to finally obtain a hydraulic concrete structure underwater crack semantic segmentation model with the model accuracy reaching a preset value;
s5: four typical working conditions are selected in the test set to test the underwater crack semantic segmentation model of the hydraulic concrete structure, and the generalization capability of the model is checked;
s6: performing binarization processing on the segmented image, and calculating a predicted result after the binarization processing by using a single-pixel-point actual-size reconstruction model and a hydraulic concrete structure underwater crack characteristic analysis method to obtain a pixel-level quantized result;
s7: and converting the pixel-level quantized result of the underwater crack into an actual physical quantized result by combining the camera imaging principle and the parameters of the camera to obtain the underwater crack segmentation result of the hydraulic concrete structure.
Preferably, in step S2, the image expansion technique includes four modes of local magnification, rotation by 90 °, stretching by 30 ° and left-right mirroring.
Preferably, the ratio of the training set, the validation set and the test set in step S3 is 8:1:1.
Preferably, the specific process of step S4 is:
s4-1, improving a deep Labv3+ segmentation model, changing an original main trunk network Xreception into a MobileNet V2, reducing the subsampling multiple of deep features to 8, and constructing a semantic segmentation model based on the MobileNet V2-deep Labv3+;
s4-2, calculating to obtain a loss function L of the segmentation model on the training set and the verification set according to the following formula:
in the formula (1), L CE Represents a cross entropy loss function, L Dice Representing the Dice loss function, y i Representing the tag value, y i 'the' value of the prediction is represented,a probability representing the predicted value; the X is the number of elements in the sample X, the Y is the number of elements in the sample Y, and the X is the intersection between X and YThe number of sets;
and dividing the parameters of the model according to the obtained loss function until the accuracy of the model reaches a preset value, and obtaining the underwater crack semantic division model of the hydraulic concrete structure.
Preferably, in step S4-1, the feature extraction process of the underwater crack image based on the semantic segmentation model of MobileNet V2-deep Labv3+ is as follows:
(1) Extracting deep features and shallow features from an input image by using a backbone network MobileNet V2, simultaneously carrying out 8 times downsampling on the deep features and then transmitting the deep features into an ASPP module, and carrying out 4 times downsampling on the shallow features and then transmitting the deep features into a decoding structure;
(2) The ASPP module carries out 5 modes of parallel processing on the input deep features, namely a 1 multiplied by 1 convolution for reducing the dimension of the input image, three cavity convolutions with expansion rates of 6, 12 and 18 and a global average pooling operation; the 5 processing modes obtain 5 corresponding feature layers, the corresponding feature layers are stacked and the deep feature layers combined with the context information are obtained after the channel number is adjusted by using 1X 1 convolution;
(3) After the up-sampling operation is finished on the deep characteristic layer processed by the encoded part, the deep characteristic layer is fused and spliced with the shallow characteristic layer with the number of the channels adjusted by 1 multiplied by 1 convolution, and a composite characteristic diagram containing shallow characteristics and deep characteristics of an input picture is obtained;
(4) And restoring the size of the composite characteristic image to be consistent with the size of the original image through 3X 3 convolution operation and 4 times up sampling operation, classifying each pixel point, and outputting a prediction result.
Preferably, in step S6, the single-pixel actual size reconstruction model is a conversion ratio of a pixel size to an actual physical size, and when the underwater robot is parallel to the underwater crack of the hydraulic concrete structure, a specific calculation formula is as follows:
in the formula (2), q is the actual size of a single pixel point, Z c Distance from camera to underwater crackDx and dy are the pixel dimensions in the X and Y directions, respectively, and f is the camera focal length.
Preferably, in step S6, the hydraulic concrete structure feature analysis method includes an circumscribed rectangle method for determining the trend of the underwater crack of the hydraulic concrete structure and a center line method for calculating the length and width of the underwater crack of the hydraulic concrete structure, and specifically includes the following steps:
extracting a skeleton from the underwater crack image to obtain an edge contour line and a central line, thinning the edge contour line and the central line into single pixel points, counting the number n1 of the pixel points on the central line of the underwater crack, wherein the length of the underwater crack is L=n1×q, and q is the actual size of the single pixel points; and (3) taking a tangent line and a vertical line of a central line at any point on the central line, and counting the number n2 of pixel points between the vertical line and the intersection points of the two edge contour lines, wherein the width of the underwater crack is W=n2×q.
Compared with the prior art, the invention has the following advantages:
1. the semantic segmentation model based on the MobileNet V2-deep Labv3+ is constructed, so that the problem that the detection accuracy is too low and the actual recognition requirement is not met due to the small number of samples in the underwater crack recognition process of the hydraulic concrete structure of the conventional convolutional neural network is effectively solved, and the generalization capability and the recognition accuracy of recognition are greatly improved.
2. The combined calculation mode based on the cross entropy loss function and the Dice loss function is adopted, so that the problem of unbalanced categories of underwater crack images is effectively solved, and the segmentation result of the adjusted segmentation model on the underwater cracks is more accurate.
3. The physical size of the underwater crack of the hydraulic concrete structure is calculated rapidly through the camera imaging principle, and the damage condition of the hydraulic concrete structure is reflected more intuitively.
Drawings
FIG. 1 is a workflow diagram of the present invention;
FIG. 2 is a diagram showing a construction of an underwater crack segmentation model of a hydraulic concrete structure according to the present invention;
FIG. 3 is a comparison graph of recognition effects of the underwater crack segmentation model of the hydraulic concrete structure.
Detailed Description
The following description of the embodiments of the present invention is provided in connection with the accompanying drawings to facilitate understanding of the present invention by those skilled in the art, and it is apparent that the present invention is not limited to the scope of the embodiments. It will be apparent to those skilled in the art that various modifications are possible without inventive step, insofar as they come within the spirit and scope of the invention as defined and defined by the appended claims, all inventions that make use of the inventive concepts of this patent are within the scope of protection.
The invention provides a semantic segmentation method for underwater cracks of a hydraulic concrete structure, referring to fig. 1, comprising the following steps:
s1: collecting data on a reservoir engineering site through an underwater robot, preprocessing a video shot by the underwater robot, and extracting clear underwater crack images which can be resolved by human eyes from the underwater crack images frame by frame;
s2: expanding the underwater crack data volume of the preprocessed image by using an image expansion technology to obtain an underwater crack data set;
s3: carrying out pixel-level labeling on the underwater crack data set by using pixel-level labeling software, and dividing the underwater crack data set into a training set, a verification set and a test set according to a proportion;
s4: constructing a convolutional neural network model, performing iterative training on a training set and a verification set, and adjusting parameters of the segmentation model by calculating a loss function between a segmentation result and a real label to finally obtain a hydraulic concrete structure underwater crack semantic segmentation model with the model accuracy reaching a preset value;
s5: four typical working conditions are selected in the test set to test the underwater crack semantic segmentation model of the hydraulic concrete structure, and the generalization capability of the model is checked;
s6: performing binarization processing on the segmented image, and calculating a predicted result after the binarization processing by using a single-pixel-point actual-size reconstruction model and a hydraulic concrete structure underwater crack characteristic analysis method to obtain a pixel-level quantized result;
s7: and converting the pixel-level quantized result of the underwater crack into an actual physical quantized result by combining the camera imaging principle and the parameters of the camera to obtain the underwater crack segmentation result of the hydraulic concrete structure.
The image expansion technology in the step S2 comprises four modes of local amplification, rotation by 90 degrees, stretching by 30 degrees and left-right mirroring.
The ratio of the training set, the verification set and the test set in the step S3 is 8:1:1.
The specific process of step S4 is as follows:
s4-1, improving a deep Labv3+ segmentation model, changing an original main trunk network Xreception into a MobileNet V2, reducing the subsampling multiple of deep features to 8, and constructing a semantic segmentation model based on the MobileNet V2-deep Labv3+, wherein the specific model structure is shown in figure 2; the characteristic extraction process of the segmentation model on the underwater crack image is as follows:
firstly, extracting deep features and shallow features from an input image by using a backbone network MobileNet V2, simultaneously carrying out 8 times downsampling on the deep features, and then transmitting the deep features into an ASPP module, and carrying out 4 times downsampling on the shallow features, and then transmitting the deep features into a decoding structure;
next, the ASPP module performs 5 parallel processing on the incoming deep features, namely a 1×1 convolution for reducing the input image, three hole convolutions with expansion rates of 6, 12 and 18, and a global average pooling operation. The 5 processing modes obtain 5 corresponding feature layers, the corresponding feature layers are stacked and the deep feature layers combined with the context information are obtained after the channel number is adjusted by using 1X 1 convolution;
then, after up-sampling operation is completed on the deep feature layer processed by the encoded part, the deep feature layer is fused and spliced with the shallow feature layer with the number of the channels adjusted by 1 multiplied by 1 convolution, and a composite feature map containing shallow features and deep features of the input picture is obtained;
and finally, restoring the size of the composite characteristic image to be consistent with the size of the original image through 3X 3 convolution operation and 4 times up-sampling operation, classifying each pixel point, and outputting a prediction result.
S4-2, according to the formula:
obtaining a loss function L of the segmentation model on the training set and the verification set; wherein L is CE Represents a cross entropy loss function, L Dice Representing a Dice loss function; y is i Representing the tag value, y' i The predicted value is represented by a value of the prediction,a probability representing the predicted value; the number of elements in the sample X is the number of elements in the sample Y, the number of elements in the sample Y is the number of intersections between X and Y.
And dividing the parameters of the model according to the obtained loss function until the accuracy of the model reaches a preset value, and obtaining the underwater crack semantic division model of the hydraulic concrete structure.
Table 1 partition model accuracy contrast table
| Numbering device | Backbone network | Downsampling multiple | MPA/(％) | MIoU/(％) |
| 1 | Xception | 16 | 89.82 | 85.09 |
| 2 | MobileNetV2 | 16 | 90.51 | 85.42 |
| 3 | Xception | 8 | 90.38 | 85.23 |
| 4 | MobileNetV2 | 8 | 91.77 | 86.33 |
Note that: MPA is the average pixel accuracy, which is obtained by calculating the accuracy of each category and then taking the average value, and the larger the value is, the more uniform the model is in different categories, and the more convincing is that the model is; MIoU is the average cross ratio, which is the proportion of the overlapped part between the predicted result of each category and the real label to the total part, and the larger the average value after summation is, the better the segmentation effect of the model on each category is explained.
The accuracy comparison table of the underwater crack segmentation model of the hydraulic concrete structure is shown in table 1. Compared with MPA (89.82%) and MIoU (85.09%) of the deep Labv3+ model, the underwater crack segmentation model of the hydraulic concrete structure is improved by 1.95% and 1.24%, respectively.
The comparison graph of the recognition effects of the underwater crack of the hydraulic concrete structure under four typical working conditions is shown in fig. 3, wherein A is an original graph of the underwater crack of the hydraulic concrete structure, B is a recognition effect graph of a deep Labv3+ model, and C is a recognition effect graph of the underwater crack segmentation model of the hydraulic concrete structure.
In step S6, the single-pixel actual size reconstruction model is a conversion ratio of a pixel size to an actual physical size, and when the underwater robot is parallel to the underwater crack of the hydraulic concrete structure, a specific calculation formula is as follows:
in the formula (2), q is the actual size of a single pixel point; z is Z c Is the distance between the underwater crack and the camera; dx and dy are the pixel dimensions in the X and Y directions, respectively; f is the camera focal length.
In step S6, the characteristic analysis method of the underwater crack of the hydraulic concrete structure comprises an external rectangle method for judging the trend of the underwater crack of the hydraulic concrete structure and a center line method for calculating the length and the width of the underwater crack of the hydraulic concrete structure. The specific operation is as follows:
extracting a skeleton from the underwater crack image to obtain an edge contour line and a central line, thinning the edge contour line and the central line into single pixel points, counting the number n1 of the pixel points on the central line of the underwater crack, wherein the length of the underwater crack is L=n1×q, and q is the actual size of the single pixel points; and (3) taking a tangent line and a vertical line of a central line at any point on the central line, and counting the number n2 of pixel points between the vertical line and the intersection points of the two edge contour lines, wherein the width of the underwater crack is W=n2×q.
According to the invention, by constructing the semantic segmentation model based on the MobileNet V2-deep Labv & lt3+ & gt, the problem that the detection accuracy is too low and the actual recognition requirement is not met due to the small number of samples in the underwater crack recognition process of the hydraulic concrete structure of the conventional convolutional neural network is effectively solved, and compared with the MPA (89.82%) and the MIoU (85.09%) of the deep Labv & lt3+ & gt model, the semantic segmentation model is increased by 1.95% and 1.24%, respectively, and the generalization capability and the recognition accuracy of recognition are greatly improved. The combined calculation mode based on the cross entropy loss function and the Dice loss function is adopted, so that the problem of unbalanced categories of underwater crack images is effectively solved, and the segmentation result of the adjusted segmentation model on the underwater cracks is more accurate. The physical size of the underwater crack of the hydraulic concrete structure is calculated rapidly through the camera imaging principle, and the damage condition of the hydraulic concrete structure is reflected more intuitively.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
What is not described in detail in the present specification belongs to the prior art known to those skilled in the art.
Claims (3)
1. The underwater crack semantic segmentation method of the hydraulic concrete structure is characterized by comprising the following steps of:
s1: collecting data on a reservoir engineering site through an underwater robot, preprocessing a video shot by the underwater robot, and extracting clear underwater crack images which can be resolved by human eyes from the underwater crack images frame by frame;
s2: expanding the underwater crack data volume of the preprocessed image by using an image expansion technology to obtain an underwater crack data set;
s3: carrying out pixel-level labeling on the underwater crack data set by using pixel-level labeling software, and dividing the underwater crack data set into a training set, a verification set and a test set according to a proportion;
s4: constructing a convolutional neural network model, performing iterative training on a training set and a verification set, and adjusting parameters of the segmentation model by calculating a loss function between a segmentation result and a real label to finally obtain a hydraulic concrete structure underwater crack semantic segmentation model with the model accuracy reaching a preset value;
the specific process of step S4 is:
s4-1, improving a deep Labv3+ segmentation model, changing an original main trunk network Xreception into a MobileNet V2, reducing the subsampling multiple of deep features to 8, and constructing a semantic segmentation model based on the MobileNet V2-deep Labv3+;
s4-2, calculating to obtain a loss function L of the segmentation model on the training set and the verification set according to the following formula:
in the formula (1), L CE Represents a cross entropy loss function, L Dice Representing the Dice loss function, y i Representing the tag value, y i 'the' value of the prediction is represented,a probability representing the predicted value; the I X I is the number of elements in the sample X, the I Y I is the number of elements in the sample Y, and the I X n Y I is the number of intersections between X and Y;
dividing parameters of the model according to the obtained loss function until the accuracy of the model reaches a preset value, and obtaining a hydraulic concrete structure underwater crack semantic division model;
in the step S4-1, the characteristic extraction process of the underwater crack image based on the semantic segmentation model of the MobileNet V2-deep Labv3+ is as follows:
(1) Extracting deep features and shallow features from an input image by using a backbone network MobileNet V2, simultaneously carrying out 8 times downsampling on the deep features and then transmitting the deep features into an ASPP module, and carrying out 4 times downsampling on the shallow features and then transmitting the deep features into a decoding structure;
(2) The ASPP module carries out 5 modes of parallel processing on the input deep features, namely a 1 multiplied by 1 convolution for reducing the dimension of the input image, three cavity convolutions with expansion rates of 6, 12 and 18 and a global average pooling operation; the 5 processing modes obtain 5 corresponding feature layers, the corresponding feature layers are stacked and the deep feature layers combined with the context information are obtained after the channel number is adjusted by using 1X 1 convolution;
(3) After the up-sampling operation is finished on the deep characteristic layer processed by the encoded part, the deep characteristic layer is fused and spliced with the shallow characteristic layer with the number of the channels adjusted by 1 multiplied by 1 convolution, and a composite characteristic diagram containing shallow characteristics and deep characteristics of an input picture is obtained;
(4) Restoring the size of the composite feature map to be consistent with the size of the original image through 3X 3 convolution operation and 4 times up-sampling operation, classifying each pixel point, and outputting a prediction result;
s5: four typical working conditions are selected in the test set to test the underwater crack semantic segmentation model of the hydraulic concrete structure, and the generalization capability of the model is checked;
s6: performing binarization processing on the segmented image, and calculating a predicted result after the binarization processing by using a single-pixel-point actual-size reconstruction model and a hydraulic concrete structure underwater crack characteristic analysis method to obtain a pixel-level quantized result;
in step S6, the single-pixel actual size reconstruction model is a conversion ratio of a pixel size to an actual physical size, and when the underwater robot is parallel to the underwater crack of the hydraulic concrete structure, a specific calculation formula is as follows:
in the formula (2), q is the actual size of a single pixel point, Z c Dx and dy are pixel sizes in the X and Y directions respectively for the distance between the underwater crack and the camera, and f is the focal length of the camera;
in step S6, the hydraulic concrete structure feature analysis method includes an circumscribed rectangle method for determining the trend of the underwater crack of the hydraulic concrete structure and a center line method for calculating the length and width of the underwater crack of the hydraulic concrete structure, and the specific operations are as follows:
extracting a skeleton from the underwater crack image to obtain an edge contour line and a central line, thinning the edge contour line and the central line into single pixel points, counting the number n1 of the pixel points on the central line of the underwater crack, wherein the length of the underwater crack is L=n1×q, and q is the actual size of the single pixel points; counting the number n2 of pixel points between the vertical line and the intersection point of the two edge contour lines by taking a tangent line and the vertical line of the central line at any point on the central line, wherein the width of the underwater crack is W=n2×q;
s7: and converting the pixel-level quantized result of the underwater crack into an actual physical quantized result by combining the camera imaging principle and the parameters of the camera to obtain the underwater crack segmentation result of the hydraulic concrete structure.
2. The method for semantic segmentation of underwater cracks of a hydraulic concrete structure according to claim 1, wherein in the step S2, the image expansion technology comprises four modes of local amplification, rotation by 90 °, stretching by 30 ° and mirroring around.
3. The method for semantic segmentation of underwater cracks of a hydraulic concrete structure according to claim 1, wherein the ratio of the training set, the verification set and the test set in the step S3 is 8:1:1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311549825.4A CN117274789B (en) | 2023-11-21 | 2023-11-21 | Underwater crack semantic segmentation method for hydraulic concrete structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311549825.4A CN117274789B (en) | 2023-11-21 | 2023-11-21 | Underwater crack semantic segmentation method for hydraulic concrete structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117274789A CN117274789A (en) | 2023-12-22 |
| CN117274789B true CN117274789B (en) | 2024-04-09 |
Family
ID=89210889
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311549825.4A Active CN117274789B (en) | 2023-11-21 | 2023-11-21 | Underwater crack semantic segmentation method for hydraulic concrete structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117274789B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118711001A (en) * | 2024-08-30 | 2024-09-27 | 成都理工大学 | Intelligent analysis method for underground water state of tunnel face |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010230421A (en) * | 2009-03-26 | 2010-10-14 | Nakanihon Highway Engineering Nagoya Kk | Method of dividing region of concrete image |
| CN102680480A (en) * | 2012-05-03 | 2012-09-19 | 中南大学 | Intelligent detecting method for cracks of concrete structures |
| CN111612846A (en) * | 2020-04-29 | 2020-09-01 | 南京震坤物联网科技有限公司 | Concrete crack width measuring method based on U-net CNN image recognition and pixel calibration |
| CN112785579A (en) * | 2021-01-26 | 2021-05-11 | 西京学院 | Concrete crack identification method based on image processing technology |
| CN112964712A (en) * | 2021-02-05 | 2021-06-15 | 中南大学 | Method for rapidly detecting state of asphalt pavement |
| CN115880571A (en) * | 2022-12-08 | 2023-03-31 | 齐丰科技股份有限公司 | Water level gauge reading identification method based on semantic segmentation |
| CN115953578A (en) * | 2022-12-07 | 2023-04-11 | 北京科技大学 | Method and device for evaluating inference result of fracture semantic segmentation model |
| CN116416244A (en) * | 2023-04-26 | 2023-07-11 | 山东大学 | Crack detection method and system based on deep learning |
| CN117058386A (en) * | 2023-08-23 | 2023-11-14 | 重庆邮电大学 | Asphalt road crack detection method based on improved deep Labv3+ network |
-
2023
- 2023-11-21 CN CN202311549825.4A patent/CN117274789B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010230421A (en) * | 2009-03-26 | 2010-10-14 | Nakanihon Highway Engineering Nagoya Kk | Method of dividing region of concrete image |
| CN102680480A (en) * | 2012-05-03 | 2012-09-19 | 中南大学 | Intelligent detecting method for cracks of concrete structures |
| CN111612846A (en) * | 2020-04-29 | 2020-09-01 | 南京震坤物联网科技有限公司 | Concrete crack width measuring method based on U-net CNN image recognition and pixel calibration |
| CN112785579A (en) * | 2021-01-26 | 2021-05-11 | 西京学院 | Concrete crack identification method based on image processing technology |
| CN112964712A (en) * | 2021-02-05 | 2021-06-15 | 中南大学 | Method for rapidly detecting state of asphalt pavement |
| CN115953578A (en) * | 2022-12-07 | 2023-04-11 | 北京科技大学 | Method and device for evaluating inference result of fracture semantic segmentation model |
| CN115880571A (en) * | 2022-12-08 | 2023-03-31 | 齐丰科技股份有限公司 | Water level gauge reading identification method based on semantic segmentation |
| CN116416244A (en) * | 2023-04-26 | 2023-07-11 | 山东大学 | Crack detection method and system based on deep learning |
| CN117058386A (en) * | 2023-08-23 | 2023-11-14 | 重庆邮电大学 | Asphalt road crack detection method based on improved deep Labv3+ network |
Non-Patent Citations (3)
| Title |
|---|
| 基于深度协同训练的肝脏CT图像自动分割方法;姜威威;刘祥强;韩金仓;;电子设计工程(14);全文 * |
| 堤防缺陷水下检测技术综述和展望;田金章 等;《中国水运》;20221130;全文 * |
| 李双 等.《电子测量技术》.2018,全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117274789A (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2020100200A4 (en) | Content-guide Residual Network for Image Super-Resolution | |
| CN111325751B (en) | CT image segmentation system based on attention convolution neural network | |
| CN113888550B (en) | Remote sensing image road segmentation method combining super-resolution and attention mechanism | |
| CN115049936A (en) | High-resolution remote sensing image-oriented boundary enhancement type semantic segmentation method | |
| CN117274789B (en) | Underwater crack semantic segmentation method for hydraulic concrete structure | |
| CN111951289B (en) | Underwater sonar image data segmentation method based on BA-Unet | |
| CN117809181B (en) | High-resolution remote sensing image water body extraction network model and method | |
| CN111797920B (en) | Remote sensing extraction method and system for depth network impervious surface with gate control feature fusion | |
| CN114419449A (en) | Self-attention multi-scale feature fusion remote sensing image semantic segmentation method | |
| CN111640116B (en) | Aerial photography graph building segmentation method and device based on deep convolutional residual error network | |
| CN114022408A (en) | Remote sensing image cloud detection method based on multi-scale convolution neural network | |
| CN115660979A (en) | Attention mechanism-based double-discriminator image restoration method | |
| CN114092467A (en) | Scratch detection method and system based on lightweight convolutional neural network | |
| CN115908793A (en) | Coding and decoding structure semantic segmentation model based on position attention mechanism | |
| CN113436198A (en) | Remote sensing image semantic segmentation method for collaborative image super-resolution reconstruction | |
| CN114821316A (en) | Three-dimensional ground penetrating radar crack disease identification method and system | |
| CN114463340B (en) | Agile remote sensing image semantic segmentation method guided by edge information | |
| CN117649448A (en) | Intelligent recognition and segmentation method for leakage water of tunnel working face | |
| CN114581789A (en) | Hyperspectral image classification method and system | |
| CN113963271A (en) | Model for identifying impervious surface from remote sensing image and method for training model | |
| CN115063410B (en) | Steel pipe counting method based on anchor-free target detection | |
| CN115456957B (en) | Method for detecting change of remote sensing image by full-scale feature aggregation | |
| CN111080533A (en) | Digital zooming method based on self-supervision residual error perception network | |
| CN113223006A (en) | Lightweight target semantic segmentation method based on deep learning | |
| CN114758387B (en) | Lightweight face anti-fraud method and device based on single-frame RGB image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |