CN116563608A - Method and device for identifying typical defect image of tunnel lining - Google Patents
Method and device for identifying typical defect image of tunnel lining Download PDFInfo
- Publication number
- CN116563608A CN116563608A CN202310382266.6A CN202310382266A CN116563608A CN 116563608 A CN116563608 A CN 116563608A CN 202310382266 A CN202310382266 A CN 202310382266A CN 116563608 A CN116563608 A CN 116563608A
- Authority
- CN
- China
- Prior art keywords
- defect
- lining
- tunnel
- image
- apparent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 244
- 238000000034 method Methods 0.000 title claims abstract description 101
- 238000013528 artificial neural network Methods 0.000 claims abstract description 41
- 230000011218 segmentation Effects 0.000 claims abstract description 31
- 238000002372 labelling Methods 0.000 claims abstract description 26
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 201000010099 disease Diseases 0.000 claims description 51
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 51
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 5
- 230000003902 lesion Effects 0.000 claims description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 abstract description 14
- 238000001514 detection method Methods 0.000 description 40
- 238000010586 diagram Methods 0.000 description 39
- 230000008569 process Effects 0.000 description 27
- 238000007689 inspection Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 5
- 239000003973 paint Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007790 scraping Methods 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005336 cracking Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The method and device for identifying the tunnel lining typical defect image provided by the application comprise the following steps: acquiring a sample lining image and a tunnel lining image to be tested; performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels; constructing a target neural network based on the sample lining image and the corresponding sample class label; inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result; determining the apparent defect type of the tunnel of the lining image prediction result; and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel. On the basis of the existing tunnel apparent defect sample library, the semantic segmentation labeling method is utilized to enrich sample types and quantity, optimize the defect recognition capability of the neural network, and realize recognition and feature extraction of lining apparent defects such as cracks, flaking (chipping) and water leakage.
Description
Technical Field
The invention relates to the technical field of iron detection, in particular to a method and a device for identifying typical defect images of tunnel lining.
Background
At present, a hand-push type tunnel inspection trolley and a vehicle-mounted tunnel lining apparent inspection trolley shown in fig. 1 are adopted in China to periodically inspect the lining surface so as to determine lining apparent defects or diseases. However, whether the vehicle is a hand-push type tunnel inspection trolley or a vehicle-mounted tunnel lining apparent inspection trolley, the detection precision and the detection speed of the intelligent detection system for the apparent tunnel lining defects carried on the vehicle-mounted tunnel inspection trolley are required to be improved.
Disclosure of Invention
The invention provides a method and a device for identifying a typical defect image of a tunnel lining, which solve the technical problems that the detection precision and the detection speed of the existing intelligent detection system for apparent defects of the tunnel lining are limited.
In a first aspect, the present invention provides a method for identifying a typical defect image of a tunnel lining, including:
acquiring a sample lining image and a tunnel lining image to be tested;
performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels;
constructing a target neural network based on the sample lining image and the corresponding sample class label;
inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
determining the apparent defect type of the tunnel of the lining image prediction result;
and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel.
Optionally, the tunnel apparent defect types include: regional disease and linear disease; according to the apparent defect type of the tunnel, determining the apparent defect result of the tunnel lining to be detected comprises the following steps:
when the apparent defect type of the tunnel is the regional defect, merging the adjacent defect areas in the image phase of the tunnel lining to be detected, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
and when the apparent tunnel defect type is the linear defect, extracting a plurality of defect single pixel trunks, and connecting based on the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
Optionally, when the apparent defect type of the tunnel is the linear defect, extracting a plurality of defect single pixel trunks, and connecting based on the defect single pixel trunks to obtain an apparent defect result of the tunnel lining to be tested, including:
extracting a plurality of disease single pixel trunks by using a Zhang-Suen refinement algorithm;
determining the end point of the single-pixel trunk of the disease based on an eight-neighborhood end point positioning algorithm;
and performing polynomial fitting on the endpoints of all the single pixel trunks of the defects to obtain apparent defect results of the tunnel lining to be detected.
Optionally, constructing a target neural network based on the sample lining image and the corresponding sample class label includes:
inputting the sample lining image into an initial neural network to generate a corresponding sample category;
determining a training error according to the sample category label and the sample category corresponding to the sample lining image;
based on the training error, the initial neural network is adjusted to obtain an optimal network parameter, and the optimal network parameter is adopted to generate the target neural network.
In a second aspect, the present invention provides a device for identifying a typical defect image of a tunnel lining, comprising:
the acquisition module is used for acquiring a sample lining image and a tunnel lining image to be tested;
the marking module is used for carrying out semantic segmentation marking on all the sample lining images by using a marking tool to obtain corresponding sample category labels;
the construction module is used for constructing a target neural network based on the sample lining image and the corresponding sample category label;
the input module is used for inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
the type determining module is used for determining the apparent tunnel defect type of the lining image prediction result;
and the apparent defect result determining module is used for determining the apparent defect result of the tunnel lining to be detected according to the type of the apparent defect of the tunnel.
Optionally, the tunnel apparent defect types include: regional disease and linear disease; the apparent disease result determination module comprises:
the first apparent defect result determining submodule is used for merging adjacent defect areas in the image phase of the tunnel lining to be detected when the apparent defect type of the tunnel is the regional defect, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
and the second apparent defect result determining submodule is used for extracting a plurality of defect single pixel trunks when the apparent defect type of the tunnel is the linear defect, and connecting the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
Optionally, the second apparent disease result determination submodule includes:
the extraction submodule is used for extracting a plurality of disease single pixel trunks through a Zhang-Suen refinement algorithm;
an endpoint determination submodule, configured to determine an endpoint of the disease single-pixel trunk based on an endpoint positioning algorithm of the eight neighborhood;
and the apparent defect determination submodule is used for carrying out polynomial fitting on the endpoints of all the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
In a third aspect, the present application provides an electronic device comprising a processor and a memory storing computer readable instructions which, when executed by the processor, perform the steps of the method as provided in the first aspect above.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method as provided in the first aspect above.
In a fifth aspect, the present application provides a computer program product having a computer program stored thereon, which when executed by the processor performs the steps of the method as provided in the first aspect above.
From the above technical scheme, the invention has the following advantages:
the invention provides a method and a device for identifying a tunnel lining typical defect image, comprising the following steps: acquiring a sample lining image and a tunnel lining image to be tested; performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels; constructing a target neural network based on the sample lining image and the corresponding sample class label; inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result; determining the apparent defect type of the tunnel of the lining image prediction result; and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel. On the basis of the existing tunnel apparent defect sample library, the semantic segmentation labeling method is utilized to enrich sample types and quantity, optimize the defect recognition capability of the neural network, and realize recognition and feature extraction of lining apparent defects such as cracks, flaking (chipping) and water leakage.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a vehicle-mounted tunnel lining appearance inspection vehicle;
FIG. 2 is a flowchart illustrating a method for identifying a typical defect image of a tunnel lining according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a layout scheme of a camera of a vehicle-mounted tunnel lining appearance inspection vehicle according to an embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 4 is a second schematic diagram of a camera layout scheme of a vehicle-mounted tunnel lining appearance inspection vehicle according to an embodiment of a method for identifying typical defect images of tunnel lining of the present invention;
FIG. 5 is a block diagram of a synchronization system of an embodiment of a method for identifying images of typical defects of a tunnel lining according to the present invention;
FIG. 6 is a flowchart illustrating a method for identifying a typical defect image of a tunnel lining according to a second embodiment of the present invention;
FIG. 7 is a schematic diagram of a process for creating a structural joint dataset according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 8 is a second schematic diagram of a construction joint dataset creation process for a second exemplary method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 9 is a third schematic diagram of a construction joint dataset creation process according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 10 is a schematic diagram of a process for establishing a block-dropping dataset according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 11 is a second schematic diagram of a process for establishing a block-dropping dataset according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 12 is a third schematic diagram of a process for creating a missing block dataset according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 13 is a schematic diagram of a process for creating a data set of dropping and scraping according to a second exemplary method for identifying a typical defect image of a tunnel lining;
FIG. 14 is a schematic diagram of a second exemplary process for creating a data set of dropping and scraping according to a second exemplary method for identifying a typical defect image of a tunnel lining;
FIG. 15 is a third schematic view of a process for creating a data set of dropping and scraping according to a second exemplary method for identifying a typical defect image of a tunnel lining;
FIG. 16 is a schematic diagram of a crack dataset creation process according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 17 is a second schematic diagram of a crack dataset creation process for a second exemplary method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 18 is a third exemplary process for creating a fracture dataset for a second exemplary method for identifying a typical defect image for a tunnel lining according to the present invention;
FIG. 19 is a schematic diagram of a black paint dataset creation process according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 20 is a second schematic diagram of a black paint dataset creation process for a method for identifying a typical defect image of a tunnel lining according to the second embodiment of the present invention;
FIG. 21 is a third schematic diagram of a black paint dataset creation process for a method for identifying a typical defect image of a tunnel lining according to the second embodiment of the present invention;
FIG. 22 is a schematic diagram of a water stain data set creation process according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 23 is a second schematic diagram of a water stain data set creation process according to a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining;
FIG. 24 is a third schematic view of a water stain dataset creation process for a second exemplary method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 25 is a schematic diagram of a crack classification of a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 26 is a second exemplary embodiment of a crack classification diagram of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 27 is a schematic diagram of a crack target detection of a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 28 is a schematic diagram of identifying the same defect as a plurality of defects in a second exemplary method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 29 is a schematic view of the actual effect of a polynomial fit crack of a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 30 is a schematic diagram of a fitting process of crack boundary points in a second exemplary embodiment of a method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 31 is one of the original high definition test tunnel lining images of an exemplary lesion image recognition method embodiment II of the tunnel lining of the present invention;
FIG. 32 is a second original high definition image of a tunnel lining to be tested according to a second exemplary method of identifying a typical defect image of a tunnel lining of the present invention;
FIG. 33 is a third original high definition test tunnel lining image of a second exemplary method for identifying a typical defect image of a tunnel lining according to the present invention;
FIG. 34 is a schematic view of a lesion development of a second exemplary embodiment of a method for identifying a typical lesion image of a tunnel lining according to the present invention;
fig. 35 is a block diagram illustrating an exemplary embodiment of a device for recognizing a typical defect image of a tunnel lining according to the present invention.
Detailed Description
The embodiment of the invention provides a method and a device for identifying typical defect images of tunnel lining, which solve the technical problems of limited detection precision and detection speed of the existing intelligent detection system for apparent defects of tunnel lining.
In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions in the embodiments of the present invention are described in detail below with reference to the accompanying drawings, and it is apparent that the embodiments described below are only some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 2, fig. 2 is a flowchart of a method for identifying a typical defect image of a tunnel lining according to an embodiment of the present invention, including:
s101, acquiring a sample lining image and a tunnel lining image to be tested;
the existing intelligent detection system for apparent tunnel lining damage carried on the hand-push type tunnel inspection trolley or the vehicle-mounted tunnel lining apparent detection trolley is divided into a damage acquisition module and a post-processing module. The defect acquisition module is a linear array formed by 8 camera models and is used for acquiring a tunnel lining surface image so as to identify lining surface defects, tunnel facilities and structural characteristics. The technical parameters of the system performance of the disease acquisition module are as follows: when the detection speed is 40km/h, the resolution of the image reaches 1mm multiplied by 1mm, and the crack of 0.5mm can be identified; when the detection speed is 80km/h, the resolution of the image reaches 2mm multiplied by 2mm, and 1mm of cracks can be identified; the crack identification accuracy is only 70%; the lining contour detection precision is 10mm; the mileage positioning accuracy is 1%.
In the embodiment of the invention, the intelligent detection system for the apparent damage of the tunnel lining, which is related to the identification method of the typical damage image of the tunnel lining, consists of a vehicle-mounted detection subsystem and a data processing system, wherein the vehicle-mounted detection subsystem adopts a machine vision and laser scanning technology to realize the rapid acquisition of the apparent image and the space size of the lining. Specifically, the in-vehicle monitoring subsystem includes: the system comprises an image acquisition module, a limit detection module, a vehicle body motion compensation module, an infrared camera module, a ballast bed state detection module, a data acquisition and storage module, a mileage and synchronization module, a power supply system and the like, wherein the acquisition module is composed of a linear array camera, an industrial lens and a laser illumination integrated imaging component, and the modules realize sensor data acquisition, control and data storage through a pre-deployed tunnel lining apparent state detection program.
In a specific implementation, on the basis of analyzing the cross section size of a full-line single-line double-line tunnel where a tunnel lining image to be detected is located in detail, a camera layout scheme of a vehicle-mounted tunnel lining apparent detection vehicle, which is shown in fig. 3 and 4, is designed by reasonably setting the angle of a camera and the focal length of a lens, wherein the camera layout scheme is shown in fig. 3 and 4, the vehicle-mounted tunnel lining apparent detection vehicle adopts 10-channel cameras to detect full-width tunnels, every 5-channel cameras detect half-width tunnels, the two cameras are arranged in two rows and are not mutually interfered, the object distance of each camera in an acquisition module is 1500-4200mm, the camera view is 30-70 degrees, the overlapping range of the camera view is about 200mm, and the image resolution is 0.5mm.
Meanwhile, as the linear array camera is adopted for detection, in order to realize that the resolution of the tunnel lining image to be detected along the line direction is fixed and avoid the influence of the speed change on the tunnel lining image to be detected, the embodiment of the invention adopts the mode of triggering the camera to acquire the image according to the distance sampling, the rotary encoder arranged at the shaft end provides a pulse triggering signal, and adopts the frequency modulation and frequency division device to synchronously trigger the camera to work and synchronously calculate mileage according to the pulse number and the pulse width. Therefore, the vehicle-mounted scheme research (apparent image and tunnel limit) of the full-section multichannel detection system of the yellow heavy-duty railway tunnel can be realized based on the actual condition of the railway site, and the safety protection level of the detection system of the severe environment of the heavy-duty railway tunnel is greatly improved.
In addition, the intelligent detection system for apparent damage of tunnel lining in the embodiment of the invention adopts two computers to cooperatively work, as shown in a synchronous system block diagram of the first embodiment of the identification method of the typical damage image of tunnel lining in the invention in fig. 5, a storage system in which the tunnel lining image to be detected is stored is connected with an industrial computer (host) and an industrial computer (slave), and the tunnel lining image to be detected is respectively stored in the industrial computer through an encoder and a synchronous control system by different image acquisition modules.
Step S102, carrying out semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample category labels;
step S103, constructing a target neural network based on the sample lining image and the corresponding sample class label;
step S104, inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
step S105, determining the apparent defect type of the tunnel of the lining image prediction result;
and S106, determining the apparent defect result of the tunnel lining to be tested according to the apparent defect type of the tunnel.
The method for identifying the typical defect image of the tunnel lining provided by the embodiment of the invention comprises the steps of obtaining a sample lining image and a tunnel lining image to be tested; performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels; constructing a target neural network based on the sample lining image and the corresponding sample class label; inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result; determining the apparent defect type of the tunnel of the lining image prediction result; and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel. On the basis of the existing tunnel apparent defect sample library, the semantic segmentation labeling method is utilized to enrich sample types and quantity, optimize the defect recognition capability of the neural network, and realize recognition and feature extraction of lining apparent defects such as cracks, flaking (chipping) and water leakage.
Referring to fig. 6, fig. 6 is a flowchart illustrating a method for identifying a typical defect image of a tunnel lining according to a second embodiment of the present invention, including:
step S201, a sample lining image and a tunnel lining image to be tested are obtained;
step S202, carrying out semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample category labels;
in the field of deep learning, the creation of data sets is critical to training models. The main characteristic information of tunnel cracks, flaking, water leakage and other diseases is quite different from most existing public data sets. In order to solve the problem of segmentation and identification of tunnel defects, in the embodiment of the invention, a labeling tool label-image-CARS is used for carrying out pixel-level semantic segmentation labeling on each image, and a special data set of tunnel defects, a mask-v 3, is constructed by using unused color labeling defect data, and the method comprises the following steps: the defect data sets such as cracks, flaking, water stains, icing and the like are shown in fig. 7 to 24, wherein fig. 7 to 9 are schematic diagrams of a structure seam data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention, fig. 10 to 12 are schematic diagrams of a block falling data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention, fig. 13 to 15 are schematic diagrams of a block falling and scratch data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention, fig. 16 to 18 are schematic diagrams of a crack data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention, fig. 19 to 21 are schematic diagrams of a black paint data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention, and fig. 22 to 24 are schematic diagrams of a water stain data set establishing process of a second embodiment of a method for identifying a typical defect image of a tunnel lining of the present invention.
Step S203, inputting the sample lining image into an initial neural network to generate a corresponding sample category;
step S204, determining training errors according to the sample category labels and the sample categories corresponding to the sample lining images;
step S205, based on the training error, the initial neural network is adjusted to obtain an optimal network parameter, and the optimal network parameter is adopted to generate the target neural network;
step S206, inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
aiming at the identification of tunnel cracks, flaking and other defects, the conventional algorithm comprises target classification and target detection.
Specifically, the object classification is to distinguish the types of objects in the image, when only a single defect exists in the image, the image can be classified, such as a crack classification schematic diagram of fig. 25 and 26, but the tunnel defect image is often complex and contains a plurality of defects, and cannot be classified into a certain type, so that the image object classification cannot well identify all tunnel defects, and the defects in the tunnel are identified as spatial levels, which cannot be achieved only by the image classification.
In the target detection, the external frame is used to frame and classify an object in a specific position of an object in a detected image, as shown in a crack target detection schematic diagram of fig. 27, a common detection network is Faster-RCNN, ssd, FCOS, the detection effect of the existing detection networks on a tiny target is not ideal, diseases in the image are often irregular in shape, and the external rectangular frame cannot well reflect the area, shape, size and other attributes of the diseases. In summary, object detection is not suitable for solving such tasks as crack disease identification.
Therefore, the existing defect detection and identification algorithm is difficult to accurately identify tunnel lining defects, defects which are not defects or less harmful are identified as normal defects, and some partial characteristics of the defects are not obvious or are blocked by other objects, so that the algorithm can identify the same defect as a plurality of defects, as shown in a schematic diagram of fig. 28, wherein only one crack is shown in the diagram, but 4 defects are identified because of the fact that the blocking objects ((1) and (2)) and the shielding objects have the inconspicuous characteristics ((3)) are identified, and thus the result statistical analysis is seriously influenced.
The embodiment of the invention realizes the defect recognition based on the actual complex scene by adopting a semantic segmentation method, the semantic segmentation is to carry out pixel-level classification recognition on a single image, namely, after inputting a picture, a network gives a category prediction to each pixel point of the image, and the output result can better reflect the position, the shape and other characteristics of tunnel defects, and the effect is shown in fig. 7-24.
Step S207, determining the apparent defect type of the tunnel of the lining image prediction result;
and step S208, determining the apparent defect result of the tunnel lining to be tested according to the apparent defect type of the tunnel.
In an alternative embodiment, the tunnel apparent defect types include: regional disease and linear disease; according to the apparent defect type of the tunnel, determining the apparent defect result of the tunnel lining to be detected comprises the following steps:
when the apparent defect type of the tunnel is the regional defect, merging the adjacent defect areas in the image phase of the tunnel lining to be detected, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
when the apparent tunnel defect type is the linear defect, extracting a plurality of defect single pixel trunks, and connecting based on the defect single pixel trunks to obtain an apparent tunnel lining defect result to be detected, wherein the method specifically comprises the following steps of:
extracting a plurality of disease single pixel trunks by using a Zhang-Suen refinement algorithm;
determining the end point of the single-pixel trunk of the disease based on an eight-neighborhood end point positioning algorithm;
and performing polynomial fitting on the endpoints of all the single pixel trunks of the defects to obtain apparent defect results of the tunnel lining to be detected.
The apparent tunnel defects mainly exist in two types of area shapes (such as flaking and water leakage) and line shapes (such as cracks), and processing algorithms are respectively designed for the two types of defects. Specifically, the regional diseases such as flaking, water leakage and the like and the linear diseases such as cracks and the like are treated respectively, the regional diseases are combined and deleted, the adjacent diseases are combined into the same place, and meanwhile, the diseases with smaller areas are deleted; for the linear diseases, a Zhang-Suen refinement algorithm is used for extracting a single pixel trunk of the diseases, an endpoint positioning algorithm based on eight neighborhood is designed for rapidly searching endpoints, finally, a polynomial is used for fitting trend of the linear diseases such as cracks and the like, stable transitional connection of the broken diseases is achieved, namely, two ends of two broken cracks are respectively provided with a plurality of coordinate points, a curve is fitted by the coordinate polynomials, the curve is used for replacing breaking positions in the middle of the two cracks, and the practical effect is shown in a polynomial fitting crack practical effect schematic diagram shown in fig. 29. Green points on the left and right sides of the figure are discrete points taken from two cracks respectively, the points are subjected to polynomial fitting, and a middle green curve is the position of the fitted curve in the middle of the two fracture cracks.
In addition, in order to perform fine calculation on the crack width, on the basis of the crack pixel area divided by the algorithm, a crack width calculation method based on a central axis perpendicular line is used, the crack edge is fitted by combining three times of Cardinal spline interpolation, the concrete crack edge with sub-pixel precision is detected, a more accurate crack is calculated, and a specific fitting process is shown as a schematic diagram of a fitting process of a crack boundary point in fig. 30.
In the embodiment of the invention, after the vehicle-mounted detection is finished, the collected original image data, namely the tunnel lining image to be detected, is subjected to pretreatment, manual check, disease grade assessment and other processing works, and a detection result is given. The specific implementation working content comprises:
(1) And (5) data analysis and preprocessing. After the original high-definition tunnel lining image to be measured is obtained and subjected to data analysis as shown in fig. 31-33, preprocessing including image segmentation, normalization and data enhancement is performed on the original high-definition tunnel lining image, so that identification accuracy is ensured.
(2) And (5) intelligent identification. Inputting the preprocessed tunnel lining image to be detected into a target neural network for semantic segmentation, namely automatically identifying diseases such as lining cracking, water leakage, flaking (blocking) and the like, obtaining a lining image prediction result, generating corresponding xml files and RGB pictures, facilitating data management, recording and historical data query, and greatly improving the intelligent level of railway on-site detection.
(3) Checking the identification result. Aiming at the prediction results of lining images comprising preliminary identification reports and identification result graphs of the diseases, checking and confirming the diseases such as lining cracking, water leakage, flaking (chipping) and the like, and giving out space information such as disease mileage, positions and the like, geometric characteristics such as length, width and the like.
(4) And generating different types of detection results according to the comprehensive information. Statistical analysis is performed on diseases in different sections and different degrees, and a disease development schematic diagram shown in fig. 34 is generated and is used as a detection result.
The method for identifying the typical defect image of the tunnel lining provided by the embodiment of the invention comprises the steps of obtaining a sample lining image and a tunnel lining image to be tested; performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels; constructing a target neural network based on the sample lining image and the corresponding sample class label; inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result; determining the apparent defect type of the tunnel of the lining image prediction result; and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel. On the basis of the existing tunnel apparent defect sample library, the semantic segmentation labeling method is utilized to enrich sample types and quantity, optimize the defect recognition capability of the neural network, and realize recognition and feature extraction of lining apparent defects such as cracks, flaking (chipping) and water leakage.
Referring to fig. 35, fig. 35 is a block diagram of an embodiment of a device for identifying a typical defect image of a tunnel lining according to the present invention, including:
an acquisition module 401, configured to acquire a sample lining image and a tunnel lining image to be tested;
the labeling module 402 is configured to perform semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels;
a construction module 403, configured to construct a target neural network based on the sample lining image and the corresponding sample class label;
the input module 404 is configured to input the tunnel lining image to be tested into the target neural network for semantic segmentation, so as to obtain a lining image prediction result;
a type determining module 405, configured to determine a tunnel apparent defect type of the lining image prediction result;
and the apparent defect result determining module 406 is configured to determine an apparent defect result of the tunnel lining to be tested according to the apparent defect type of the tunnel.
In an alternative embodiment, the tunnel apparent defect types include: regional disease and linear disease; the apparent disease result determination module 406 includes:
the first apparent defect result determining submodule is used for merging adjacent defect areas in the image phase of the tunnel lining to be detected when the apparent defect type of the tunnel is the regional defect, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
and the second apparent defect result determining submodule is used for extracting a plurality of defect single pixel trunks when the apparent defect type of the tunnel is the linear defect, and connecting the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
In an alternative embodiment, the second apparent disease result determination submodule includes:
the extraction submodule is used for extracting a plurality of disease single pixel trunks through a Zhang-Suen refinement algorithm;
an endpoint determination submodule, configured to determine an endpoint of the disease single-pixel trunk based on an endpoint positioning algorithm of the eight neighborhood;
and the apparent defect determination submodule is used for carrying out polynomial fitting on the endpoints of all the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
In an alternative embodiment, the building block 403 includes:
the class generation sub-module is used for inputting the sample lining image into an initial neural network to generate a corresponding sample class;
the training error determining submodule is used for determining training errors according to the sample category labels and the sample categories corresponding to the sample lining images;
and the neural network generation sub-module is used for obtaining optimal network parameters by adjusting the initial neural network based on the training error, and generating the target neural network by adopting the optimal network parameters.
The embodiment of the invention also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the steps of the method for identifying the tunnel lining typical defect image according to any embodiment.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by the processor, implements the steps of the method for identifying a tunnel lining typical defect image according to any of the above embodiments.
An embodiment of the present invention further provides a computer program product, which when executed by the processor, implements the steps of the method for identifying a tunnel lining typical defect image according to any of the above embodiments.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the embodiments provided in the present application, it should be understood that the methods, apparatuses, electronic devices and storage media disclosed in the present application may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a readable storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned readable storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The method for identifying the typical defect image of the tunnel lining is characterized by comprising the following steps of:
acquiring a sample lining image and a tunnel lining image to be tested;
performing semantic segmentation labeling on all the sample lining images by using a labeling tool to obtain corresponding sample class labels;
constructing a target neural network based on the sample lining image and the corresponding sample class label;
inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
determining the apparent defect type of the tunnel of the lining image prediction result;
and determining the apparent defect result of the tunnel lining to be detected according to the apparent defect type of the tunnel.
2. The method for identifying a tunnel lining representative defect image according to claim 1, wherein the tunnel apparent defect type comprises: regional disease and linear disease; according to the apparent defect type of the tunnel, determining the apparent defect result of the tunnel lining to be detected comprises the following steps:
when the apparent defect type of the tunnel is the regional defect, merging the adjacent defect areas in the image phase of the tunnel lining to be detected, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
and when the apparent tunnel defect type is the linear defect, extracting a plurality of defect single pixel trunks, and connecting based on the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
3. The method for identifying a tunnel lining typical defect image according to claim 2, wherein when the tunnel apparent defect type is the linear defect, extracting a plurality of defect single pixel trunks, and connecting based on the defect single pixel trunks to obtain the tunnel lining apparent defect result to be tested, comprises:
extracting a plurality of disease single pixel trunks by using a Zhang-Suen refinement algorithm;
determining the end point of the single-pixel trunk of the disease based on an eight-neighborhood end point positioning algorithm;
and performing polynomial fitting on the endpoints of all the single pixel trunks of the defects to obtain apparent defect results of the tunnel lining to be detected.
4. The method for identifying a tunnel lining representative lesion image according to claim 1, wherein constructing a target neural network based on the sample lining image and a corresponding sample class label comprises:
inputting the sample lining image into an initial neural network to generate a corresponding sample category;
determining a training error according to the sample category label and the sample category corresponding to the sample lining image;
based on the training error, the initial neural network is adjusted to obtain an optimal network parameter, and the optimal network parameter is adopted to generate the target neural network.
5. A tunnel lining representative defect image recognition apparatus, comprising:
the acquisition module is used for acquiring a sample lining image and a tunnel lining image to be tested;
the marking module is used for carrying out semantic segmentation marking on all the sample lining images by using a marking tool to obtain corresponding sample category labels;
the construction module is used for constructing a target neural network based on the sample lining image and the corresponding sample category label;
the input module is used for inputting the tunnel lining image to be detected into the target neural network for semantic segmentation to obtain a lining image prediction result;
the type determining module is used for determining the apparent tunnel defect type of the lining image prediction result;
and the apparent defect result determining module is used for determining the apparent defect result of the tunnel lining to be detected according to the type of the apparent defect of the tunnel.
6. The apparatus for identifying a tunnel lining representative defect image according to claim 5, wherein the tunnel apparent defect type comprises: regional disease and linear disease; the apparent disease result determination module comprises:
the first apparent defect result determining submodule is used for merging adjacent defect areas in the image phase of the tunnel lining to be detected when the apparent defect type of the tunnel is the regional defect, and deleting the defect areas with the area smaller than a preset area threshold value to obtain a defect representing result of the tunnel lining to be detected;
and the second apparent defect result determining submodule is used for extracting a plurality of defect single pixel trunks when the apparent defect type of the tunnel is the linear defect, and connecting the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
7. The apparatus for identifying a tunnel lining representative defect image of claim 6, wherein the second apparent defect result determination submodule comprises:
the extraction submodule is used for extracting a plurality of disease single pixel trunks through a Zhang-Suen refinement algorithm;
an endpoint determination submodule, configured to determine an endpoint of the disease single-pixel trunk based on an endpoint positioning algorithm of the eight neighborhood;
and the apparent defect determination submodule is used for carrying out polynomial fitting on the endpoints of all the defect single pixel trunks to obtain the apparent defect result of the tunnel lining to be detected.
8. An electronic device comprising a processor and a memory storing computer readable instructions that, when executed by the processor, perform the method of any of claims 1-4.
9. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, performs the method according to any of claims 1-4.
10. A computer program product, characterized in that it performs the method according to any of claims 1-4 when run on a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310382266.6A CN116563608A (en) | 2023-04-11 | 2023-04-11 | Method and device for identifying typical defect image of tunnel lining |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310382266.6A CN116563608A (en) | 2023-04-11 | 2023-04-11 | Method and device for identifying typical defect image of tunnel lining |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563608A true CN116563608A (en) | 2023-08-08 |
Family
ID=87490715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310382266.6A Pending CN116563608A (en) | 2023-04-11 | 2023-04-11 | Method and device for identifying typical defect image of tunnel lining |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563608A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117351241A (en) * | 2023-10-18 | 2024-01-05 | 中交路桥科技有限公司 | Intelligent detection and assessment method, device, terminal and storage medium for tunnel defect |
| CN118154516A (en) * | 2024-01-30 | 2024-06-07 | 浙江华展研究设计院股份有限公司 | Tunnel seepage and crack detection method |
| CN118212178A (en) * | 2024-02-05 | 2024-06-18 | 中铁长江交通设计集团有限公司 | Data analysis method and system for tunnel apparent defect development trend |
-
2023
- 2023-04-11 CN CN202310382266.6A patent/CN116563608A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117351241A (en) * | 2023-10-18 | 2024-01-05 | 中交路桥科技有限公司 | Intelligent detection and assessment method, device, terminal and storage medium for tunnel defect |
| CN117351241B (en) * | 2023-10-18 | 2024-05-03 | 中交路桥科技有限公司 | Intelligent detection and assessment method, device, terminal and storage medium for tunnel defect |
| CN118154516A (en) * | 2024-01-30 | 2024-06-07 | 浙江华展研究设计院股份有限公司 | Tunnel seepage and crack detection method |
| CN118212178A (en) * | 2024-02-05 | 2024-06-18 | 中铁长江交通设计集团有限公司 | Data analysis method and system for tunnel apparent defect development trend |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Peng et al. | A UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning | |
| CN116563608A (en) | Method and device for identifying typical defect image of tunnel lining | |
| Huang et al. | A pavement crack detection method combining 2D with 3D information based on Dempster‐Shafer theory | |
| CN111476781B (en) | Concrete crack identification method and device based on video semantic segmentation technology | |
| Min et al. | Real time detection system for rail surface defects based on machine vision | |
| Lei et al. | A novel tunnel-lining crack recognition system based on digital image technology | |
| Wang et al. | Automated estimation of reinforced precast concrete rebar positions using colored laser scan data | |
| Medina et al. | Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques | |
| Gao et al. | Detection and segmentation of cement concrete pavement pothole based on image processing technology | |
| Akagic et al. | Pothole detection: An efficient vision based method using rgb color space image segmentation | |
| CN104048969A (en) | Tunnel defect recognition method | |
| CN110660072B (en) | Method and device for identifying straight line edge, storage medium and electronic equipment | |
| CN112966665A (en) | Pavement disease detection model training method and device and computer equipment | |
| Wu et al. | Concrete spalling detection for metro tunnel from point cloud based on roughness descriptor | |
| CN110969610A (en) | Power equipment infrared chart identification method and system based on deep learning | |
| CN111046950A (en) | Image processing method and device, storage medium and electronic device | |
| CN113240623A (en) | Pavement disease detection method and device | |
| CN112001880A (en) | Characteristic parameter detection method and device for planar component | |
| CN118172046B (en) | Vehicle maintenance management method and system | |
| Lang et al. | Pavement cracking detection and classification based on 3d image using multiscale clustering model | |
| CN114280075A (en) | Online visual inspection system and method for surface defects of pipe parts | |
| CN113781436B (en) | High-voltage switch state monitoring method based on camera and angle measurement | |
| Parrany et al. | A new image processing strategy for surface crack identification in building structures under non‐uniform illumination | |
| CN104878679A (en) | Vehicle-mounted rut real-time detection system and detection method | |
| Chen et al. | 3D pavement data decomposition and texture level evaluation based on step extraction and pavement-transformer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |