CN112967255A - Shield segment defect type identification and positioning system and method based on deep learning - Google Patents
Shield segment defect type identification and positioning system and method based on deep learning Download PDFInfo
- Publication number
- CN112967255A CN112967255A CN202110255249.7A CN202110255249A CN112967255A CN 112967255 A CN112967255 A CN 112967255A CN 202110255249 A CN202110255249 A CN 202110255249A CN 112967255 A CN112967255 A CN 112967255A
- Authority
- CN
- China
- Prior art keywords
- defect
- segment
- image
- network
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 242
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013135 deep learning Methods 0.000 title claims abstract description 18
- 238000012549 training Methods 0.000 claims description 27
- 230000002950 deficient Effects 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 238000009877 rendering Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000013136 deep learning model Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000007405 data analysis Methods 0.000 claims description 7
- 238000013079 data visualisation Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000010276 construction Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000004807 localization Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a shield segment defect type identification and positioning system based on deep learning and a method thereof. Through shield constructs section of jurisdiction defect type discernment and positioning system can realize the automatic quick and accurate statistics of section of jurisdiction defect information, has greatly reduced the amount of manual labor, has shortened section of jurisdiction defect from discovering to prosthetic time.
Description
Technical Field
The invention relates to the field of application of computer vision image processing technology, in particular to a shield segment defect type identification and positioning system based on deep learning.
Technical Field
In China, the tunnel and underground engineering in the world is the largest in scale and the largest in quantity, the most complex in geological conditions and structural forms and the fastest in building technology development. The shield construction method is one of important construction methods suitable for modern tunnel and underground engineering construction, has a plurality of advantages and plays an important role. The shield segment is a main assembly component for shield construction, is the innermost barrier of the tunnel and plays a role in resisting soil layer pressure, underground water pressure and some special loads. The shield segment is a permanent lining structure of a shield tunnel, and the quality of the shield segment is directly related to the overall quality and safety of the tunnel, so that the waterproof performance and the durability of the tunnel are influenced. And if the defects of the shield segment are not found in time, huge potential safety hazards and economic losses are brought, and the importance of finding and repairing the segment defects in time can be seen. The investigation shows that the quality defects of the segment used in the subway shield tunnel construction still have some problems which are not well solved, such as leakage, cracks, slab staggering, corner collapse, torsion and the like of the segment.
At traditional manual work segment defect and patrol and examine and discern categorised in-process, on the one hand, to the shield constructs the quality problems that patrols and examines the in-process appearance of section of jurisdiction, often by artifical manual paper record and the sample of shooing. On the other hand, the processing of the segment quality information is also manually analyzed by the past experience. However, the pure manual method has the problems of large human input, low processing speed, long time consumption, low intelligent degree and the like, and the accuracy rate cannot be guaranteed, so that the quality maintenance of the wafer is influenced.
In recent years, computer technology has been advanced rapidly, and particularly in the field of artificial intelligence deep learning, image recognition technology is becoming mature. The computer can classify and identify the shield segment through the 2D image captured by the external architecture camera, and the automatic inspection of the shield segment for the position and the type of the defect can be identified by the computer along with the improvement of computer power and the enhancement of processing capacity.
Disclosure of Invention
In view of the defects of the prior art, the invention aims to provide a shield segment defect type identification and positioning system based on deep learning and a method thereof, which combine the deep learning technology with a segment defect identification method, can realize automatic identification and detection of the type and position of the shield segment defect, and greatly improve the speed and precision of segment defect information statistics and maintenance.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a shield constructs section of jurisdiction defect type discernment and positioning system based on degree of deep learning, the system includes:
the image acquisition module is used for shooting shield segment images from an external framework;
a metadata input module for inputting data describing defect information;
the data synchronization module is used for synchronizing data of the mobile terminal and the server terminal;
an image preprocessing module for preprocessing an image obtained from an external architecture;
the deep learning model training module is used for training a neural network to obtain a weight file;
the deep learning model loading module is used for loading the trained convolutional neural network;
the defect type identification module is used for identifying the defective segment and judging the defect type of the defective segment;
the defect positioning module is used for positioning the defect position of the defective segment;
the information storage module is used for storing the defect information of the segment;
and the data analysis and visualization module is used for comprehensively analyzing the segment defect position and type information, generating a report and visualizing the report in a graphic form.
Based on the system, the invention also provides a shield segment defect type identification and positioning system identification and positioning method based on deep learning, which comprises the following steps:
s110, inputting metadata such as ring numbers, segment types, k point positions and the like into a data synchronization module;
s120, acquiring an image of the wall of the subway shield segment pipe by an external framework, and inputting the image into a data synchronization module;
s130, loading the metadata input in the step S1 and the image acquired in the step S2, and uploading the metadata and the image to a server;
s140, receiving the data uploaded by the mobile equipment, downloading the data to a server, and storing the data in an information storage module;
s150, preprocessing a segment defect picture acquired from the outside, and performing data enhancement on image data in a traditional image random affine transformation mode to obtain image data which is easier to process;
s160, carrying out defect detection on the image characteristic graph, comparing the obtained detection result with the label, carrying out error calculation, updating the parameter of the deep convolutional network according to the error, and stopping training when the error is smaller than the set error threshold value of the deep convolutional network to obtain the trained deep convolutional network;
s170, loading the trained deep learning model stored on the server, wherein the deep learning model comprises a defect type identification module and a defect positioning module;
s180, delivering the preprocessed pictures to a defect type identification module, outputting the identified defect types, and classifying the defect types;
s190, delivering the preprocessed picture to a defect positioning module, and outputting the position of the defect in the picture; then, calculating the relative distance and angle between the positioning defect and the approximate position of the bolt hole according to the positioning defect and the approximate position of the bolt hole so as to determine the specific position of the defect on the duct piece;
s1100, delivering the defect type information and the defect positioning information output by the network to an information storage module for storing data and synchronizing the data with the mobile equipment;
s1110, summarizing and summarizing the information in the information storage module to obtain information useful for a user, visualizing various types of information of the segment defects to the user according to the requirements of the user using the system, and providing assessment and summary for the user.
It should be noted that, the defect type identification module in step S8 is implemented by using a ResNet50-FPN-RPN network structure, and includes the following steps:
s210, acquiring a segment defect image through an image acquisition device, marking the defect type of a defect object, and establishing a defect data set of various defects;
s220, initializing training parameters, loading a pre-training network model and weights, and setting defect classification types and number; presetting training times, and obtaining a trained network weight file under the condition of ensuring to meet constraint conditions such as a loss function and the like;
s230, editing a training data set and a test set, wherein the training data set comprises a defective image, an image file path and a defect type, and performing primary secondary classification on an image acquired by an image acquisition terminal by adopting a pre-trained convolutional neural network to obtain a result set of the defective image and a result set of a non-defective image;
s240, screening the images with defects in the binary classification result set, and performing fine classification operation in the subsequent steps and removing the images without defects;
s250, extracting feature map by using a ResNet50 network architecture as a feature extractor; meanwhile, a characteristic pyramid network FPN is added to expand a backbone network, so that multi-scale information of segment defects is obtained;
s260, a light-weight region suggestion network RPN is adopted to search candidate ROIs, each scanned region is an anchor while a target region is searched, and finally the anchor with the highest score, namely the position where the defect is most likely to be located, is selected according to the specific score of each anchor and transmitted to the next stage;
s270, performing ROI pooling operation on the transferred ROI, sampling at different points of the feature map by adopting ROI Align operation proposed in Mask RCNN, and finally obtaining the feature map with a fixed size by applying a bilinear interpolation method; then inputting the defect type of the image to a classifier for classifying the defect type;
s280, the predicted defect type information output by the neural network is stored in an information storage module and is delivered to a server to maintain data for a user to use.
It should be noted that, in the step S9, the defect location of the die is implemented by using a YOLO _ V3 network; the method comprises the following steps:
s310, acquiring the segment defects through an external acquisition device, marking the detailed coordinate positions of the defect objects, numbering the detailed coordinate positions, and establishing a defect position data set from 1;
s320, initializing parameters of the defect positioning convolution neural network, and training the network by using the established data set to generate a weight file;
s330, receiving a segment defect map which is acquired and preprocessed by an image acquisition terminal, calling a defect positioning convolutional neural network which is trained in advance, setting network parameters and loading the trained network weights;
s340, selecting the size of an input image of the design network to be 416 multiplied by 416, and carrying out uniform size on the image by adopting a bilinear interpolation method;
s350, normalizing the serialized images, and normalizing the RGB color modes of 0-255 to black and white binary modes of 0-1;
s360, dividing the processed image into S multiplied by S grids, and detecting the object by the grid unit only when the center of the target defect falls into a certain grid unit, so as to obtain the position detection result of the target defect; predicting a fixed number of prediction frames by each grid, and finally calculating the relative distance and angle between the prediction frames and the approximate positions of the bolts according to the prediction frames for positioning the defects and the approximate positions of the bolts so as to position the specific positions of the defects on the duct piece;
s370, the predicted defect position information output by the neural network is stored in the information storage module and is delivered to the server to maintain data for users to use.
It should be noted that the prediction box includes 5 basic data of (x, y, w, h, cre); wherein (x, y) is the offset of the center of the prediction box with respect to the current mesh and (w, h) is the length and width of the prediction box; the value of cre reflects whether the bounding box contains the target probability and the current bounding box coincides with the real bounding box.
It should be noted that, the segment defect visualization in step S11 needs to render a perspective view convenient for a user to view according to the segment type, the K block position, the segment defect type and the position, and includes the following steps:
s410, rendering a segment perspective view according to a segment type and K block positions, wherein the segment type comprises: 10 point location, 16 point location, 19 point location, large shield and the like, wherein the K block can be positioned at different positions according to the type of the segment and the number of segment pushing rings;
s420, rendering the segment connecting bolt according to the segment type and the K block position; the segment connecting bolt comprises a longitudinal bolt and a transverse bolt, the longitudinal bolt is used for connecting shield segments of the front ring and the rear ring, and the transverse bolt is used for connecting adjacent segments in the same ring;
s430, rendering a defect legend according to the quality defect type and the position of the duct piece, wherein different defect types are rendered by different legends, and the method specifically comprises the following steps: leakage, cracks, slab staggering, etc.; the approximate location of the defect is calculated and rendered based on the segment in which the defect is located and the relative location of the segment to the bolt.
The invention has the beneficial effects that:
1. the shield segment defect type identification and positioning system can quickly acquire segment defect types and position information only by inputting segment defect pictures into the system by a user. The user can view all defect records and the system automatically generates a record table. The user can check the distribution of all defects on the duct piece, and the system automatically generates a positioning table and a map. The user can check the daily statistics and monthly statistics defect results, namely the defect confirmation conditions of each day and each month, and the system automatically generates a segment quality report and displays the segment quality report to the user in a pie chart and a histogram.
2. Compared with the traditional manual defect classification and positioning, the system realizes an automatic defect type identification module and a positioning module, namely, a convolutional neural network capable of identifying the defects is trained for identifying the type of the segment defects and determining the defect positions by the convolutional neural network capable of positioning the defect positions. Before the shooting defect image of the external architecture is input into the network, sufficient preprocessing and image enhancement processing are carried out so as to improve the accuracy of judgment of the neural network. Through inspection, the method has high accuracy in classifying and detecting the defects of the duct pieces, overcomes the defects of manual operation, avoids loss caused by misjudgment, has high economic benefit and has high practical application value.
Drawings
FIG. 1 is a schematic diagram of an example system configuration of the present invention;
FIG. 2 is a flow chart of the shield segment surface defect type identification based on deep learning according to the present invention;
FIG. 3 is a flow chart of the shield segment surface defect location based on deep learning of the present invention;
FIG. 4 is a flow chart of shield segment perspective and surface defect rendering of the present invention;
fig. 5 is a schematic diagram of shield segment perspective and surface defect rendering of the present invention.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
The present invention will be further described with reference to the accompanying drawings, and it should be noted that the present embodiment is based on the technical solution, and the detailed implementation and the specific operation process are provided, but the protection scope of the present invention is not limited to the present embodiment.
The invention relates to a shield segment defect type identification and positioning system based on deep learning, which comprises:
the image acquisition module is used for shooting shield segment images from an external framework;
a metadata input module for inputting data describing defect information;
the data synchronization module is used for synchronizing data of the mobile terminal and the server terminal;
an image preprocessing module for preprocessing an image obtained from an external architecture;
the deep learning model training module is used for training a neural network to obtain a weight file;
the deep learning model loading module is used for loading the trained convolutional neural network;
the defect type identification module is used for identifying the defective segment and judging the defect type of the defective segment;
the defect positioning module is used for positioning the defect position of the defective segment;
the information storage module is used for storing the defect information of the segment;
and the data analysis and visualization module is used for comprehensively analyzing the segment defect position and type information, generating a report and visualizing the report in a graphic form.
Examples
Based on the system, the method for identifying and positioning the duct piece defect types comprises the following steps:
step S110: the step inputs the metadata such as ring number, segment type, k point position and the like into the data synchronization module. The ring numbers total No. 10, standard ring one total 7 rings, adjacent ring 2 ring, capping ring 1 ring. The segment types comprise 10 point positions, 16 point positions, 19 point positions, large shields and the like.
Step S120: in the step, an external framework is used for carrying out image acquisition on the wall of the subway shield segment pipe and then inputting the image into a data synchronization module. The external framework can be a mobile phone, a tablet personal computer and the like, and on the basis that the external framework provides support for shooting and the system program is installed, the shooting mode is started from the inside of the system, and the defective parts of the chips can be shot.
Step S130: the steps are used for loading the collected images and the input metadata and then uploading the images and the input metadata to a server.
Step S140: the steps are used for receiving the data uploaded by the mobile equipment, downloading the data to the server and storing the data in the information storage module.
Step S150: the method comprises the following steps of preprocessing a segment defect picture acquired from the outside, and performing data enhancement on image data to acquire more easily processed image data mainly in a conventional random affine transformation mode of the image, wherein the random affine transformation mode comprises one or more of cutting, rotating, scaling, denoising and sharpening.
Step S160: the method comprises the steps of carrying out defect detection on an image characteristic graph, comparing a detection result with a label, carrying out error calculation, updating parameters of the deep convolution network according to errors, stopping training when the errors are smaller than a set error threshold value of the deep convolution network, obtaining the trained deep convolution network, and storing a weight file.
Step S170: the method comprises the step of loading a trained deep learning model stored on a server, wherein the model comprises a defect type identification module and a defect positioning module.
Step S180: the step delivers the preprocessed pictures to a defect type identification module, and the module network finally outputs the identified defect types and classifies the defect types.
Step S190: the step is to deliver the preprocessed picture to a defect positioning module, and the module network finally outputs the position of the defect in the picture. And then calculating the relative distance and angle between the positioning defect and the bolt hole according to the approximate position of the positioning defect and the bolt hole so as to position the specific position of the defect on the tube piece.
Step S1100: and delivering the defect type information and the defect positioning information output by the two networks to an information storage module for storing data and synchronizing the data with the mobile equipment.
Step S1110: the information in the information storage module is summarized and summarized to obtain information useful for users, various kinds of information of the duct piece defects are visualized to the users according to the requirements of the users using the system, evaluation and summary are provided for the users, and column diagrams, pie charts, report forms and other forms of display are visually provided.
Further, the segment defect type identification in step S180 is implemented by using a ResNet50-FPN-RPN network structure, and the specific steps are as follows:
step S210: and collecting the segment defects through an image collecting device and marking the defect objects to establish a defect data set. The defect data set contains a large number of complete defect surface image types including dislocation (three types including previous ring high, previous ring low and ring dislocation), fragmentation, leakage, cracks (two types including transverse cracks and longitudinal cracks) and the like.
Step S220: initializing training parameters, loading a pre-training network model and weights, and setting defect classification types and quantity. Preset ofAnd (5) determining the training times, and obtaining the trained network weight file under the condition of ensuring to meet the constraint conditions of a loss function and the like. The loss constraint is L ═ Lcls+Lbbox+Lmask。Wherein L isclsTo classify the loss, LbboxTo return loss, LmaskIs a segmentation loss.
Step S230: and editing a training data set and a test set, wherein the training data set comprises a defect image, an image file path and a defect type, and performing primary secondary classification on the image acquired by the image acquisition terminal by adopting a pre-trained convolutional neural network, namely acquiring a result set of the defect image and the non-defect image.
Step S240: and screening images with defects in the binary classification result set, and performing fine classification operation in the subsequent steps and removing the images without defects.
Step S250: the network implementation adopts a ResNet50 network architecture as a feature extractor to extract feature maps. Meanwhile, a feature pyramid network FPN extension trunk network is added, so that specific features are not lost while the receptive field is enlarged, and multi-scale information of segment defects is obtained better.
Step S260: and (3) searching candidate ROIs by adopting a lightweight region suggestion network RPN, searching a target region, simultaneously scanning each region by using an anchor, finally selecting the anchor with the highest score, namely the position where the defect is most likely to be located according to the specific score of each anchor, and transmitting the anchor to the next stage.
Step S270: because the classifier can not process the input with different sizes well, but only can process the fixed input size, and the ROI frame can be any size, the ROI pooling operation needs to be carried out on the transmitted ROI. Then, the data is input into a classifier to be classified into N classes (N is the number of defect types). The specific method of bilinear interpolation is as follows: if we want to get the value of the unknown function f at point P ═ x, y, we assume that we know the function f at Q11＝(x1,y1)、Q12＝(x1,y2),Q21＝(x2,y1) And Q22＝(x2,y2) The values of the four points, the final result of bilinear interpolation is:
step S280: and storing the predicted defect type information output by the neural network into an information storage module, and delivering the information to a server to maintain data for a user to use.
Further, the segment defect location in step S190 is implemented by using a YOLO _ V3 network. The method comprises the following specific steps:
step S310: and (3) acquiring the segment defects through an external acquisition device, marking the detailed coordinate positions of the defect objects, numbering the defect objects, and establishing a defect position data set from 1. Labeling the image by using labelImg software, labeling the position of a defect in the image by using a rectangular real frame, and recording the upper left corner (x) of the rectangular frameL,yL) And the lower right corner (x)R,yR) The coordinate information of (2) is marked by adopting a dense marking method in the marking process, namely, each individual defect is marked.
Step S320: initializing parameters of the defect localization convolutional neural network, and training the network by utilizing the established data set to generate a weight file.
Step S330: receiving a segment defect map which is acquired and preprocessed by an image acquisition terminal, calling a defect positioning convolutional neural network which is trained in advance, setting network parameters and loading the trained network weights.
Step S340: since the YOLO _ V3 network requires that the input image size is an integer multiple of 32, the designed network selects the input image size of 416 × 416, and the processing method is to unify the size of the images by using a bilinear interpolation method.
Step S350: and (4) normalizing the serialized images, and normalizing the RGB color modes of 0-255 to black-white binary modes of 0-1, so that the subsequent processing of a neural network is facilitated. The normalized formula is:
step S360: and dividing the processed image into S multiplied by S grids, and detecting the object only when the center of the target defect falls into a certain grid unit by the grid unit, thereby obtaining the position detection result of the target defect. Each grid predicts a fixed number of prediction boxes, which contain 5 basic data of (x, y, w, h, cre), (x, y) is the offset of the center of the prediction box relative to the current grid, and (w, h) is the length and width of the prediction box. The value of cre reflects whether the bounding box contains the target probability and the current bounding box coincides with the real bounding box. And finally, calculating the relative distance and angle between the positioning defect prediction frame and the bolt hole according to the approximate positions of the positioning defect prediction frame and the bolt hole so as to position the specific position of the defect on the tube piece. Design of the loss function: loss is Lc + Lp, where Lc represents the loss function with defect free classification, responsible for the accuracy of the classification. Lp represents a loss function of defect localization, responsible for the accuracy of defect localization. In the invention, Lp is 1-I0U, wherein IOU is an intersection ratio, namely the ratio of the intersection area of the prediction result and the position of the real target to the combination area of the prediction result and the real target.
Step S370: and storing the predicted defect position information output by the neural network into an information storage module, and delivering the information to a server to maintain data for a user to use.
Step S1110: and summarizing the information in the information storage module to obtain useful information for users, visualizing various types of information of the segment defects to the users according to the requirements of the users using the system, and providing evaluation and summary for the users. Mainly need according to section of jurisdiction type, K piece position, section of jurisdiction defect type and position come the rendering convenient to user's perspective of watching, specific flow is as follows specifically:
step S410: rendering a segment perspective view according to a segment type and K block positions, wherein the segment type comprises: 10-point, 16-point, 19-point and big shield, wherein the lining of 10-point is K, B, A1, A2, A3 and C in sequence. The 16-point linings are K, B1, A1, A2, A3 and B2 in sequence; the lining of 19-point position is F, L1, B1, B2, B3, B4 and L2 in sequence, and the lining of large shield position is F, L1, B1, B2, B3, B4, B5, B6, B7 and L2 in sequence. The angles of lining of different segment types are as follows (the K/F block positions can be located at different positions according to the segment type and the number of segment pushing rings):
step S420: and rendering the segment connecting bolt according to the segment type and the K block position. The segment connecting bolt comprises a longitudinal bolt and a transverse bolt, the longitudinal bolt is used for connecting shield segments of the front ring and the rear ring, and the transverse bolt is used for connecting adjacent segments in the same ring.
Step S430: rendering a defect legend according to the quality defect type and the position of the duct piece, wherein different defect types are rendered by different legends, and the method specifically comprises the following steps: leaks, cracks, dislocations, etc. The approximate location of the defect is calculated and rendered based on the segment in which the defect is located and the relative location of the segment to the bolt. The rendering effect is schematically shown in FIG. 5.
Further, the information storage module is used for storing the segment defect information. The storage module is used for storing a plurality of segment expansion diagrams, is connected with the data analysis and visualization module, and is used for acquiring and displaying the segment expansion diagram to be recorded from the storage module according to a segment selection instruction of a user, and storing modified new data to the information storage module again according to different modification operations of the user.
The defect information comprises defect reporting time, interval, line, ring number, lining block, defect type, reporting personnel, remarks and picture information. The segment expansion diagram comprises 10 circumferential point positions, 16 circumferential point positions, 19 circumferential point positions of the segment and expansion diagrams of a large shield, a rectangular shield and the like. The plurality of segment development drawings comprise original segment development drawings of a plurality of ring numbers in tunnel construction engineering. The segment selection instruction comprises the ring number of the segment, the K block point position and the type of the segment ring and shield tail clearance information. The visualization module can display the segment expansion diagram corresponding to the specific ring number.
The data analysis and visualization module is used for displaying the integral information of all the segments to a user. The data analysis includes three basic functions: a defect record management function, a defect position distribution management function and a quality report management function. The defect record management function is used for acquiring segment defect information corresponding to a first query instruction from the information storage module according to the first query instruction input by a user and generating a defect record list; the defect position distribution management function is used for acquiring segment defect information corresponding to a second query instruction from the information storage module according to the second query instruction input by a user, carrying out statistical analysis on the segment defect information, and generating a defect position distribution statistical table; and the quality report management function is used for acquiring segment defect information corresponding to a third query instruction from the information storage module according to the third query instruction input by the user, performing statistical analysis, and generating a quality report. The third query instruction comprises a daily report query instruction and a monthly report query instruction, and the quality report comprises a quality daily report and a quality monthly report. The various recording lists provide clear forms such as column diagrams, pie charts and the like to be displayed to a user, so that the user can more conveniently perform comprehensive analysis on the overall situation of the defects of the chips.
The data analysis and visualization module may export the plurality of defect records in the various types of record lists selected by the user as EXCEL, WORD, TXT or PDF documents, and simultaneously support the plurality of defect records in the record lists selected by the user to be sent to an external printer, so that the external printer prints the plurality of records in the defect record lists selected by the user.
Various modifications may be made by those skilled in the art based on the above teachings and concepts, and all such modifications are intended to be included within the scope of the present invention as defined in the appended claims.
Claims (6)
1. The utility model provides a shield constructs section of jurisdiction defect type discernment and positioning system based on degree of deep learning, its characterized in that, the system includes:
the image acquisition module is used for shooting shield segment images from an external framework;
a metadata input module for inputting data describing defect information;
the data synchronization module is used for synchronizing data of the mobile terminal and the server terminal;
an image preprocessing module for preprocessing an image obtained from an external architecture;
the deep learning model training module is used for training a neural network to obtain a weight file;
the deep learning model loading module is used for loading the trained convolutional neural network;
the defect type identification module is used for identifying the defective segment and judging the defect type of the defective segment;
the defect positioning module is used for positioning the defect position of the defective segment;
the information storage module is used for storing the defect information of the segment;
and the data analysis and visualization module is used for comprehensively analyzing the segment defect position and type information, generating a report and visualizing the report in a graphic form.
2. The method for identifying and positioning the type of the defect of the shield segment based on the deep learning of claim 1, wherein the method comprises the following steps:
s110, inputting metadata such as ring numbers, segment types, k point positions and the like into a data synchronization module;
s120, acquiring an image of the wall of the subway shield segment pipe by an external framework, and inputting the image into a data synchronization module;
s130, loading the metadata input in the step S1 and the image acquired in the step S2, and uploading the metadata and the image to a server;
s140, receiving the data uploaded by the mobile equipment, downloading the data to a server, and storing the data in an information storage module;
s150, preprocessing a segment defect picture acquired from the outside, and performing data enhancement on image data in a traditional image random affine transformation mode to obtain image data which is easier to process;
s160, carrying out defect detection on the image characteristic graph, comparing the obtained detection result with the label, carrying out error calculation, updating the parameter of the deep convolutional network according to the error, and stopping training when the error is smaller than the set error threshold value of the deep convolutional network to obtain the trained deep convolutional network;
s170, loading the trained deep learning model stored on the server, wherein the deep learning model comprises a defect type identification module and a defect positioning module;
s180, delivering the preprocessed pictures to a defect type identification module, outputting the identified defect types, and classifying the defect types;
s190, delivering the preprocessed picture to a defect positioning module, and outputting the position of the defect in the picture; then, calculating the relative distance and angle between the positioning defect and the approximate position of the bolt hole according to the positioning defect and the approximate position of the bolt hole so as to determine the specific position of the defect on the duct piece;
s1100, delivering the defect type information and the defect positioning information output by the network to an information storage module for storing data and synchronizing the data with the mobile equipment;
s1110, summarizing and summarizing the information in the information storage module to obtain information useful for a user, visualizing various types of information of the segment defects to the user according to the requirements of the user using the system, and providing assessment and summary for the user.
3. The method for identifying and positioning the defect type of the shield segment based on the deep learning of claim 2, wherein the defect type identification module in the step S8 is implemented by using a ResNet50-FPN-RPN network structure, and comprises the following steps:
s210, acquiring a segment defect image through an image acquisition device, marking the defect type of a defect object, and establishing a defect data set of various defects;
s220, initializing training parameters, loading a pre-training network model and weights, and setting defect classification types and number; presetting training times, and obtaining a trained network weight file under the condition of ensuring to meet constraint conditions such as a loss function and the like;
s230, editing a training data set and a test set, wherein the training data set comprises a defective image, an image file path and a defect type, and performing primary secondary classification on an image acquired by an image acquisition terminal by adopting a pre-trained convolutional neural network to obtain a result set of the defective image and a result set of a non-defective image;
s240, screening the images with defects in the binary classification result set, and performing fine classification operation in the subsequent steps and removing the images without defects;
s250, extracting feature map by using a ResNet50 network architecture as a feature extractor; meanwhile, a characteristic pyramid network FPN is added to expand a backbone network, so that multi-scale information of segment defects is obtained;
s260, a light-weight region suggestion network RPN is adopted to search candidate ROIs, each scanned region is an anchor while a target region is searched, and finally the anchor with the highest score, namely the position where the defect is most likely to be located, is selected according to the specific score of each anchor and transmitted to the next stage;
s270, performing ROI pooling operation on the transmitted ROI, sampling at different points of the feature map by ROIAlign operation provided in Mask RCNN, and finally obtaining the feature map with a fixed size by applying a bilinear interpolation method; then inputting the defect type of the image to a classifier for classifying the defect type;
s280, the predicted defect type information output by the neural network is stored in an information storage module and is delivered to a server to maintain data for a user to use.
4. The method for identifying and positioning the shield segment defect type based on the deep learning of claim 2, wherein in the step S9, a YOLO _ V3 network is adopted for positioning the segment defects; the method comprises the following steps:
s310, acquiring the segment defects through an external acquisition device, marking the detailed coordinate positions of the defect objects, numbering the detailed coordinate positions, and establishing a defect position data set from 1;
s320, initializing parameters of the defect positioning convolution neural network, and training the network by using the established data set to generate a weight file;
s330, receiving a segment defect map which is acquired and preprocessed by an image acquisition terminal, calling a defect positioning convolutional neural network which is trained in advance, setting network parameters and loading the trained network weights;
s340, selecting the size of an input image of the design network to be 416 multiplied by 416, and carrying out uniform size on the image by adopting a bilinear interpolation method;
s350, normalizing the serialized images, and normalizing the RGB color modes of 0-255 to black and white binary modes of 0-1;
s360, dividing the processed image into S multiplied by S grids, and detecting the object by the grid unit only when the center of the target defect falls into a certain grid unit, so as to obtain the position detection result of the target defect; predicting a fixed number of prediction frames by each grid, and finally calculating the relative distance and angle between the prediction frames and the approximate positions of the bolts according to the prediction frames for positioning the defects and the approximate positions of the bolts so as to position the specific positions of the defects on the duct piece;
s370, the predicted defect position information output by the neural network is stored in the information storage module and is delivered to the server to maintain data for users to use.
5. The method for identifying and positioning the defect type of the shield segment based on the deep learning of claim 4, wherein the prediction frame comprises 5 basic data of (x, y, w, h, cre); wherein (x, y) is the offset of the center of the prediction box with respect to the current mesh and (w, h) is the length and width of the prediction box; the value of cre reflects whether the bounding box contains the target probability and the current bounding box coincides with the real bounding box.
6. The method for identifying and positioning the shield segment defect type and system based on the deep learning of claim 2, wherein the visualization of the segment defects in the step S11 requires rendering a perspective view convenient for a user to view according to the segment type, the K block position, the segment defect type and the position, and comprises the following steps:
s410, rendering a segment perspective view according to a segment type and K block positions, wherein the segment type comprises: 10 point location, 16 point location, 19 point location, large shield and the like, wherein the K block can be positioned at different positions according to the type of the segment and the number of segment pushing rings;
s420, rendering the segment connecting bolt according to the segment type and the K block position; the segment connecting bolt comprises a longitudinal bolt and a transverse bolt, the longitudinal bolt is used for connecting shield segments of the front ring and the rear ring, and the transverse bolt is used for connecting adjacent segments in the same ring;
s430, rendering a defect legend according to the quality defect type and the position of the duct piece, wherein different defect types are rendered by different legends, and the method specifically comprises the following steps: leakage, cracks, slab staggering, etc.; the approximate location of the defect is calculated and rendered based on the segment in which the defect is located and the relative location of the segment to the bolt.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110255249.7A CN112967255A (en) | 2021-03-09 | 2021-03-09 | Shield segment defect type identification and positioning system and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110255249.7A CN112967255A (en) | 2021-03-09 | 2021-03-09 | Shield segment defect type identification and positioning system and method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112967255A true CN112967255A (en) | 2021-06-15 |
Family
ID=76277240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110255249.7A Pending CN112967255A (en) | 2021-03-09 | 2021-03-09 | Shield segment defect type identification and positioning system and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112967255A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112967221A (en) * | 2020-12-04 | 2021-06-15 | 江苏龙冠新型材料科技有限公司 | Shield constructs section of jurisdiction production and assembles information management system |
| CN113450357A (en) * | 2021-09-01 | 2021-09-28 | 南昌市建筑科学研究所（南昌市建筑工程质量检测中心） | Segment image online analysis subsystem and subway shield detection system |
| CN113935982A (en) * | 2021-10-27 | 2022-01-14 | 征图新视(江苏)科技股份有限公司 | Printing quality detection and analysis system based on deep learning |
| CN113989255A (en) * | 2021-11-05 | 2022-01-28 | 中国地质大学（北京） | Subway tunnel lining shedding recognition model training method and recognition method based on Mask-RCNN |
| CN116188877A (en) * | 2023-04-23 | 2023-05-30 | 昆山润石智能科技有限公司 | Method and system for detecting and classifying unknown wafer defect categories |
| CN116758073A (en) * | 2023-08-17 | 2023-09-15 | 粤芯半导体技术股份有限公司 | Mask plate data detection method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106022560A (en) * | 2016-05-05 | 2016-10-12 | 四川铁安达科技有限公司 | Subway shield segment attitude monitoring and management system based on Internet of Things and cloud computing |
| CN107818372A (en) * | 2017-11-30 | 2018-03-20 | 广州轨道交通建设监理有限公司 | A kind of shield duct piece quality cruising inspection system |
| CN110220909A (en) * | 2019-04-28 | 2019-09-10 | 浙江大学 | A kind of Shield-bored tunnels Defect inspection method based on deep learning |
| CN112446852A (en) * | 2019-08-30 | 2021-03-05 | 成都唐源电气股份有限公司 | Tunnel imaging plane display method and intelligent defect identification system |
-
2021
- 2021-03-09 CN CN202110255249.7A patent/CN112967255A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106022560A (en) * | 2016-05-05 | 2016-10-12 | 四川铁安达科技有限公司 | Subway shield segment attitude monitoring and management system based on Internet of Things and cloud computing |
| CN107818372A (en) * | 2017-11-30 | 2018-03-20 | 广州轨道交通建设监理有限公司 | A kind of shield duct piece quality cruising inspection system |
| CN110220909A (en) * | 2019-04-28 | 2019-09-10 | 浙江大学 | A kind of Shield-bored tunnels Defect inspection method based on deep learning |
| CN112446852A (en) * | 2019-08-30 | 2021-03-05 | 成都唐源电气股份有限公司 | Tunnel imaging plane display method and intelligent defect identification system |
Non-Patent Citations (1)
| Title |
|---|
| 董洪义: "《深度学习之PyTorch物体检测实战》", 31 March 2020 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112967221A (en) * | 2020-12-04 | 2021-06-15 | 江苏龙冠新型材料科技有限公司 | Shield constructs section of jurisdiction production and assembles information management system |
| CN112967221B (en) * | 2020-12-04 | 2024-05-14 | 江苏龙冠新型材料科技有限公司 | Shield segment production and assembly information management system |
| CN113450357A (en) * | 2021-09-01 | 2021-09-28 | 南昌市建筑科学研究所（南昌市建筑工程质量检测中心） | Segment image online analysis subsystem and subway shield detection system |
| CN113450357B (en) * | 2021-09-01 | 2021-12-17 | 南昌市建筑科学研究所（南昌市建筑工程质量检测中心） | Segment image online analysis subsystem and subway shield detection system |
| CN113935982A (en) * | 2021-10-27 | 2022-01-14 | 征图新视(江苏)科技股份有限公司 | Printing quality detection and analysis system based on deep learning |
| CN113989255A (en) * | 2021-11-05 | 2022-01-28 | 中国地质大学（北京） | Subway tunnel lining shedding recognition model training method and recognition method based on Mask-RCNN |
| CN116188877A (en) * | 2023-04-23 | 2023-05-30 | 昆山润石智能科技有限公司 | Method and system for detecting and classifying unknown wafer defect categories |
| CN116188877B (en) * | 2023-04-23 | 2023-08-04 | 昆山润石智能科技有限公司 | Method and system for detecting and classifying unknown wafer defect categories |
| CN116758073A (en) * | 2023-08-17 | 2023-09-15 | 粤芯半导体技术股份有限公司 | Mask plate data detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210319561A1 (en) | Image segmentation method and system for pavement disease based on deep learning | |
| CN112967255A (en) | Shield segment defect type identification and positioning system and method based on deep learning | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN111881730A (en) | Wearing detection method for on-site safety helmet of thermal power plant | |
| US20030156739A1 (en) | Rock fragmentation analysis system | |
| CN106610969A (en) | Multimodal information-based video content auditing system and method | |
| CN111967313B (en) | Unmanned aerial vehicle image annotation method assisted by deep learning target detection algorithm | |
| CN109800698A (en) | Icon detection method based on depth network | |
| CN110781882A (en) | License plate positioning and identifying method based on YOLO model | |
| CN110910360B (en) | Positioning method of power grid image and training method of image positioning model | |
| CN113435407B (en) | Small target identification method and device for power transmission system | |
| CN112528979B (en) | Transformer substation inspection robot obstacle distinguishing method and system | |
| CN111339902A (en) | Liquid crystal display number identification method and device of digital display instrument | |
| CN113033516A (en) | Object identification statistical method and device, electronic equipment and storage medium | |
| CN113255590A (en) | Defect detection model training method, defect detection method, device and system | |
| CN110728269B (en) | High-speed rail contact net support pole number plate identification method based on C2 detection data | |
| CN113077438B (en) | Cell nucleus region extraction method and imaging method for multi-cell nucleus color image | |
| CN113487570A (en) | High-temperature continuous casting billet surface defect detection method based on improved yolov5x network model | |
| CN112418033B (en) | Landslide slope surface segmentation recognition method based on mask rcnn neural network | |
| CN112966698A (en) | Freshwater fish image real-time identification method based on lightweight convolutional network | |
| CN111738264A (en) | Intelligent acquisition method for data of display panel of machine room equipment | |
| CN116385758A (en) | Detection method for damage to surface of conveyor belt based on YOLOv5 network | |
| CN114863274A (en) | Surface green net thatch cover extraction method based on deep learning | |
| CN106886609B (en) | Block type rural residential area remote sensing quick labeling method | |
| CN112861861B (en) | Method and device for recognizing nixie tube text and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210615 |