CN114627106A - Weld defect detection method based on Cascade Mask R-CNN model - Google Patents

Weld defect detection method based on Cascade Mask R-CNN model Download PDF

Info

Publication number
CN114627106A
CN114627106A CN202210349016.8A CN202210349016A CN114627106A CN 114627106 A CN114627106 A CN 114627106A CN 202210349016 A CN202210349016 A CN 202210349016A CN 114627106 A CN114627106 A CN 114627106A
Authority
CN
China
Prior art keywords
training
defect detection
weld defect
feature
weld
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210349016.8A
Other languages
Chinese (zh)
Inventor
梁丽红
陈赡舒
郭文明
代淮北
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Special Equipment Inspection and Research Institute
Original Assignee
China Special Equipment Inspection and Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Special Equipment Inspection and Research Institute filed Critical China Special Equipment Inspection and Research Institute
Priority to CN202210349016.8A priority Critical patent/CN114627106A/en
Publication of CN114627106A publication Critical patent/CN114627106A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30152—Solder

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

A weld defect detection method based on a Cascade Mask R-CNN model comprises the steps of firstly, obtaining a weld ray detection digital image containing five defects of round defects, strip defects, cracks, incomplete penetration and incomplete fusion and a corresponding label file thereof, and dividing the weld ray detection digital image into a training set, a verification set and a test set; and then training a weld defect detection model which is built and optimized based on a Cascade Mask R-CNN model by using a training set after image preprocessing, and testing a test set after acquiring an optimal weight file by using a verification set so as to evaluate the performance of the weld defect detection model. The method improves the problem of low precision of the target detection model based on a single threshold value and the problem of overfitting caused by directly improving the threshold value, and improves the detection precision of the weld defects.

Description

Weld defect detection method based on Cascade Mask R-CNN model
Technical Field
The invention relates to the field of target detection, in particular to a weld defect detection method based on a Cascade Mask R-CNN model.
Background
Welding is one of the most important technologies in industrial production and manufacturing processes, and nondestructive testing needs to be performed on a welding seam in order to ensure the welding quality, wherein ray detection is one of the commonly used nondestructive testing technologies. At present, the defect qualification and the positioning of the ray detection mainly depend on manual evaluation, namely manual film evaluation. The manual film evaluation is greatly influenced by subjective factors such as professional level and self condition of film evaluation personnel, and the efficiency is low, so that the automatic and intelligent detection requirements of modern industry cannot be met.
In recent years, the development of deep learning not only breaks through a plurality of difficult-to-solve visual problems, improves the level of image cognition, but also accelerates the progress of related technologies in the field of target detection, and the method for automatically learning image features in deep learning is applied to the mainstream research direction in the detection of the weld defects of industrial products. The target detection model based on deep learning is mainly divided into a one-stage target detection model and a two-stage target detection model, wherein the one-stage target detection model directly predicts the category and the position of a target by using a convolutional neural network to realize classification and regression in one step, and the models have the characteristics of high speed and low precision; the two-stage target detection model is characterized in that a suggested candidate frame which possibly contains a target is generated firstly, and then classification and regression are carried out through a prediction network. In addition, when the intersection ratio (IoU) of the proposed candidate frame and the real labeling frame is calculated by a common target detection model, the proposed candidate frame is divided into positive and negative samples by comparing the intersection ratio (IoU) with a set single threshold, usually the positive sample is far larger than the negative sample, the positive and negative samples are sampled in a test stage to enable the proportion of the positive and negative samples to meet a certain ratio, but the quality of the proposed candidate frame is low and the detection precision is low due to the fact that the real labeling frame is not compared in the test stage; therefore, a model with a cascade structure is needed to break through the bottleneck of the detection precision of the weld defects under a single threshold value, so that a sample can be closer to the real position of the weld defects after each regression of the model, and the distribution of different suggested candidate frames is adapted.
Disclosure of Invention
The invention aims to provide a weld defect detection method based on a Cascade Mask R-CNN model, so that the problems in the prior art are solved.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a weld defect detection method based on a Cascade Mask R-CNN model comprises the following steps:
s1, acquiring a welding seam radiographic inspection digital image containing five defects of round defect, strip defect, crack, incomplete penetration and incomplete fusion and a corresponding label file thereof; dividing the welding seam ray detection digital image and the corresponding label file into a training set, a verification set and a test set;
s2, carrying out image preprocessing on the welding line ray detection digital image to obtain an enhanced and unified image;
s3, constructing and optimizing a weld defect detection model based on a Cascade Mask R-CNN model, wherein the weld defect detection model comprises a convolutional neural network for feature extraction and a prediction network for classification and regression;
s4, training the weld defect detection model by using the training set, and verifying the weight file obtained by each training by using the verification set to obtain the weight file which is optimal to be expressed on the verification set; the method comprises the following steps:
s41, performing feature extraction on the welding line ray detection digital image through the convolutional neural network to form a feature map;
s42, improving the RPN by utilizing a multi-scale detection algorithm (FPN) to generate a suggested candidate box for the feature map;
s43, mapping the suggested candidate box to the feature map to obtain a corresponding feature matrix, and uniformly scaling the feature matrix to a specified size through ROI Align;
s44, obtaining corresponding defect categories and boundary box regression parameters under three set thresholds through a three-level cascade detector consisting of a target classifier and a boundary box regressor by the suggested candidate box; finally, a final detection result is obtained by non-maximum value inhibition and low probability target filtering;
s45, setting training parameters of the weld defect detection model, training the weld defect detection model through the steps S41-S44 by adopting the training set, verifying the weight file obtained by each training by adopting the verification set, and acquiring the weight file which is optimal to be expressed on the verification set;
and S5, testing the weight file which is obtained by applying the step S45 and has the optimal performance on the verification set by using the test set, so as to evaluate the performance of the weld defect detection model.
Preferably, the image preprocessing method in step S2 includes image enhancement and image denoising, where the image enhancement employs an AHE algorithm to perform image enhancement on the weld ray detection digital image, and performs detail sharpening on the weld ray detection digital image to highlight the defect feature, and the calculation formula of the image enhancement is as follows:
Figure BDA0003578531740000031
in the above formula: y isi,jRepresenting the central pixel before transformation, Yi,jRepresenting the transformed central pixel, mi,jIs expressed as yi,jThe gray level mean value of a local area of a central point, T represents a cumulative distribution transformation function of the point, k represents an adaptive function and is obtained by pixel characterization calculation of the local area;
and the image denoising adopts a DMB algorithm to perform image denoising on the welding line ray detection digital image, so that the image noise is reduced, and the defect characteristics are reserved and enhanced.
Preferably, the convolutional neural network in step S41 is resenex-101, and the convolutional neural network includes a convolutional layer, a pooling layer, and an activation layer, and the convolutional layer extracts features from the input image to generate a feature map; the pooling layer is used for removing redundant information, reducing the number of parameters and expanding an acceptance domain; the activation layer increases the output nonlinearity, and the result of the output layer is convolved by using the activation function to obtain nonlinear mapping.
Preferably, the step S42 of generating the candidate suggestion box for the feature map by using the FPN to improve the RPN includes the following steps:
s421: feeding forward a part of ResNeXt-101 in the forward process of the convolutional neural network, and recording the output of the last residual block of each stage of ResNeXt-101 as { C1, C2, C3, C4 and C5}, firstly, performing a bottom-up process, and reducing the sampling of a set step length upwards at each stage to form a stage without changing the size of a feature map so as to form a feature pyramid;
s422: the small feature map of the top layer is enlarged to the same size as the feature map of the last stage from top to bottom in an up-sampling mode;
s423: the cross-concatenation fuses the upsampled results with feature maps of the same size generated from bottom to top, and convolves each fused result with a 3 × 3 convolution kernel to obtain the final feature level P ═ P2, P3, P4, P5 }.
Preferably, the step S421 further includes: a high efficiency attention module was introduced in { C3, C4, C5} of ResNeXt-101, first using a 1 × 1 convolution WkAnd softmax obtains attention weight, acquires global context features by attention pooling, and then by a 1 × 1 convolution Wv1After normalization of the sum layer, the sum layer is activated by the ReLU function and is subjected to a convolution W of 1 × 1v2Obtaining the importance degree of each channel, and finally utilizing addition to aggregate the global context characteristics to the characteristics of each position to form a long-distance dependency relationship; the attention module calculation formula is as follows:
Figure BDA0003578531740000041
in the above formula: z is a radical ofiIndicating the input of the attention module, ZiIndicating the output of the attention module, NpExpressed as a number of locations in the feature map,
Figure BDA0003578531740000042
weights representing global attention pooling, LN stands for layer normalization, Wv2ReLU(LN(Wv1(.))) represents a calculationThe importance of each channel.
Preferably, after step S423, the method further includes: the area and the length-width ratio of a rectangular marking box in the marking file are counted through a K-means clustering algorithm, and five areas {32 } are set2,642,1282,2562,5122The anchors of the areas correspond to five feature layers of { P2, P3, P4, P5 and P6} respectively, wherein the P6 feature layer is obtained by downsampling the P5 feature layer, seven aspect ratios of {1:10,1:5,1:2,1:1,2:1,5:1 and 10:1} are set for the anchors of each area, and the generated anchors are traversed on the feature layers in a sliding mode to generate the suggested candidate frames.
Preferably, the boundary box regressor in the three-stage cascade detector in step S44 is defined as a cascade regression problem, the cascade regression changes the sample distribution to be processed in different stages by resampling, and the boundary box regressor is defined as follows:
Figure BDA0003578531740000043
in the above formula: x denotes the subimage block, b denotes the sample distribution, f denotes the bounding box regressor, f1、f2、f3The set threshold values are 0.4, 0.5, 0.6, respectively, f1As the output of f2Input of f2As the output of f3Input of { f }1,f2,f3Optimizing the resampling distribution of different stages, and simultaneously acting on training and testing stages;
the total loss function L defining the training model comprises two parts: the total loss function calculation formula comprises a bounding box regression loss and a target classification loss, wherein the total loss function calculation formula comprises the following steps:
L(xt,g)＝Lcls(ht(xt),Xt)+μ[Xt≥1]Lloc(ft(xt,bt),g)
in the above formula: l is a radical of an alcoholclsLoss function, L, representing the classification of objectslocLoss function representing regression of bounding box, { btDenotes the sample distribution for different training phases t andhas b at＝ft-1(xt-1,bt-1)，htRepresenting the object classifier, ftRepresenting a bounding box regressor, g representing the correspondence xtMu represents a compromise coefficient, XtDenotes xtCorresponding label [. C]An indicator function is represented.
Preferably, the training parameter settings in step S45 include values of learning rate, momentum, weight attenuation, batch _ size, and total number of training rounds, training weights are used as pre-training weights of the weld defect detection model, the weld ray detection digital image is scaled uniformly before entering the weld defect detection model, and parameters of the weld defect detection model are updated by using a stochastic gradient descent method.
The invention has the beneficial effects that: the invention discloses a weld defect detection method based on a Cascade Mask R-CNN model, which solves the problems of low precision of a target detection model based on a single threshold and overfitting caused by directly improving the threshold by introducing a three-level Cascade structure; RPN is improved through a multi-scale detection algorithm FPN, and low-layer feature information with high resolution and high-layer feature information with high semantics are fused; by introducing the high-efficiency attention module into the Cascade Mask R-CNN model, on the basis of simplifying the calculation amount of the Non-Local attention mechanism, a close precision result is maintained, and the original characteristics are strengthened by aggregating the same characteristics at each position of the characteristic diagram. Compared with the existing two-stage target detection model and the original Cascade Mask R-CNN model, the method provided by the invention has the advantage that the detection precision of the weld defects is obviously improved.
Drawings
FIG. 1 is a flowchart of a weld defect detection method based on a Cascade Mask R-CNN model;
FIG. 2 is a collected weld ray inspection digital image defect portion;
FIG. 3 is a schematic diagram of a RPN modification using FPN;
FIG. 4 is a schematic diagram of a high efficiency attention module;
FIG. 5 is a schematic structural diagram of a Cascade Mask R-CNN model;
figure 6 is a graph relating loss function values to the number of iterations in the training process.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
A weld defect detection method based on a Cascade Mask R-CNN model comprises the steps of firstly, dividing a weld ray detection digital image containing five defects of round defects, strip defects, cracks, incomplete penetration and incomplete fusion and a corresponding label file thereof into a training set, a verification set and a test set; and training a weld defect detection model which is built and optimized based on a Cascade Mask R-CNN model by using a training set after image preprocessing, and testing a test set after acquiring an optimal weight file by using a verification set so as to evaluate the performance of the weld defect detection model. A method for performing weld ray detection based on a Cascade Mask R-CNN model is shown in FIG. 1 and specifically comprises the following steps:
s1, acquiring a welding seam ray detection digital image containing five defects of a circular defect, a strip defect, a crack, incomplete penetration and incomplete fusion and a corresponding label file thereof; dividing the welding seam ray detection digital image and the corresponding label file into a training set, a verification set and a test set;
in the embodiment, 2600 metal weld ray detection digital images are selected in total, the pixel size of the weld image is 400 × 3050, the defect types of the weld image are 5, namely, circle defect, strip defect, crack, incomplete penetration and incomplete fusion, and various defects in the weld ray detection digital images are shown in fig. 2 and are according to 9: 1, randomly dividing all the weld ray detection digital images and the corresponding label files into a training set and a test set, and randomly dividing 10% of data from the training set as a verification set.
S2, carrying out image preprocessing on the welding line ray detection digital image to obtain an enhanced and unified image; the image preprocessing method comprises image enhancement and image denoising, wherein the image enhancement adopts an AHE algorithm to carry out image enhancement on the welding line radiographic inspection digital image, the detail sharpening is carried out on the welding line radiographic inspection digital image, the defect characteristics are highlighted, and the image enhancement calculation formula is as follows:
Figure BDA0003578531740000061
in the above formula: y isi,jRepresenting the central pixel before transformation, Yi,jRepresenting the transformed central pixel, mi,jIs expressed as yi,jThe gray level mean value of a local area of a central point, T represents a cumulative distribution transformation function of the point, k represents an adaptive function and is calculated by pixel characteristics of the local area;
and the image denoising adopts a DMB algorithm to perform image denoising on the welding line ray detection digital image, so that the image noise is reduced, and the defect characteristics are reserved and enhanced.
S3, building and optimizing a weld defect detection model based on a Cascade Mask R-CNN original model, wherein the weld defect detection model comprises a convolutional neural network for feature extraction and a prediction network for classification and regression;
the convolutional neural network is ResNeXt-101, the convolutional neural network comprises a convolutional layer, a pooling layer and an activation layer, and the convolutional layer extracts features from the input image to generate a feature map; the pooling layer is used for removing redundant information, reducing the number of parameters and expanding an acceptance domain; the activation layer increases the output nonlinearity, and the result of the output layer is convolved by using the activation function to obtain nonlinear mapping.
The network structure of ResNeXt-101 is shown in Table 1:
table 1: network structure of ResNeXt-101
Figure BDA0003578531740000071
S4, training the weld defect detection model by using the training set, and verifying the weight file obtained by each training by using the verification set to obtain the weight file which is optimal to be expressed on the verification set; the method comprises the following steps:
s41, performing feature extraction on the welding line ray detection digital image through the convolutional neural network to form a feature map;
s42, improving the RPN by utilizing a multi-scale detection algorithm (FPN) to generate a suggested candidate box for the feature map, wherein the transverse connection in the FPN is fused with low-level feature information with high resolution and high-level feature information with high semantic;
the embodiment utilizes a multi-scale detection algorithm FPN to improve RPN to generate the suggested candidate box for the feature map, and comprises the following steps:
s421: feeding forward a part of ResNeXt-101 in the forward process of the network, and keeping the output of the last residual block of each stage of ResNeXt-101 as { C1, C2, C3, C4, C5}, as shown in FIG. 3, firstly, a bottom-up process is carried out, each stage adopts down-sampling with set step size upwards, and the layers without changing the size of the feature map are classified into a stage to form a feature pyramid;
to enhance the original features, an efficient attention module is introduced into { C3, C4, C5} of ResNeXt-101, as shown in FIG. 4, first a convolution W of 1 × 1 is usedkAnd softmax obtains attention weight, acquires global context features by attention pooling, and then by a 1 × 1 convolution Wv1After normalization of the sum layer, the sum layer is activated by the ReLU function and is subjected to a convolution W of 1 × 1v2Obtaining the importance degree of each channel, and finally utilizing addition to aggregate the global context characteristics to the characteristics of each position to form a long-distance dependency relationship; the attention module calculation formula is as follows:
Figure BDA0003578531740000081
in the above formula: z is a radical ofiIndicating the input of the attention module, ZiIndicating the output of the attention module, NpExpressed as a number of locations in the feature map,
Figure BDA0003578531740000082
weights representing global attention pooling, LN stands for layer normalization, Wv2ReLU(LN(Wv1(.))) represents the degree of importance of computing each channel.
S422: the small feature map of the top layer is enlarged to the same size as the feature map of the last stage from top to bottom in an up-sampling mode;
s423: the cross-concatenation fuses the upsampled results with feature maps of the same size generated from bottom to top, and convolves each fused result with a 3 × 3 convolution kernel to obtain the final feature level P ═ P2, P3, P4, P5 }.
After the feature layer P subjected to feature fusion is obtained, the area and the length-width ratio of a rectangular marking box in a marking file are counted through a K-means clustering algorithm, and five areas {32 } are set2,642,1282,2562,5122The anchors of the areas correspond to five feature layers of { P2, P3, P4, P5 and P6} respectively, wherein the P6 feature layer is obtained by downsampling the P5 feature layer, seven aspect ratios of {1:10,1:5,1:2,1:1,2:1,5:1 and 10:1} are set for the anchors of each area, and the generated anchors are traversed on the feature layers in a sliding mode to generate the suggested candidate frames.
S43: mapping the suggested candidate box to the feature map to obtain a corresponding feature matrix, and uniformly scaling the feature matrix to a specified size through ROI Align;
s44: passing the suggested candidate frame through a three-level cascade detector composed of a target classifier and a boundary frame regressor, as shown in fig. 5, to obtain three defect categories and boundary frame regression parameters corresponding to the set threshold values; finally, a final detection result is obtained by non-maximum value inhibition and low probability target filtering;
the boundary box regressor in the three-level cascade detector is defined as a cascade regression problem, the cascade regression changes the sample distribution to be processed in different stages through resampling, and the boundary box regressor is defined as follows:
Figure BDA0003578531740000091
in the above formula: x denotes the subimage block, b denotes the sample distribution, f denotes the bounding box regressor, f1、f2、f3The set threshold values are 0.4, 0.5, 0.6, respectively, f1As output of f2Input of f2As the output of f3Input of (c) { f1,f2,f3The resampling distribution for different phases is optimized and acts on both training and testing phases.
The total loss function L defining the training model comprises two parts: the total loss function calculation formula comprises a bounding box regression loss and a target classification loss, wherein the total loss function calculation formula comprises the following steps:
L(xt,g)＝Lcls(ht(xt),Xt)+μ[Xt≥1]Lloc(ft(xt,bt),g)
in the above formula: l isclsLoss function, L, representing the classification of the objectlocLoss function representing regression of bounding box, { btDenotes the distribution of samples for different training phases t and has bt＝ft-1(xt-1,bt-1)，htRepresenting the object classifier, ftDenotes a bounding box regressor, g denotes the correspondence xtMu represents the compromise coefficient, XtDenotes xtCorresponding label [. C]An indicator function is represented.
S45: setting training parameters of the welding seam defect detection model: the initial learning rate is set to 0.00125, the momentum is set to 0.9, the weight attenuation is set to 1e-4, the value of batch _ size is set to 1, and the total number of rounds of training is set to 40; training weights by utilizing ImageNet data sets to serve as pre-training weights of the weld defect detection model, uniformly scaling the weld ray detection digital image to 350 x 2600 before entering the weld defect detection model, and updating parameters of the weld defect detection model by using a random gradient descent method; training the weld defect detection model by adopting the training set through the steps S41-S44, and verifying the weight file obtained by each training by adopting the verification set to obtain the weight file which is optimal to be expressed on the verification set; the correlation curve of the loss function value and the iteration number in the process of training the weld defect detection model is shown in fig. 6.
And S5, testing the weight file which is obtained by applying the step S45 and has the optimal performance on the verification set by using the test set, so as to evaluate the performance of the weld defect detection model.
In this embodiment, the weld defect detection model and other two-stage target detection models are trained and tested by using a training set and a testing set, and the average Accuracy (AP) of each type of defect and the average value (mAP) of each type of defect AP are introduced as evaluation indexes of this embodiment. The results of the experiment and the comparison are shown in table 2:
table 2: comparison of model test results (IoU ═ 0.5)
Figure BDA0003578531740000101
Analyzing the above experimental results can obtain: compare traditional two-stage target detection model, the welding seam defect detection model described in this embodiment all has obvious promotion about five defect detection's of crackle, strip lack, round lack, incomplete penetration and not fusing AP, and ultimate mAP is also higher than other models.
By adopting the technical scheme disclosed by the invention, the following beneficial effects are obtained:
the invention discloses a weld defect detection method based on a Cascade Mask R-CNN model, which solves the problems of low precision of a target detection model based on a single threshold and overfitting caused by directly improving the threshold by introducing a three-level Cascade structure; RPN is improved through a multi-scale detection algorithm FPN, and low-layer feature information with high resolution and high-layer feature information with high semantics are fused; by introducing the high-efficiency attention module into the Cascade Mask R-CNN model, on the basis of simplifying the calculation amount of the Non-Local attention mechanism, a close precision result is maintained, and the original characteristics are strengthened by aggregating the same characteristics at each position of the characteristic diagram. Compared with the existing two-stage target detection model and the original Cascade Mask R-CNN model, the method provided by the invention has the advantage that the detection precision of the weld defects is obviously improved.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements should also be considered within the scope of the present invention.

Claims (8)

1. A weld defect detection method based on a Cascade Mask R-CNN model is characterized by comprising the following steps:
s1, acquiring a welding seam radiographic inspection digital image containing five defects of round defect, strip defect, crack, incomplete penetration and incomplete fusion and a corresponding label file thereof; dividing the welding seam ray detection digital image and the corresponding label file into a training set, a verification set and a test set;
s2, carrying out image preprocessing on the welding line ray detection digital image to obtain an enhanced and unified image;
s3, building and optimizing a weld defect detection model based on a Cascade Mask R-CNN model, wherein the weld defect detection model comprises a convolutional neural network for feature extraction and a prediction network for classification and regression;
s4, training the weld defect detection model by using the training set, and verifying the weight file obtained by each training by using the verification set to obtain the weight file which is optimal to be expressed on the verification set; the method comprises the following steps:
s41, performing feature extraction on the welding line ray detection digital image through the convolutional neural network to form a feature map;
s42, improving the RPN by utilizing a multi-scale detection algorithm (FPN) to generate a suggested candidate box for the feature map;
s43, mapping the suggested candidate box to the feature diagram to obtain a corresponding feature matrix, and uniformly scaling the feature matrix to a specified size through ROIAlign;
s44, obtaining corresponding defect categories and boundary box regression parameters under three set thresholds through a three-level cascade detector consisting of a target classifier and a boundary box regressor by the suggested candidate box; finally, a final detection result is obtained by non-maximum value inhibition and low probability target filtering;
s45, setting training parameters of the weld defect detection model, training the weld defect detection model through the steps S41-S44 by adopting the training set, verifying the weight file obtained by each training by adopting the verification set, and acquiring the weight file which is optimal to be expressed on the verification set;
and S5, testing the weight file which is obtained by applying the step S45 and has the optimal performance on the verification set by using the test set, so as to evaluate the performance of the weld defect detection model.
2. The Cascade Mask R-CNN model-based weld defect detection method of claim 1, wherein the image preprocessing method in step S2 comprises image enhancement and image denoising, wherein the image enhancement adopts an AHE algorithm to perform image enhancement on the weld ray detection digital image, the weld ray detection digital image is subjected to detail sharpening to highlight defect features, and the image enhancement has the following calculation formula:
Figure FDA0003578531730000021
in the above formula: y isi,jRepresenting the central pixel before transformation, Yi,jRepresenting the transformed center pixel, mi,jIs expressed as yi,jThe gray level mean value of a local area of a central point, T represents a cumulative distribution transformation function of the point, k represents an adaptive function and is obtained by pixel characterization calculation of the local area;
the image denoising adopts a DMB algorithm to perform image denoising on the welding line ray detection digital image, so that the image noise is reduced, and the defect characteristics are reserved and enhanced.
3. The Cascade Mask R-CNN model-based weld defect detection method according to claim 1, characterized in that the convolutional neural network in step S41 is ResNeXt-101, the convolutional neural network comprises a convolutional layer, a pooling layer and an activation layer, and the convolutional layer extracts features from the input image to generate a feature map; the pooling layer is used for removing redundant information, reducing the number of parameters and expanding an acceptance domain; the activation layer increases the output nonlinearity, and the result of the output layer is convolved by using the activation function to obtain nonlinear mapping.
4. The Cascade Mask R-CNN model-based weld defect detection method according to claim 1, wherein in step S42, a proposed candidate frame is generated for the feature map by using FPN to improve RPN, and the method comprises the following steps:
s421: feeding forward a part of ResNeXt-101 in the forward process of the convolutional neural network, and recording the output of the last residual block of each stage of ResNeXt-101 as { C1, C2, C3, C4 and C5}, firstly, performing a bottom-up process, and reducing the sampling of a set step length upwards at each stage to form a stage without changing the size of a feature map so as to form a feature pyramid;
s422: the small feature map of the top layer is enlarged to the same size as the feature map of the last stage from top to bottom in an up-sampling mode;
s423: the cross-concatenation fuses the upsampled results with feature maps of the same size generated from bottom to top, and convolves each fused result with a 3 × 3 convolution kernel to obtain the final feature level P ═ P2, P3, P4, P5 }.
5. The Cascade Mask R-CNN model-based weld defect detection method according to claim 4, wherein the step S421 further comprises: a high efficiency attention module was introduced in { C3, C4, C5} of ResNeXt-101, first using a 1 × 1 convolution WkAnd softmax obtains attention weight, acquires global context features by attention pooling, and then by a 1 × 1 convolution Wv1After normalization of the sum layer, activation by the ReLU function, and a further 1 × 1 convolution Wv2Obtaining the importance degree of each channel, and finally adding the obtained data to the original dataThe local context features are aggregated to the features of each position to form a long-distance dependency relationship; the attention module calculation formula is as follows:
Figure FDA0003578531730000031
in the above formula: z is a radical ofiIndicating the input of the attention module, ZiIndicating the output of the attention module, NpExpressed as a number of locations in the feature map,
Figure FDA0003578531730000032
weights representing global attention pooling, LN stands for layer normalization, Wv2ReLU(LN(Wv1(.))) represents the degree of importance of computing each channel.
6. The Cascade Mask R-CNN model-based weld defect detection method according to claim 4, further comprising, after the step S423: the area and the length-width ratio of a rectangular marking box in the marking file are counted through a K-means clustering algorithm, and five areas {32 } are set2,642,1282,2562,5122The anchors of the areas correspond to five feature layers of { P2, P3, P4, P5 and P6} respectively, wherein the P6 feature layer is obtained by downsampling the P5 feature layer, seven aspect ratios of {1:10,1:5,1:2,1:1,2:1,5:1 and 10:1} are set for the anchors of each area, and the generated anchors are traversed on the feature layers in a sliding mode to generate the suggested candidate frames.
7. The Cascade Mask R-CNN model-based weld defect detection method of claim 1, wherein a boundary box regressor in the three-level Cascade detector in the step S44 is defined as a Cascade regression problem, the Cascade regression changes the sample distribution to be processed in different stages through resampling, and the boundary box regressor is defined as follows:
Figure FDA0003578531730000033
in the above formula: x denotes the subimage block, b denotes the sample distribution, f denotes the bounding box regressor, f1、f2、f3The set threshold values are 0.4, 0.5, 0.6, respectively, f1As output of f2Input of f2As output of f3Input of { f }1,f2,f3Optimizing the resampling distribution of different stages, and simultaneously acting on training and testing stages;
the total loss function L defining the training model includes two parts: the total loss function calculation formula comprises a bounding box regression loss and a target classification loss, wherein the total loss function calculation formula comprises the following steps:
L(xt,g)＝Lcls(ht(xt),Xt)+μ[Xt≥1]Lloc(ft(xt,bt),g)
in the above formula: l isclsLoss function, L, representing the classification of the objectlocLoss function representing regression of bounding box, { btDenotes the distribution of samples for different training phases t and has bt＝ft-1(xt-1,bt-1)，htRepresenting object classifier, ftRepresenting a bounding box regressor, g representing the correspondence xtMu represents the compromise coefficient, XtDenotes xtCorresponding label, [ ·]An indicator function is represented.
8. The Cascade Mask R-CNN model-based weld defect detection method according to claim 1, wherein the training parameter settings in step S45 include learning rate, momentum, weight attenuation, value of batch _ size and total number of training rounds, the training weights are used as pre-training weights of the weld defect detection model, the weld ray detection digital image is uniformly scaled before entering the weld defect detection model, and parameters of the weld defect detection model are updated by using a random gradient descent method.
CN202210349016.8A 2022-04-01 2022-04-01 Weld defect detection method based on Cascade Mask R-CNN model Pending CN114627106A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210349016.8A CN114627106A (en) 2022-04-01 2022-04-01 Weld defect detection method based on Cascade Mask R-CNN model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210349016.8A CN114627106A (en) 2022-04-01 2022-04-01 Weld defect detection method based on Cascade Mask R-CNN model

Publications (1)

Publication Number Publication Date
CN114627106A true CN114627106A (en) 2022-06-14

Family

ID=81905833

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210349016.8A Pending CN114627106A (en) 2022-04-01 2022-04-01 Weld defect detection method based on Cascade Mask R-CNN model

Country Status (1)

Country Link
CN (1) CN114627106A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115526874A (en) * 2022-10-08 2022-12-27 哈尔滨市科佳通用机电股份有限公司 Round pin of brake adjuster control rod and round pin split pin loss detection method
CN117218121A (en) * 2023-11-08 2023-12-12 北京航空航天大学江西研究院 Casting DR image defect identification method
CN117252862A (en) * 2023-11-10 2023-12-19 北京航空航天大学江西研究院 SE-ResNeXt-based casting defect identification method
CN117789184A (en) * 2024-02-26 2024-03-29 沈阳派得林科技有限责任公司 Unified weld joint ray image intelligent identification method
CN118096644A (en) * 2023-12-19 2024-05-28 苏州大学 Tunnel water leakage detection method and system based on deep learning

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115526874A (en) * 2022-10-08 2022-12-27 哈尔滨市科佳通用机电股份有限公司 Round pin of brake adjuster control rod and round pin split pin loss detection method
CN115526874B (en) * 2022-10-08 2023-05-12 哈尔滨市科佳通用机电股份有限公司 Method for detecting loss of round pin and round pin cotter pin of brake adjuster control rod
CN117218121A (en) * 2023-11-08 2023-12-12 北京航空航天大学江西研究院 Casting DR image defect identification method
CN117252862A (en) * 2023-11-10 2023-12-19 北京航空航天大学江西研究院 SE-ResNeXt-based casting defect identification method
CN118096644A (en) * 2023-12-19 2024-05-28 苏州大学 Tunnel water leakage detection method and system based on deep learning
CN117789184A (en) * 2024-02-26 2024-03-29 沈阳派得林科技有限责任公司 Unified weld joint ray image intelligent identification method
CN117789184B (en) * 2024-02-26 2024-05-17 沈阳派得林科技有限责任公司 Unified weld joint ray image intelligent identification method

Similar Documents

Publication Publication Date Title
CN114627106A (en) Weld defect detection method based on Cascade Mask R-CNN model
CN109977793B (en) Roadside image pedestrian segmentation method based on variable-scale multi-feature fusion convolutional network
US20230360390A1 (en) Transmission line defect identification method based on saliency map and semantic-embedded feature pyramid
CN112967243A (en) Deep learning chip packaging crack defect detection method based on YOLO
CN109671071B (en) Underground pipeline defect positioning and grade judging method based on deep learning
CN112232371B (en) American license plate recognition method based on YOLOv3 and text recognition
CN112488025B (en) Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion
CN110659601B (en) Depth full convolution network remote sensing image dense vehicle detection method based on central point
CN112633149B (en) Domain-adaptive foggy-day image target detection method and device
CN113256601B (en) Pavement disease detection method and system
CN113469951B (en) Hub defect detection method based on cascade region convolutional neural network
CN117854072B (en) Automatic labeling method for industrial visual defects
CN116310785B (en) Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN109615604A (en) Accessory appearance flaw detection method based on image reconstruction convolutional neural networks
CN113313000B (en) Gas-liquid two-phase flow intelligent identification method based on optical image
CN109840483A (en) A kind of method and device of landslide fissure detection and identification
CN111738114A (en) Vehicle target detection method based on anchor-free accurate sampling remote sensing image
CN112819748A (en) Training method and device for strip steel surface defect recognition model
CN114897802A (en) Metal surface defect detection method based on improved fast RCNN algorithm
CN118351380A (en) Weather detection method based on multi-mode data fusion framework
CN117011222A (en) Cable buffer layer defect detection method, device, storage medium and equipment
CN116523897A (en) Semi-supervised enteromorpha detection method and system based on transconductance learning
CN117670855A (en) RoadU-Net-based intelligent recognition and classification method for asphalt pavement diseases
CN117274355A (en) Drainage pipeline flow intelligent measurement method based on acceleration guidance area convolutional neural network and parallel multi-scale unified network
CN111612803B (en) Vehicle image semantic segmentation method based on image definition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination