CN115953387A - Radiographic image weld defect detection method based on deep learning - Google Patents
Radiographic image weld defect detection method based on deep learning Download PDFInfo
- Publication number
- CN115953387A CN115953387A CN202310093032.XA CN202310093032A CN115953387A CN 115953387 A CN115953387 A CN 115953387A CN 202310093032 A CN202310093032 A CN 202310093032A CN 115953387 A CN115953387 A CN 115953387A
- Authority
- CN
- China
- Prior art keywords
- defect
- network layer
- layer
- image
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a radiographic image weld defect detection method based on deep learning, which utilizes a fast RCNN network to carry out target detection on a radiographic image, on the basis of the traditional fast RCNN, a background subtraction network layer is added to obtain a five-module fast RCNN network structure, the background subtraction network layer has updated parameters, the effect of reducing the background can be continuously optimized along with network training, the defect characteristics are highlighted, meanwhile, a three-branch region recommendation network layer is utilized to replace the traditional region recommendation network, two regression branches are provided to subdivide a task of predicting defect position information, one branch is responsible for predicting the horizontal coordinate and width of a central point of a defect, and the other branch is responsible for predicting the vertical coordinate and height of the central point of the defect, so that the accuracy of defect position prediction is enhanced.
Description
Technical Field
The invention relates to the technical field of pipeline weld defect detection and deep learning, in particular to a radiographic image weld defect detection method based on deep learning.
Background
The welding technology has important application value in pipeline processing. However, with the existing welding technology, welding defects such as circular defects, strip defects, cracks, unfused, undercut, and undercut still occur on the welding seam, so that the detection of the welding defects is important to ensure the reliability and safety of the welded product. At present, manual detection is still a common method for detecting defects, but the detection efficiency of the method is low, and the accuracy is limited by the expertise of detection personnel. Then, researchers have started research on automatic detection of weld defects on X-ray nondestructive inspection images of welds in combination with machine learning algorithms.
The initial Defect Detection algorithm identifies candidate regions generated in an image processing stage by Using a machine learning algorithm, determines Defect positions and types, for example, a potential Defect region in an image is obtained in image preprocessing by Duan Feng et al (Duan Feng, YIN shift, son Peipei, et al automatic Welding Defect Detection of X-Ray Images by Using cassette Access with a latent test [ J ]. IEEE Access,2019, 7. However, this method is not accurate enough for location prediction. After that, with the application of the deep learning algorithm, fast RCNN and YOLOv3 are widely used for the target detection task of weld defects. For example, a welding defect detection system based on Faster R-CNN model X-ray image is designed by Guo civiling et al (Guo civiling, liukai, quhui Sai Hui Sai. J. The academic newspaper of Beijing post and telecommunications university, 2019,42 (06): 20-28.) and simple attempts are made on defect detection; wehouyu et al (wehouyu, lissutent, wupinrong, et al. Track fastener defect detection based on the modified YOLO V3 algorithm [ J ]. Railroad standard design, 2020,64 (12): 30-36.) detected defects using two prediction branches in YOLOv 3. However, the accuracy of the model still needs to be improved, and research directions mainly focus on the problems of small-defect target detection, multi-scale defect target detection and detection efficiency. For the problem of small defect target detection, it is a common way to predict a multi-layer feature map by using a Feature Pyramid Network (FPN). Plum dongjie et al (plum dongjie, plum lohao, improved Faster RCNN-based method for detecting a defect in a mug [ J ] laser and optoelectronics progress, 2020,57 (4): 353-360.) improved the problem of detecting a small target in the detection of a defect in a mug by combining with FPN on the basis of a Faster RCNN detection network. Chen Yongbin et al (CHEN Yongbin, WANG Jingen, WANG Guitang. Intelligent shaping Detection Model on Improved R-CNN [ J ]. IETE Journal of Research, 2022). Liu Weipeng et al (LIU Weipeng, SHAN Shengqi, CHEN Haiyong, et al.X-ray Weld Defect Detection Based on AF-RCNN [ J ]. Welding in the World,2022,66 (6): 1165-1177.) propose AF-RCNN network to detect Weld defects, the algorithm uses ResNet and FPN as the backbone network of fast RCNN, combines with high efficiency convolution attention module and CIOU loss function, and improves the Detection capability of fast RCNN for small defects. Aiming at the problem of multi-scale defects, chenhai Yong et al (Chenhai Yong, zhao Peng, yan bright. Crack detection of multi-scale Faster RCNN with attention [ J ] photoelectric engineering, 2021,48 (1): 64-74.) fuse FPNs with bottom-to-top paths in a backbone network of Faster RCNN, and combines with Focal Loss functions to improve the detection precision of the defects. Zhu 'an kang et al (Zhu' an kang, enchanting elegance, zhang Ganyu, aluminum surface defect detection system design [ J ] sensor and micro system based on deep learning, 2022,41 (08): 96-99+ 103.) aiming at the problem of low detection effect caused by uneven defect size, add deformable convolution in Faster RCNN network model to improve detection accuracy. In the aspect of detection efficiency, improved application of fast-RCNN in weld defect detection [ J ] of combined machine tool and automatic processing technology, 2021, (12): 83-86) of Tang Maojun et al (Tang Maojun, yellow sea pine, zhangsong, etc.) increases decoupling classification refinement structure and improves detection speed on the basis of increasing FPN structure. Wuweihao et al (Wuweihao, liqing. Electrical connector Defect detection based on improved Yolov3 [ J ]. Sensory technical report, 2020,33 (2): 299-307.) to increase defect detection speed, the number of DBL units in Yolov3 was reduced and single scale feature maps were used for prediction. The above research has done a lot of work around defect detection, but the existence of small defects and multi-size defects, and the requirement of the detection task on the efficiency of the model, make the detection model still need to continuously improve the accuracy and optimize the detection speed.
Disclosure of Invention
The invention mainly overcomes the defects in the prior art and provides a radiographic image weld defect detection method based on deep learning.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
1. a radiographic image weld defect detection method based on deep learning is characterized by comprising the following steps:
the method comprises the following steps: performing image preprocessing on an original image, wherein the image preprocessing comprises gray level stretching, median filtering and mean filtering to obtain an image after the image preprocessing, and a gray level stretching formula is as follows:
wherein g is the pixel value after gray stretching, and the value range is the interval [0, g max ]Integer of inner, g max To be stretchedThe maximum pixel value of the back image is in a positive integer range, and g max Is greater than f max F is the pixel value of the original image, f max Is the maximum pixel value of the original image, f min Is the minimum pixel value of the original image,
in order to round downwards, the median filtering replaces the original pixel value by the median of all pixel values in the mxn window size of the original pixel value, the mean filtering replaces the original pixel value by the mean of all pixel values in the mxn window size of the original pixel value, and the value ranges of m and n are both positive integers;
step two: taking the preprocessed image as an input image, zooming the input image into a tensor of a multiplied by b multiplied by c, wherein the value ranges of a, b and c are positive integers, and then normalizing, wherein the normalized formula is as follows:
wherein u is a value after standardization, the value range is an interval [ -1,1], v is a value before standardization, the value range is any real number, m is a mean value of pixel values of all images to be standardized, the value range is any real number, delta is a standard deviation of the pixel values of all the images to be standardized, and the value range is any real number;
step three: taking the normalized tensor as input, building a fast RCNN target detection network to detect the defects in the image, adding a background subtraction network layer, changing the four-part structure of the traditional fast RCNN network into a five-part structure, modifying regression branches in a region recommendation network layer, and independently predicting the position information of the defects in the image, wherein the position information of the defects comprises the abscissa of the center point of the defects, the ordinate of the center point of the defects, the height of the defects and the width of the defects, the fast RCNN network comprises five parts of the background subtraction network layer, a feature extraction network layer, a region recommendation network layer, an interest region pooling layer and a classification/position regression layer, and the building steps are as follows:
(1) Building a background subtraction network layer, reducing the influence of the background in the image, highlighting defect characteristics, and consisting of m x n large-size mean pooling, convolution and jump connection, wherein the value ranges of m and n are positive integers, the input of the layer is subjected to the large-size mean pooling and convolution to obtain a simulated background, and the input of the layer is subtracted from the simulated background through the jump connection to obtain the output of the layer;
(2) A feature extraction network layer is built to extract defect features, and the feature extraction network layer consists of ResNet50, FPN and SE attention mechanisms, wherein a feature pyramid network is a typical method for extracting feature graphs with different sizes and is used for a subsequent task, in the defect detection task, N feature graphs are used as output, N is a positive integer and is used for the prediction process of the subsequent part, each layer of feature graph is responsible for anchor frames with A sizes, A is a positive integer, and the size of each anchor frame is h a ×w a Is represented by the formula (I), wherein h a The height of the anchor frame is represented, the value range is positive integer, w a The width of an anchor frame is represented, the value range is a positive integer, the output of the layer obtained by subtracting the output of the network layer from the background is used as the input, and the output is N characteristic graphs;
(3) Taking the feature map as input, building a regional recommendation network layer, and generating a regional recommendation, wherein the regional recommendation network subdivides responsibility of regression branches on the basis of classification and regression double branches of a traditional regional recommendation network layer, and independently predicts position information of defects in an image, the regional recommendation network comprises three branches, namely a front/background classification branch, a wide/central point abscissa regression branch and a high/central point ordinate regression branch, and is hereinafter referred to as a three-branch regional recommendation network layer;
(4) Building an interest region pooling layer, adjusting the number K of convolution layers, wherein the value range of K is a positive integer, extracting features from the corresponding feature graph according to the positive example, the negative example and the offset obtained in the step (3), and outputting the features to be uniform, wherein a formula for selecting the feature graph when extracting the features is as follows:
wherein p is the number of the selected characteristic diagram, the value range is positive integer, and p is 0 The initial characteristic diagram number is taken as the numeric value range, the numeric value range is positive integer, h is the height of the anchor frame after the deviation, the numeric value range is positive number, w is the width of the anchor frame after the deviation, the numeric value range is positive number, T is constant, the numeric value range is positive number,
to round down;
(5) Building a classification/position regression layer, wherein the classification/position regression layer is formed by two full-connection layers, one full-connection layer classifies the defect types, and the other full-connection layer regresses the defect positions to obtain the defect types and the defect positions, so that a frame with the defect types is formed and is called as a prediction frame;
step four: training the model on a training set, wherein the image of the training set is I train Radiographic image of a weld in a pipeline, I train The value range of (1) is positive integer, the labels of the training set are the types C and positions of the defects in the image, the number of the defect types is C, the value range of C is positive integer, and the value range of C is the interval [0, C-1 ]]The location of the defect includes the abscissa x of the center point of the defect t Center point ordinate y of the defect t Height w of the defect t And width h of the defect t Each image correspondingly comprises a plurality of defects, each defect corresponds to a position, the label is marked in a manual mode, and the training process is as follows:
(1) Calculating the offset of the defect position in the label relative to the anchor frame, wherein the offset comprises the offset of an abscissa, an ordinate, a height and a width, and the offset calculation formula of the abscissa is as follows:
wherein, t x Is the offset of the abscissa, x t Is the abscissa, x, of the center point of the defect in the label a Is the abscissa of the center point of the anchor frame, w a Is the width, x, of the anchor frame t 、x a 、w a The value range of (1) is a positive integer, and the offset calculation formula of the ordinate is as follows:
wherein, t y Is the offset of the ordinate, y t As the ordinate of the centre point of the defect in the label, y a Is the abscissa of the center point of the anchor frame, h a Is the height, y, of the anchor frame t 、y a 、h a The value range of (a) is a positive integer, and the wide offset calculation formula is as follows:
wherein, t w For a wide offset, w t For the width of the defect in the label, the value range is positive integer, and the high offset calculation formula is as follows:
wherein, t h Is a high offset, h t The number of the defects in the label is high, and the value range is positive integer;
(2) In the training model, the network layer is subtracted from the background, the feature extraction network layer and the three-branch regional recommendation network layer are extracted, and the trained parameters are selectedSelecting the model with the minimum loss value in the training process, and selecting N for each image a Calculating by the anchor frame, wherein the calculation formula of the loss value is as follows:
wherein L is a loss value, N a The number of anchor frames used for training for each image is positive integer, q i The maximum value of the model output vector of the ith anchor frame is in the range of [0,1]]，t x,i For the ith anchor frame corresponding to the offset of the abscissa in the tag, t y,i For the ith anchor frame corresponding to the offset of the ordinate in the label, t w,i For the ith anchor frame, which corresponds to a wide offset in the tag, t h,i For the ith anchor frame corresponding to a high offset in the tag, r x,i Offset of the abscissa, r, output for the ith anchor frame corresponding to the model y,i For the ith anchor frame corresponding to the offset of the ordinate, r, of the model output w,i For the ith anchor frame corresponding to the wide offset of the model output, r h,i For the high offset output by the model corresponding to the ith anchor frame, the value ranges of the offsets are real numbers, S (..) is the loss of the smooth L1, and the calculation formula is as follows:
wherein S (x) is any input;
(3) Training the whole model, importing the parameters of the three-branch region recommended network layer trained in the step (2), selecting the model with the minimum loss value in the training process by the trained model, wherein the loss value calculation formula is the same as that in the step (2), the offset of smooth L1 loss input is converted into the offset based on the three-branch region recommended network layer output, and N is selected for each image r A candidate box is calculated, N r Substitution of N a Ensuring that the parameters of the recommended network layer of the three-branch region are not updated in the training process;
(4) The background minus network layer, the feature extraction network layer and the three-branch regional recommended network layer in the training model are imported with the parameters of the background minus network layer, the feature extraction network layer and the three-branch regional recommended network layer which are trained in the step (3), the parameters of the three-branch regional recommended network layer are only updated in the training process, and the rest of the training processes are the same as those in the step (2);
(5) Training the whole model, importing the parameters of the background subtraction network layer, the feature extraction network layer, the interest region pooling layer and the classification/position regression layer which are trained in the step (3), importing the parameters of the three-branch region recommendation network layer which is trained in the step (4), only updating the parameters of the interest region pooling layer and the classification/position regression layer in the training process, and keeping the rest of the training processes the same as those in the step (3);
step five: the model needs to carry out post-processing on the prediction frame output by the last layer of the model during testing, screens the prediction frames meeting the requirements, and removes unqualified prediction frames output by the detection network, and the processing process is as follows: firstly, removing a prediction frame with a classification label being non-defect, then removing a prediction frame with a left upper corner coordinate and a right lower corner coordinate exceeding an image boundary, finally dividing the residual prediction frames according to the classification label, sorting the prediction frames of the same class according to a classification score descending order, removing the prediction frames with a score lower than that of the prediction frames with a score higher than that of the prediction frames with the value range of [0,1], taking the residual prediction frames as a final prediction result, wherein the prediction frames in the prediction result comprise information of two aspects of defect type and defect position.
The invention provides a radiographic image weld defect detection method based on deep learning, which utilizes a fast RCNN network to carry out target detection on a radiographic image, on the basis of the traditional fast RCNN, a background subtraction network layer is added to obtain a five-module fast RCNN network structure, the background subtraction network layer has updated parameters, the effect of reducing the background can be continuously optimized along with network training, the defect characteristics are highlighted, meanwhile, a three-branch region recommendation network layer is utilized to replace the traditional region recommendation network, two regression branches are provided to subdivide a task of predicting defect position information, one branch is responsible for predicting the horizontal coordinate and width of a central point of a defect, and the other branch is responsible for predicting the vertical coordinate and height of the central point of the defect, so that the accuracy of defect position prediction is enhanced.
Has the advantages that:
compared with the prior art, the invention has the following beneficial effects:
the background minus network layer can continuously improve the background elimination effect along with training, reduce the influence of the background on defect detection and highlight the defect area; the three-branch regional recommended network layer comprises two regression branches, the prediction task of the defect position is subdivided, the obtained candidate frames are more accurate, the number of the candidate frames output by the regional recommended network layer can be reduced, and the detection speed is improved.
Drawings
FIG. 1 is a flow chart of a deep learning-based radiographic image weld defect detection method, which includes image preprocessing, scaling and standardization, a Faster RCNN network, and post-processing, wherein the Faster RCNN network is modified into a five-part structure, and is composed of five parts, namely a background subtraction network layer, a feature extraction network layer, a three-branch region recommendation network layer, an interest region pooling layer, and a classification/position regression layer;
FIG. 2 is a diagram of a background minus network layer structure consisting of large-size mean pooling, convolution and jump concatenation, the jump concatenation being the subtraction part of the diagram;
FIG. 3 is a diagram of a three-branch area recommended network layer structure, consisting of six convolutions, for a total of three branches;
FIG. 4 is a diagram of an example of weld defect inspection, in which the inspection result includes the type of defect and the position of the defect, and forms a frame with the type of defect, and the number above the frame in the diagram represents the type of defect, "1" represents a circular defect, "2" represents a strip defect, "3" represents an unfused defect, "4" represents a concave defect, "5" represents an undercut defect, and "6" represents a crack defect.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example (b):
a radiographic image weld defect detection method based on deep learning comprises the following specific implementation methods:
the method comprises the following steps: performing image preprocessing on an original image to obtain an image after the image preprocessing, wherein the image comprises gray level stretching, median filtering and mean filtering, and a gray level stretching formula is as follows:
wherein g is the pixel value after gray stretching, g max The maximum pixel value of the stretched image is 255, f is the pixel value of the original image, f max Is the maximum pixel value of the original image, f min Is the minimum pixel value of the original image,
in order to get the whole downwards, a window with the size of 3x3 is selected for median filtering, and a window with the size of 3x3 is selected for mean filtering;
step two: taking the preprocessed image as an input image, zooming the input image into a 224x224x3 tensor, and then normalizing, wherein the normalized formula is as follows:
wherein u is a normalized value, v is a value before normalization, m is a mean value of pixel values of all images to be normalized, according to the scaling size, the input to be normalized has 3 channels, the mean values are respectively 0.485, 0.456, 0.406, δ is a standard deviation of pixel values of all images to be normalized, according to the scaling size, the input to be normalized has 3 channels, and the standard deviations are respectively 0.229, 0.224, 0.225;
step three: taking the normalized tensor as input, building a fast RCNN target detection network to detect the defects in the image, adding a background subtraction network layer, changing the four-part structure of the traditional fast RCNN network into a five-part structure, modifying regression branches in a region recommendation network layer, and independently predicting the position information of the defects in the image, wherein the position information of the defects comprises the abscissa of the center point of the defects, the ordinate of the center point of the defects, the height of the defects and the width of the defects, the fast RCNN network comprises five parts of the background subtraction network layer, a feature extraction network layer, a region recommendation network layer, an interest region pooling layer and a classification/position regression layer, and the building steps are as follows:
(1) Building a background subtraction network layer, reducing the influence of the background in the image, highlighting defect characteristics, and consisting of 25x25 large-size mean pooling, convolution and jump connection, wherein the input of the layer is subjected to the large-size mean pooling and convolution to obtain a simulated background, and the input of the layer is subtracted from the simulated background through the jump connection to obtain the output of the layer;
(2) Constructing a feature extraction network layer, extracting defect features, wherein the feature extraction network layer comprises a ResNet50, a Feature Pyramid Network (FPN) and an SE attention mechanism, the feature pyramid network takes 5 feature maps as output for a prediction process of a subsequent part, the 5 feature maps are marked as P2, P3, P4, P5 and P6, each feature map is responsible for anchor frames with 3 sizes, the anchor frame size responsible for each feature map is obtained through clustering, the anchor frames are respectively the anchor frames responsible for the P2 layer 8x49, 13x30 and 19x20, the P3 layer is responsible for the anchor frames of 16x98, 26x60 and 38x41, the P4 layer is responsible for the anchor frames of 26x159, 42x98 and 63x66, the P5 layer is responsible for the anchor frames of 38x236, 62x145 and 93x98, the P6 layer is responsible for the anchor frames of 51x311, 83x191 and 122x129, the layer subtracts the output of the background network layer as input, and the output is 5 feature maps;
(3) Taking the feature map as input, building a regional recommendation network layer, and generating a regional recommendation, wherein the regional recommendation network subdivides responsibility of regression branches on the basis of classification and regression double branches of a traditional regional recommendation network layer, and independently predicts position information of defects in an image, the regional recommendation network comprises three branches, namely a front/background classification branch, a wide/central point abscissa regression branch and a high/central point ordinate regression branch, and is hereinafter referred to as a three-branch regional recommendation network layer;
(4) Building an interest region pooling layer, adjusting the number of convolution layers, adding 4 convolution layers in the interest region pooling layer, extracting features from the corresponding feature graph according to the positive example, the negative example and the offset obtained in the step (3), and outputting the features to be uniform, wherein a formula for selecting the feature graph when extracting the features is as follows:
wherein p is the selected feature map number, p 0 For the initial feature map number, where the value 5, h is the height of the anchor frame after the migration, w is the width of the anchor frame after the migration, T is a constant, where the value 224,
to round down; (6) Building a classification/position regression layer, wherein the classification/position regression layer is formed by two full-connection layers, one full-connection layer classifies the defect types, and the other full-connection layer regresses the defect positions to obtain the defect types and the defect positions, so that a frame with the defect types is formed and is called as a prediction frame;
step four: training the model on a training set, wherein the image of the training set is 1200 radiographic images of the pipeline weld joint, the label of the training set is the type c and the position of the defect in the image, the number of the defect types is 7, and the value of the c is the interval [0,6 ]]Positive integers in (1) respectively corresponding to non-defect and circularDefects, strip defects, unfused defects, dishing defects, undercut defects, and crack defects, the location of the defect including the center point abscissa x of the defect t Center point ordinate y of the defect t Height w of the defect t And width h of the defect t Each image correspondingly comprises a plurality of defects, each defect corresponds to a position, the label is marked in a manual mode, and the training process comprises the following steps:
(1) Calculating the offset of the defect position in the label relative to the anchor frame, wherein the offset comprises the offset of an abscissa, an ordinate, a height and a width, and the offset calculation formula of the abscissa is as follows:
wherein, t x Is the offset of the abscissa, x t Is the abscissa, x, of the center point of the defect in the label a Is the abscissa of the center point of the anchor frame, w a The offset of the ordinate for the width of the anchor frame is calculated by the formula:
wherein, t y Is the offset of the ordinate, y t As the ordinate of the center point of the defect in the label, y a Is the abscissa of the center point of the anchor frame, h a The offset of the height and width of the anchor frame is calculated by the formula:
wherein, t w Is a wide offset, w t For a wide defect in a label, the high offset is calculated by the formula:
wherein, t h Is a high offset, h t Is high for defects in the label;
(2) Subtracting a network layer, a feature extraction network layer and a three-branch region recommendation network layer from a background in a training model, selecting a model with the minimum loss value in the training process according to parameters after training, selecting 16 anchor frames for each image to calculate, wherein the calculation formula of the loss value is as follows:
wherein L is a loss value, N a The number of anchor frames used for training for each image is 16, q i For the maximum value of the model output vector, t, of the ith anchor frame x,i For the ith anchor frame corresponding to the offset of the abscissa in the label, t y,i For the ith anchor frame corresponding to the offset of the ordinate in the label, t w,i For the ith anchor frame corresponding to a wide offset in the label, t h,i For the ith anchor frame corresponding to a high offset in the tag, r x,i Offset of the abscissa, r, output for the ith anchor frame corresponding to the model y,i Offset of ordinate, r, of model output for the ith anchor frame w,i For the ith anchor frame corresponding to the wide offset of the model output, r h,i For the ith anchor frame corresponding to the high offset of the model output, S (..) is the smoothed L1 penalty, and the formula is:
wherein S (x) is any input;
(3) Training the whole model, importing the parameters of the three-branch area recommended network layer trained in the step (2), selecting the model with the minimum loss value in the training process by the trained model, wherein the loss value calculation formula is the same as that in the step (2), the offset input by smoothing L1 loss is converted into the offset output by the three-branch area recommended network layer, each image selects 16 candidate frames for calculation, and the parameters of the three-branch area recommended network layer in the training process are ensured not to be updated;
(4) The background minus network layer, the feature extraction network layer and the three-branch regional recommended network layer in the training model are imported with the parameters of the background minus network layer, the feature extraction network layer and the three-branch regional recommended network layer which are trained in the step (3), the parameters of the three-branch regional recommended network layer are only updated in the training process, and the rest of the training processes are the same as those in the step (2);
(5) Training the whole model, importing the parameters of the background subtraction network layer, the feature extraction network layer, the interest region pooling layer and the classification/position regression layer which are trained in the step (3), importing the parameters of the three-branch region recommendation network layer which is trained in the step (4), only updating the parameters of the interest region pooling layer and the classification/position regression layer in the training process, and keeping the rest of the training processes the same as those in the step (3);
step five: the model needs to carry out post-processing on the prediction frame output by the last layer of the model during testing, the prediction frames meeting the requirements are screened, unqualified prediction frames for detecting network output are removed, and the processing process is as follows: firstly, removing prediction frames with classification labels being non-defects, then removing prediction frames with upper-left and lower-right coordinates exceeding image boundaries, finally dividing the residual prediction frames according to the classification labels, sorting the prediction frames of the same category according to classification scores in a descending order, removing the prediction frames with low scores and higher scores, wherein the intersection ratio of the prediction frames with the higher scores exceeds 0.5, taking the residual prediction frames as a final prediction result, and the prediction frames in the prediction result comprise information of two aspects of defect types and defect positions;
step six: evaluating the performance of different models under the same test set, wherein the image of the test set is a radiographic image of 300 pipeline welding seams, the label of the test set is the type c and the position of the defect in the image, the number of the defect types is 7, and the value range of the c is an interval [0,6 ]]The positive integer in (1) corresponds to non-defect, circular defect, strip defect, non-fusion defect, concave defect, undercut defect and crack defect respectively, and the position of the defect comprises the abscissa x of the central point of the defect t Center point ordinate y of the defect t Height w of the defect t And width h of the defect t Each imageCorrespondingly containing a plurality of defects, each defect corresponds to a position, the label is marked in an artificial way, each image corresponds to a kind label, the label is marked in an artificial way, the evaluation index is mean average precision, the larger the mean average precision of the model is, the better the defect Detection effect of the model is, and the mean average precision is utilized to carry out Detection on original fast RCNN, fast RCNN based on ResNet50 and FPN, the algorithm of the invention and YOLOv3 (wherein the original fast RCNN is literature (REN Shaoqing, HE Kaiming, GIRSHICK Ross, et al. Fast R-CNN: towards read-Time Object Detection with Region sample Networks [ J ] J]IEEE Transactions on Pattern Analysis and Machine understanding, 2017,39 (6): 1137-1149), faster RCNN based on ResNet50 and FPN is a document (LIN Tsungyi, DOLLARPiott, GIRSHICK Ross, et al]30th model of IEEE Conference on Computer Vision and Pattern recognition,2017, 936-944), yolov3 is literature (REDCON Joseph, FARHODI Ali. Yolov3: an incorporated Improvement [ C]Model of IEEE Conference on Computer Vision and Pattern Recognition, honolulu, USA: IEEE,2017, 6517-6525), wherein the mean average accuracy is expressed as:
wherein, mAP represents the average precision, N represents the number of categories, AP represents the average precision, and the calculation formula is:
wherein, P (R) represents the curve of the accuracy rate relative to the recall rate, and in practice, because a continuous curve cannot be obtained, the AP value here is the average value of the maximum accuracy rate in a certain number of equidistant recall rate intervals, and the formula of the accuracy rate is as follows:
wherein, P is the precision rate, TP is the sample number of positive case for prediction, and actual positive case for prediction, FP is the sample number of positive case for prediction, and actual negative case for prediction, and the formula of the call rate is as follows:
wherein, R is recall rate, FN is sample number of prediction negative case and actual positive case, and the comparison result is:
although the present invention has been described with reference to the above embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the present invention.
Claims (1)
1. A radiographic image weld defect detection method based on deep learning is characterized by comprising the following steps:
the method comprises the following steps: performing image preprocessing on an original image, wherein the image preprocessing comprises gray level stretching, median filtering and mean filtering to obtain an image after the image preprocessing, and a gray level stretching formula is as follows:
wherein g is the pixel value after gray stretching, and the value range is the interval [0, g max ]Integer of (i), g max The value of the maximum pixel value of the stretched image is in a range of positive integers, and g max Is greater than f max F is the pixel value of the original image, f max Is the maximum pixel value of the original image, f min Is the minimum pixel value of the original image,
in order to round downwards, the median filtering replaces the original pixel value by the median of all pixel values in the mxn window size of the original pixel value, the mean filtering replaces the original pixel value by the mean of all pixel values in the mxn window size of the original pixel value, and the value ranges of m and n are positive integers;
step two: taking the preprocessed image as an input image, zooming the input image into a tensor of a multiplied by b multiplied by c, wherein the value ranges of a, b and c are positive integers, and then normalizing, wherein the normalized formula is as follows:
wherein u is a value after standardization, the value range is an interval [ -1,1], v is a value before standardization, the value range is any real number, m is the mean value of pixel values of all images to be standardized, the value range is any real number, delta is the standard deviation of the pixel values of all images to be standardized, and the value range is any real number;
step three: taking the normalized tensor as input, building a fast RCNN target detection network to detect the defects in the image, adding a background subtraction network layer, changing the four-part structure of the traditional fast RCNN network into a five-part structure, modifying regression branches in a region recommendation network layer, and independently predicting the position information of the defects in the image, wherein the position information of the defects comprises the abscissa of the center point of the defects, the ordinate of the center point of the defects, the height of the defects and the width of the defects, the fast RCNN network comprises five parts of the background subtraction network layer, a feature extraction network layer, a region recommendation network layer, an interest region pooling layer and a classification/position regression layer, and the building steps are as follows:
(1) Building a background subtraction network layer, reducing the influence of the background in the image, highlighting defect characteristics, and consisting of m x n large-size mean pooling, convolution and jump connection, wherein the value ranges of m and n are positive integers, the input of the layer is subjected to the large-size mean pooling and convolution to obtain a simulated background, and the input of the layer is subtracted from the simulated background through the jump connection to obtain the output of the layer;
(2) A feature extraction network layer is built to extract defect features, and the feature extraction network layer consists of ResNet50, FPN and SE attention mechanisms, wherein a feature pyramid network is a typical method for extracting feature graphs with different sizes and is used for a subsequent task, in the defect detection task, N feature graphs are used as output, N is a positive integer and is used for the prediction process of the subsequent part, each layer of feature graph is responsible for anchor frames with A sizes, A is a positive integer, and the size of each anchor frame is h a ×w a Is represented by the formula (I), wherein h a The height of the anchor frame is represented, the value range is positive integer, w a The width of an anchor frame is represented, the value range is a positive integer, the output of the layer obtained by subtracting the output of the network layer from the background is used as the input, and the output is N characteristic graphs;
(3) Taking the feature map as input, building a regional recommendation network layer, and generating a regional recommendation, wherein the regional recommendation network subdivides responsibility of regression branches on the basis of classification and regression double branches of a traditional regional recommendation network layer, and independently predicts position information of defects in an image, the regional recommendation network comprises three branches, namely a front/background classification branch, a wide/central point abscissa regression branch and a high/central point ordinate regression branch, and is hereinafter referred to as a three-branch regional recommendation network layer;
(4) Building an interest region pooling layer, adjusting the number K of convolution layers, wherein the value range of K is a positive integer, extracting features from the corresponding feature graph according to the positive example, the negative example and the offset obtained in the step (3), and outputting the features to be uniform, wherein a formula for selecting the feature graph when extracting the features is as follows:
wherein p is the number of the selected characteristic diagram, the value range is positive integer, and p is 0 The initial characteristic diagram number is taken as the numeric value range, the numeric value range is positive integer, h is the height of the anchor frame after the deviation, the numeric value range is positive number, w is the width of the anchor frame after the deviation, the numeric value range is positive number, T is constant, the numeric value range is positive number,
to round down;
(5) Building a classification/position regression layer, wherein the classification/position regression layer is formed by two full connections, one full connection layer classifies the defect types, the other full connection layer regresses the defect positions to obtain the defect types and the defect positions, and a frame with the defect types is formed and is called a prediction frame;
step four: training the model on a training set, wherein the image of the training set is I train Radiographic image of a weld in a pipeline, I train The value range of (1) is positive integer, the labels of the training set are the types C and the positions of the defects in the image, the number of the defect types is C, the value range of C is positive integer, and the value range of C is the interval [0, C-1 ]]The location of the defect includes the abscissa x of the center point of the defect t Center point ordinate y of the defect t Height w of the defect t And width h of the defect t Each image correspondingly comprises a plurality of defects, each defect corresponds to a position, the label is marked in a manual mode, and the training process comprises the following steps:
(1) Calculating the offset of the defect position in the label relative to the anchor frame, wherein the offset comprises the offset of an abscissa, an ordinate, a height and a width, and the offset calculation formula of the abscissa is as follows:
wherein, t x Is the offset of the abscissa, x t Is the abscissa, x, of the center point of the defect in the label a Is the abscissa of the center point of the anchor frame, w a Width of anchor frame, x t 、x a 、w a The value range of (a) is a positive integer, and the offset calculation formula of the ordinate is as follows:
wherein, t y Is the offset of the ordinate, y t As the ordinate of the centre point of the defect in the label, y a Is the central point abscissa of the anchor frame, h a Is the height, y, of the anchor frame t 、y a 、h a The value range of (a) is a positive integer, and the wide offset calculation formula is as follows:
wherein, t w Is a wide offset, w t For the width of the defect in the label, the value range is positive integer, and the high offset calculation formula is as follows:
wherein, t h Is a high offset, h t The number of the defects in the label is high, and the value range is positive integer;
(2) Subtracting a network layer, a feature extraction network layer and a three-branch region recommendation network layer from a background in a training model, selecting a model with the minimum loss value in the training process according to the trained parameters, and selecting N for each image a Calculating by using the anchor frame, wherein the calculation formula of the loss value is as follows:
wherein L is a loss value, N a The number of the anchor frames used for training for each image is within a positive integer,
q i the maximum value of the model output vector of the ith anchor frame is in the range of [0,1]]，t x,i For the ith anchor frame corresponding to the offset of the abscissa in the tag, t y,i For the ith anchor frame corresponding to the offset of the ordinate in the label, t w,i For the ith anchor frame, which corresponds to a wide offset in the tag, t h,i For the ith anchor frame corresponding to a high offset in the tag, r x,i Offset of the abscissa, r, output for the ith anchor frame corresponding to the model y,i For the ith anchor frame corresponding to the offset of the ordinate, r, of the model output w,i For the ith anchor box to correspond to a wide shift in the model output,
r h,i the value range of the high offset output by the model corresponding to the ith anchor frame is real,
s (..) is the smoothed L1 penalty, the formula is:
wherein S (x) is any input;
(3) Training the whole model, importing the parameters of the three-branch region recommendation network layer trained in the step (2), and selecting the trained modelSelecting a model with the minimum loss value in the training process, wherein the calculation formula of the loss value is the same as the formula (2), the offset of the smooth L1 loss input is converted into the offset of the recommended network layer output based on the three-branch region, and N is selected for each image r A candidate box is calculated, N r Substitution of N a Ensuring that the parameters of the recommended network layer of the three-branch region are not updated in the training process;
(4) The three parts of the background subtraction network layer, the feature extraction network layer and the three-branch regional recommendation network layer in the training model are imported, parameters of the background subtraction network layer, the feature extraction network layer and the three-branch regional recommendation network layer which are trained in the step (3) are imported, the parameters of the three-branch regional recommendation network layer are only updated in the training process, and the rest training processes are the same as those in the step (2);
(5) Training the whole model, importing the parameters of the background subtraction network layer, the feature extraction network layer, the interest region pooling layer and the classification/position regression layer which are trained in the step (3), importing the parameters of the three-branch region recommendation network layer which is trained in the step (4), only updating the parameters of the interest region pooling layer and the classification/position regression layer in the training process, and keeping the rest of the training processes the same as those in the step (3);
step five: the model needs to carry out post-processing on the prediction frame output by the last layer of the model during testing, the prediction frames meeting the requirements are screened, unqualified prediction frames for detecting network output are removed, and the processing process is as follows: firstly, removing a prediction frame with a classification label being non-defect, then removing a prediction frame with a left upper corner coordinate and a right lower corner coordinate exceeding an image boundary, finally dividing the residual prediction frames according to the classification label, sorting the prediction frames of the same class according to a classification score descending order, removing the prediction frames with a score lower than that of the prediction frames with a score higher than that of the prediction frames with the value range of [0,1], taking the residual prediction frames as a final prediction result, wherein the prediction frames in the prediction result comprise information of two aspects of defect type and defect position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310093032.XA CN115953387A (en) | 2023-02-10 | 2023-02-10 | Radiographic image weld defect detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310093032.XA CN115953387A (en) | 2023-02-10 | 2023-02-10 | Radiographic image weld defect detection method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115953387A true CN115953387A (en) | 2023-04-11 |
Family
ID=87297424
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310093032.XA Pending CN115953387A (en) | 2023-02-10 | 2023-02-10 | Radiographic image weld defect detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115953387A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117218129A (en) * | 2023-11-09 | 2023-12-12 | 四川大学 | Esophageal cancer image identification and classification method, system, equipment and medium |
-
2023
- 2023-02-10 CN CN202310093032.XA patent/CN115953387A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117218129A (en) * | 2023-11-09 | 2023-12-12 | 四川大学 | Esophageal cancer image identification and classification method, system, equipment and medium |
| CN117218129B (en) * | 2023-11-09 | 2024-01-26 | 四川大学 | Esophageal cancer image identification and classification method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111223088B (en) | Casting surface defect identification method based on deep convolutional neural network | |
| CN112967243B (en) | Deep learning chip packaging crack defect detection method based on YOLO | |
| CN113362326B (en) | Method and device for detecting defects of welding spots of battery | |
| US10803573B2 (en) | Method for automated detection of defects in cast wheel products | |
| CN109191459B (en) | Automatic identification and rating method for continuous casting billet macrostructure center segregation defect | |
| CN111223093A (en) | AOI defect detection method | |
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN108492291B (en) | CNN segmentation-based solar photovoltaic silicon wafer defect detection system and method | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| CN113554631B (en) | Chip surface defect detection method based on improved network | |
| CN112819748B (en) | Training method and device for strip steel surface defect recognition model | |
| CN111382785A (en) | GAN network model and method for realizing automatic cleaning and auxiliary marking of sample | |
| CN113392849A (en) | R-CNN-based complex pavement crack identification method | |
| CN112991271A (en) | Aluminum profile surface defect visual detection method based on improved yolov3 | |
| CN115830004A (en) | Surface defect detection method, device, computer equipment and storage medium | |
| CN115294033A (en) | Tire belt layer difference level and misalignment defect detection method based on semantic segmentation network | |
| CN115953387A (en) | Radiographic image weld defect detection method based on deep learning | |
| CN117252815A (en) | Industrial part defect detection method, system, equipment and storage medium based on 2D-3D multi-mode image | |
| CN117455917B (en) | Establishment of false alarm library of etched lead frame and false alarm on-line judging and screening method | |
| CN113962929A (en) | Photovoltaic cell assembly defect detection method and system and photovoltaic cell assembly production line | |
| CN115830302B (en) | Multi-scale feature extraction fusion power distribution network equipment positioning identification method | |
| CN117095155A (en) | Multi-scale nixie tube detection method based on improved YOLO self-adaptive attention-feature enhancement network | |
| CN116934696A (en) | Industrial PCB defect detection method and device based on YOLOv7-Tiny model improvement | |
| CN116934685A (en) | Steel surface defect detection algorithm based on Focal module and deformable convolution | |
| CN110889418A (en) | Gas contour identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |