CN113034478B - Weld defect identification positioning method and system based on deep learning network - Google Patents

Abstract

The invention relates to the field of image recognition, in particular to a weld defect recognition positioning method and system based on a deep learning network. The method comprises the following steps: s1: acquiring a weld defect ray image, taking part of the image as a training data set and the rest images as a test data set; s2: carrying out normalization pretreatment on the images in the data set; s3: constructing an identification positioning network model for identifying and positioning the weld defect image; s4: detecting and processing the weld defect image by using the identification positioning network model; s5: training the identification positioning network model until a required training termination condition is reached; s6: based on the acquired new test data set, a trained identification positioning network is adopted to identify and position the weld defects, and the detection performance of the network module is evaluated. The method overcomes the defects of the traditional weld defect recognition method that the accuracy is low, the recognition positioning accuracy is insufficient and the detection efficiency is low.

Description

Weld defect identification positioning method and system based on deep learning network

Technical Field

The invention relates to the field of image recognition, in particular to a weld defect recognition positioning method and system based on a deep learning network.

Background

The identification of the radiation defect image is an important nondestructive testing method. In the field of ray weld defect image recognition, a manual on-line detection method is adopted, and is influenced by subjective experience of a quality inspector, so that the problems of missing detection, false detection and the like are easily caused when the detection task amount is large, and the accuracy of a detection result is influenced.

In order to solve the problems, researchers are continuously exploring to realize automatic identification of welding defects in the radiographic welding seam images by adopting an artificial intelligence algorithm. At present, two main types of methods for identifying and positioning the ray weld defect image based on an artificial intelligence algorithm exist, one type of method is based on a traditional neural network algorithm, and the other type of method is based on a deep learning convolutional neural network (convolution neural network) algorithm. The traditional neural network algorithm is used for identifying the ray weld defect image, firstly, the image is segmented, the weld part in the image is separated, then, the characteristics such as geometric dimensions, textures and the like of the weld defect are extracted and screened based on manual experience, and finally, the characteristic parameters are used as input, and the identification and the positioning of the weld defect in the ray image are realized through the traditional neural network. The conventional neural network algorithms mainly include SVM, BP, ANFIS, adaBoost and RBF, PCA, ANN, a multi-layer perceptron (MLP), and the like. The main problems of this type of method are: the radiographic image is complex, accurate segmentation of the image is difficult, extraction and screening of features are affected by artificial factors, rich information contained in the image cannot be fully utilized, and the diversity of the image cannot be comprehensively expressed.

The convolutional neural network based on deep learning directly takes an image as input, does not need to manually extract target features, and can automatically learn complex depth features in the ray weld defect image. Based on these depth features, convolutional neural networks, through their good fault tolerance, parallelism, and generalization capabilities, can find the location of defective targets and classify each target. The intelligent identification and positioning from the input of the original ray weld image to the output of the weld defect classification and the end-to-end of the defect position are truly realized.

At present, target recognition and positioning algorithms based on deep learning convolutional neural networks are divided into one-stage and two types of methods. The one-stage method classifies and locates the generated anchor directly, while the two-stage method firstly generates a candidate region (region candidates) and then maps the candidate region to the feature map for classification and location. Therefore, the two-stage method is very time-consuming, and the performance of the two-stage method is difficult to meet the requirement of real-time detection; the one-stage method represented by the YOLO series algorithm and the SSD algorithm has the defects of lower accuracy and recall rate and insufficient recognition and positioning accuracy, although the detection efficiency is obviously improved compared with that of two-stage; it is difficult to meet the requirements of accuracy and instantaneity of weld defect detection.

Disclosure of Invention

In order to overcome the problems in the prior art, the method and the system for identifying and positioning the weld defects based on the deep learning network provided by the invention overcome the defects of low accuracy, insufficient identification and positioning precision and low detection efficiency of the traditional weld defect identification method.

The technical scheme provided by the invention is as follows:

a weld defect identification positioning method based on a deep learning network comprises the following steps:

s1: acquiring weld defect ray images containing air holes, slag inclusions, cracks, unfused and incomplete penetration defects, taking part of the images as a training data set and the rest images as a test data set;

s2: carrying out normalization preprocessing on the images in the test data set or the training data set, and obtaining a uniform-resolution image block of the weld defect image after preprocessing;

s3: constructing an identification positioning network model for identifying and positioning the weld defect image; the identifying and positioning network model comprises a feature extraction module, a target detection module and an output module;

s4: detecting and processing the preprocessed weld defect image by using an identification positioning network model, and outputting a prediction conclusion; the process comprises the following steps:

S41: extracting shallow layer features and deep layer features in the weld defect image by using a feature extraction module in an identification positioning network;

s42: reconstructing the features extracted from different layers with an original image by using a target detection module and an image gradient rising method to obtain low-layer features rich in detail information and high-layer features rich in semantic information, and realizing transverse and longitudinal short connection through FPN backstbone and Bottom-up path augmentation in the target detection module;

s43: extracting features from CSPDens_bolck in a feature extraction module, performing double up-sampling operation on branches with different resolutions, cascading an up-sampled feature layer with a shallow feature layer, and respectively performing independent detection on a fused feature map with multiple scales;

s44: an anchor mechanism is introduced into the YOLO of the output module, and an anchor value is acquired by using a K-means clustering method, so that the initializing stage of the network training is more in line with the parameters of the object to be detected; finally, combining the target positions and the category information extracted on different scales by adopting a maximum suppression algorithm to obtain a final detection result;

s5: adjusting parameters of network model training, and training the identification positioning network model in the step S3 by adopting the method in the step S4 and the preprocessed training data set obtained in the step S2 until a required training termination condition is reached;

S6: based on the acquired new test data set, the trained recognition positioning network in the step S5 is adopted to recognize and position the weld defect result, and the detection performance of the network module is evaluated.

Further, in step S2, the image preprocessing method includes: cutting the collected original image according to the specification of 320 multiplied by 320 pixels to generate an image as an input image block, and for the original image with the width and the height which are not 320 multiplied by 320, completing cutting in a mode of partially reserving an overlapping area in the image, so that all the cut image blocks keep the same specification, and finally numbering all the image blocks belonging to the same original image data according to the sequence.

Further, the identification and positioning network model takes the whole image as input data, the input image is divided into N multiplied by N grids, each divided unit grid is responsible for detecting a target with a center point falling in the grid, and the generated anchors are directly classified and positioned; wherein, the liquid crystal display device comprises a liquid crystal display device,

the feature extraction module adopts a CSPDens_block module combined with a CSPNet network and a DensNet network, and the CSPDens_block module is applied to a main network to realize feature extraction of the ray weld defect image; the target detection module adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; the output module adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; and performing NMS processing on the bounding box with higher confidence coefficient obtained by calculation to obtain a final detection result.

Further, in step S41, the processing procedure of the cspdens_block module in the feature extraction stage is as follows:

s411: the CSPDens_block divides the feature map obtained by the convolution of the upper layer into 2 parts, one part passes through a Dens module, and the other part directly performs connection expansion of the feature map with the output of the Dens so as to enable gradient flows to propagate on different network paths;

s412: the trans-layer characteristic information transmission is realized through the Dens module, and the characteristic information is directly transmitted to the following network layer by skipping part of the network layers, so that the network learns the characteristic connection between more layers; and then, a network for densely connecting all the front layers with the rear layer is established on the basis of ResNet through DensNet, so that the characteristic reuse is realized;

the calculation formula of the channel connection is as follows:

x _l ＝H _l ([x ₀ ,x ₁ ,......,x _l-1 ])

in the above formula: [ x ] ₀ ,x ₁ ,......,x _l-1 ]Output feature map of layer 0, … …, l-1, H _l Representing channel merge operations, H _l Including 3 x 3 convolutions, BN and leakey ReLU;

s413: the separable convolution is adopted to replace the traditional convolution, and the separable convolution decomposes a complete convolution operation into two steps, namely Depthwise Convolution and Pointwise Convolution; the Depthwise Convolution convolution is performed entirely in a two-dimensional plane, the number of filters is the same as the Depth of the previous layer, and the Pointwise Convolution uses a 1×1 convolution kernel Depthwise Convolution to perform weighted combination on the feature map in the Depth direction.

Further, in step S43, the scale fusion process of the target detection stage is as follows:

s431: the multi-scale detection module in the YOLOv4 is improved, and the original 3 scales are expanded to 4 scales;

s432: the original input size is 320×320, and the resolution size and convolution kernel of the Dens module operation in the cspdens_bolck are 160×160, 32 in order; 80 x 80, 64;40 x 40, 128;20×20, 256; each branch of the target detection module detects the feature map after CSPDens_bolck multiscale fusion;

s433: the operation and convolution kernel of the Dens module in the 2, 3, 4, 5 layers CSPDens_bolck is reduced by 1/2 relative to the YOLO; double up-sampling operation is carried out on branches with the resolution of 10 multiplied by 10, 20 multiplied by 20 and 40 multiplied by 40, the up-sampled feature layer and the shallow feature layer are cascaded, and independent detection is carried out on the fusion feature images with 4 scales respectively;

s434: the improved multi-scale fusion is expanded into predicting the object to be detected for the four scale feature images of 10×10, 20×20, 40×40 and 80×80, learning the position features from the shallow feature layer, and carrying out exact fine granularity detection on the deep features after the fusion and the up-sampling.

Further, in step S44, the dimension clustering is performed again by the K-means algorithm, and the IOU value of the anchor box and the group trunk needs to be made as large as possible, so that the objective function of the distance measure adopts the ratio DIOU of the intersection and union of the prediction bounding box and the real bounding box as a measure, and the formula of the measure function is as follows:

In the above formula, targ_box is the target box of the sample label, cent is the cluster center, d represents the measurement distance, and DIOU represents the ratio of the intersection and union of the prediction bounding box and the real bounding box.

Further, in the target detection stage, the detected defects in the image blocks determine their positions in the original image through coordinate conversion, and for any image block, the image block is divided into s×s grids, each of which predicts B rectangular bounding boxes containing target defects and C probability values belonging to a certain class; each rectangular bounding box contains 5 data values, namely: (x, y, w, h, confidence), where (x, y) is the offset of the center of the rectangular bounding box relative to the cells, (w, h) is the width and height of the rectangular bounding box, confidence is the confidence that the target belongs to a certain class of defects in a certain grid;

then, for s×s grids into which the image with width W and height H is divided, the coordinates of one grid in the image are set to (x _i ,y _j )，x _i And y _j The value range of (5) is 0,S-1, and the coordinates of the central point of the prediction boundary frame are (x) _c ,y _c ) The final predicted position (x, y) normalization process formula is as follows:

the confidence value is used to represent the probability of whether the bounding box contains the object and the overlap ratio of the current bounding box and the real bounding box, and the calculation formula is as follows:

In the above, P _r (obj) represents the probability of the target defect being present in the grid, if so, P _r (obj) =1, if not, P _r (obj) =0; DIOU denotes the ratio of the intersection and union of the prediction bounding box and the real bounding box;

the formula of the output probability P of each grid prediction is as follows:

in the above, P _r (obj) represents the probability of the existence of a target defect in the grid, P _r (class _i I obj) indicates that the grid contains conditional probabilities of target defects belonging to the i-th category, P _r (class _i ) Representing the probability of a class i target defect; DIOU represents the intersection and union of the prediction bounding box and the real bounding boxIs a ratio of (c).

Further, in step S5, during training of the network model, the leak ReLU is used as an activation function, and the coefficient when x is less than or equal to 0 is adjusted to 0.01 according to the detected target feature, where the formula is as follows:

defining a loss function for a training network includes three parts: the bounding box loss, the confidence loss and the classification loss are calculated according to the following formulas:

loss＝loss _coord +loss _conf +loss _class

wherein loss is _coord Representing a bounding box loss function, the calculation formula is as follows:

in the above formula:

values representing the abscissa, ordinate, width, height of the center of the real target bounding box, x _i ,y _i ,w _i ,h _i Values representing the abscissa, ordinate, width, height of the prediction target bounding box, s×s is the number of divided meshes, B is the number of prediction bounding boxes per mesh,/S >

Judging whether an ith grid where a jth bounding box is positioned is responsible for detecting the defect, if so, selecting the responsibility with the largest DIOU value with the real bounding box; lambda (lambda) _coord Is a punishment coefficient of coordinate prediction, and has the function that when a network traverses the whole image, each grid does not necessarily contain target defects, and when the grid does not contain the target defects, the confidence coefficient is 0, so that the training gradient spans greatly, the final model is unstable, and in order to solve the problem, a super-parameter lambda is arranged in a loss function _coord The method is used for controlling the loss of the predicted position of the target frame; />

Is the adjustment parameter of the convergence speed of the network training;

loss _conf the confidence loss function is represented by the following calculation formula:

in the above formula:

representing the true confidence that the target defect belongs to a certain category in the ith grid, c is the prediction confidence,/o>

No target defect, lambda is contained in the jth bounding box representing the ith grid _noobj A penalty coefficient representing the confidence level when the detection target is not included in the grid;

loss _class the classification loss function is represented, and the calculation formula is as follows:

in the above formula: c represents the predicted target defect class,

true probability value, p, representing that the object in the ith grid belongs to a certain class of defect _i (c) Predictive probability value representing that the object in the ith grid belongs to a certain class of defect,/for >

Indicating whether the ith grid is responsible for the target defect.

The invention also comprises a weld defect identification and positioning system based on the deep learning network, which adopts the weld defect identification and positioning method based on the deep learning network to finish the identification and positioning of the weld defect in the weld ray image and give out a prediction result; the system comprises: the system comprises an image acquisition module, an image preprocessing module and an identification positioning network module.

The image acquisition module is used for acquiring a weld defect ray image containing air holes, slag inclusions, cracks, unfused and unskibbled defects, taking the image as a training set or a testing set, and completing training of a system or completing recognition and positioning tasks of the weld defects in the image based on the image in the training set or the testing set.

The image preprocessing module is used for carrying out normalization preprocessing on images in a training set or a testing set, so that tiles with uniform resolution about weld defect images are obtained after preprocessing.

The recognition positioning network module takes the processed image as input data, divides the input image into N multiplied by N grids, and enables each divided unit grid to be responsible for detecting a target with a center point falling in the grid, and directly classifies and positions the generated anchors; the identifying and positioning network module comprises a feature extraction sub-module, a target detection sub-module and an output sub-module, wherein the feature extraction sub-module adopts a CSPDens_block module combined with a CSPNet network and a DensNet network, and the CSPDens_block module is applied to a main network to realize feature extraction of a ray weld defect image; the target detection submodule adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; the output submodule adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; and performing NMS processing on the bounding box with higher confidence coefficient obtained by calculation to obtain a final detection result.

The invention also includes a weld defect identification and location terminal based on a deep learning network, the terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the weld defect identification and location method based on the deep learning network as described above.

The weld defect identification and positioning method based on the deep learning network has the following beneficial effects:

1. the invention provides a one-stage weld defect identification positioning method based on a deep learning network. The method improves the YOLO network by adopting the methods of feature pyramid, reducing network depth, introducing jump connection convolution blocks, K-means algorithm and the like, and improves the accuracy and speed of the network for identifying and positioning the weld defects. The method has the online real-time identification and positioning capability of weld defects, and improves the engineering application value of the method.

2. The method improves the original YOLO network, so that the accuracy of the algorithm in identifying and positioning the weld defect target is improved. And the feature extraction capability is effectively improved by adopting CSPDense block and separable convolution, shallow feature information is fully utilized by upward shift concatat operation, deep semantic features are fused, the characterization capability of a feature pyramid is enhanced, unnecessary convolution and modules are reduced, and the operation amount of a network is greatly reduced.

3. Compared with the two-stage target detection algorithm, the method provided by the invention has the advantages that the accuracy and recall rate of weld defect detection are improved to a certain extent, and compared with the original YOLO algorithm, the method has the advantages that the detection speed and the recognition accuracy are improved, and the requirements of the weld defect detection accuracy and the instantaneity can be met.

Drawings

FIG. 1 is a flowchart of a weld defect identification and localization method based on a deep learning network provided in this embodiment 1;

fig. 2 is a schematic diagram of the structure of the identifying and locating network model in embodiment 1;

fig. 3 is a schematic structural diagram of a cspdens_block module in this embodiment 1, where part (a) in the figure is a schematic flow diagram of a cspdens_block process, part (b) in the figure is a schematic flow diagram of a separable convolution process, and part (c) in the figure is a schematic diagram of a Dens module implementing cross-layer feature information transfer;

FIG. 4 is a partial image sample of the weld defect radiographic image acquired in this example 2;

FIG. 5 is a distribution curve of the K-means cluster analysis result in the present example 2;

FIG. 6 is a graph showing the correlation between the loss function value and the iteration number of the control group in the network training process of the present embodiment 2;

FIG. 7 is an average cross-correlation curve of the method of the present embodiment with the control group during the network training process of the present embodiment 2;

FIG. 8 is a graph showing the comparison of the defect results at the detection of the method and the control group according to the embodiment 2 for the same input image;

FIG. 9 is a schematic block diagram of a weld defect identification and localization system based on a deep learning network provided in this example 3;

marked in the figure as:

1. an image acquisition module; 2. an image preprocessing module; 3. identifying a positioning module; 31. a feature extraction sub-module; 32. a target detection sub-module; 33. and outputting a sub-module.

Detailed Description

The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Example 1

As shown in fig. 1, the embodiment provides a weld defect identifying and positioning method based on a deep learning network, which includes the following steps:

s1: and acquiring weld defect radiographic images containing air holes, slag inclusions, cracks, unfused and unwelded defects, taking part of the images as a training data set and the rest images as a test data set.

S2: carrying out normalization preprocessing on the images in the test data set or the training data set, and obtaining a uniform-resolution image block of the weld defect image after preprocessing;

The preprocessing method of the image comprises the following steps: cutting the collected original image according to the specification of 320 multiplied by 320 pixels to generate an image as an input image block, and for the original image with the width and the height which are not 320 multiplied by 320, completing cutting in a mode of partially reserving an overlapping area in the image, so that all the cut image blocks keep the same specification, and finally numbering all the image blocks belonging to the same original image data according to the sequence.

S3: and constructing an identification and positioning network model for identifying and positioning the weld defect image.

The convolutional neural network based on deep learning can learn and extract abundant shallow features such as edges, textures and the like in the image through convolution and downsampling operations, and can also learn and extract deep features such as structures, semantics and the like. However, when the network depth is large and the downsampling operation is more, the detail information of the image can be lost, so that part of small target features disappear, and the recognition and positioning effects on the small targets are poor. The object identified and positioned in the invention not only comprises small target weld defects such as air holes, slag inclusions, tiny cracks and the like, but also comprises larger weld defects such as unfused and incomplete penetration and the like. The design of the identification positioning network model is therefore fully designed to take into account the large differences in defect scale.

Based on the above reasons, the weld defect image recognition positioning network constructed in the embodiment is composed of a feature extraction module, a target detection module and an output module.

In the deep learning neural network, CSPNet has the advantages of less parameters, small calculated amount, strong generalization performance, capability of effectively improving the characteristic learning capability of the network and the like, and DensNet can effectively relieve gradient dispersion, so that information can be smoothly propagated forwards and backwards.

The target detection module adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; so as to improve the accuracy of restoring the characteristic image pixels during the identification and positioning.

The output module adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; and performing NMS processing on the bounding box with higher confidence coefficient obtained by calculation to obtain a final detection result.

The overall structure of the network model is shown in fig. 2, in which Conv represents Convolution, concat represents feature map connection, and CBL represents Convolume, batch Normalization, and leak ReLU.

S4: detecting and processing the preprocessed weld defect image by using an identification positioning network model, and outputting a prediction conclusion; the process comprises the following steps:

s41: and extracting shallow layer features and deep layer features in the weld defect image by using a feature extraction module in the identification and positioning network.

In general, a convolutional neural network based on deep learning regards an input picture as being formed by overlapping pictures with various different features, the features of the whole input picture are scanned through a plurality of filters, the structured deep semantic features are extracted through downsampling, and more filters and deeper networks can extract more features of the picture. However, in the practical application process, the deeper the network is, the more errors are accumulated, so that the problem of gradient dissipation occurs, and the performance of the network is reduced.

Therefore, the processing procedure of the improved cspdens_block module in the feature extraction stage in this embodiment is as follows:

s411: the CSPDens_block divides the feature map obtained by the convolution of the upper layer into 2 parts, one part passes through a Dens module, and the other part directly performs connection expansion of the feature map with the output of the Dens so as to enable gradient flows to propagate on different network paths; therefore, the network calculated amount is reduced, and meanwhile, richer gradient fusion information is obtained, and the reasoning speed and accuracy are improved. The structure of the CSPDens_block is shown in part (a) of FIG. 3.

S412: as shown in part (c) of fig. 3, cross-layer feature information transmission is realized through a Dens module, and feature information is directly transmitted to a following network layer by skipping part of the network layers, so that the network learns the feature connection between more layers; and then, a network for densely connecting all the front layers with the rear layer is established on the basis of ResNet through DensNet, so that the characteristic reuse is realized; the Dens operation allows the network to learn the feature connections between more levels. Thus, the loss and gradient disappearance of the characteristic information in the layer-by-layer transmission are reduced, and the transmission of the characteristic in the network is quickened.

The method adopted by the Dens module is not addition among pixels of the feature map, but connection among channels, and the calculation formula is as follows:

x _l ＝H _l ([x ₀ ,x ₁ ,......,x _l-1 ])

in the above formula: [ x ] ₀ ,x ₁ ,......,x _l-1 ]Output feature map of layer 0, … …, l-1, H _l Representing channel merge operations, H _l Including 3 x 3 convolutions, BN and leakey ReLU;

s413: as the number of network layers increases, the conventional convolution of the multi-channel input image causes an exponential increase in the computational effort.

Therefore, in order to further improve the network performance, the inter-channel connection tensor is controlled, in this embodiment, a separable convolution is used instead of the conventional convolution for extracting the features, and the separable convolution decomposes a complete convolution operation into two steps, as shown in part (b) of fig. 3, which are Depthwise Convolution and Pointwise Convolution respectively; depthwise Convolution convolution is performed entirely in a two-dimensional plane, with the same number of filters as the Depth of the previous layer, and Pointwise Convolution weighted combination in the Depth direction using the feature map output by the 1 x 1 convolution kernel Depthwise Convolution. This ensures feature delivery and reduces computation.

S42: and reconstructing the features extracted from different layers with the original image by using a target detection module and an image gradient rising method to obtain low-layer features rich in detail information and high-layer features rich in semantic information, and realizing transverse and longitudinal short connection through FPN backstbone and Bottom-up path augmentation in the target detection module.

Because the deep layer neurons are activated by the structural features of the wider receptive field, and the shallow layer neurons are feature images generated by the activation of the features such as local edges, textures and the like, the network realizes the transverse and longitudinal short connection through the FPN backstbone and the Bottom-up path augmentation, so that the deep and shallow layer feature information can circulate faster, and the positioning and detection capability of the feature structure is further improved. After improvement, by fusing more characteristic information, the characteristic feature characterization capability of the characteristic pyramid is enhanced, so that the detection precision of the small defect target is improved, and the omission ratio is reduced.

S43: extracting features from CSPDens_bolck in a feature extraction module, performing double up-sampling operation on branches with different resolutions, cascading an up-sampled feature layer with a shallow feature layer, and respectively performing independent detection on a fused feature map with multiple scales;

The scale fusion process of the target detection stage is as follows:

s431: in the ray weld defect image detection, small target defects generally have only tens or even fewer pixels, and semantic information which can be extracted from the network by the few pixels is very limited. In the feature extraction process, shallow features have higher resolution and are rich in stronger position information; deep features have strong semantic information, but location information is coarser. According to the image gradient rising method, the original image is reconstructed by utilizing the features extracted from different layers to obtain the conclusion that the low-layer features rich in detail information and the high-layer features rich in semantic information can better assist in target detection.

The traditional YOLOv4 network uses 3 feature images with different scales to predict the target to be detected, the feature images output by the last two residual_bolck blocks are up-sampled, and the feature images with corresponding sizes of the shallow layers of the network are fused into effective information to be predicted.

In this embodiment, in order to more fully utilize shallow features and position information, the multi-scale detection module in YOLOv4 is improved, and the original 3 scales are expanded to 4 scales.

S432: the original input size is 320×320, and the resolution size and convolution kernel of the Dens module operation in the cspdens_bolck are 160×160, 32 in order; 80 x 80, 64;40 x 40, 128;20×20, 256; each branch of the target detection module detects the feature map after CSPDens_bolck multiscale fusion;

S433: in order to reduce the operation amount and improve the detection speed, the operation and convolution kernel of the Dens module in the 2, 3, 4 and 5 layers CSPDens_bolck are reduced by 1/2 relative to the YOLO in the embodiment; double up-sampling operation is carried out on branches with the resolution of 10 multiplied by 10, 20 multiplied by 20 and 40 multiplied by 40, the up-sampled feature layer and the shallow feature layer are cascaded, and independent detection is carried out on the fusion feature images with 4 scales respectively;

s434: the improved multi-scale fusion is expanded into predicting the object to be detected for the four scale feature images of 10×10, 20×20, 40×40 and 80×80, learning the position features from the shallow feature layer, and carrying out exact fine granularity detection on the deep features after the fusion and the up-sampling.

In the embodiment, the characterization capability of the feature pyramid is enhanced by fusing shallow feature information with more scales, the detection precision of small target defects is improved, the omission ratio is reduced, and finally the redundant frame is removed through a non-maximum suppression algorithm.

S44: an anchor mechanism is introduced into the YOLO of the output module, and an anchor value is acquired by using a K-means clustering method, so that the initializing stage of the network training is more in line with the parameters of the object to be detected; and finally, combining the target positions and the category information extracted on different scales by adopting a maximum suppression algorithm to obtain a final detection result. The dimension clustering is performed again through a K-means algorithm, and IOU values of an anchor box and a group trunk are required to be made to be as large as possible, so that an objective function of distance measurement adopts a ratio DIOU of an intersection and a union of a prediction boundary box and a real boundary box as a measurement standard, and a formula of the measurement function is as follows:

In the above formula, targ_box is the target box of the sample label, cent is the cluster center, d represents the measurement distance, and DIOU represents the ratio of the intersection and union of the prediction bounding box and the real bounding box.

S5: and (3) adjusting parameters of the network model training, and training the identification positioning network model in the step (S1) by adopting the method in the step (S4) and the preprocessed training data set obtained in the step (S2) until the required training termination condition is reached.

During network model training, the leak ReLU is adopted as an activation function, and the coefficient when x is less than or equal to 0 is adjusted to be 0.01 according to the detected target characteristic, and the formula is as follows:

defining a loss function for a training network includes three parts: the bounding box loss, the confidence loss and the classification loss are calculated according to the following formulas:

loss＝loss _coord +loss _conf +loss _class

wherein loss is _coord Representing a bounding box loss function, the calculation formula is as follows:

in the above formula:

values representing the abscissa, ordinate, width, height of the center of the real target bounding box, x _i ,y _i ,w _i ,h _i Values representing the abscissa, ordinate, width, height of the prediction target bounding box, s×s is the number of divided meshes, B is the number of prediction bounding boxes per mesh,/S>

Judging whether an ith grid where a jth bounding box is positioned is responsible for detecting the defect, if so, selecting the responsibility with the largest DIOU value with the real bounding box; lambda (lambda) _coord Is a penalty factor for coordinate prediction, which functions such that each grid does not have to be used when traversing the entire imageThe confidence level of 0 when the target defect is not included is included, so that the training gradient is greatly crossed, the final model is unstable, and in order to solve the problem, the super parameter lambda is set in the loss function _coord The method is used for controlling the loss of the predicted position of the target frame; />

Is the adjustment parameter of the convergence speed of the network training; />

loss _conf The confidence loss function is represented by the following calculation formula:

in the above formula:

representing the true confidence that the target defect belongs to a certain category in the ith grid, c is the prediction confidence,/o>

No target defect, lambda is contained in the jth bounding box representing the ith grid _noobj A penalty coefficient representing the confidence level when the detection target is not included in the grid;

loss _class the classification loss function is represented, and the calculation formula is as follows:

in the above formula: c represents the predicted target defect class,

true probability value, p, representing that the object in the ith grid belongs to a certain class of defect _i (c) Predictive probability value representing that the object in the ith grid belongs to a certain class of defect,/for>

Indicating whether the ith grid is responsible for the target defect.

S6: based on the acquired new test data set, the trained recognition positioning network in the step S5 is adopted to recognize and position the weld defect result, and the detection performance of the network module is evaluated.

In this embodiment, the recognition and positioning network model uses the whole image as input data, the input image is divided into n×n grids, each divided cell is responsible for detecting the target with the center point falling in the grid, and the generated anchor is directly classified and positioned.

In the embodiment, target defects with different scales and types exist in the weld defect image, different target defects have different characteristics, and the real characteristics of the weld defects can be more accurately reflected by fusing the characteristic information and the spatial information of the deep layer and the shallow layer. In the process of extracting the characteristics, the image blocks are subjected to multi-channel convolution and downsampling operation to extract target characteristics, and then various simple targets and complex targets are accurately identified and positioned on the net. Therefore, in the network of the embodiment, the feature fusion is realized by upsampling and connecting the feature graphs of different layers.

Thus, in the target detection phase, the detected defects in the tile determine their position in the original image by coordinate conversion, for any tile the tile is divided into s×s grids, each grid predicting B rectangular bounding boxes containing target defects and C probability values belonging to a certain class; each rectangular bounding box contains 5 data values, namely: (x, y, w, h, confidence), where (x, y) is the offset of the center of the rectangular bounding box relative to the cells, (w, h) is the width and height of the rectangular bounding box, confidence is the confidence that the target belongs to a certain class of defects in a certain grid;

Then, for s×s grids into which the image with width W and height H is divided, the coordinates of one grid in the image are set to (x _i ,y _j )，x _i And y _j The value range of (5) is 0,S-1, and the coordinates of the central point of the prediction boundary frame are (x) _c ,y _c ) The final predicted position (x,y) normalization formula is as follows:

the confidence value is used to represent the probability of whether the bounding box contains the object and the overlap ratio of the current bounding box and the real bounding box, and the calculation formula is as follows:

in the above, P _r (obj) represents the probability of the existence of a target defect in the grid, if so, P _r (obj) =1, if not, P _r (obj) =0; DIOU denotes the ratio of the intersection and union of the prediction bounding box and the real bounding box;

the formula of the output probability P of each grid prediction is as follows:

in the above, P _r (obj) represents the probability of the existence of a target defect in the grid, P _r (class _i I obj) indicates that the grid contains conditional probabilities of target defects belonging to the i-th category, P _r (class _i ) Representing the probability of a class i target defect; DIOU denotes the ratio of the intersection and union of the prediction bounding box and the real bounding box.

Example 2

The present embodiment provides a simulation experiment (in other embodiments, the simulation experiment may not be performed, and other experimental schemes may be adopted to perform experiments to determine the influence of the network model and the related parameters thereof on the weld joint identification and positioning performance of the method) of the weld joint defect identification positioning method based on the deep learning network as in embodiment 1.

Experimental conditions

In the embodiment, the detection experiment adopts an operating system Windows10, a CPU 7-8700k, a GPU 1080ti, a memory 16GB and a deep learning framework tensorf low.

Initialization parameter setting of network training: the maximum iteration is 50 000 times, the learning rate is 0.001, the batch_size is set to be 32, the weight attenuation coefficient is 0.0005, the impulse constant is 0.9, and the learning rate and the value of the batch_size are properly adjusted according to the descending trend of the loss, and training is stopped until the loss function value is smaller than or equal to the experience threshold value.

(II) data set acquisition

In this embodiment, the experimental image data is from an obstetric and research corporation. And acquiring 920 radiographic images of 5 common weld defects, namely air holes, slag inclusions, cracks, unfused welding and incomplete penetration. 800 images of each defect are randomly extracted and used as network training, and the rest 120 images are used as test set images.

In addition, in order to avoid network overfitting caused by small data size of the training set, the embodiment also cuts, turns over, translates, adjusts contrast and noise disturbance changes on 800 original images, expands various defect images, and generates 50 images, wherein 10 076 images containing air hole defects, 9847 images containing slag defects, 10 150 images containing crack defects, 10 326 images containing unfused defects and 9763 images containing unfused defects, and a part of weld X-ray image samples in the experiment are shown in fig. 4.

(III) Cluster analysis

In the one-stage target recognition positioning network of the embodiment, the YOLO introduces an anchor mechanism, and acquires an anchor value by using a K-means clustering method, so that the initializing stage of network training is more in line with the parameters of the object to be detected, and the deviation between the initializing parameters and the optimized parameters is reduced.

The number and size of the anchor boxes directly affect the accuracy and speed of defect target identification and location, so it is particularly important to set appropriate anchor parameters. However, the YOLO original algorithm is an anchor value obtained after training on the COCO and VOC data set, and is not suitable for weld defect detection studied in this example. Therefore, the dimension clustering is carried out again through the K-means algorithm, the result of the clustering analysis of the labels by the K-means algorithm is shown in figure 5,

in the image, the first 12 anchor values are selected: (7, 9), (13, 17), (21, 37), (36, 52), (69, 48), (12, 48), (96, 18), (24, 265), (180, 22), (57, 258), (168, 63), (132, 265) are assigned to feature maps of 4 scales by area size, the feature map of larger scale uses smaller anchor boxes, and 3 prediction boxes are calculated per grid.

(IV) model test results

And (3) retrieving a defect image training set from a database of the weld X-ray images, and training the YOLO network module and the CSPDensNet network module for 12.6h and 13.8h respectively. And then, calling the testing set data image to input the testing set data image into the YOLO network module and the CSPDensNet network module for detection.

The curves of the loss function values loss and the average intersection ratio IoU of the two network models are compared respectively. Wherein, the correlation curve of the loss function value and the iteration number in the network training process is shown in fig. 6. The average cross-ratio curve of training is shown in figure 7.

The loss function values for the YOLO model iterated over the training set and the test set are shown in fig. 6 as curves yolo_train and yolo_test. The loss function values iterated over the training set and the test set using CSPDensNet are shown in FIG. 6 as curves CSPDnet_train and CSPDnet_test.

Analysis of the results in fig. 6 can find that: the CSPDENNet network model trains the value of the loss function on the validation set representing the recognition accuracy better than YOLO. The value of the loss function of the CSPDensNet model is stable gradually and finally reaches about 2%, while the value of the loss function of the YOLO model drops rapidly, but after reaching the minimum value for 4500 times, the value of the loss function rises after oscillation, and finally reaches about 4%.

Meanwhile, as can be seen from analysis of fig. 7, the average cross-over ratio of the anchor box and the ground trunk is also significantly higher than YOLO in the training process of the CSPDensNet network model.

(V) Performance evaluation

In the field of target detection, accuracy (Precision) and Recall (Recall) are important criteria for evaluating the quality of a detection network model, in this embodiment, the two indexes and detection time are adopted to evaluate an experimental result, and a traditional YOLOv4 network is used as a control group to perform a comparison experiment, so that the performance of the test network and the traditional YOLOv4 network provided in this embodiment are compared.

The formulas of accuracy (Precision) and Recall (Recall) are as follows:

in the above formula, TP represents the number of positive samples detected, i.e., the number of samples for which the detected defect classification is correct; FP represents the negative number of samples detected as positive samples, i.e. the number of samples detecting a defect classification error; FN is detected as a positive number of samples of negative samples, i.e. a number of samples which are not detected but actually contain defects.

The test results obtained by statistics in the simulation test are shown in table 1:

table 1: weld defect detection result statistical table of network and comparison group of embodiment

According to analysis of the test results, compared with a traditional YOLOv4 algorithm, the method has the advantages that on detection of 5 common weld defects of air holes, slag inclusion, cracks, incomplete fusion and incomplete penetration, the accuracy rate or recall rate is improved obviously.

FIG. 8 is a diagram showing a partial detection result in a simulation experiment according to the present embodiment; lines (a) through (e) are images of pinholes (Pore), slag inclusions (Slag), cracks (Crack), unfused (LOF), and lack of penetration (LOP) defects, respectively. The (1) column in fig. 8 is a partial image of a defect of a weld joint to be detected, the (2) column is a detection result of YOLOv4 algorithm, and the (3) column is a detection result in this embodiment.

Comparing the identification and location results of the air holes (Pore), slag inclusion (Slag) and Crack (Crack) defects in the (2) th and (3) th columns in fig. 8, it can be found that: the method in the embodiment has higher accuracy in identifying and positioning various defects under the condition of missed detection of YOLOv 4.

In addition, in this embodiment, the same data set is used to compare the method with some classical CNN target detection algorithms based on candidate regions, and the evaluation index is the average accuracy (mAP) of various defects. The higher mAP value indicates that the algorithm has better identification and positioning effects on various weld defects.

The experimental results of the average accuracy of the methods in this example and the methods in the control group are shown in table 2:

table 2 the method and statistics of the detection results of different algorithms in this embodiment

Algorithm name	mAP(％)	Recall (%)	Detection time (ms)
				R-CNN	70.6	70.9	29500
Fast R-CNN	80.9	81.7	2380
				Faster R-CNN	93.1	93.6	1650
YOLOv4	87.7	88.5	24.89
				Algorithm herein	94.9	95.7	19.58

From table 2 it can be seen that: compared with the two R-CNN and Fast R-CNN algorithms based on two-stage, the method and the YOLOv4 algorithm provided by the embodiment based on one-stage have obvious advantages in detection speed and precision in weld defect identification and positioning. Compared with Faster R-CNN, the YOLOv4 is inferior in indexes of accuracy and recall, but is far superior to the algorithm in detection speed.

In addition, compared with the Faster R-CNN, the method provided by the embodiment has obvious advantages in terms of detection speed and detection precision.

Example 3

The embodiment provides a weld defect identification positioning system based on a deep learning network, which adopts the weld defect identification positioning method based on the deep learning network as in the embodiment 1 to finish the identification and positioning of the weld defect in the weld ray image and give out a prediction result; the system comprises: an image acquisition module 1, an image preprocessing module 2, and an identification positioning network module 3.

The image acquisition module 1 is used for acquiring a weld defect ray image containing air holes, slag inclusions, cracks, unfused and unskibbled defects, taking the image as a training set or a testing set, and completing training of a system or completing recognition and positioning tasks of the weld defects in the image based on the image in the training set or the testing set.

The image preprocessing module 2 is used for carrying out normalization preprocessing on images in a training set or a testing set, so that tiles with uniform resolution about weld defect images are obtained after preprocessing.

The recognition and positioning network module 3 takes the processed image as input data, divides the input image into N multiplied by N grids, and enables each divided unit grid to be responsible for detecting a target with a center point falling in the grid, and directly classifies and positions the generated anchors; the identifying and positioning network module 3 comprises a feature extraction sub-module 31, a target detection sub-module 32 and an output sub-module 33, wherein the feature extraction sub-module 31 adopts a CSPDens_block module combined with a CSPNet network and a DensNet network, and the CSPDens_block module is applied to a main network to realize feature extraction of a ray weld defect image; the target detection submodule 32 adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; the output sub-module 33 adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; and performing NMS processing on the bounding box with higher confidence coefficient obtained by calculation to obtain a final detection result.

Example 4

The present embodiment provides a weld defect recognition and location terminal based on a deep learning network, the terminal including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the weld defect recognition and location method based on the deep learning network as in embodiment 1.

The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.

Claims (9)

1. The weld defect identification and positioning method based on the deep learning network is characterized by comprising the following steps of:

s1: acquiring weld defect ray images containing air holes, slag inclusions, cracks, unfused and incomplete penetration defects, taking part of the images as a training data set and the rest images as a test data set;

s2: carrying out normalization pretreatment on images in a test data set or a training data set, and obtaining a uniform-resolution image block of the weld defect image after pretreatment;

s3: constructing an identification positioning network model for identifying and positioning the weld defect image; the identification positioning network model comprises a feature extraction module, a target detection module and an output module; the identification positioning network model takes the whole image as input data, the input image is divided into N multiplied by N grids, each divided unit grid is responsible for detecting a target of which the center point falls in the grid, and the generated anchors are directly classified and positioned; the feature extraction module adopts a CSPDens_block module combined with a CSPNet network and a DensNet network, and the CSPDens_block module is applied to a backbone network to realize feature extraction of the ray weld defect image; the target detection module adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; the output module adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; NMS processing is carried out on the bounding box with higher confidence coefficient obtained through calculation, and a final detection result is obtained;

S4: detecting and processing the preprocessed weld defect image by using the identification positioning network model, and outputting a prediction conclusion; the process comprises the following steps:

s41: extracting shallow layer features and deep layer features in the weld defect image by using a feature extraction module in an identification positioning network;

s42: reconstructing the features extracted from different layers with an original image by using a target detection module and an image gradient rising method to obtain low-layer features rich in detail information and high-layer features rich in semantic information, and realizing transverse and longitudinal short connection through FPN backstbone and Bottom-up path augmentation in the target detection module;

s43: extracting features from CSPDens_bolck in a feature extraction module, performing double up-sampling operation on branches with different resolutions, cascading an up-sampled feature layer with a shallow feature layer, and respectively performing independent detection on a fused feature map with multiple scales;

s44: an anchor mechanism is introduced into the YOLO of the output module, and an anchor value is acquired by using a K-means clustering method, so that the initializing stage of the network training is more in line with the parameters of the object to be detected; finally, combining the target positions and the category information extracted on different scales by adopting a maximum suppression algorithm to obtain a final detection result;

S5: adjusting parameters of network model training, and training the identification positioning network model in the step S3 by adopting the method in the step S4 and the preprocessed training data set obtained in the step S2 until a required training termination condition is reached;

defining a loss function for a training network includes three parts: the bounding box loss, the confidence loss and the classification loss are calculated according to the following formulas:

loss＝loss _coord +loss _conf +loss _class

wherein loss is _coord Representing a bounding box loss function, the calculation formula is as follows:

in the above formula:

values representing the abscissa, ordinate, width, height of the center of the real target bounding box, x _i ,y _i ,w _i ,h _i Values representing the abscissa, ordinate, width, height of the prediction target bounding box, s×s is the number of divided meshes, B is the number of prediction bounding boxes per mesh,/S>

Judging whether an ith grid where a jth bounding box is positioned is responsible for detecting the defect, if so, selecting the responsibility with the largest DIOU value with the real bounding box; lambda (lambda) _coord Is a punishment coefficient of coordinate prediction, and has the function that when a network traverses the whole image, each grid does not necessarily contain target defects, and when the grid does not contain the target defects, the confidence coefficient is 0, so that the training gradient spans greatly, the final model is unstable, and in order to solve the problem, a super-parameter lambda is arranged in a loss function _coord The method is used for controlling the loss of the predicted position of the target frame; />

Is the adjustment parameter of the convergence speed of the network training, wherein theta ₁ And theta ₂ Is an initial parameter set during network training;

the loss is _conf The confidence loss function is represented by the following calculation formula:

in the above formula:

representing the true confidence that the target defect belongs to a certain category in the ith grid, c _i In order to predict the degree of confidence,

no target defect, lambda is contained in the jth bounding box representing the ith grid _noobj A penalty coefficient representing the confidence level when the detection target is not included in the grid;

the loss is _class The classification loss function is represented, and the calculation formula is as follows:

in the above formula: c represents the predicted target defect class,

true probability value, p, representing that the object in the ith grid belongs to a certain class of defect _i (c) Predictive probability value representing that the object in the ith grid belongs to a certain class of defect,/for>

Indicating whether the ith grid is responsible for the target defect;

s6: based on the acquired new test data set, the trained recognition positioning network in the step S5 is adopted to recognize and position the weld defect result, and the detection performance of the network module is evaluated.

2. The deep learning network-based weld defect identification and localization method of claim 1, wherein: in the step S2, the image preprocessing method includes: cutting the collected original image according to the specification of 320 multiplied by 320 pixels to generate an image as an input image block, and for the original image with the width and the height which are not 320 multiplied by 320, completing cutting in a mode of partially reserving an overlapping area in the image, so that all the cut image blocks keep the same specification, and finally numbering all the image blocks belonging to the same original image data according to the sequence.

3. The deep learning network-based weld defect identification and localization method of claim 1, wherein: in the step S41, the processing procedure of the cspdens_block module in the feature extraction stage is as follows:

s411: the CSPDens_block divides the feature map obtained by the convolution of the upper layer into 2 parts, one part passes through a Dens module, and the other part is directly connected with the output of the Dens to realize the connection expansion of the feature map, so that gradient flows are propagated on different network paths;

s412: the trans-layer characteristic information transmission is realized through the Dens module, and the characteristic information is directly transmitted to the following network layer by skipping part of the network layers, so that the network learns the characteristic connection between more layers; and then, a network for densely connecting all the front layers with the rear layer is established on the basis of ResNet through DensNet, so that the characteristic reuse is realized;

the calculation formula of the channel connection is as follows:

x _l ＝H _l ([x ₀ ,x ₁ ,......,x _l-1 ])

in the above formula: [ x ] ₀ ,x ₁ ,......,x _l-1 ]Output feature map of layer 0, … …, l-1, H _l Representing channel merge operations, H _l Including 3 x 3 convolutions, BN and leakey ReLU;

s413: the method comprises the steps of replacing traditional convolution with separable convolution, wherein the separable convolution is carried out by decomposing a complete convolution operation into two steps, namely Depthwise Convolution and Pointwise Convolution; the Depthwise Convolution convolution is performed entirely in a two-dimensional plane, the number of filters is the same as the Depth of the previous layer, and the Pointwise Convolution uses a 1×1 convolution kernel Depthwise Convolution to perform weighted combination on the feature map in the Depth direction.

4. The deep learning network-based weld defect identification and localization method of claim 3, wherein: in the step S43, the scale fusion process in the target detection stage is as follows:

s431: the multi-scale detection module in the YOLOv4 is improved, and the original 3 scales are expanded to 4 scales;

s432: the original input size is 320×320, and the resolution size and convolution kernel of the Dens module operation in the cspdens_bolck are 160×160, 32 in order; 80 x 80, 64;40 x 40, 128;20×20, 256; each branch of the target detection module detects the feature map after CSPDens_bolck multiscale fusion;

s433: the operation and convolution kernel of the Dens module in the 2, 3, 4, 5 layers CSPDens_bolck is reduced by 1/2 relative to the YOLO; double up-sampling operation is carried out on branches with the resolution of 10 multiplied by 10, 20 multiplied by 20 and 40 multiplied by 40, the up-sampled feature layer and the shallow feature layer are cascaded, and independent detection is carried out on the fusion feature images with 4 scales respectively;

s434: the improved multi-scale fusion is expanded into predicting the object to be detected for the four scale feature images of 10×10, 20×20, 40×40 and 80×80, learning the position features from the shallow feature layer, and carrying out exact fine granularity detection on the deep features after the fusion and the up-sampling.

5. The deep learning network-based weld defect identification and localization method of claim 4, wherein: in the step S44, the dimension clustering is performed again by the K-means algorithm, and the IOU value of the anchor box and the group trunk needs to be made as large as possible, so that the objective function of the distance measurement adopts the ratio DIOU of the intersection and union of the prediction bounding box and the real bounding box as the measurement standard, and the formula of the measurement function is as follows:

in the above formula, targ_box is the target box of the sample label, cent is the cluster center, d represents the measurement distance, and DIOU represents the ratio of the intersection and union of the prediction bounding box and the real bounding box.

6. The deep learning network-based weld defect identification and localization method of claim 5, wherein: in the target detection stage, the detected defects in the image blocks are subjected to coordinate conversion to determine the positions of the defects in the original image, and for any image block, the image block is divided into S multiplied by S grids, and each grid predicts B rectangular bounding boxes containing target defects and C probability values belonging to a certain class; each rectangular bounding box contains 5 data values, namely: (x, y, w, h, confidence), where (x, y) is the offset of the center of the rectangular bounding box relative to the cells, (w, h) is the width and height of the rectangular bounding box, confidence is the confidence that the target belongs to a certain class of defects in a certain grid;

Then, for s×s grids into which the image with width W and height H is divided, the coordinates of one grid in the image are set to (x _i ,y _j )，x _i And y _j The value range of (5) is 0,S-1, and the coordinates of the central point of the prediction boundary frame are (x) _c ,y _c ) The final predicted position (x, y) normalization process formula is as follows:

the confidence value is used to represent the probability of whether the bounding box contains the object and the overlap ratio of the current bounding box and the real bounding box, and the calculation formula is as follows:

/>

in the above, P _r (obj) represents the probability of the target defect being present in the grid, if so, P _r (obj) =1, if not, P _r (obj) =0; DIOU denotes the ratio of the intersection and union of the prediction bounding box and the real bounding box;

the formula of the output probability P of each grid prediction is as follows:

in the above, P _r (obj) represents the probability of the existence of a target defect in the grid, P _r (class _i I obj) indicates that the grid contains conditional probabilities of target defects belonging to the i-th category, P _r (class _i ) Representing the probability of a class i target defect; DIOU denotes the ratio of the intersection and union of the prediction bounding box and the real bounding box.

7. The deep learning network-based weld defect identification and localization method of claim 6, wherein:

in the step S5, the training of the network model uses the teakey ReLU as the activation function, and adjusts the coefficient when x is less than or equal to 0 to 0.01 according to the detected target feature, and the formula is as follows:

8. A weld defect identification and positioning system based on a deep learning network, which is characterized in that the weld defect identification and positioning method based on the deep learning network as claimed in any one of claims 1-7 is adopted to finish the identification and positioning of the weld defect in a weld ray image and give a prediction result; the system comprises:

the image acquisition module is used for acquiring a weld defect ray image containing air holes, slag inclusions, cracks, unfused and incomplete welding defects, taking the image as a training set or a testing set, and completing training of a system or completing recognition and positioning tasks of the weld defects in the image based on the image in the training set or the testing set;

the image preprocessing module is used for carrying out normalization preprocessing on images in a training set or a testing set so as to obtain tiles with uniform resolution on weld defect images after preprocessing; and

the recognition positioning network module takes the processed image as input data, divides the input image into N multiplied by N grids, and enables each divided unit grid to be responsible for detecting a target of which the center point falls in the grid, and directly classifies and positions the generated anchors; the identifying and positioning network comprises a feature extraction submodule, a target detection submodule and an output submodule, wherein the feature extraction submodule adopts a CSPDens_block module combined with a CSPNet network and a DensNet network, and the CSPDens_block module is applied to a main network to realize feature extraction of a ray weld defect image; the target detection submodule adopts FPN backbond and Bottom-up path augmentation in PANet to realize fusion of shallow features and deep features; the output submodule adopts a YOLO layer in YOLOv4 to realize classification and regression of the multi-scale targets; and performing NMS processing on the bounding box with higher confidence coefficient obtained by calculation to obtain a final detection result.

9. A weld defect identification locating terminal based on a deep learning network, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: the processor performs the deep learning network-based weld defect identification localization method of any one of claims 1-7.

Images