CN113643228A - Nuclear power station equipment surface defect detection method based on improved CenterNet network - Google Patents

Nuclear power station equipment surface defect detection method based on improved CenterNet network Download PDF

Info

Publication number
CN113643228A
CN113643228A CN202110575217.5A CN202110575217A CN113643228A CN 113643228 A CN113643228 A CN 113643228A CN 202110575217 A CN202110575217 A CN 202110575217A CN 113643228 A CN113643228 A CN 113643228A
Authority
CN
China
Prior art keywords
detection
nuclear power
network
defect
loss function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110575217.5A
Other languages
Chinese (zh)
Other versions
CN113643228B (en
Inventor
吕建成
郎九霖
商佳程
汤臣薇
贺喆南
徐褀
赵毅
赵宇翔
高毅
胡宴清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN202110575217.5A priority Critical patent/CN113643228B/en
Publication of CN113643228A publication Critical patent/CN113643228A/en
Application granted granted Critical
Publication of CN113643228B publication Critical patent/CN113643228B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Monitoring And Testing Of Nuclear Reactors (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a nuclear power station equipment surface defect detection method based on an improved CenterNet network, which comprises the steps of firstly obtaining a nuclear power station equipment surface picture and preprocessing the nuclear power station equipment surface picture; secondly, performing feature extraction on the preprocessed picture by using a DLAseg network to obtain a down-sampling feature map, performing convolution operation on the down-sampling feature map obtained by checking with various different convolutions to obtain a heat map, an offset value and a size of a defect point on the surface of the equipment, converting the obtained heat map into a prediction frame, obtaining a plurality of detection frames of the defect point according to the offset value and the size, and removing redundant detection frames by using Soft _ NMS and a confidence threshold value to obtain a prediction result of the defect of the equipment; and finally, inputting the obtained prediction result into a neural network for training to obtain a detection result of the equipment defect.

Description

Nuclear power station equipment surface defect detection method based on improved CenterNet network
Technical Field
The invention relates to a nuclear power station equipment surface defect detection method based on an improved CenterNet network.
Background
Nuclear power plants are facilities that convert nuclear energy into electrical energy by means of appropriate devices, and nuclear power is a very clean and low-carbon energy source, and therefore nuclear power generation is a very important electrical energy source. The nuclear power station is generally built at sea by comprehensively considering a series of factors such as cooling, discharging, regional economic development and the like, a very good environment is created for corrosion in a severe environment with high salt and high humidity along the sea, and the nuclear power station equipment material is in contact with a high-temperature, high-pressure and high-heat-flow medium, a chemical medium and a seawater medium and is influenced by factors such as irradiation, stress, vibration and the like, so that higher requirements are provided for the anticorrosion work of the nuclear power station. Besides corrosion, the surfaces of nuclear power plant equipment can also have various other defects, such as cracking, peeling, blistering and the like. In order to deal with the corrosion problem of nuclear power plant equipment, when the nuclear power plant equipment is selected, the radiation resistance requirement is considered, and according to the importance degree of the equipment, stainless steel or a carbon plate is generally selected as a main structure, but the mode cannot completely avoid the corrosion and the like. When defects on the surface of the equipment are inevitable, the defects are discovered in time and different remedial measures are taken for different defects. If the defects on the surface of the nuclear power station equipment are not repaired in time, the service life of the equipment is seriously influenced, and the production efficiency and quality are reduced, so that the defect problem on the surface of the nuclear power station equipment is detected in time, and the method has important significance for guaranteeing safe and stable operation of nuclear power.
Nowadays, the common method for detecting defects of nuclear power plant equipment is still manual visual detection, and the type and the position of the defects are recorded by experienced inspectors to form an inspection report. However, the efficiency of the method is very low, and manual visual inspection has many difficulties, for example, visual fatigue is easily generated in long-time inspection, so that some defects are correspondingly missed; the detection of small defects is also difficult, and very small defects can cause great hidden dangers on very important equipment, and if the small defects cannot be found by a manual visual detection method, the equipment and the production in the scheme can be lost in an immeasurable way; finally, the judgment criteria of different inspectors for defects may be different, so that the final detection result may be controversial. Therefore, computer-aided device defect detection is of great significance.
Before the appearance of deep learning, a plurality of classic machine learning algorithms are applied to the industrial field for defect detection, and the specific method mainly comprises three stages: the method comprises the steps of region selection, feature extraction and classification, wherein the step of region selection is to locate the position of a defect, a sliding window strategy is generally adopted to traverse the whole image, the size, the length-width ratio and the like of the defect are not determined, and therefore sliding windows with different scales and aspect ratios need to be set, although the method comprises the defects with various sizes and all possible positions of the defects, the time complexity is high, most of the sliding windows are not necessary, and the speed and the performance of the subsequent feature extraction and classification can be influenced; common features in the feature extraction stage include SIFT, HOG and the like, good features can greatly improve the detection performance, but the defects are various in form and different in illumination intensity, so that the manual design of a robust feature is not easy, the prior knowledge of a designer is required, and the advantage of big data is difficult to utilize; the classifiers commonly used in the classification stage are SVM, Adaboost and the like. In summary, the conventional defect detection method based on machine learning mainly has two problems: 1. the time complexity of the region selection strategy based on the sliding window is high, and the window is redundant; 2. manual design of features is difficult and not very robust.
After the occurrence of deep learning, the defect detection method based on deep learning is widely applied to the industry, such as faster-CNN, YOLO, SSD, etc. Such methods can be broadly divided into two categories: the two-stage method and the one-stage method. The two-stage method needs to generate a series of candidate frames by an algorithm, and then classify the candidate frames through a convolutional neural network, such as R-CNN, FasterR-CNN, and the like. The one-stage method does not need to generate a candidate frame, and directly converts the positioning problem of the target frame into a regression problem for processing, such as YOLO, SSD, and the like. The two methods have difference in detection performance, the two-stage method is more advantageous in detection accuracy and positioning accuracy, and the one-stage method is more advantageous in speed. Whether the two-stage method or the one-stage method is adopted, most of methods for detecting defects in the industry at the present stage are detection methods based on an anchor mechanism, and the method has a relatively obvious defect that the scale and the aspect of the anchor frame related to the anchor frame are relatively difficult to design, and in different industrial application scenes, the scale and the aspect of the anchor need to be designed according to the data situation in the scheme, so that strong prior knowledge is needed, and the setting of the hyperparameter of the IOU threshold is also a problem when the anchor frame is used for carrying out target category classification. And the target detection method based on the Anchor mechanism can generate a great number of redundant frames in one image, and the defects in one image are limited, so that a great number of anchors become background frames without defects, thereby causing the problem of serious imbalance of positive and negative samples.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a nuclear power station equipment surface defect detection method based on an improved CenterNet network.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
a nuclear power station equipment surface defect detection method based on an improved CenterNet network comprises the following steps:
s1, obtaining a surface picture of the nuclear power station equipment and preprocessing the surface picture;
s2, performing feature extraction on the picture preprocessed in the step S1 by using a DLAseg network to obtain a down-sampling feature map;
s3, performing convolution operation on the downsampled feature map obtained in the step S2 by using a plurality of different convolution checks respectively to obtain a heat map, an offset value and a size of the defect point on the surface of the equipment;
s4, converting the heat map obtained in the step S3 into a prediction frame, and obtaining a plurality of detection frames of defect points according to the offset value and the size;
s5, removing redundant detection frames by using Soft _ NMS and a confidence threshold value to obtain a prediction result of the equipment defect;
and S6, inputting the prediction result obtained in the step S6 into a neural network for training to obtain the detection result of the equipment defect.
The beneficial effect of above-mentioned scheme does:
the traditional target detection algorithm based on an anchor mechanism is avoided, so that any hyper-parameter related to the anchor point, such as scale, aspect ratio and the like of an anchor point frame, does not need to be set.
Further, the preprocessing in step S1 specifically includes:
s11, turning, zooming, cutting and turning the acquired nuclear power plant equipment surface picture by using an image processing tool;
and S12, performing Gaussian nuclear scattering on the picture processed in the step S11 through a Gaussian function.
The beneficial effects of the further scheme are as follows:
and data enhancement is carried out, the more the number of samples is, the better the training effect of the model is, and the stronger the generalization ability of the model is. The gaussian function is to transform the tag.
Further, the step S3 specifically includes:
s31, the number of channels of the tensor output by the heat map is 80, each channel represents the heat map of the corresponding category, and the operations are as follows:
Sequential(
(0):Conv2d(64,64,kernel_size＝(3,3),stride＝(1,1),padding＝(1,1))
(1):ReLU(inplace)
(2):Conv2d(64,80,kernel_size＝(1,1),stride＝(1,1))
)
s32, the number of channels of the tensor of width and height output is 2, which respectively represents the length and width of the center of the output frame, and the operation is as follows:
Sequential(
(0):Conv2d(64,64,kernel_size＝(3,3),stride＝(1,1),padding＝(1,1))
(1):ReLU(inplace)
(2):Conv2d(64,2,kernel_size＝(1,1),stride＝(1,1)))
s33, the number of channels of the tensor of offset value output is 2, which are the offsets in the width and height directions, respectively, and the operation is as follows:
Sequential(
(0):Conv2d(64,64,kernel_size＝(3,3),stride＝(1,1),padding＝(1,1))
(1):ReLU(inplace)
(2):Conv2d(64,2,kernel_size＝(1,1),stride＝(1,1)))
the beneficial effects of the further scheme are as follows:
and obtaining the category and the confidence of the prediction frame and the specific position of the prediction frame.
Further, the step S4 specifically includes:
s41, performing maximum pooling operation on the heatmap obtained in the step S3 to obtain a plurality of neighbor maximum value points of the defect point, and arranging the obtained neighbor maximum value points in a descending order according to the score;
s42, selecting the corresponding neighbor maximum value point with the ranking higher than the score threshold value and the corresponding number according to the score size of the neighbor maximum value point;
and S43, planning a detection frame for the neighboring maximum value point selected in the step S2 according to the offset value and the size obtained in the step S3, and obtaining a plurality of detection frames for the defective point.
The beneficial effects of the further scheme are as follows:
and deleting the redundant prediction frame to obtain a final detection result.
Further, the step S5 specifically includes:
s51, arranging the detection frames obtained in the step S4 in a descending order according to the confidence level;
and S52, judging whether the object accuracy standard of the detection frame with the arrangement number in the step S51 is larger than a set threshold value, if so, reducing the confidence of the detection frame, and if not, keeping the detection frame.
The beneficial effects of the further scheme are as follows:
more candidate detection boxes that are likely to contain targets are retained.
Further, the calculation method of the confidence level in step S51 is as follows:
the heat map after the pooling operation is the confidence level in S51.
The beneficial effects of the further scheme are as follows:
when the defects are dense, soft _ nms can reserve more detection frames, and the detection precision of the algorithm is improved.
Further, in step S52, the calculation method for reducing the confidence of the detection frame is as follows:
Figure RE-GDA0003274068020000061
wherein d isiDenotes the ith detection frame, dmAnd sigma is a constant value, and represents the detection box with the maximum confidence coefficient.
The beneficial effects of the further scheme are as follows:
when the defects are more dense, the method can reserve more detection frames instead of directly setting the confidence of the detection frames which possibly contain the defects to be 0.
Further, the step S6 specifically includes:
s61, inputting the prediction result obtained in the step S5 into a CenterNet network, and constructing a positive and negative sample pair;
s62, reducing the weight of the negative sample in the heat map by using the loss function, increasing the weight of the heat map loss function and reducing the loss function of the offset value to obtain the loss function of the CenterNet network;
and S63, training the prediction result obtained in the step S5 by using the loss function obtained in the step S62 to obtain the detection result of the surface defects of the nuclear power plant equipment.
The beneficial effects of the further scheme are as follows:
and the capability of classifying and positioning the surface defects of the model nuclear power station equipment is improved.
Further, the loss function of the centret network in step S62 is expressed as:
Ldet＝λkLk+λsizeLsize+λoffLoff；
wherein L isdetAs a loss function of the CenterNet network, LkAs a function of heat map loss, λkIs the weight of the heat map loss function and lambdak>1，LsizeAs a function of size loss, λsizeWeight of the size loss function, LoffAs a loss function of the offset value, λoffIs the weight of the offset loss function andoff<1。
the beneficial effects of the further scheme are as follows:
different weights are distributed to different loss functions, so that the network is better optimized, and the defect detection precision is improved.
Further, the heat map loss function LkThe calculation formula of (2) is as follows:
Figure RE-GDA0003274068020000071
wherein, therein
Figure RE-GDA0003274068020000072
C represents the number of categories,
Figure RE-GDA0003274068020000073
represents the predicted value of the model in (x, Y) coordinates on the c-th heatmap channel, YxycIs a true tag, α and β are hyper-parameters;
size loss function LsizeThe calculation formula of (2) is as follows:
Figure RE-GDA0003274068020000081
wherein,
Figure RE-GDA0003274068020000082
is the predicted value of the model, N is the number of key points, skIn order to require the actual value of the regression,
Figure RE-GDA0003274068020000083
k is the target number.
Offset loss function LoffThe calculation formula of (2) is as follows:
Figure RE-GDA0003274068020000084
wherein p represents the center point of the target frame, R represents the multiple of the down-sampling,
Figure RE-GDA0003274068020000085
Figure RE-GDA0003274068020000086
the deviation value is indicated.
The beneficial effects of the further scheme are as follows:
when the annotation information is mapped to the output characteristic diagram from the input image, errors on coordinates can be brought by rounding operation, and the offset can improve the positioning accuracy of the detection model.
Drawings
Fig. 1 is a schematic flow chart of a method for detecting surface defects of nuclear power plant equipment based on an improved centrnet network.
FIG. 2 is a graph comparing the detection effects of CenterNet and SSD, YOLOv3 in accordance with the present invention.
FIG. 3 is a diagram illustrating the loss variation of different models during the training phase according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
A nuclear power plant equipment surface defect detection method based on an improved centrnet network, as shown in fig. 1, includes the following steps:
s1, obtaining a surface picture of the nuclear power station equipment and preprocessing the surface picture;
data preprocessing: the method comprises the steps of turning over, affine transformation (including scaling, cutting and the like), turning over and the like of a surface picture of the nuclear power station equipment; in the training phase, the labels of the defect targets need to be converted into heat maps through a gaussian function, namely, gaussian kernel scattering needs to be carried out, in an unmodified cenet, the radius of the gaussian kernel is determined according to the IOU threshold of a prediction frame and a GTbox, in the unmodified cenet, the IOU threshold of the prediction frame and the GTbox is set to be 0.7, then the radius of the gaussian kernel is determined according to the threshold of 0.7, and two-dimensional gaussian kernel scattering is carried out, only the frames with the IOU of the prediction frame and the GTbox being greater than 0.7 are reserved, but for the application scenario of surface defect detection of nuclear power plant equipment, the outlines of the defects are not determined, and therefore, the labeling of the defects does not have a determined standard.
S2, performing feature extraction on the picture preprocessed in the step S1 by using a DLAseg network to obtain a down-sampling feature map;
and sending the preprocessed picture into a DLASeg network for feature extraction, wherein DLA is deep LayerAggregation, and DlAseg is a segmentation network added with deformable convolution on the basis of DLA-34, and the deformable convolution is convolution operation with stronger feature extraction capability after improving the traditional CNN. The deformable convolution is very suitable for a task that a target to be identified has certain geometric deformation, and in an application scene of defect detection, a defect has various shapes, and the deformable convolution can be considered as an object identification task with various geometric deformations in the scheme, so that the deformable convolution can have good performance in the application scene of defect detection.
The Aggregation is a network structure Aggregation mode capable of fusing information among different depths, different stages and blocks, and like jump connection used by ResNet, the Aggregation can be regarded as a connection mode, but the fusion mode of ResNet can only be performed in a block, and is also performed in a simple superposition mode, and a DLA structure can iteratively fuse the characteristic information of a network structure, so that the model can be represented with higher precision under the condition of less parameter quantity.
However, in an unmodified DLA-34 network, the model only spatially fuses more features, and does not display information between modeling channels, and in order to enable the model to automatically learn the importance of different Channel features, an eca (efficient Channel attachment) module is embedded in the DLA-34 network in the scheme, so that the performance of the neural network is significantly improved. The ECA is an extremely light channel attention module, after channel-level global average pooling without dimension reduction, local cross-channel interaction information is captured by considering each channel and K neighbors of each channel, although a small number of parameters are added, obvious performance gains can be obtained in tasks of classification, detection and segmentation, and the ECA is an attention mechanism which is very suitable for being applied to the field of industrial detection. The ECA module enables the model to learn the characteristics of the defects more effectively, and the accuracy of defect detection is improved.
By the feature extraction of the backbone network, a 4-time down-sampling feature map can be obtained, and the resolution of the feature map is much higher than that of a feature map obtained by a general network, so that a small target can be well predicted. By performing convolution operation on the feature map by using three different convolution kernels, three feature maps with different dimensions can be obtained, which correspond to the heat map, the offset value and the width and height respectively. Since the model is regressed by thermodynamic diagram, the receptive field of the model is very important for detecting defects, and 3 × 3 convolution is replaced by 5 × 5 convolution in the scheme in order to make the model have a larger receptive field.
S3, performing convolution operation on the downsampled feature map obtained in the step S2 by using a plurality of different convolution checks respectively to obtain a heat map, an offset value and a size of the defect point on the surface of the equipment;
s4, converting the heat map obtained in the step S3 into a prediction frame, and obtaining a plurality of detection frames of defect points according to the offset value and the size;
first, a 3 × 3 maxporoling operation is performed on the backbone network output, which is similar to the effect of NMS (Non-Maximum Suppression) based anchor point detection method, i.e. to find 8-neighbor Maximum points. Then, the first K maximum scores and corresponding id of the heatmap are taken out without considering the influence of the category, wherein the K points are central points of K defects predicted in the scheme, and then according to the offset value and the width and the height output by the main network, K detection boxes can be obtained in the scheme, wherein K in the CenterNet takes 100, for most cases, 100 detection boxes have a large amount of redundancy, so that a specific algorithm is needed in the scheme to remove the redundant detection boxes, and Soft-NMS is used in the CenterNet to remove the redundant detection boxes. Soft-NMS improves the traditional NMS (Non-MaximumSuppresion) method.
S5, removing redundant detection frames by using Soft _ NMS and a confidence threshold value to obtain a prediction result of the equipment defect;
the traditional NMS algorithm firstly sorts the detection frames from high to low according to the confidence level, then reserves the detection frame with the highest confidence level, and removes the detection frame with the IOU larger than the set threshold value. The above steps are then repeated again in the remaining detection boxes. However, this method has an obvious disadvantage that when two defects are too close to each other, a missing detection situation may occur, because the detection frame with lower confidence is removed due to the larger overlapping area with the frame with higher confidence. In the application scenario of surface defect detection of nuclear power plant equipment, the positions of defects are close to each other and even overlap with each other, so that the conventional NMS algorithm is not suitable for the scenario. The Soft-NMS used in the CenterNet is an improvement of the traditional NMS algorithm, for the detection box with the IOU larger than the threshold, the Soft-NMS adopts a mode that the detection box is not directly removed, but the confidence coefficient of the detection box is reduced, so that more detection boxes are reserved, and the detection precision is improved, and the calculation formulas of the traditional NMS and the Soft-NMS are as follows:
NMS sets the confidence coefficient of the detection box with the iou of the detection box with the highest confidence coefficient larger than the threshold value as 0
Figure RE-GDA0003274068020000121
And the Soft-NMS does not directly set the confidence level to 0 for the detection box with the iou larger than the threshold value, but reduces the confidence level through a certain function.
Figure RE-GDA0003274068020000122
The confidence is reduced in the centrnet using gaussian weighting:
Figure RE-GDA0003274068020000123
s in the formulas (1), (2) and (3)iRepresenting the confidence of the ith detection frame;
diindicates the ith detection frame；
dmRepresenting the detection box with the maximum confidence;
σ is constant, 0.5 in CenterNet;
and S6, inputting the prediction result obtained in the step S6 into a neural network for training to obtain the detection result of the equipment defect.
In this embodiment, the method specifically includes the following steps:
s61, inputting the prediction result obtained in the step S5 into a CenterNet network, and constructing a positive and negative sample pair;
s62, reducing the weight of the negative sample in the heat map by using the loss function, increasing the weight of the heat map loss function and reducing the loss function of the offset value to obtain the loss function of the CenterNet network;
and S63, training the prediction result obtained in the step S5 by using the loss function obtained in the step S62 to obtain the detection result of the surface defects of the nuclear power plant equipment.
Loss function
Ldet＝Lk+λsizeLsize+λoffLoff (4)
The loss function of CenterNet is shown in equation 4, where LdetExpressed is the overall loss function of the CenterNet, LdetFrom Lk(loss of heatmap portion), Lsize(loss of width) and Loff(loss of bias portion) three-part loss composition, where λsizeAnd λoffIs a constant, representing the weight of different loss, an unmodified loss function λsizeAnd λoffTake 0.1 and 1, respectively. The three part loss function formula is as follows:
Figure RE-GDA0003274068020000131
Figure RE-GDA0003274068020000132
Figure RE-GDA0003274068020000133
n in the formula (5) represents the number of image key points, wherein
Figure RE-GDA0003274068020000134
(C represents the number of categories),
Figure RE-GDA0003274068020000135
represents the predicted value of the model in (x, Y) coordinates on the c-th heatmap channel, YxycIs a true tag, α and β are hyperparameters, 2 and 4 are taken in the centrnet, respectively. L iskThe loss function reduces the weight of a large number of simple negative samples in training, so that the model pays more attention to the difficult and difficultly-classified samples.
From the formula (6), LsizeThe length and width were trained using the L1Loss function, assuming the kth target, class ckIs represented as (x 1)(k),y1(k),x2(k),y2(k)) Then the coordinates of the center point of the target frame are
Figure RE-GDA0003274068020000136
The length and width of the target frame are sk＝(x2(k)- x1(k),y2(k)-y1(k)),
Figure RE-GDA0003274068020000141
And N is the number of key points for the predicted value of the model.
Equation (7) is also trained on the bias using L1Loss, where p represents the center point of the target box, R represents the multiple of down-sampling 4,
Figure RE-GDA0003274068020000142
the deviation value is indicated.
As explained in step 1 in the present solution, since the outline of the defect is not determined and the labeling manner of the defect is not unique, the position output of the model does not need to be strictly aligned to the groudtruth in the present solution, so that the loss function in the present solution is improved as follows:
Ldet＝λkLk+λsizeLsize+λoffLoff (8)
in the scheme, a lambda is addedkThe parameter defaults to 2 and L is reducedoffOccupied weight, will beoffSet to 0.1.
Experimental verification
The ENPP divides the defective data into a training set and a test set, wherein the training set comprises 1822 pictures, and the test set comprises 217 pictures.
The experiment used SSD, YOLOv3, RetinaNet, and the method in the present scheme to train 200 epochs on the training set, respectively. In the scheme, several pictures are randomly selected from the verification set, and a comparison graph of the detection results of the SSD, the YOLOv3 and the model in the scheme is shown in fig. 2.
As can be seen from fig. 2, the detection effect of the method in the present scheme is not better than that of SSD and YOLOv3 which have been widely used in the industry, and even in some cases, the detection effect of the method in the present scheme is better than that of SSD and YOLOv 3. Meanwhile, the method in the scheme can also have good performance for detecting the small defects.
The experiment used the single-stage detection models SSD, YOLOv3, unmodified centrnet as comparative experiments, respectively, and the results were as follows:
experiments were performed with 200 epochs training on the training set using different models, whose loss variation is shown in fig. 3:
it can be seen from the loss variation graph that the loss training based on the centrnet method converges first, which greatly saves the training time in the scheme.
In addition, the model in the scheme is tested by referring to the test method of the cocoapi, the test set in the scheme is 217 pictures, and as explained in the scheme in 4.1, the scheme needs to give a higher tolerance to the prediction frame, so that the scheme is provided with five IOU thresholds of 0.3, 0.35, 0.4, 0.45 and 0.5 in total, the final mAP is an average value of results of the five IOU thresholds, the mAR is an average recall rate, the Inferencetime is an average inference time of each picture, and the test results are respectively shown in Table 1:
TABLE 1
mAP mAR Inference time Parameters(M)
YOLOv3 0.48 0.59 0.016s 61.54
RetinaNet 0.30 0.60 0.037 36.39
SSD 0.45 0.56 0.09s 24.15
CenterNet_Resdcn18 0.38 0.54 0.028s 14.43
CenterNet_Resdcn101 0.473 0.60 0.043s 49.67
CenterNet_Hourglass104 0.551 0.65 0.068 191.24
ours 0.515 0.76 0.038 20.17
From the test results, the inference time of the YolOv3 is shortest, but the accuracy and the recall rate are not ideal, the inference time of the SSD is longest, and the detection model based on the CenterNet generally has better accuracy and recall rate than the YolOv3 and the SSD although the inference time is not equal to the inference time of the YolOv3, so the detection model based on the CenterNet has a very wide application prospect in the field of defect detection.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Those skilled in the art can make various other specific changes and combinations based on the teachings of the present invention without departing from the spirit of the invention, and these changes and combinations are within the scope of the invention.

Claims (8)

1. A nuclear power station equipment surface defect detection method based on an improved CenterNet network is characterized by comprising the following steps:
s1, obtaining a surface picture of the nuclear power station equipment and preprocessing the surface picture;
s2, performing feature extraction on the picture preprocessed in the step S1 by using a DLAseg network to obtain a down-sampling feature map;
s3, performing convolution operation on the downsampled feature map obtained in the step S2 by using a plurality of different convolution checks respectively to obtain a heat map, an offset value and a size of the defect point on the surface of the equipment;
s4, converting the heat map obtained in the step S3 into a prediction frame, and obtaining a plurality of detection frames of defect points according to the offset value and the size;
s5, removing redundant detection frames by using Soft _ NMS and a confidence threshold value to obtain a prediction result of the equipment defect;
and S6, inputting the prediction result obtained in the step S5 into a neural network for training to obtain the detection result of the equipment defect.
2. The method for detecting the surface defects of the nuclear power plant equipment based on the improved centret network as claimed in claim 1, wherein the preprocessing in the step S1 is specifically:
s11, turning, zooming, cutting and turning the acquired nuclear power plant equipment surface picture by using an image processing tool;
and S12, performing Gaussian nuclear scattering on the picture processed in the step S11 through a Gaussian function.
3. The method for detecting surface defects of nuclear power plant equipment based on the improved centret network as claimed in claim 2, wherein said step S4 specifically includes:
s41, performing maximum pooling operation on the heatmap obtained in the step S3 to obtain a plurality of neighbor maximum value points of the defect point, and arranging the obtained neighbor maximum value points in a descending order according to the confidence degree;
s42, selecting corresponding neighbor maximum value points with the ranking higher than the confidence threshold value and corresponding numbers according to the confidence level in the step S41;
and S43, planning detection frames for the neighboring maximum value points selected in the step S42 according to the offset value and the size obtained in the step S3, and obtaining a plurality of detection frames for the defect points.
4. The method for detecting surface defects of nuclear power plant equipment based on the improved CenterNet network as claimed in claim 3, wherein the step S5 specifically includes:
s51, arranging the detection frames obtained in the step S4 in a descending order according to the confidence level;
and S52, judging whether the object accuracy standard of the arranged detection frames in the step S51 is larger than a set threshold value, if so, reducing the confidence of the detection frames, and if not, keeping the detection frames.
5. The method for detecting surface defects of nuclear power plant equipment based on the improved centret network of claim 4, wherein in the step S52, the calculation method for reducing the confidence of the detection box is as follows:
Figure FDA0003084043020000021
wherein S isiConfidence of the i-th detection box, diDenotes the ith detection frame, dmRepresents the detection box with the maximum confidence coefficient, sigma is a constant, iou (d)i，dm) And (4) representing the intersection ratio of the ith detection frame and the detection frame with the maximum confidence coefficient.
6. The method for detecting surface defects of nuclear power plant equipment based on the improved CenterNet network as claimed in claim 5, wherein said step S6 specifically includes:
s61, inputting the prediction result obtained in the step S5 into a CenterNet network, and constructing a positive and negative sample pair;
s62, reducing the weight of the negative sample in the heat map by using the loss function, increasing the weight of the heat map loss function and reducing the loss function of the offset value to obtain the loss function of the CenterNet network;
and S63, training the prediction result obtained in the step S5 by using the loss function obtained in the step S62 to obtain the detection result of the surface defects of the nuclear power plant equipment.
7. The improved centret network based nuclear power plant equipment surface defect detection method of claim 6, wherein the loss function of the centret network in step S62 is expressed as:
Ldet＝λkLk+λsizeLsize+λoffLoff；
wherein L isdetAs a loss function of the CenterNet network, LkAs a function of heat map loss, λkIs the weight of the heat map loss function and lambdak＞1，LsizeAs a function of size loss, λsizeWeight of the size loss function, LoffAs a loss function of the offset value, λoffIs the weight of the offset loss function andoff＜1。
8. the improved CenterNet network based nuclear power plant equipment surface defect detection method of claim 7,
the heat map loss function LkThe calculation formula of (2) is as follows:
Figure FDA0003084043020000031
wherein c is the category of the c-th heatmap channel,
Figure FDA0003084043020000032
denotes the predicted value at coordinate (x, Y) in the c-th heatmap channel, YxycIs a true tag, α and β are hyper-parameters;
size loss function LsizeThe calculation formula of (2) is as follows:
Figure FDA0003084043020000033
wherein,
Figure FDA0003084043020000034
is the predicted value of the model, N is the number of defect points, K is the index of the defect points, skAnd k is the regression true value and the target number.
Offset loss function LoffThe calculation formula of (2) is as follows:
Figure FDA0003084043020000035
wherein p represents the center point of the target frame, R represents the multiple of the down-sampling,
Figure FDA0003084043020000036
represent a network at
Figure FDA0003084043020000037
The offset value of the output of (a) is,
Figure FDA0003084043020000041
representing the central point of the target frame on the feature map after down sampling,
Figure FDA0003084043020000042
Figure FDA0003084043020000043
the deviation value is indicated.
CN202110575217.5A 2021-05-26 2021-05-26 Nuclear power station equipment surface defect detection method based on improved CenterNet network Active CN113643228B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110575217.5A CN113643228B (en) 2021-05-26 2021-05-26 Nuclear power station equipment surface defect detection method based on improved CenterNet network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110575217.5A CN113643228B (en) 2021-05-26 2021-05-26 Nuclear power station equipment surface defect detection method based on improved CenterNet network

Publications (2)

Publication Number Publication Date
CN113643228A true CN113643228A (en) 2021-11-12
CN113643228B CN113643228B (en) 2024-01-19

Family

ID=78415823

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110575217.5A Active CN113643228B (en) 2021-05-26 2021-05-26 Nuclear power station equipment surface defect detection method based on improved CenterNet network

Country Status (1)

Country Link
CN (1) CN113643228B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114359153A (en) * 2021-12-07 2022-04-15 湖北工业大学 Insulator defect detection method based on improved CenterNet
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术（天津）有限公司 Roof panel defect detection method and system
CN114594103A (en) * 2022-04-12 2022-06-07 四川大学 Method and system for automatically detecting surface defects of nuclear industrial equipment and automatically generating reports
CN114612803A (en) * 2022-01-25 2022-06-10 安徽理工大学 Transmission line insulator defect detection method for improving CenterNet
CN114782311A (en) * 2022-03-14 2022-07-22 华南理工大学 Improved multi-scale defect target detection method and system based on CenterNet
CN116309587A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Cloth flaw detection method and device, electronic equipment and storage medium
CN116309586A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Defect detection method, device, equipment and medium based on convolutional neural network
CN116862886A (en) * 2023-07-17 2023-10-10 天合光能股份有限公司 Solar cell defect detection method, device and readable medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110766681A (en) * 2019-10-28 2020-02-07 福建帝视信息科技有限公司 Bamboo strip surface defect detection method based on triple loss network
CN111062413A (en) * 2019-11-08 2020-04-24 深兰科技（上海）有限公司 Road target detection method and device, electronic equipment and storage medium
CN111275171A (en) * 2020-01-19 2020-06-12 合肥工业大学 Small target detection method based on parameter sharing and multi-scale super-resolution reconstruction
CN111738995A (en) * 2020-06-10 2020-10-02 苏宁云计算有限公司 RGBD image-based target detection method and device and computer equipment
CN112257815A (en) * 2020-12-03 2021-01-22 北京沃东天骏信息技术有限公司 Model generation method, target detection method, device, electronic device, and medium
CN112446302A (en) * 2020-11-05 2021-03-05 杭州易现先进科技有限公司 Human body posture detection method and system, electronic equipment and storage medium
CN112446327A (en) * 2020-11-27 2021-03-05 中国地质大学（武汉） Remote sensing image target detection method based on non-anchor frame
CN112507845A (en) * 2020-12-02 2021-03-16 余姚市浙江大学机器人研究中心 Pedestrian multi-target tracking method based on CenterNet and depth correlation matrix
CN112613564A (en) * 2020-12-25 2021-04-06 桂林汉璟智能仪器有限公司 Target detection post-processing method for eliminating overlapped frames
US20210118136A1 (en) * 2019-10-22 2021-04-22 Novateur Research Solutions LLC Artificial intelligence for personalized oncology

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210118136A1 (en) * 2019-10-22 2021-04-22 Novateur Research Solutions LLC Artificial intelligence for personalized oncology
CN110766681A (en) * 2019-10-28 2020-02-07 福建帝视信息科技有限公司 Bamboo strip surface defect detection method based on triple loss network
CN111062413A (en) * 2019-11-08 2020-04-24 深兰科技（上海）有限公司 Road target detection method and device, electronic equipment and storage medium
CN111275171A (en) * 2020-01-19 2020-06-12 合肥工业大学 Small target detection method based on parameter sharing and multi-scale super-resolution reconstruction
CN111738995A (en) * 2020-06-10 2020-10-02 苏宁云计算有限公司 RGBD image-based target detection method and device and computer equipment
CN112446302A (en) * 2020-11-05 2021-03-05 杭州易现先进科技有限公司 Human body posture detection method and system, electronic equipment and storage medium
CN112446327A (en) * 2020-11-27 2021-03-05 中国地质大学（武汉） Remote sensing image target detection method based on non-anchor frame
CN112507845A (en) * 2020-12-02 2021-03-16 余姚市浙江大学机器人研究中心 Pedestrian multi-target tracking method based on CenterNet and depth correlation matrix
CN112257815A (en) * 2020-12-03 2021-01-22 北京沃东天骏信息技术有限公司 Model generation method, target detection method, device, electronic device, and medium
CN112613564A (en) * 2020-12-25 2021-04-06 桂林汉璟智能仪器有限公司 Target detection post-processing method for eliminating overlapped frames

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
XINGYI ZHOU 等: "Objects as Points", pages 1 - 12 *
ZHIYONG DAI 等: "Cascade CenterNet: Robust Object Detection for Power Line Surveillance", 《IEEE》, pages 1 - 14 *
史亚秋: "机器视觉技术在模具检测与定位中的应用研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅰ辑》, no. 08, pages 022 - 342 *
王璇 等: "基于改进 CenterNet的AFP铺层表面缺陷检测", pages 1 - 16 *
行路南 等: "Soft-NMS: 用一行代码改进目标检测", pages 1 - 14 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114359153A (en) * 2021-12-07 2022-04-15 湖北工业大学 Insulator defect detection method based on improved CenterNet
CN114359153B (en) * 2021-12-07 2024-08-02 湖北工业大学 Insulator defect detection method based on improvement CENTERNET
CN114612803A (en) * 2022-01-25 2022-06-10 安徽理工大学 Transmission line insulator defect detection method for improving CenterNet
CN114612803B (en) * 2022-01-25 2024-09-24 安徽理工大学 Improved CENTERNET transmission line insulator defect detection method
CN114782311A (en) * 2022-03-14 2022-07-22 华南理工大学 Improved multi-scale defect target detection method and system based on CenterNet
CN114782311B (en) * 2022-03-14 2024-05-14 华南理工大学 CENTERNET improvement-based multi-scale defect target detection method and system
CN114594103A (en) * 2022-04-12 2022-06-07 四川大学 Method and system for automatically detecting surface defects of nuclear industrial equipment and automatically generating reports
CN114594103B (en) * 2022-04-12 2023-05-16 四川大学 Automatic detection and report generation method and system for surface defects of nuclear industrial equipment
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术（天津）有限公司 Roof panel defect detection method and system
CN116309587A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Cloth flaw detection method and device, electronic equipment and storage medium
CN116309586A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Defect detection method, device, equipment and medium based on convolutional neural network
CN116862886A (en) * 2023-07-17 2023-10-10 天合光能股份有限公司 Solar cell defect detection method, device and readable medium

Also Published As

Publication number Publication date
CN113643228B (en) 2024-01-19

Similar Documents

Publication Publication Date Title
CN113643228A (en) Nuclear power station equipment surface defect detection method based on improved CenterNet network
Xie et al. Automatic detection and classification of sewer defects via hierarchical deep learning
EP3379491B1 (en) Surface defect detection
Liu et al. A rail surface defect detection method based on pyramid feature and lightweight convolutional neural network
Abu et al. The performance analysis of transfer learning for steel defect detection by using deep learning
Dong et al. Defect detection and classification by training a generic convolutional neural network encoder
WO2020046213A1 (en) A method and apparatus for training a neural network to identify cracks
US11699224B2 (en) Neural network training device, system and method
CN105740891B (en) Target detection based on multi level feature selection and context model
US10922807B2 (en) Wafer manufacturing system, device and method
CN111652853A (en) Magnetic powder flaw detection method based on deep convolutional neural network
CN112700444A (en) Bridge bolt detection method based on self-attention and central point regression model
CN107784288A (en) A kind of iteration positioning formula method for detecting human face based on deep neural network
CN112766170B (en) Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image
CN111382709A (en) Insulator image detection method based on unmanned aerial vehicle inspection
CN116563250A (en) Recovery type self-supervision defect detection method, device and storage medium
Sruthy et al. Waste collection & segregation using computer vision and convolutional neural network for vessels
CN113762151B (en) Fault data processing method, system and fault prediction method
Yasir et al. Faster metallic surface defect detection using deep learning with channel shuffling
Tunali et al. Steel Surface Defect Classification Via Deep Learning
Wangli et al. Foxtail Millet ear detection approach based on YOLOv4 and adaptive anchor box adjustment
CN117173550A (en) Method and system for detecting underwater small target of synthetic aperture sonar image
CN115861956A (en) Yolov3 road garbage detection method based on decoupling head
García et al. A configuration approach for convolutional neural networks used for defect detection on surfaces
CN114882428A (en) Target detection method based on attention mechanism and multi-scale fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant