KR20230137788A - A multi-class pipeline defect detection, tracking and counting method based on self-attention mechanism - Google Patents

Abstract

The present invention is an inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism, and includes the following procedures. S1. Collect images of various types of plumbing defects. S2. Building a pipe defect image dataset. S3. Building and training piping defect inspection models. S4. Building pipe defect tracking and counting models. S5. Testing and output of results. The present invention is an object inspection, tracking, or counting algorithm targeting pipe defect characteristics based on a deep learning learning method based on a self-attention mechanism. Model training based on plumbing defect big data improves robustness and generalization ability. The present invention improves the accuracy of model inspection and realizes automatic recognition and counting of piping defects by using a transformer neural network instead of the existing convolutional neural network.

Description

Inspection, tracking and counting method for various types of pipeline defects based on self-attention mechanism {A multi-class pipeline defect detection, tracking and counting method based on self-attention mechanism}

The present invention relates to piping defect inspection technology, and to a method for inspecting, tracking, and counting various piping defects based on a self-attention mechanism.

Drain pipes are the lifeline of a city and have the function of water collection, drainage, or drainage. In recent years, various piping defect problems have arisen due to phenomena such as subsidence, microbial corrosion, and contaminant deposition due to ground imbalance. Leaks and joint failures on the internal surfaces of drain pipes can cause contamination, affecting the surrounding water and soil. This is harmful to people's health. Water leaks cause surrounding soil to be lost and roads to collapse. These problems have a negative impact on people's travel safety. If there are obstacles or tree roots in the space inside the drain pipe, the pipe will be blocked and the drainage ability will be weakened. This causes the problem of urban flooding. Therefore, pipe defect inspection and quantity statistics should be conducted regularly and preventive measures should be taken. Through this, the extent of pipe damage must be determined and an accurate repair method for the defect must be taken.

Existing drain pipe inspections focus on visual inspection techniques such as closed-circuit television cameras or periscopes and involve manual identification. However, manual identification requires a lot of time, the evaluation is subjective, and problems of missing or incorrect testing may occur, which reduces the accuracy of the test.

Existing drain pipe defect inspection methods rely on experience-based artificial workers or data-based unsupervised learning algorithms. Traditional algorithms can be divided into three types. First, the gray level of the plumbing defects is lower than the background. For example, there are gradient histograms and threshold methods. Second, there is a large change in the gray level value at the edge of the crack. Examples include artificial edge checking operators and discrete wavelet transform algorithms. Third, plumbing defects have the same characteristics that can be obtained through statistics and regression. For example, there are support vector machines, artificial neural networks, random forests, etc. Traditional algorithms can be used independently as feature extractors or classifiers and can achieve excellent results. However, the internal environment of the drain pipe is complex and the types of defects are diverse. Because existing algorithms extract only a small amount of features, they cannot satisfy the requirements for accuracy and robustness in drain pipe inspection.

The patent document, patent number CN202110737415.7, discloses a method and system for inspecting drain pipe defects based on deep learning. This method processes the data set for the original defect image through sinGAN and creates an anchor frame using the GA-RPN algorithm. Lastly, inspection and segmentation processing of pipe defect images are performed using a neural network. This method obtains the specific location coordinates of the pipe defect, but there is a possibility that the problem of double calculation of the same defect in consecutive frames of the image may occur. Therefore, considering the problem of high correlation of images generated by existing data augmentation, a method to increase defect inspection accuracy is needed.

CN202110737415.7

In order to solve the problems existing in existing technology, the present invention is a self-attention mechanism-based inspection, tracking and counting method for various piping defects, a piping defect database built based on a generative adversarial neural network, and a self-attention mechanism. It consists of three main parts: a piping defect inspection model or a piping defect coefficient model built based on a self-attention mechanism. This method solves the problem of existing inspection algorithms repeatedly calculating the same defect and has the advantage of high accuracy, increased generalization ability, and good robustness by optimizing the inspection model through a self-attention mechanism. Therefore, it solves the problem mentioned in the background technology section above.

To achieve the above object, the present invention provides technical solutions such as inspection, tracking, and counting methods for various types of piping defects based on a self-attention mechanism. Specifically, it includes the following procedures.

S1. Collect images of various types of plumbing defects;

S2. Building a pipeline defect image dataset;

S3. Building and training piping defect inspection models;

S4. Piping defect tracking and counting model construction;

S5. Testing and output of results.

Preferably, the image collection of various types of pipe defects, which is procedure S1, includes the following detailed procedures:

S11. Collecting images or videos of surface defects in pipes through pipe crawling robots;

S12. Preprocessing of the original image is performed, the video is divided into frame images, the resolution of all images is unified to 256 × 256, and saved in jpg format;

S13. Select an image that contains one or more defects.

Preferably, the construction of the pipe defect image data set, which is procedure S2, includes the following detailed procedures:

S21. Through procedure S1, pipe defect images are classified into three defect types: joint failure, rupture, and obstruction;

S22. Construct a generative adversarial network (GAN). A GAN neural network consists of a generator and a discriminator. The generator randomly receives noise and creates a new image. The discriminator is to determine whether the image is true. Conduct training according to the type of piping defect. A total of three image generation models are built, and the images are created and saved in jpg format.

S23. Raw images and generated images are integrated into the same file and renamed in image order.

S24. The pixels in the display image background area are set to 0, the pixels in the splicing error area are set to 1, the pixels in the rupture area are set to 2, and the pixels in the obstacle area are set to 3, and are saved in xml format.

S25. A pipe defect image database is constructed by combining the corrected jpg format images and the generated xml format display information.

S26. The image database according to procedure S25 is randomly divided into a training set, validation set, and test set at a ratio of 6:2:2.

Preferably, building and training the pipe defect inspection model mentioned in Procedure S3 above includes the following detailed procedures. You must build a plumbing defect inspection model based on a self-attention mechanism, initialize the model using a transfer learning method, input training set data into the training model, compare the loss values and accuracy rates recorded during the model training process, and find the optimal hyperparameters. Data from the verification set are input into the model, verification is performed, and the model is saved after reaching the verified prediction.

Preferably, the desired optimal hyperparameters should satisfy the following conditions. When setting other hyperparameters, the stability and convergence of the maximum accuracy rate increase curve and the loss value decrease curve must be compared during the training process. The above verification estimate shows that the model's accuracy reaches 88.860% and the loss curve shows a easing trend. The optimal hyperparameter has an initial learning rate of 0.00001, the number of image batches repeated each time is 2, the weighted attenuation value is 0.0005, and the cycle is repeated a total of 61 times.

Preferably, the self-attention mechanism computes interactions between sequences to obtain attention weights relative to current expectations. The specific process is as follows. First, the original image is divided into nine 2-dimensional blocks, and the 2-dimensional blocks are vectorized to obtain a self-attention model. This can be accepted as a one-dimensional valid series (x ₁ , x ₂ , x ₃ ,..., x ₉ ). Converts one-dimensional series into fixed-length content vectors through a fully connected layer. Position coding information is added to each block, content vectors are inserted, semantic series (z ₁ , z ₂ , z ₃ , ,,, , z ₉ ) are obtained, and finally attention weights are obtained.

The self-attention mechanism includes a lookup vector, a key vector, and a value vector. The lookup vector is the preceding output y _{i -1} , the key vector influences the importance of the input value x _i to the output value y _i , and the value vector expresses the size of each input value x _i . Based on the value multiplied by the key vector and the lookup vector, the weight vector a _i is obtained using the Softmax function. This means the degree of attention of the input vector in the process of predicting the output value y _i .

Preferably, building a pipe defect tracking and counting model according to procedure S4 above adds a self-attention mechanism to the tracking network and trains this neural network with a pipe defect image database. We build a piping tracking and counting model based on a self-attention mechanism. Once a specific type of pipe defect is detected, it will be tracked using a tracking algorithm until the defect disappears. Add 1 to the corresponding defect coefficient quantity. The specific process is as follows:

S41. The pipe image series is input into the model, and the pipe defect inspection model detects defects in the first frame, sets the initial bounding box, and adds 1 to the defect quantity;

S42. In a pipe defect tracking network, the tracking algorithm continues tracking the detected defect until it disappears;

S43. After the tracked defect disappears from the frame series, the next defect is inspected and calculated;

S44. Finally, the quantity of defects by type is totaled.

Preferably, the tracking algorithm according to procedure S42 above is applied as follows. First, the features of the previous frame and the current frame are obtained using a convolutional neural network, then the inspection framework and tracking framework are obtained through decoding using a self-attention mechanism, and finally, the inspection frame is obtained by using the IoU function on the same frame. Complete the work and tracking framework tracking.

Preferably, the inspection and output results by the above procedure S5 are specifically as follows. Check the test set above. Determine whether numerical indicators can reach expected accuracy rates by inputting them into a tracking or counting model and inspecting, tracking, or counting pipe defects. After reaching the estimate, the pipe defect results will be output.

Preferably, the test numerical indicators include accuracy rate, F1 score and PR curve. The PR curve uses recall as the abscissa and accuracy as the ordinate.

Preferably, the predicted accuracy rate is greater than or equal to 85% for each type of piping defect.

The beneficial effects of the present invention are as follows. The present invention builds a deep learning-based drain pipe defect image generation network and adds new defect image quantities. To solve the redundant counting problem caused by consecutive video frames, we propose a tracking or counting model based on a self-attention mechanism. This applies to calculating the quantity and type of drain defects. To solve the problem of limitations in the convolution operator acceptance area, the accuracy of defect inspection is improved by utilizing a self-attention mechanism. Because the types of piping defects are diverse and the situations are complex, existing algorithms have limitations in extracting defect characteristic information, making it difficult to apply them to various piping environments. The present invention utilizes deep learning to increase generalization ability and improve identification accuracy by learning multiple pipe defect characteristics.

Figure 1 is a flow chart of the inspection, tracking and counting method for various types of piping defects based on the self-attention mechanism of the present invention.
Figure 2 is a structural diagram of the self-attention mechanism of the present invention.
Figure 3 is a diagram showing the recognition results of the inspection, tracking, and counting method for various types of piping defects based on the self-attention mechanism of the present invention.

Hereinafter, technical solutions in embodiments of the present invention will be clearly and completely described with reference to the attached drawings. Obviously, the described embodiments are some, but not all, embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by a person skilled in the art without creative efforts will fall within the protection scope of the present invention.

Definitions of terminology of the present invention are as follows:

F1 Score (F1 Score) is an indicator that evaluates the accuracy of a binary classification model in statistics and simultaneously considers the accuracy and recall of the classification model. The F1 score is a weight of the model's accuracy and recall, and the maximum value is 1 and the minimum value is 0;

PR curve (P stands for precision, R stands for recall, and expresses the relationship between precision and recall);

Robustness (the property of a system representation to maintain a certain performance under unstable parameter perturbations);

Attention model (AM, Attention model, refers to the attention function of the human brain and creates different points of interest and different weights depending on the data area input by the neural network);

Generative adversarial network (GAN)

Referring to Figures 1 to 3, the present invention is a self-attention mechanism-based inspection, tracking or counting method for various types of piping defects and includes the following procedures as shown in Figure 1:

Procedure 1: Collect images of different types of pipe defects

By utilizing a pipe crawling robot, images and videos of internal surface defects of underground drain pipes in Zhengzhou and Tianjin cities were collected. However, because the internal environment of the drain pipe is complex, there are problems such as ambiguity in the original image and uneven quantity of each type of defect. Therefore, preprocessing is performed on the original image and images with one or several defects are selected. The specific process is as follows:

1.1 The method of collecting road surface defect images is as follows. Collect images or videos of surface defects within pipes through a pipe crawling robot;

1.2 The person involved performs pre-processing on the original image, divides the video into frame images, unifies the resolution of all images to 256×256, and saves them in jpg format;

1.3 The person concerned selects an image containing one or more defects.

Specifically, this solves the problem of redundantly increasing the model's attention to a single defect by storing only one image of the same defect in the video.

To be specific, the person involved performs filters based on resolution, exposure, ambiguity, brightness, and quantity balance so that defects appear clearly in the photo.

Procedure 2: Building a Pipe Defect Image Dataset

By using pipe defect images generated based on a generative adversarial neural network, a defect data set is constructed by combining the original image and the generated image. By classifying and displaying pipeline defect data sets, stakeholders build a high-resolution, quantitatively balanced pipeline defect database. Also, it is divided into training set, validation set and test set according to the ratio. The specific process is as follows:

2.1 Classify true piping defect images into three defect types: joint failure, rupture, and obstruction.

2.2 By building a generative adversarial network, we train pipe defects by type and create a model. It also creates defect images and saves them in jpg format.

2.3 Select the created image. The original image and the created image are saved as the same file, and the image is renamed according to the image order;

2.4 Display image The pixels in the background area are marked as 0, the pixels in the splicing error zone are marked as 1, the pixels in the rupture zone are marked as 2, and the pixels in the obstacle zone are marked as 3 and saved in xml format;

2.5 Create a pipe defect image database by combining the corrected jpg format images and the generated xml format display information;

2.6 The pipe defect image database is randomly divided into a training set, validation set, and test set at a ratio of 6:2:2.

Specifically, targeted inspection based on deep learning requires large amounts of images. It is difficult to obtain defect image data due to the complex environment of drain pipes (e.g. dark environment, water quality environment). Data enrichment methods based on traditional affine transformations (e.g. translation, rotation and scaling) are applied to numerous image creations. However, this method can only create new images that are highly related to the original image. A generative adversarial network consists of a generator and a discriminator. The generator randomly receives noise and creates a new image. The discriminator is to determine whether the image is true.

Specifically, during the training period, the generator generates fake photos to deceive the discriminator, and the discriminator serves to distinguish between fake images generated by the generator and true images. Therefore, it is a dynamic game consisting of a generator and a discriminator. Under good circumstances, the generator can produce photos that look like true images. In this case, it is difficult for the discriminator to determine whether the image generated by the generator is authentic.

Specifically, during the training process, 499 splicing error images and 472 obstacles are input to the GAN. In the rupture type, the number of local images is 117 and the total number of images is 315. If local and global images are trained simultaneously, the GAN model cannot distinguish between local and global features. In some images, local features and global features are present, and the performance is confused. For this reason, training is conducted using only 315 local rupture images. Finally, we build a generation model for three defect images.

Specifically, if the images in the training set, validation set, and test set do not overlap each other, the robustness and generalization ability of the validation model are improved.

Specifically, a drain pipe defect image library is constructed with 1403 real images and 1500 fake images. To ensure the veracity of the images in the validation and test sets, 1500 fake images are added to the training set. For this reason, the number of images in the validation set and test set is kept constant. There are 842 splicing errors, 116 obstacles and 74 ruptures in the training set. There are 114 joint errors, 88 obstructions and 101 ruptures in the test set.

Procedure 3: Building and training a pipe defect inspection model

A piping defect test model based on a self-attention mechanism is applied, and the transfer learning method is used to initialize the model and train the model, as shown in Figure 2. We need to compare the loss values and accuracy rates recorded during the model training process and find the optimal hyperparameters. The specific process is as follows:

3.1 A piping defect test model based on a self-attention mechanism is applied. The self-attention mechanism includes lookup vector, key vector and value vector;

3.2 Transfer learning method is used to initialize the neural network, which improves training efficiency and improves inspection accuracy;

3.3 The loss and accuracy rates recorded during the model training process must be compared and the optimal hyperparameters must be found.

Specifically, the attention model is applied to machine translation for the first time and solves the long-term request problem. This has become an important concept in the field of neural networks. The self-attention mechanism calculates interactions between sequences to obtain an attention weight related to the current prediction. In this process, the self-attention mechanism reduces the complexity of the neural network model. Depending on the value of the weight, the attention mechanism simulates the concentration of attention in human information processing, improves model performance and reduces the amount of computation.

Specifically, to process a 2D image, the original image is divided into 9 2D blocks and the 2D blocks are vectorized to obtain a self-attention model. This can be accepted as a one-dimensional valid series (x ₁ , x ₂ , x ₃ ,..., x ₉ ). Converts one-dimensional series into fixed-length content vectors through a fully connected layer. Position coding information is added to each block, a content vector is inserted, and a semantic series (z ₁ ,z ₂ ,z ₃ ,...,z ₉ ) is obtained.

Specifically, the lookup vector is the previous output y _{i -1} , the key vector influences the importance of the input value x _i to the output value y _i , and the value vector represents the size of each input value x _i . . Based on the value multiplied by the key vector and the lookup vector, the weight vector a _i is obtained using the Softmax function. This means the degree of attention of the input vector in the process of predicting the output value y _i . Finally, the output probability distribution is obtained through the Softmax function, and the output result is obtained through label coding. The attention mechanism improves the model's performance by simulating the concentration of attention when processing human information using a weight vector.

Specifically, the desired optimal hyperparameter must satisfy the following conditions. When setting other hyperparameters, validation set data must be entered into the model and verified. The model should be evaluated by indicators such as the maximum accuracy increase curve and the stability and convergence of the loss decrease curve in the validation set.

Specifically, as training continues, the model's loss gradually decreases and accuracy gradually increases. Through training in 61 time periods, the accuracy rate of the model (expected value verification) reaches 88.860%, and the loss curve shows a easing trend. The model is saved so that it can be used in the next step.

In the embodiment of the present invention that should be selected first, the hyperparameters of the final model through several tunings are as follows. The initial learning rate is 0.00001, the number of image batches repeated each time is 2, the weighted attenuation value is 0.0005, and the cycle is repeated a total of 61 times.

Procedure 4: Building a pipe defect tracking and counting model.

By adding a self-attention mechanism to the tracking network and training this neural network with a database of pipe defect images, a pipe tracking and counting model based on the self-attention mechanism is built. Once a pipe defect of any type is detected, it will be tracked using tracking algorithms until the defect disappears. Add 1 to the corresponding defect coefficient quantity. The specific process is as follows;

4.1 Input the piping image series into the model, detect the defects in the first frame with the piping defect inspection model, determine the initial bounding box, and add 1 to the defect quantity;

4.2 In the pipe defect tracking network, the tracking algorithm continues tracking the detected defect until it disappears;

4.3 After the tracked defect disappears from the frame series, the next defect is inspected and calculated;

4.4 Finally, sum up the quantity of defects by type.

Specifically, the type and location of the defect can be obtained through an existing deep learning-based piping defect inspection neural network. However, the same defect may exist in successive frames of video. Existing inspection methods do not recognize the same defects, and problems that are difficult to calculate may arise. Testing is a static task: finding a target in a single image using features learned from similar images. On the other hand, tracking is a dynamic task that finds the same target in successive frames by utilizing the characteristics of previous frames. The tracking task finds the target's movement trajectory by comparing the similarity between the previous frame and the current frame.

Furthermore, the tracking algorithm includes the following procedures. First, the features of the previous frame and the current frame are obtained using a convolutional neural network, then the inspection framework and tracking framework are obtained through decoding using a self-attention mechanism, and finally, the IoU function is used to inspect the same frame. Framework and Tracking Framework tracking has been completed.

Procedure 5: Test and print results.

Images of surface defects within the pipe are captured using a pipe crawling robot and transmitted to a high-performance server. A test set can be input into the model and inspect and track pipe defects to determine if inspection metrics can reach expected accuracy rates. When the expected value is reached, the pipe defect result is output.

Specifically, piping defect results include the defect type, its quantity, and its relative location.

Specifically, FIG. 3 shows a 12-second pipe defect image with a resolution of 720×408 and a playback speed of 25 frames per second. This model is used to track defects and the location of the defect in the image is determined using the tracking framework. Among them, the letters in the upper left indicate the defect type, and the numbers indicate the coefficient number. The defects shown in this video are obstacles, and there are a total of two. All inspection and counting results are accurately counted.

Specifically, test numerical indicators include accuracy rate, F1 score, and PR curve.

Specifically, by adjusting the reliability threshold, the validation set image can output a recall rate and a corresponding accuracy rate. The PR curve uses recall as the abscissa and accuracy as the ordinate. By comparing the size of the area under the curve, the strengths and weaknesses of the model can be analyzed. When the test set samples are unbalanced, the PR curve is a more objective way to evaluate a model than accuracy and F1 score.

Specifically, we need to ensure that an accuracy rate of 85% or higher for each type of defect can be achieved by collecting accurate or incorrect statistical information. This ensures the ability of the present invention to be applied in a realistic environment. If the accuracy rate does not reach the expected requirement, the test error information must be statisticized and deducted. Images that find error defects are processed by the method of the present invention, then added to the pipe defect image database, retrained, and tuned until each indicator reaches the expected value. The present invention improves the accuracy of model inspection and implements automatic recognition and counting of piping defects by using a transformer neural network instead of the existing convolutional neural network.

Although the present invention has been described in detail with reference to the above-described embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the above-described embodiments or make equivalent replacements for some technical features. Modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (10)

S1. Procedure for collecting images of various types of pipe defects;
S2. Pipe defect image dataset construction procedure;
S3. Piping defect inspection model construction and training procedures;
S4. Piping defect tracking and counting model building procedures; and
S5. Inspection, tracking and counting method for various types of piping defects based on self-attention mechanism including test and result output procedures. In claim 1,
The above procedure S1, collecting images of various types of pipe defects,
S11. Procedure for collecting images or videos of surface defects in pipes through pipe crawling robots;
S12. The procedure of pre-processing the original image, dividing the video into frame images, unifying the resolution of all images to 256×256, and saving them in jpg format; and
S13. An inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism that includes a procedure for selecting an image containing one or several defects. In claim 1,
The construction of the pipe defect image data set, which is procedure S2, is as follows:
S21. The procedure of classifying pipe defect images into three defect types: joint failure, rupture, and obstruction through procedure S1;
S22. A generative adversarial neural network (GAN) is constructed. The GAN neural network consists of a generator and a discriminator. The generator randomly receives noise and creates a new image, and the discriminator determines whether the image is true, and trains according to the type of pipe defect. A procedure in which a total of three image generation models are built and the images are created and saved in jpg format;
S23. The original image and the generated image are integrated into the same file, and renaming is performed in the order of the images;
S24. The pixels in the display image background area are marked as 0, the pixels in the junction error zone are marked as 1, the pixels in the rupture zone are marked as 2, and the pixels in the obstacle zone are marked as 3, and the procedure is saved in xml format;
S25. A procedure for building a pipe defect image database by combining the corrected jpg format images and the generated xml format display information; and
S26. Inspection and tracking of various types of piping defects based on a self-attention mechanism, which includes the procedure of randomly dividing the image database according to procedure S25 into a training set, validation set, and test set at a ratio of 6:2:2. and counting method. In claim 1,
Building and training the piping defect inspection model mentioned in procedure S3 above,
Build a plumbing defect inspection model based on a self-attention mechanism, initialize the model using a transfer learning method, input training set data into the training model, compare the loss values and accuracy rates recorded during the model training process, and find the optimal hyperparameters. An inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism that inputs data from the verification set into the model, performs verification, and saves the model after reaching the verified estimate. In claim 4,
The desired optimal hyperparameters are:
When setting other hyperparameters, the stability and convergence of the maximum accuracy rising curve and the loss value falling curve are compared during the training process, and the verification estimate shows that the accuracy of the model reaches 88.860% and the loss curve shows a tendency to relax. The optimal hyperparameter has an initial learning rate of 0.00001, the number of image batches repeated each time is 2, the weighted attenuation value is 0.0005, and inspection of various types of piping defects based on the self-attention mechanism, which is characterized by repeating a total of 61 cycles. Tracking and counting methods. In claim 4,
The self-attention mechanism calculates interactions between sequences to obtain an attention weight related to the current prediction. The specific process is:
First, the raw image is divided into 9 2-dimensional blocks, and the 2-dimensional blocks are vectorized to obtain a self-attention model, which receives a 1-dimensional valid series (x ₁ , x ₂ , x ₃ ,..., x ₉ ). It is possible, converting the one-dimensional series into a fixed-length content vector through a fully connected layer, adding position coding information to each block, inserting the content vector, and converting the semantic series (z ₁ , z ₂ , z ₃ , ..., z ₉ ) is obtained and finally the attention weight is obtained; and
The self-attention mechanism includes a lookup vector, a key vector and a value vector, where the lookup vector is the previous item output y _{i -1} , and the key vector influences the degree to which the input value x _i is important to the output value y _i , The value vector expresses the size of each input value x _i , and based on the value multiplied by the key vector and the lookup vector, a weight vector a _i is obtained using the Softmax function, which is the process of predicting the output value y _i . An inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism, which refers to the degree of attention of the input vector. In claim 1,
Building a pipe defect tracking and counting model according to the above procedure S4 adds a self-attention mechanism to the tracking network, trains this neural network with a pipe defect image database, builds a pipe tracking and counting model based on the self-attention mechanism, and determines which type of pipe. When a defect is detected, it will be tracked until the defect disappears by using a tracking algorithm, and 1 will be added to the defect count quantity.
The specific process is,
S41. A procedure to input a pipe image series into the model, detect defects in the first frame with the pipe defect inspection model, determine the initial bounding box, and add 1 to the defect quantity;
S42. A procedure in which a tracking algorithm in a piping defect tracking network continues to track a detected defect until it disappears;
S43. A procedure for inspecting and calculating the next defect after a tracked defect has disappeared from the frame series; and
S44. Lastly, an inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism that includes a procedure for summing the quantity of defects by type. In claim 7,
The tracking algorithm according to procedure S42 is,
First, obtain the features of the previous frame and the current frame using a convolutional neural network, then use the self-attention mechanism to obtain the inspection framework and tracking framework through decoding, and finally, use the IoU function for the same frame to obtain the inspection frame. An inspection, tracking, and counting method for various types of piping defects based on a self-attention mechanism that completes the work and tracking framework tracking. In claim 1,
The results of the inspection and output according to procedure S5 above are:
By inputting a test set into the inspection, tracking, or counting model and inspecting, tracking, or counting pipe defects, it is determined whether the numerical indicator can reach the expected accuracy rate, and after reaching the expectation, the pipe defect result is output. An inspection, tracking and counting method for various types of piping defects based on a self-attention mechanism. In claim 8,
The inspection numerical index includes accuracy rate, F1 score, and PR curve. The PR curve has the recall rate as the abscissa and the accuracy rate as the ordinate, and the expected accuracy rate is characterized in that the accuracy rate for each type of piping defect is 85% or more. An inspection, tracking and counting method for various types of piping defects based on a self-attention mechanism.