CN117218101A - Composite wind power blade defect detection method based on semantic segmentation - Google Patents
Composite wind power blade defect detection method based on semantic segmentation Download PDFInfo
- Publication number
- CN117218101A CN117218101A CN202311261096.2A CN202311261096A CN117218101A CN 117218101 A CN117218101 A CN 117218101A CN 202311261096 A CN202311261096 A CN 202311261096A CN 117218101 A CN117218101 A CN 117218101A
- Authority
- CN
- China
- Prior art keywords
- matrix
- attention
- wind power
- semantic segmentation
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 58
- 230000007547 defect Effects 0.000 title claims abstract description 48
- 239000002131 composite material Substances 0.000 title claims abstract description 18
- 238000001514 detection method Methods 0.000 title claims abstract description 14
- 238000004458 analytical method Methods 0.000 claims abstract description 20
- 238000010191 image analysis Methods 0.000 claims abstract description 6
- 239000011159 matrix material Substances 0.000 claims description 57
- 238000000034 method Methods 0.000 claims description 33
- 238000012545 processing Methods 0.000 claims description 29
- 230000006870 function Effects 0.000 claims description 27
- 238000010586 diagram Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 18
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 208000023445 Congenital pulmonary airway malformation Diseases 0.000 claims description 12
- 238000013519 translation Methods 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 2
- 238000009659 non-destructive testing Methods 0.000 abstract description 19
- 230000000694 effects Effects 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 5
- 239000003086 colorant Substances 0.000 abstract description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000009977 dual effect Effects 0.000 description 5
- 238000007689 inspection Methods 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 230000007306 turnover Effects 0.000 description 5
- 230000001965 increasing effect Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000003733 fiber-reinforced composite Substances 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000037303 wrinkles Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a composite wind power blade defect detection method based on semantic segmentation; the invention collects the defect detection image of the industrial wind power blade by using ultrasonic nondestructive detection equipment, and automatically segments the defect area on the wind power blade by using a semantic segmentation network; dividing the image can endow different colors to different defect positions in the image, separating out different color areas by using an OpenCV related function, and carrying out specific image analysis by using connected domain analysis and contour detection. The semantic segmentation network provided by the invention has a good segmentation effect on wind power blade ultrasonic nondestructive testing, so that automatic analysis of defect characteristics can be performed; the artificial intelligence and computer vision technology is introduced into the industrial nondestructive testing field to carry out auxiliary analysis, so that the artificial participation is greatly reduced, the subjectivity of defect analysis is reduced, and the analysis efficiency and accuracy are improved.
Description
Technical Field
The invention relates to the technical field of wind power blade detection, in particular to a composite material wind power blade defect detection method based on semantic segmentation.
Background
The energy source in the modern society, especially the demand of electric energy is very big, and wind energy is as a green renewable energy source, and with the advantages such as its development degree of difficulty is low, endless and pollution-free, become the people and solve the current first choice of the increasingly deficient problem of traditional energy source. The device for completing wind power generation is a wind turbine generator, and comprises wind power blades, a cabin and a tower, wherein the wind power blades are used for bearing wind load and transmitting mechanical energy to the cabin, and the wind power blades are vital components of the wind turbine generator under severe conditions. In order to achieve the large-scale, light-weight and low cost, a fiber-reinforced composite material composed of fibers and a matrix is a main material used for the composite material, and the defects generated by the production process are mainly as follows: inclusion defects, air gap defects (layering, glue shortage, cracks), fat-rich, air holes, bubbles, wrinkles, dry spots, dry fibers, cracks and the like, directly influence the quality of raw materials of the wind power blade, and further influence the normal operation of the wind power device. Therefore, the industry often needs participation in non-destructive inspection techniques to ensure the quality safety of materials.
The ultrasonic nondestructive testing technology is the most widely used technical means for detecting defects of wind power blades at the present stage, ultrasonic nondestructive testing equipment transmits a wave source to an object to be tested by utilizing an ultrasonic probe, echo is obtained according to the transmission and reflection of ultrasonic waves in the object to be tested, and then ultrasonic testing equipment obtains an ultrasonic testing diagram through analysis of the echo, so that the type, the size and the position information of the defects are obtained, but the ultrasonic nondestructive testing diagram is complex and special personnel are needed for analysis.
Semantic segmentation, which is one of the most popular research hot spots in recent years as a classical computer vision problem, recognizes pictures at the pixel level and marks the category to which each pixel point in an image belongs, and usually marks the image as different colors in a predicted image, so that the image is segmented into areas with different semantic information. The technology is introduced into the nondestructive testing field to process the ultrasonic testing image, the defect area in the image can be automatically segmented, the image processing related technology is used for analysis, the defect part can be more accurately analyzed, the dependence on the experience of a testing worker is eliminated, the influence of subjective elements in the testing analysis is reduced, and meanwhile, the labor is saved.
Disclosure of Invention
The invention aims to provide a wind power blade composite material defect detection method based on semantic segmentation and OpenCV.
The invention discloses a composite wind power blade defect detection method based on semantic segmentation, which comprises the following steps of:
step one, detecting wind power blades by using ultrasonic nondestructive detection equipment, and constructing a data set with a label.
And step two, constructing a semantic segmentation module. The semantic segmentation module comprises an encoder, a decoder and an attention module; the encoder takes a ResNeXt50 network as a backbone extraction network; five different ResnextBlock blocks are arranged in the ResNeXt50 network; feature layers output by the plurality of ResnextBlock blocks are respectively transmitted into corresponding attention modules; and adding the feature layers before and after the input attention module to obtain a target feature layer. The decoder sequentially performs up-sampling and jump connection feature fusion on each target feature layer; each up-sampling is preceded by an attention module. The attention module comprises a CAM attention layer and a PAM attention layer; the feature map of the input attention module first passes through a 3 x 3 convolution layer before entering the CAM attention layer; after the output result of the CAM attention layer is processed by Dropout processing and 1 multiplied by 1 convolution, the output result is input into the PAM attention layer; the output result of the PAM attention layer is processed by Dropout twice and convolved by 1X 1 to obtain the final output result of the dual self-attention CPAM module.
Training the semantic segmentation module constructed in the third step by using the data set amplified in the second step; the trained semantic segmentation module processes the ultrasonic data acquired from the wind power blade to be tested by the segmentation image analysis module to obtain a segmentation image.
And fourthly, identifying the segmented image to obtain the position and the size of the defect of the detected wind power blade.
Preferably, the data set in the first step is enhanced and amplified by a data enhancing module; the amplified image is amplified by gamma change, blurring original image processing, noise addition, translation and overturn processing, rotation processing, contrast and brightness adjustment and saturation adjustment.
Preferably, in the amplification process, a random probability P is generated by a random function aiming at an original image; when P is greater than the preset probability threshold P 0 And when the method is used, the original blurring processing, noise adding, translation and turnover processing are carried out, wherein the probability of horizontal or vertical turnover is 0.5. For the image subjected to fuzzy original image processing, noise adding, translation and overturning processing, randomly generating probability P 'through a random function, and when P' is larger than a preset probability threshold value P 0 At this time, image brightness, saturation, and contrast adjustment are performed.
Preferably, in the CAM attention layer, a feature map x εR is entered B×C×H×W After the CAM attention layer is input, a matrix Q with the dimension of B multiplied by C multiplied by HW is obtained through dimension conversion, a matrix K with the dimension of B multiplied by HW multiplied by C is obtained through dimension conversion, a matrix V with the dimension of B multiplied by C multiplied by HW is obtained through dimension conversion, a matrix E with the dimension of B multiplied by C is obtained after the matrix Q and the matrix K are subjected to inner product, and a corresponding element at each position of the matrix E is subtracted by the maximum value of the last dimension of the matrix E to reconstruct the matrix E, so that each element of the matrix E cannot have a too large value, and numerical overflow taking an index is avoided. The reconstructed matrix E is subjected to a softmax function to obtain an attention matrix A. Multiplying the attention matrix A with the matrix V, multiplying the multiplied result with the learning parameter beta, and then multiplying the multiplied result with the input characteristic diagram x epsilon R B×C×H×W And adding to obtain a final output characteristic. B is the batch size, C is the channel number, H is the height, and W is the width. The initial value of the learning parameter β is 0, and gradually increases by learning.
Preferably, in the PAM attention layer, a feature map x∈r is input B×C×H×W After being input into a PAM attention layer, the PAM attention layer is subjected to a group of average pooling with different scales to obtain a group of downsampled characteristics, and all the obtained characteristics are further subjected toStacking the features in the channel dimension to obtain a feature map F epsilon R B×C×M The method comprises the steps of carrying out a first treatment on the surface of the Will input a feature map x e R B×C×H×W Performing dimension conversion and transposition, and then performing inner product with the transposed feature map F to obtain a matrix with dimensions of B multiplied by H multiplied by W multiplied by M; then the matrix is subjected to a softmax function to obtain an attention matrix E; after the attention matrix E is transposed and the transposed feature diagram F is subjected to inner product, the result is multiplied by the learning parameter alpha and then is multiplied by x epsilon R B×C×H×W And adding to obtain an output result of the PAM attention layer.
Preferably, in the third step, the training period is 100; in the training process, an Adam optimizer is utilized for optimization, the momentum is set to be 0.9, the initial learning rate is set to be 0.0001, and the loss function is a cross entropy loss function.
Preferably, in the fourth step, median filtering noise reduction processing is sequentially performed on the separated images, and then the area and the center point coordinates of the connected domain are extracted through median filtering noise reduction, gray level conversion, binary conversion, expansion convolution operation and connected domain analysis.
The invention has the advantages that:
the semantic segmentation network provided by the invention has a good segmentation effect on wind power blade ultrasonic nondestructive testing, so that automatic analysis of defect characteristics can be performed; the artificial intelligence and computer vision technology is introduced into the industrial nondestructive testing field to carry out auxiliary analysis, so that the artificial participation is greatly reduced, the subjectivity of defect analysis is reduced, and the analysis efficiency and accuracy are improved.
Drawings
Fig. 1 is a flow chart of the operation of the present invention.
FIG. 2 is a diagram of the process of constructing a dataset in the present invention.
Fig. 3 is a diagram showing the effect of data enhancement in the second step of the present invention.
Fig. 4 is a schematic diagram of a network structure of a semantic segmentation module according to the present invention.
Fig. 5 is a block diagram of a dual self-attention CPAM module in accordance with the present invention.
Fig. 6 is a diagram of the attention layer structure of a CAM in accordance with the present invention.
Fig. 7 is a structural diagram of PAM attention layer in the present invention.
Fig. 8 is a schematic diagram of packet convolution performed by the ResNeXt50 network in accordance with the present invention.
Fig. 9 is a semantic segmentation effect diagram of the semantic segmentation module according to the present invention.
FIG. 10 is a flowchart illustrating the operation of the split image analysis module of the present invention.
Fig. 11 is a schematic diagram illustrating connected domain analysis of a segmented image analysis module according to the present invention.
FIG. 12 is a schematic diagram illustrating analysis of defect profiles obtained in the present invention.
Detailed Description
The objects and effects of the present invention will become more apparent by describing the present invention with reference to the accompanying drawings and specific examples, it being understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to be limiting.
As shown in FIG. 1, the method for detecting the defects of the wind power blade made of the composite material based on semantic segmentation comprises the following steps:
step one, as shown in fig. 2, a dataset is constructed. Ultrasonic data acquisition is carried out on a plurality of different wind power blades (part of the blades have surface or internal defects) by using ultrasonic nondestructive testing equipment, so that a wind power blade composite material defect image is obtained, and a label file is obtained after labelme labeling is used; and the original image and the label file are in one-to-one correspondence and are used as a semantic segmentation data set. Inputting the data set into a semantic segmentation network for training to obtain a semantic segmentation model, and can be used for the segmentation of nondestructive testing images of wind turbine blade composite materials. In the figure 2, (a) is a side view of a wind power blade workpiece object, (b) is an air hole defect ultrasonic nondestructive testing image, (c) is an air hole defect ultrasonic nondestructive testing labeling image, (d) is a top view of the wind power blade workpiece object, (e) is a surface defect ultrasonic nondestructive testing image, and (f) is a surface defect ultrasonic nondestructive testing labeling image.
And step two, enhancing the data set by using a data enhancement module.
As shown in the figure 3 of the drawings, performing gamma change, blurred original image processing, noise adding, and image processing,Translation and turnover treatment, rotation treatment, contrast and brightness adjustment and saturation adjustment to obtain an amplified image; the amplified image and the original image are integrated into the data set, so that the data volume of the data set is expanded, and the diversity of the data set images is improved. In the amplification process, firstly, generating random probability P aiming at an original image through a random function; when P is greater than the preset probability threshold P 0 And when the method is used, the original blurring processing, noise adding, translation and turnover processing are carried out, wherein the probability of horizontal or vertical turnover is 0.5. For the image subjected to fuzzy original image processing, noise adding, translation and overturning processing, randomly generating probability P 'through a random function, and when P' is larger than a preset probability threshold value P 0 At this time, image brightness, saturation, and contrast adjustment are performed. The brightness, saturation and contrast of the image are randomly obtained in an adjusting range (0-1 in the embodiment); meanwhile, the adjustment of brightness, saturation and contrast can be randomly overlapped.
In fig. 3, (a) is a nondestructive inspection image of the air hole defect, (b) is a nondestructive inspection labeling image of the air hole defect, (c) is an enhancement processed image of nondestructive inspection data of the air hole defect, and (d) is an enhancement processed image of nondestructive inspection labeling data of the air hole defect.
In this embodiment, the gamma change process is as follows: firstly, normalizing pixel values, then taking an index to enable an output image gray value to be in an index relation with an input image gray value, and completing the process through an LUT function in OpenCV.
The processing process of the fuzzy original image comprises the following steps: the use of the cv2.Blu function is achieved by mean filtering the image.
The noise adding process comprises the following steps: salt and pepper noise is added by randomly setting some pixels to 0 or 255.
The translation and inversion processes are: the maximum moving distance of all the target frames in the up-down left-right direction is regulated, and translation is realized through a cv2.Warp Affine function; the horizontal or flip-up operation on the image is achieved by a cv2.Flip function.
The rotation treatment process comprises the following steps: and randomly generating a rotation angle of 0-360 degrees and rotating the radian through a random function, and then realizing image coordinate rotation through a rotation transformation formula.
The contrast and brightness adjustment and saturation adjustment processes are as follows: wherein contrast adjustment converts the image into HSV format, and the median value of all brightness is selected first. When the contrast ratio is to be increased, the value of the brightness value smaller than the median value is reduced, and the value of the brightness value larger than the median value is increased; when the contrast ratio is to be reduced, increasing the value with the brightness value smaller than the median value, and reducing the value with the brightness value larger than the median value so that all the brightness values are close to the median value; brightness adjustment converts an image into an HLS format, changes the brightness of the image by adjusting the value of an L channel in the image, and directly adds or subtracts a fixed value to each pixel under an RGB color space; converting the image into HSV format, changing saturation of the picture color by adjusting S value in the image, and converting the image into HSV format is realized by cv2.cvtColor function.
And thirdly, constructing a semantic segmentation module ResNeXt-CPAM-Unet.
As shown in fig. 4, the semantic segmentation module includes an encoder, a decoder, and an attention module; the encoder takes a ResNeXt50 network as a backbone extraction network; the ResNeXt50 network includes five different ResnextBlock blocks; the feature layers output by the five ResnextBlock blocks are respectively transmitted into five attention modules; and adding the feature layers before and after the input attention module to obtain a target feature layer. The decoder sequentially carries out up-sampling and jump connection feature fusion on the five target feature layers four times, and the output of the last layer is the output feature diagram; and an attention module is arranged before each upsampling to obtain multi-scale feature association and richer abstract semantic information.
As shown in table 1, the ResNeXt50 and the ResNet50 are similar in structure, and there are 5 convolution modules in this embodiment, wherein the conv1 modules of the two are identical, while in the last four convolution modules, the number of convolution kernels of the first two convolution layers of each ResnextBlock block of the ResNeXt50 is 2 times the number of the ResNet50, and the third convolution layer is identical; meanwhile, the middle layer 3×3 convolution res net50 uses packet convolution of group=32. As shown in fig. 8, the block convolution adopts the idea of division integration, and Gao Weijuan is integrated into a plurality of identical low-dimensional convolutions, and a plurality of features of the block convolution are fused after the convolution operation is completed. The adoption of the packet convolution reduces a plurality of parameters compared with the original ResNet50 network structure, and the network performance is improved.
TABLE 1ResNet50 vs. ResNext50
The attention module is a double self-attention CPAM module with a residual error structure; specifically, the dual self-attention CPAM module includes a CAM attention layer and a PAM attention layer; the former focuses on mining the spatial dependency relationship between the feature graphs, and the position features of the feature graphs are aggregated and updated through weighted summation; the latter focuses on mining the channel dependency between feature graphs, and uses the weighting of each channel graph for channel information update.
As shown in fig. 5, the dual self-attention CPAM module adopts an attention mechanism of serial connection of CAM and PAM, and before entering the attention layer of CAM, the input feature map first passes through a 3×3 convolution layer, so as to obtain more abstract semantic information, and meanwhile, prevent the overfitting problem; after the output result of the CAM attention layer is processed by Dropout processing and 1 multiplied by 1 convolution, the output result is input into the PAM attention layer; the output result of the PAM attention layer is processed by Dropout twice and convolved by 1X 1 to obtain the final output result of the dual self-attention CPAM module.
As shown in fig. 6, the CAM attention layer operates as follows: input feature map x e R B×C×H×W After the CAM attention layer is input, a matrix Q with the dimension of B multiplied by C multiplied by HW is obtained through dimension conversion, a matrix K with the dimension of B multiplied by HW multiplied by C is obtained through dimension conversion, a matrix V with the dimension of B multiplied by C multiplied by HW is obtained through dimension conversion, a matrix E with the dimension of B multiplied by C is obtained after the inner product of the matrix Q and the matrix K is made, and the corresponding element of each position of the matrix E is subtracted from the maximum value of the last dimension of the matrix E to reconstructThe matrix E is such that each element of the matrix E will not have too large a value, avoiding overflow of the exponential value. The reconstructed matrix E is subjected to a softmax function to obtain an attention matrix a, which is substantially the cosine similarity of the matrix Q and the matrix K. The attention matrix A and the input characteristic diagram x epsilon R B×C×H×W Multiplying the multiplication result by a matrix V with dimension of BXCXHW obtained by dimension conversion, multiplying the multiplication result by a learning parameter beta (the initial value of the beta parameter is 0, and gradually obtaining a larger weight value through learning), and then multiplying the multiplication result by an input feature map xE R B ×C×H×W And adding to obtain a final output characteristic. B is Batchsize, the batch size; c is Channels, namely the number of Channels; h is the height; w is the width.
As shown in fig. 7, the PAM attention layer works as follows: input feature map x e R B×C×H×W After being input into a PAM attention layer, the PAM attention layer is subjected to a group of different-scale average pooling to obtain a group of downsampled features, and all the obtained features are stacked in the channel dimension to obtain a feature map F epsilon R B×C×M Thereby reducing the amount of calculation; will input a feature map x e R B×C×H×W Performing dimension conversion and transposition, and then performing inner product with the transposed feature map F to obtain a matrix with dimensions of B multiplied by H multiplied by W multiplied by M; then the matrix is subjected to a softmax function to obtain an attention matrix E; after the attention matrix E is transposed and the transposed feature diagram F is subjected to inner product, the result is multiplied by a learning parameter alpha (the initial value of the alpha parameter is 0, and a larger weight is gradually obtained through learning) and then multiplied by x epsilon R B×C×H×W And adding to obtain a final output characteristic.
Training the semantic segmentation module constructed in the third step by using the data set amplified in the second step; the training period is 100; the training process is optimized by using an Adam optimizer, the momentum (momentum) is set to be 0.9, the initial learning rate is set to be 0.0001, and the loss function is a cross entropy loss function.
In order to evaluate the segmentation accuracy of the nondestructive testing images of the wind power blades, the intersection ratio (IoU) and the Class Pixel Accuracy (CPA) are used as segmentation evaluation indexes, and each time 5 epochs are trained for evaluation and the result is stored. The same random number seeds are set when different models are trained, the same random number sequence is generated in the training process, and network fluctuation caused by different random parameters is avoided. Compared with IoU and CPA index comparison conditions of other segmentation algorithms, the semantic segmentation module provided by the invention is shown in Table 2.
Table 2 index contrast for segmentation algorithm
In table 2, resNeXt-CPAM-Unet refers to the semantic segmentation module provided in this embodiment; resNeXt-CBAM-Unet refers to a network model in which a backbone part of a Unet network is replaced by ResNeXt, and a CBAM attention mechanism is added in a feature fusion part; the Resunate-CBAM refers to a network model which replaces a backbone part of a Unet network with Resnet and adds a CBAM attention mechanism in a feature fusion part; the Resunat-CPAM is a network model which replaces a backbone part of a Unet network with a Resnet network and adds a CPAM attention mechanism in a feature fusion part; resunate-CoortdA entry is a model that replaces the backbone part of the Unet network with Resnet and adds a CoortdA entry attention mechanism. Table 2 shows that the semantic segmentation model ResNeXt-CPAM-Unet provided by the implementation has the best defect image segmentation effect.
As shown in fig. 9, the semantic segmentation model obtained in this embodiment can obtain a good segmentation effect for the segmentation of the defect image. In FIG. 9, (a) is a part of the ultrasonic nondestructive testing chart of the air hole, (b) is a part of the ultrasonic nondestructive testing chart of the surface defect, (c) is a part of the semantic segmentation result of the ultrasonic nondestructive testing chart of the air hole, and (d) is a part of the semantic segmentation result of the nondestructive testing chart of the surface defect
And fifthly, constructing a segmentation image analysis module.
Separating and storing the segmentation images obtained by the semantic segmentation model according to different colors of different types of defects; as shown in fig. 10, median filtering and noise reduction are performed through a cv2.median blue function, gray conversion and binarization conversion are performed through a cv2.cvtcolor and a cv2.threshold function to obtain a binary image, and after expansion convolution operation, connected domain analysis is performed through a cv2.connectiedcomponents withstats function to obtain the area and center point coordinates of the connected domain; the result of the connected domain analysis is shown in fig. 11.
Multiple defects of the same type are also endowed with different colors and output results are counted respectively; as shown in fig. 12, the contour detection analysis is performed on the binary image by using a cv.findcontours function, so as to obtain the perimeter and the aspect ratio of the image; in addition, the above-mentioned obtained size data are all pixel sizes, and the actual size is obtained by combining the image resolution and the original image scale.
Claims (8)
1. A composite wind power blade defect detection method based on semantic segmentation is characterized by comprising the following steps of: the method comprises the following steps:
step one, detecting wind power blades by using ultrasonic nondestructive detection equipment, and constructing a data set with a label;
step two, constructing a semantic segmentation module; the semantic segmentation module comprises an encoder, a decoder and an attention module; the encoder takes a ResNeXt50 network as a backbone extraction network; five different ResnextBlock blocks are arranged in the ResNeXt50 network; feature layers output by the plurality of ResnextBlock blocks are respectively transmitted into corresponding attention modules; adding the feature layers before and after the input attention module to obtain a target feature layer; the decoder sequentially performs up-sampling and jump connection feature fusion on each target feature layer; before each upsampling, passing through an attention module; the attention module comprises a CAM attention layer and a PAM attention layer; the feature map of the input attention module first passes through a 3 x 3 convolution layer before entering the CAM attention layer; after the output result of the CAM attention layer is processed by Dropout processing and 1 multiplied by 1 convolution, the output result is input into the PAM attention layer; the output result of the PAM attention layer is processed by Dropout twice and convolved by 1X 1 to obtain the final output result of the double self-attention CPAM module;
training the semantic segmentation module constructed in the third step by using the data set amplified in the second step; the trained semantic segmentation module processes the ultrasonic data acquired from the wind power blade to be tested by the segmentation image analysis module to obtain a segmentation image;
and fourthly, identifying the segmented image to obtain the position and the size of the defect of the detected wind power blade.
2. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: the data set in the first step is enhanced and amplified by a data enhancement module; the amplified image is amplified by gamma change, blurring original image processing, noise addition, translation and overturn processing, rotation processing, contrast and brightness adjustment and saturation adjustment.
3. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: the data set in the first step is enhanced and amplified by a data enhancement module; the amplified image is amplified by gamma change, blurring original image processing, noise addition, translation and overturn processing, rotation processing, contrast and brightness adjustment and saturation adjustment.
4. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: in the amplification process, firstly, generating random probability P aiming at an original image through a random function; when P is greater than the preset probability threshold P 0 When the method is used, fuzzy original image processing, noise adding, translation and overturning processing are carried out, wherein the probability of horizontal or up-down overturning is 0.5; for the image subjected to fuzzy original image processing, noise adding, translation and overturning processing, randomly generating probability P 'through a random function, and when P' is larger than a preset probability threshold value P 0 At this time, image brightness, saturation, and contrast adjustment are performed.
5. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: in the CAM attention layer, a feature map x εR is input B×C×H×W After being input into the CAM attention layer, the dimension is obtained through dimension conversionFor a matrix Q of B×C×HW, converting and converting the dimension into a matrix K of B×HW×C, converting the dimension into a matrix V of B×C×HW, performing inner product on the matrix Q and the matrix K to obtain a matrix E of B×C×C, and subtracting the corresponding element at each position of the matrix E from the maximum value of the last dimension of the matrix E to reconstruct the matrix E, so that each element of the matrix E cannot have a too large value, and numerical overflow taking an index is avoided; the reconstructed matrix E is subjected to a softmax function to obtain an attention matrix A; multiplying the attention matrix A with the matrix V, multiplying the multiplied result with the learning parameter beta, and then multiplying the multiplied result with the input characteristic diagram x epsilon R B ×C×H×W Adding to obtain a final output characteristic; b is the batch size, C is the channel number, H is the height, and W is the width; the initial value of the learning parameter β is 0, and gradually increases by learning.
6. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: in the PAM attention layer, a feature map x ε R is input B×C×H×W After being input into a PAM attention layer, the PAM attention layer is subjected to a group of different-scale average pooling to obtain a group of downsampled features, and all the obtained features are stacked in the channel dimension to obtain a feature map F epsilon R B×C×M The method comprises the steps of carrying out a first treatment on the surface of the Will input a feature map x e R B×C×H×W Performing dimension conversion and transposition, and then performing inner product with the transposed feature map F to obtain a matrix with dimensions of B multiplied by H multiplied by W multiplied by M; then the matrix is subjected to a softmax function to obtain an attention matrix E; after the attention matrix E is transposed and the transposed feature diagram F is subjected to inner product, the result is multiplied by the learning parameter alpha and then is multiplied by x epsilon R B ×C×H×W And adding to obtain an output result of the PAM attention layer.
7. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: in the third step, the training period is 100; in the training process, an Adam optimizer is utilized for optimization, the momentum is set to be 0.9, the initial learning rate is set to be 0.0001, and the loss function is a cross entropy loss function.
8. The method for detecting defects of the composite wind power blade based on semantic segmentation according to claim 1, which is characterized by comprising the following steps: and step four, sequentially performing median filtering noise reduction treatment on the separated images, and extracting the area and the center point coordinates of the connected domain through median filtering noise reduction, gray level conversion, binarization conversion, expansion convolution operation and connected domain analysis.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311261096.2A CN117218101A (en) | 2023-09-27 | 2023-09-27 | Composite wind power blade defect detection method based on semantic segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311261096.2A CN117218101A (en) | 2023-09-27 | 2023-09-27 | Composite wind power blade defect detection method based on semantic segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117218101A true CN117218101A (en) | 2023-12-12 |
Family
ID=89044218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311261096.2A Pending CN117218101A (en) | 2023-09-27 | 2023-09-27 | Composite wind power blade defect detection method based on semantic segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218101A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952983A (en) * | 2024-03-27 | 2024-04-30 | 中电科大数据研究院有限公司 | Intelligent manufacturing production process monitoring method and system based on artificial intelligence |
-
2023
- 2023-09-27 CN CN202311261096.2A patent/CN117218101A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117952983A (en) * | 2024-03-27 | 2024-04-30 | 中电科大数据研究院有限公司 | Intelligent manufacturing production process monitoring method and system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113052210B (en) | Rapid low-light target detection method based on convolutional neural network | |
| CN108961235B (en) | Defective insulator identification method based on YOLOv3 network and particle filter algorithm | |
| CN111223088B (en) | Casting surface defect identification method based on deep convolutional neural network | |
| CN109800631B (en) | Fluorescence coding microsphere image detection method based on mask region convolution neural network | |
| CN109145872B (en) | CFAR and Fast-RCNN fusion-based SAR image ship target detection method | |
| CN112733950A (en) | Power equipment fault diagnosis method based on combination of image fusion and target detection | |
| CN109034184B (en) | Grading ring detection and identification method based on deep learning | |
| CN107145846A (en) | A kind of insulator recognition methods based on deep learning | |
| CN117218101A (en) | Composite wind power blade defect detection method based on semantic segmentation | |
| CN114973032B (en) | Deep convolutional neural network-based photovoltaic panel hot spot detection method and device | |
| CN111178177A (en) | Cucumber disease identification method based on convolutional neural network | |
| CN115082744B (en) | Artificial intelligence-based solar heat collection efficiency analysis method and system | |
| CN111080574A (en) | Fabric defect detection method based on information entropy and visual attention mechanism | |
| CN116645328A (en) | Intelligent detection method for surface defects of high-precision bearing ring | |
| CN116703812A (en) | Deep learning-based photovoltaic module crack detection method and system | |
| CN116485802B (en) | Insulator flashover defect detection method, device, equipment and storage medium | |
| CN116205832A (en) | Metal surface defect detection method based on STM R-CNN | |
| CN117745697A (en) | Steel surface defect detection algorithm based on improved YOLOv8 model | |
| CN111539434B (en) | Infrared weak and small target detection method based on similarity | |
| CN117451716A (en) | Industrial product surface defect detection method | |
| CN113256563A (en) | Method and system for detecting surface defects of fine product tank based on space attention mechanism | |
| CN110415816B (en) | Skin disease clinical image multi-classification method based on transfer learning | |
| CN117392065A (en) | Cloud edge cooperative solar panel ash covering condition autonomous assessment method | |
| CN111402223B (en) | Transformer substation defect problem detection method using transformer substation video image | |
| CN116309429A (en) | Chip defect detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |