CN115239034B - Method and system for predicting early defects of wind driven generator blade - Google Patents
Method and system for predicting early defects of wind driven generator blade Download PDFInfo
- Publication number
- CN115239034B CN115239034B CN202211169343.1A CN202211169343A CN115239034B CN 115239034 B CN115239034 B CN 115239034B CN 202211169343 A CN202211169343 A CN 202211169343A CN 115239034 B CN115239034 B CN 115239034B
- Authority
- CN
- China
- Prior art keywords
- early
- model
- self
- image
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 134
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000011218 segmentation Effects 0.000 claims abstract description 97
- 238000012549 training Methods 0.000 claims abstract description 81
- 238000012795 verification Methods 0.000 claims abstract description 14
- 210000003128 head Anatomy 0.000 claims description 29
- 230000006870 function Effects 0.000 claims description 19
- 238000002372 labelling Methods 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 18
- 238000010276 construction Methods 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 8
- 230000007797 corrosion Effects 0.000 claims description 7
- 238000005260 corrosion Methods 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 6
- 238000005065 mining Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 239000011800 void material Substances 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 5
- 239000011796 hollow space material Substances 0.000 claims description 3
- 230000000873 masking effect Effects 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims 2
- 238000010248 power generation Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 11
- 238000013135 deep learning Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Marketing (AREA)
- Databases & Information Systems (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Development Economics (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for predicting early defects of a wind driven generator blade, and belongs to the field of wind power generation equipment. The method comprises the steps of firstly collecting images of early, middle and late scenes of a leaf, and carrying out manual annotation on the late images to be used as an inversion semantic segmentation training set sample; dividing the corresponding leaf early-middle image set as a model sample set into a self-supervision training set and a verification set, and manually marking the verification set; constructing a base network and an early defect self-supervision learning model, and adopting a self-supervision training set to train to obtain a mature model and obtain a model weight; constructing a segmentation head model, loading weights on a base network, connecting the base network with the segmentation head model, inputting an inversion semantic segmentation training set for training, and outputting semantic segmentation labels to obtain a mature downstream task segmentation model; and acquiring an early image of the blade to be predicted, inputting the early image into a downstream task segmentation model, and outputting position information of an early defect. The method improves the accuracy of the early-stage defect prediction of the blade.
Description
Technical Field
The invention belongs to the field of wind power generation equipment, and particularly relates to a method and a system for predicting early defects of a wind driven generator blade.
Background
In industrial production, the defect detection of equipment plays a crucial role in guiding the smooth production. Wind power generation is an important part of industrial power generation, and the damage rate of equipment is high due to the influence of environment, wind power and the like. Especially, the wind driven generator blade is easy to generate crack defects in severe environment, so that the service life of equipment is shortened, and large crack defects are developed from small cracks, so that the defects can be accurately detected at an early stage, and the defects can be prevented from being developed to cause larger loss in a mode of timely maintaining or replacing parts and the like.
In the prior art, the defect detection method of the wind driven generator blade comprises a non-machine vision method, a traditional machine learning method, a deep learning method and the like. The non-machine vision method comprises manual detection and sound wave nondestructive detection, and is low in safety coefficient, long in consumed time and low in accuracy; the traditional machine learning method is based on a support vector machine model, has low identification precision, is difficult to achieve higher accuracy in the aspect of defect identification tasks, cannot well process small early defects, and is generally applied to middle and late defects with obvious characteristics; the existing deep learning-based method generally takes a strong supervision convolutional neural network as a main part, such as AlexNet, googleNet, resNet, U-Net and the like, however, early defects of wind driven generator blades have adverse factors such as difficulty in observation of human eyes, difficulty in data annotation, influence of human factors on distinction and the like, under the strong supervision network, early scene information is lacking, image information is incomplete, sufficient semantic information cannot be provided for supervised training, meanwhile, manual annotation of data consumes a large amount of time and is difficult to achieve a unified annotation standard, so that the data effectiveness is poor, and the prediction accuracy is low.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and a system for predicting an early defect of a wind turbine blade, where a late easily-labeled defect sample is used for performing network training to invert characteristics of the early defect, so as to improve segmentation accuracy and improve accuracy and precision of early defect prediction.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
on one hand, the embodiment of the invention provides a method for predicting early defects of a wind driven generator blade, which comprises the following steps:
the method comprises the following steps of S1, collecting images of scenes of a plurality of wind driven generator blades in the early stage, the middle stage and the late stage of the whole life cycle, wherein the early stage, the middle stage and the late stage respectively account for one third of the life cycle;
s2, selecting a late image with defects capable of being recognized by naked eyes to establish an image set to be annotated; taking the corresponding early and middle leaf image sets as model sample sets, selecting samples in a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set;
s3, constructing a base network based on a Transformer structure, constructing an early-stage defect self-supervision learning model based on the base network, and setting hyper-parameters of the early-stage defect self-supervision learning model;
s4, training an early-stage defect self-supervision learning model by adopting the self-supervision training set, mining data information from unsupervised data, enabling the model to learn the relation among all pixels, extracting deep semantic representation to obtain a mature early-stage defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment;
s5, constructing a segmentation head model, loading the weight of the mature self-supervision learning model by the base network at the moment, and connecting the base network with the segmentation head model to serve as a downstream task segmentation model;
s6, manually labeling the images in the image set to be labeled, taking the areas of corrosion, cracks and perforations in the images as defect labeling areas, taking the rest parts as background labeling areas, and taking the later-stage images which are manually labeled as inversion semantic segmentation training set samples;
s7, training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting a semantic segmentation label, comparing the semantic segmentation label with an artificial label, and optimizing the weight of the downstream task segmentation model through data inversion; then, verifying by adopting a verification set to obtain a mature downstream task segmentation model;
and S8, acquiring an early image of the leaf to be predicted, inputting the image into a mature downstream task segmentation model, and outputting position information of the early defect of the leaf to be predicted.
As a preferred embodiment of the present invention, the collected image is RGB image data having 3 channels.
As a preferred embodiment of the present invention, a down-sampling module is connected in front of the base network; the downsampling module performs double downsampling by using a convolution layer with the convolution kernel size of 3 multiplied by 3 and the step length of 2, and activates the characteristic matrix after sampling by using a ReLU activation function.
As a preferred embodiment of the invention, the base network comprises a convolutional constructed Stem Block and four repeatedly stacked Transformer layers constructed Transformer structures; wherein, the first and the second end of the pipe are connected with each other,
the Stem Block consists of a convolution layer with a convolution kernel size of 4 multiplied by 4 and a step length of 4, a ReLU activation function layer, a batch normalization layer, a position coding embedded layer and a linear embedded layer;
each layer of Transformer layer consists of a layer normalization layer, a multi-head self-attention layer, a GELU activation function layer and a multi-layer perceptron layer, wherein the multi-head self-attention layer and the multi-layer perceptron layer are respectively connected on respective outputs by adopting residual errors.
As a preferred embodiment of the present invention, the building of the early-stage defect self-supervision learning model based on the base network comprises:
for each input label-free image, after the image is divided into a plurality of image blocks with the resolution of 4 x 4, masking 75% of the image blocks by using a mask randomly, reconstructing the whole original image by using a small amount of unmasked image block data through model learning, and calculating loss by using a standardized pixel loss function to perform back propagation, so that the model can learn the internal relation between the pixel points of the early image, and realize an automatic supervision auxiliary task.
As a preferred embodiment of the invention, the hyper-parameters of the early defect self-supervision learning model comprise the number of multi-head attention heads, the number of tokens and the length of tokens of each transform layer.
As a preferred embodiment of the present invention, step S4 performs preprocessing on the samples in the self-supervised training set for data enhancement before training, where the preprocessing includes random horizontal flipping, random vertical flipping, random multi-scale transformation, and random angle transformation.
As a preferred embodiment of the present invention, the segmentation head model is composed of a cavity space convolution pooling pyramid structure and a low-frequency characteristic path structure; wherein, the first and the second end of the pipe are connected with each other,
the void space convolution pooling pyramid structure is formed by connecting 1 x 1 convolution layers, 3 x 3 convolution layers with expansion coefficients of 6, 12 and 18 respectively and a global pooling layer in parallel, and all the convolution layers are activated by adopting a ReLU activation function and are connected with a batch standard layer in series; the convolution layer in the void space convolution pooling pyramid structure is spliced with the output of the pooling layer in the channel dimension, feature fusion is carried out through the convolution layer of 1 x 1, and the feature matrix resolution is restored to the output size of the base network through the upper sampling layer of 4 times;
the low-frequency characteristic path structure is composed of 1 multiplied by 1 convolutional layers, is spliced with the output of the void space convolutional pooling pyramid structure on the channel dimension, is subjected to characteristic fusion through a 3 multiplied by 3 convolutional layer, is restored to the size of the resolution of an input image through a transposed convolutional layer with the step length of 4 and the convolutional kernel size of 4 multiplied by 4, and is finally adjusted to output the channel dimension to be the number of the marked categories through the 1 multiplied by 1 convolutional layer.
As a preferred embodiment of the present invention, in step S3, a markov chain monte carlo algorithm is used to estimate and set the hyper-parameters, so as to obtain a hyper-parameter global optimal solution and a corresponding hyper-parameter confidence interval.
In another aspect, an embodiment of the present invention further provides a system for predicting an early defect of a wind turbine blade, where the system includes: the system comprises a data acquisition module, a sample labeling and dividing module, a base network construction module, a pre-training module, a segmentation head construction module and a prediction model training module; wherein, the first and the second end of the pipe are connected with each other,
the data acquisition module is used for collecting images of the early, middle and late scenes of the blades of the wind driven generator in the whole life cycle, wherein the early, middle and late scenes respectively account for one third of the life cycle; the method is also used for acquiring an early image of the blade to be predicted;
the sample labeling and dividing module is used for selecting a late-stage image with defects capable of being identified by naked eyes to establish an image set to be labeled; taking the corresponding early and middle leaf image sets as model sample sets, selecting samples in a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set; the method is also used for carrying out manual annotation on the images in the image set to be annotated, taking areas of corrosion, cracks and perforations in the images as defect type annotation areas, taking the rest parts as background type annotation areas, and taking the later-stage images which are subjected to manual annotation as inversion semantic segmentation training set samples;
the base network construction module is used for constructing a base network based on a Transformer structure, constructing an early defect self-supervision learning model based on the base network, and setting hyper-parameters of the early defect self-supervision learning model; the pre-training module is used for loading the weight of a mature self-supervision learning model after completing training, and is connected with the segmentation head model to be used as a downstream task segmentation model;
the pre-training module is used for training an early defect self-supervision learning model by adopting the self-supervision training set, mining data information from unsupervised data, enabling the model to learn the relation among all pixels, extracting deep semantic representation, obtaining a mature early defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment;
the segmentation head construction module is used for constructing a segmentation head model;
the prediction model training module is used for training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting a semantic segmentation label, comparing the semantic segmentation label with an artificial label, and optimizing the weight of the downstream task segmentation model through data inversion; then, verifying by adopting a verification set to obtain a mature downstream task segmentation model; and the method is also used for inputting the early image of the blade to be predicted into a mature downstream task segmentation model and outputting the position information of the early defect of the blade to be predicted.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
according to the method and the system for predicting the early-stage defects of the blades of the wind driven generator, provided by the embodiment of the invention, a self-supervision training method utilizing time sequence information is designed to enable a model to automatically learn image characteristics, so that the problem that the early-stage fine defect samples of the blades are difficult to manually label is solved; by utilizing the defect change process of the blades of the wind driven generator, the later-stage easily-labeled defect sample is used for network training to invert the early-stage defect characteristics, so that the deficiency of early-stage defect information is relieved; meanwhile, the method improves the application of a transform structure to semantic segmentation tasks, improves the capability of a network model in learning multi-dimensional and long-distance dependency relationship of image information, estimates the hyper-parameters of the model by using a Bayesian theory estimation method, avoids the problem that the hyper-parameters fall into local optimum, ensures that the model reaches the optimum solution globally, and improves the segmentation accuracy.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a method for early blade defect prediction in a wind turbine according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for predicting early defects of a wind turbine blade according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a base network structure according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a structure of a segmentation head model according to an embodiment of the present invention;
FIG. 5 is a flow chart of a Bayesian theory estimation hyperparameter in an embodiment of the present invention;
FIG. 6 is a flowchart illustrating the training process of an early-stage defect auto-supervised learning model according to an embodiment of the present invention;
FIG. 7 is a flow chart of semantic segmentation inversion training in an embodiment of the present invention.
Detailed Description
After finding the above problems, the inventors of the present application have conducted intensive studies on a conventional method for detecting an early defect of a blade of a wind turbine. Research finds that early-stage defects of the leaves can be detected based on an automatic supervision deep learning method. With the maturity of deep learning and artificial intelligence technology, the self-supervision deep learning method has made a remarkable progress in the aspect of large sample label-free learning clustering characteristics. In addition, the defects of the wind turbine blade have a changing process from early stage to late stage, and the change in the process has a dynamic law.
However, the deep learning model based on the CNN has limited ability to acquire global information, cannot capture long-distance dependence, and is difficult to solve the problems of small information dimension scale, large number of difficult samples, low labeling precision, poor model generalization and the like of the early state prediction data set; the uncertainty of the model hyperparameter in the deep learning network training stage causes the uncertainty of the model result, the model efficiency is reduced (the model hyperparameter is uncertain), and the local optimization rather than the global optimization of the model hyperparameter optimization can be caused by the conventional optimization method based on point estimation expansion.
It should be noted that the above prior art solutions have defects which are the results of practical and careful study by the inventors, and therefore, the discovery process of the above problems and the solutions proposed by the following embodiments of the present invention to the above problems should be the contribution of the inventors to the present invention in the course of the present invention.
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. In the description of the present invention, the terms "first", "second", "third", "fourth", and the like are used only for distinguishing the description, and are not intended to indicate or imply relative importance.
After the deep analysis, the embodiment of the invention provides a method for predicting the early defects of the blades of the wind driven generator, which is developed by aiming at the design of a base network construction and a self-supervision learning model and aims to use a small amount of easily-labeled data of late defects to invert and predict the state of the early defects, so as to realize the segmentation task of the early defects, specifically, the early defect data is subjected to self-supervision learning without labeling, and then the later easily-labeled defect data is adopted to invert the characteristics of the early defect data, so that the early defects of the blades are predicted, the prediction accuracy of the early defects of the blades is effectively improved, the blades are maintained or replaced in time, and the operation safety of the blades is improved.
Referring to fig. 1 and 2, a method for predicting an early defect of a wind turbine blade according to an embodiment of the present invention includes the following steps:
step S1, collecting images of scenes of a plurality of wind driven generator blades in the early stage, the middle stage and the late stage of the whole life cycle, wherein the early stage, the middle stage and the late stage respectively account for one third of the life cycle.
In this step, when image acquisition is performed, it is preferable to use a majiang longitude and latitude match 210 unmanned aerial vehicle which can bear the wind speed of 10m/s and carry RGB cameras to perform image acquisition on generator blades with different sizes of corrosion, cracks and perforation defects at different times (early stage, middle stage and late stage), so as to obtain an RGB image data set with 3 channels. Clipping is performed according to the uniform reference frame size.
S2, selecting a late image with defects capable of being recognized by naked eyes to establish an image set to be annotated; and taking the corresponding early-stage and middle-stage image sets of the leaf as model sample sets, selecting samples with a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set.
In this step, the state of the defect is represented as a time series data, which can be expressed by using a time series dynamic equation, for example: x _ (t + 1) = F (X _ t, U _ t), wherein X _ (t + 1) represents the defect characteristics at the moment t +1, X _ t represents the defect characteristics at the moment t, F represents the defect change process function, and U _ t represents internal and external factors influencing the defect change at the moment t. Typically, small defects are present in the early and middle images, and significant defects are present in the late images. The obvious defects can be identified by naked eyes and can be manually marked.
And S3, constructing a base network based on a Transformer structure, constructing an early defect self-supervision learning model based on the base network, and estimating and setting hyper-parameters of the early defect self-supervision learning model.
As shown in fig. 3 and 4, in this step, the base network includes a Transformer structure constructed by convolving a transform layer constituting a Stem Block and four layers of repeated stacks. The Stem Block consists of a convolution layer with a convolution kernel size of 4 multiplied by 4 and a step length of 4, a ReLU activation function layer, a batch normalization layer, a position coding embedded layer and a linear embedded layer; each layer of Transformer layer consists of a layer normalization layer, a multi-head self-attention layer, a GELU activation function layer and a multi-layer perceptron layer, wherein the multi-head self-attention layer and the multi-layer perceptron layer are respectively connected on respective outputs by adopting residual errors.
Preferably, a downsampling module can be connected to the base network to reduce the calculation amount of the base network. The downsampling module performs double downsampling by using a convolution layer with the convolution kernel size of 3 multiplied by 3 and the step length of 2, and activates the characteristic matrix after sampling by using a ReLU activation function.
The hyper-parameters related to the early defect self-supervision learning model comprise the number of multi-head attention heads, the number of tokens, the length of tokens and the like of each transform layer.
As shown in fig. 5, the hyper-parameters are estimated and set, and the markov chain monte carlo algorithm in the bayes theory estimation method is used. Firstly, defining a minimum loss function, setting a hyper-parameter to be estimated, setting the rest hyper-parameters to be unchanged, inputting the hyper-parameters to be estimated into an early defect picture expired defect self-supervision learning model, and estimating the set hyper-parameter to be estimated by using a Markov chain Monte Carlo algorithm; and selecting the result with the minimum loss function after multiple iterations as the estimated hyperparameter. Specifically, the method for estimating the hyper-parameters of the early defect self-supervision learning model by adopting the Markov chain Monte Carlo algorithm comprises the following steps: firstly, determining hyper-parameters in a model by using algorithms including but not limited to EFAST, sabo and the like, then defining a minimization loss function, and performing nonlinear model parameter estimation on each new iteration by using a light-weighted improved Markov chain Monte Carlo algorithm to obtain a hyper-parameter global optimal solution and a corresponding hyper-parameter confidence interval.
And S4, training the early-stage defect self-supervision learning model by adopting the self-supervision training set, mining the information of the data from the non-supervision data, enabling the model to learn the relation among all pixels, extracting deep semantic representation to obtain a mature early-stage defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment.
In this step, before training, the samples in the self-supervised training set may be preprocessed to enhance data, where the preprocessing includes random horizontal flipping, random vertical flipping, random multi-scale transformation, random angle transformation, and the like, so as to obtain an extended self-supervised training set.
As shown in fig. 6, in order to implement the self-supervision auxiliary task, when the early defect self-supervision learning model is trained, for each input label-free image, after the image is divided into a plurality of image blocks with the resolution of 4 × 4, 75% of the image blocks are randomly masked by using a mask, the model learning reconstructs the whole original image by using a small amount of unmasked image block data, and the normalized pixel loss function is used for calculating loss to perform back propagation, so that the model can learn the internal relation between the pixels of the early image.
The weight obtained at the moment is obtained by an auto-supervised deep learning algorithm based on early defects; and then, the weights at the moment are optimized through inversion of late-stage defect data, and features of early-stage defect data are easily inverted by later-stage labeling of the defect data, so that the accuracy of early-stage defect prediction is improved.
And S5, constructing a segmentation head model, loading the weight of the mature self-supervision learning model by the base network, and connecting the base network with the segmentation head model to serve as a downstream task segmentation model.
In this step, the segmentation head model is different from the base network, and the segmentation head model is used to restore the image to the original size and classify each pixel, and is constructed by using a convolution network to capture short-distance detail information.
The segmentation head model consists of a cavity space convolution pooling pyramid structure and a low-frequency characteristic path structure. The void space convolution pooling pyramid structure is composed of 1 x 1 convolution layers (all convolution layers are activated by a ReLU activation function and are connected with batch standard layers in series), 3 x 3 convolution layers with expansion coefficients of 6, 12 and 18 are connected with a global pooling layer in parallel, the convolution layers in the void space convolution pooling pyramid structure are spliced with the output of the pooling layer in channel dimension, feature fusion is carried out through the 1 x 1 convolution layers, and the feature matrix resolution is restored to the output size of a base network through an upper sampling layer of 4 times; the low-frequency characteristic path structure is composed of 1 multiplied by 1 convolutional layers, the low-frequency characteristic path structure is spliced with the output of the hollow space convolutional pooling pyramid structure on the channel dimension, the characteristic fusion is carried out on the convolutional layers of 3 multiplied by 3, then the output characteristic matrix is restored to the input image resolution size through the transposed convolutional layer with the step length of 4 and the convolutional kernel size of 4 multiplied by 4, and finally the output channel dimension is adjusted to be the labeled category number through the 1 multiplied by 1 convolutional layer.
The loading of the weight of the self-supervision learning model in the step is an inheritance process, and the weight obtained by training the early-stage defect self-supervision learning model is inherited into the downstream task segmentation model.
And S6, manually labeling the images in the image set to be labeled, taking the areas of corrosion, cracks and perforations in the images as defect labeling areas, taking the rest parts as background labeling areas, taking the later-stage images which are manually labeled as inversion semantic segmentation training set samples, and making semantic segmentation labels for supervised training.
S7, training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting semantic segmentation labels, comparing the early-stage defect information obtained by self-supervision learning with artificial labels, and optimizing the weight of the downstream task segmentation model through data inversion, so that the weight of the model at the moment is subjected to inverse optimization through a labeled later-stage image, and relevant information with obvious defect image characteristics is obtained; and then, verifying by adopting a verification set to obtain a mature downstream task segmentation model.
As shown in fig. 7, after the base network inherits the weight of the base network in the self-supervision training, the weight is transmitted to the segmentation head structure spliced with the base network, the segmentation result is obtained after the training is completed, the loss is calculated by using the cross entropy, the weight of the base network is propagated and optimized in the range of the iteration times in the reverse direction, and the weight is transmitted to the segmentation head repeatedly to perform the next training until the training is completed. The base network loads the weight of the mature self-supervision learning model, namely inherits the early defect information obtained in the self-supervision learning, and obtains the transmission and inheritance of the early defect information in the downstream task segmentation model at the moment.
And S8, acquiring an early image of the leaf to be predicted, inputting the image into a mature downstream task segmentation model, and outputting position information of the early defect of the leaf to be predicted.
Based on the same idea, the embodiment of the invention also provides a system for predicting the early defect of the wind turbine blade, which comprises: the system comprises a data acquisition module, a sample labeling and dividing module, a base network construction module, a pre-training module, a segmentation head construction module and a prediction model training module; wherein, the first and the second end of the pipe are connected with each other,
the data acquisition module is used for collecting images of the early, middle and late scenes of the blades of the wind driven generator in the whole life cycle, wherein the early, middle and late scenes respectively account for one third of the life cycle; the method is also used for acquiring an early image of the blade to be predicted;
the sample labeling and dividing module is used for selecting a late-stage image with defects capable of being identified by naked eyes to establish an image set to be labeled; taking the corresponding early and middle leaf image sets as model sample sets, selecting samples in a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set; the method is also used for carrying out manual annotation on the images in the image set to be annotated, taking areas of corrosion, cracks and perforations in the images as defect type annotation areas, taking the rest parts as background type annotation areas, and taking the later-stage images which are subjected to manual annotation as inversion semantic segmentation training set samples;
the base network construction module is used for constructing a base network based on a Transformer structure, constructing an early-stage defect self-supervision learning model based on the base network, and setting hyper-parameters of the early-stage defect self-supervision learning model; the pre-training module is used for loading the weight of a mature self-supervision learning model after completing training, and is connected with the segmentation head model to be used as a downstream task segmentation model;
the pre-training module is used for training an early defect self-supervision learning model by adopting the self-supervision training set, mining data information from unsupervised data, enabling the model to learn the relation among all pixels, extracting deep semantic representation, obtaining a mature early defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment;
the segmentation head construction module is used for constructing a segmentation head model;
the prediction model training module is used for training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting a semantic segmentation label, comparing the semantic segmentation label with an artificial label, and optimizing the weight of the downstream task segmentation model through data inversion; then, verifying by adopting a verification set to obtain a mature downstream task segmentation model; and the method is also used for inputting the early image of the blade to be predicted into a mature downstream task segmentation model and outputting the position information of the early defect of the blade to be predicted.
In the embodiment, each module is realized by a processor, and when the storage is needed, the storage is added appropriately. The Processor may be, but is not limited to, a microprocessor MPU, a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, and the like. The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
In addition, it should be noted that the system for predicting the early defect of the wind turbine blade in the embodiment corresponds to the method for predicting the early defect of the wind turbine blade, and the description and the limitation of the method are also applicable to the system, and are not described herein again.
According to the technical scheme, the method and the system for predicting the early-stage defects of the blades of the wind driven generator, which are provided by the embodiment of the invention, design a self-supervision training method utilizing time sequence information to enable a model to automatically learn image characteristics, so that the problem that the early-stage tiny defect samples of the blades are difficult to label manually is avoided; by utilizing the defect change process of the blades of the wind driven generator, the later-stage easily-labeled defect sample is used for network training to invert the early-stage defect characteristics, so that the deficiency of early-stage defect information is relieved; meanwhile, the method improves the Transformer structure used for semantic segmentation tasks, improves the capability of a network model for learning multi-dimensional and long-distance dependency relationship of image information, estimates the hyper-parameters of the model by using a Bayesian theory estimation method, avoids the problem that the hyper-parameters fall into local optimum, ensures that the model reaches the optimum solution globally, and improves the segmentation accuracy.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method for predicting early defects of a wind turbine blade is characterized by comprising the following steps:
the method comprises the following steps that S1, images of scenes of a plurality of wind driven generator blades in the early stage, the middle stage and the late stage of the whole life cycle are collected, wherein the early stage, the middle stage and the late stage respectively account for one third of the life cycle;
s2, selecting a late image with defects capable of being recognized by naked eyes to establish an image set to be annotated; taking the corresponding early and middle leaf image sets as model sample sets, selecting samples in a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set;
s3, constructing a base network based on a Transformer structure, constructing an early defect self-supervision learning model based on the base network, and setting hyper-parameters of the early defect self-supervision learning model;
s4, training an early-stage defect self-supervision learning model by adopting the self-supervision training set, mining data information from unsupervised data, enabling the model to learn the relation among all pixels, extracting deep semantic representation to obtain a mature early-stage defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment;
s5, constructing a segmentation head model, wherein the weight of the mature self-supervision learning model is loaded by the base network at the moment, and the base network is connected with the segmentation head model to serve as a downstream task segmentation model;
s6, manually labeling the images in the image set to be labeled, taking the areas of corrosion, cracks and perforations in the images as defect labeling areas, taking the rest parts as background labeling areas, and taking the later-stage images which are manually labeled as inversion semantic segmentation training set samples;
s7, training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting a semantic segmentation label, comparing the semantic segmentation label with an artificial label, and optimizing the weight of the downstream task segmentation model through data inversion; then, verifying by adopting a verification set to obtain a mature downstream task segmentation model;
and S8, acquiring an early image of the leaf to be predicted, inputting the image into a mature downstream task segmentation model, and outputting position information of the early defect of the leaf to be predicted.
2. The method of claim 1, wherein the collected image is RGB image data having 3 channels.
3. The method for early blade defect prediction of a wind turbine generator as claimed in claim 1, wherein a down-sampling module is connected in front of the base network; the downsampling module uses a convolution layer with convolution kernel size of 3 multiplied by 3 and step length of 2 to carry out double downsampling, and uses a ReLU activation function to activate the characteristic matrix after sampling.
4. The method for predicting the early defect of the wind turbine blade according to any one of claims 1 to 3, wherein the base network comprises a convolution constructed Stem Block and a Transformer structure constructed by four repeatedly stacked Transformer layers; wherein, the first and the second end of the pipe are connected with each other,
the Stem Block consists of a convolution layer with a convolution kernel size of 4 multiplied by 4 and a step length of 4, a ReLU activation function layer, a batch normalization layer, a position coding embedded layer and a linear embedded layer;
each layer of Transformer layer consists of a layer normalization layer, a multi-head self-attention layer, a GELU activation function layer and a multi-layer perceptron layer, wherein the multi-head self-attention layer and the multi-layer perceptron layer are respectively connected on respective outputs by adopting residual errors.
5. The method for early defect prediction of wind turbine blades according to claim 4, wherein said building an early defect self-supervised learning model based on a base network comprises:
for each input label-free image, after the image is divided into a plurality of image blocks with the resolution of 4 x 4, masking 75% of the image blocks by using a mask randomly, reconstructing the whole original image by using a small amount of unmasked image block data through model learning, and calculating loss by using a standardized pixel loss function to perform back propagation, so that the model can learn the internal relation between the pixel points of the early image, and realize an automatic supervision auxiliary task.
6. The method for predicting the early defects of the wind turbine blade according to claim 5, wherein the hyper-parameters of the early defect self-supervision learning model comprise the number of multi-head attention heads, the number of tokens and the token length of each layer of the transform layer.
7. The method for early blade defect prediction of wind turbine generator as claimed in claim 1, wherein step S4 is performed by preprocessing samples in the self-supervised training set for data enhancement before training, wherein the preprocessing comprises random horizontal flipping, random vertical flipping, random multi-scale transformation and random angle transformation.
8. The method for early blade defect prediction of a wind turbine generator as claimed in claim 1, wherein the segmentation head model is composed of a void space convolution pooling pyramid structure and a low frequency feature path structure; wherein the content of the first and second substances,
the hollow space convolution pooling pyramid structure is formed by connecting convolution layers of 1 x 1, convolution layers of 3 x 3 with expansion coefficients of 6, 12 and 18 respectively and a global pooling layer in parallel, and all the convolution layers are activated by adopting a ReLU activation function and are connected with batch standard layers in series; the convolution layer in the cavity space convolution pooling pyramid structure is spliced with the output of the pooling layer in the channel dimension, feature fusion is carried out through the convolution layer of 1 x 1, and the feature matrix resolution is restored to the output size of the base network through the upper sampling layer of 4 times;
the low-frequency characteristic path structure is composed of 1 multiplied by 1 convolutional layers, the low-frequency characteristic path structure is spliced with the output of the hollow space convolutional pooling pyramid structure on the channel dimension, the characteristic fusion is carried out on the convolutional layers of 3 multiplied by 3, then the output characteristic matrix is restored to the input image resolution size through the transposed convolutional layer with the step length of 4 and the convolutional kernel size of 4 multiplied by 4, and finally the output channel dimension is adjusted to be the labeled category number through the 1 multiplied by 1 convolutional layer.
9. The method for early defect prediction of a wind turbine blade according to claim 1, wherein the hyper-parameters are estimated and set in step S3 using markov chain monte carlo algorithm.
10. A wind turbine blade early defect prediction system, the system comprising: the device comprises a data acquisition module, a sample labeling and dividing module, a base network construction module, a pre-training module, a segmentation head construction module and a prediction model training module; wherein the content of the first and second substances,
the data acquisition module is used for collecting images of the early, middle and late scenes of the blades of the wind driven generator in the whole life cycle, wherein the early, middle and late scenes respectively account for one third of the life cycle; the method is also used for acquiring an early image of the blade to be predicted;
the sample labeling and dividing module is used for selecting a late-stage image with defects capable of being identified by naked eyes to establish an image set to be labeled; taking the corresponding early and middle leaf image sets as model sample sets, selecting samples in a preset proportion from the model sample sets as a self-supervision training set, and manually marking the rest samples as a verification set; the method is also used for carrying out manual annotation on the images in the image set to be annotated, taking areas of corrosion, cracks and perforations in the images as defect type annotation areas, taking the rest parts as background type annotation areas, and taking the later-stage images which are subjected to manual annotation as inversion semantic segmentation training set samples;
the base network construction module is used for constructing a base network based on a Transformer structure, constructing an early-stage defect self-supervision learning model based on the base network, and setting hyper-parameters of the early-stage defect self-supervision learning model; the pre-training module is used for loading the weight of a mature self-supervision learning model after completing training, and is connected with the segmentation head model to be used as a downstream task segmentation model;
the pre-training module is used for training an early defect self-supervision learning model by adopting the self-supervision training set, mining data information from unsupervised data, enabling the model to learn the relation among all pixels, extracting deep semantic representation, obtaining a mature early defect self-supervision learning model, and obtaining the weight of the mature self-supervision learning model at the moment;
the segmentation head construction module is used for constructing a segmentation head model;
the prediction model training module is used for training a downstream task segmentation model by adopting an inverse semantic segmentation training set, outputting a semantic segmentation label, comparing the semantic segmentation label with an artificial label, and optimizing the weight of the downstream task segmentation model through data inversion; then, verifying by adopting a verification set to obtain a mature downstream task segmentation model; and the method is also used for inputting the early image of the blade to be predicted into a mature downstream task segmentation model and outputting the position information of the early defect of the blade to be predicted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211169343.1A CN115239034B (en) | 2022-09-26 | 2022-09-26 | Method and system for predicting early defects of wind driven generator blade |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211169343.1A CN115239034B (en) | 2022-09-26 | 2022-09-26 | Method and system for predicting early defects of wind driven generator blade |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115239034A CN115239034A (en) | 2022-10-25 |
| CN115239034B true CN115239034B (en) | 2022-11-29 |
Family
ID=83666983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211169343.1A Active CN115239034B (en) | 2022-09-26 | 2022-09-26 | Method and system for predicting early defects of wind driven generator blade |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115239034B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861409B (en) * | 2023-02-28 | 2023-06-27 | 华南农业大学 | Soybean leaf area measuring and calculating method, system, computer equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110020122A1 (en) * | 2009-07-24 | 2011-01-27 | Honeywell International Inc. | Integrated condition based maintenance system for wind turbines |
| CN112233091B (en) * | 2020-10-15 | 2021-05-18 | 青岛理工大学 | Wind turbine blade image damage detection and positioning method |
| CN114202505A (en) * | 2021-10-27 | 2022-03-18 | 上海工程技术大学 | Detection method for surface defects of fan blade |
| CN114842026A (en) * | 2022-04-20 | 2022-08-02 | 华能新能源股份有限公司 | Real-time fan blade image segmentation method and system |
-
2022
- 2022-09-26 CN CN202211169343.1A patent/CN115239034B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115239034A (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110728654B (en) | Automatic pipeline detection and classification method based on deep residual error neural network | |
| CN111311538B (en) | Multi-scale lightweight road pavement detection method based on convolutional neural network | |
| EP3690714A1 (en) | Method for acquiring sample images for inspecting label among auto-labeled images to be used for learning of neural network and sample image acquiring device using the same | |
| Caceres et al. | A probabilistic Bayesian recurrent neural network for remaining useful life prognostics considering epistemic and aleatory uncertainties | |
| CN112488025B (en) | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion | |
| CN112613375B (en) | Tire damage detection and identification method and equipment | |
| CN111860106B (en) | Unsupervised bridge crack identification method | |
| CN112750129A (en) | Image semantic segmentation model based on feature enhancement position attention mechanism | |
| CN115239034B (en) | Method and system for predicting early defects of wind driven generator blade | |
| US11948078B2 (en) | Joint representation learning from images and text | |
| CN114283285A (en) | Cross consistency self-training remote sensing image semantic segmentation network training method and device | |
| CN110599459A (en) | Underground pipe network risk assessment cloud system based on deep learning | |
| CN116310850B (en) | Remote sensing image target detection method based on improved RetinaNet | |
| CN114842343A (en) | ViT-based aerial image identification method | |
| CN112036249A (en) | Method, system, medium and terminal for end-to-end pedestrian detection and attribute identification | |
| CN112149612A (en) | Marine organism recognition system and recognition method based on deep neural network | |
| CN116702076A (en) | Small sample migration learning fault diagnosis method, system, computer and storage medium based on CNN feature fusion | |
| CN114332075A (en) | Rapid structural defect identification and classification method based on lightweight deep learning model | |
| CN114863361A (en) | Distribution line bird nest detection method based on USRNet and YOLOv3 | |
| CN114419005A (en) | Crack automatic detection method based on improved light weight CNN and transfer learning | |
| CN117636183A (en) | Small sample remote sensing image classification method based on self-supervision pre-training | |
| CN117372854A (en) | Real-time detection method for hidden danger diseases of deep water structure of dam | |
| CN116935128A (en) | Zero sample abnormal image detection method based on learning prompt | |
| CN117011219A (en) | Method, apparatus, device, storage medium and program product for detecting quality of article | |
| CN116543153A (en) | Semi-supervised point cloud semantic segmentation method based on selective active learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |