WO2022077605A1

WO2022077605A1 - Wind turbine blade image-based damage detection and localization method

Info

Publication number: WO2022077605A1
Application number: PCT/CN2020/125752
Authority: WO
Inventors: 曹金凤; 郭继鸿
Original assignee: 青岛理工大学
Priority date: 2020-10-15
Filing date: 2020-11-02
Publication date: 2022-04-21
Also published as: CN112233091B; CN112233091A

Abstract

A wind turbine blade image-based damage detection and localization method. The method is based on a deep convolutional neural network, and comprises two processes, i.e., model training and damage detection and localization. A wind turbine blade image captured by an unmanned aerial vehicle or a monitoring camera can be interpreted automatically, so that multiple types of wind turbine blade damages can be identified and localized efficiently and accurately, the frequency of unexpected downtime of a wind turbine caused by a wind turbine blade failure is reduced, and the operation and maintenance costs of the wind turbine are reduced.

Description

An image damage detection and localization method for wind turbine blades

Technical field:

The invention belongs to the technical field of fault detection, and in particular relates to a wind turbine blade image damage detection and positioning method.

Background technique:

my country is rich in wind energy resources, with about 4,350 GW of wind energy resources that can be developed in the country, and the reserves of wind energy resources rank among the top in the world. The popularity of wind turbines (referred to as "wind turbines") is rapid in China. With the continuous expansion of the deployment scale of wind turbines in China and even around the world, its operation status monitoring and safety maintenance work have attracted more and more attention. Wind turbines are installed in places with abundant wind resources, most of which are on the seaside or on the hilltops near the seaside, and the environment is harsh. Therefore, wind turbines are usually exposed to changing and harsh environments, such as high altitude, desert, Gobi and sea. Wind turbine blades are the main components of wind turbines, and extreme conditions such as cold, hail, rain and snow, humidity, corrosion, sandstorm, vibration and high temperature will quickly lead to the loss of wind turbine blades. In addition, cracks occur in wind turbine blades due to changing loads and fatigue stress during long-term operation, which poses a serious safety hazard. According to statistics, wind turbine blade damage is one of the main faults leading to wind turbine shutdown. The blades of wind turbines are huge and installed at high altitudes. Professional tools and trained personnel are required to access them. Manual overhaul and maintenance are extremely difficult. Therefore, among all wind turbine failures, the repair cost of wind turbine blade failure is the highest. Longest repair time.

In recent years, the number of published data and literature shows that related fields such as wind turbine blade damage detection, assessment and prediction are becoming the research hotspots of domestic and foreign scientific research institutions and related enterprises. Scholars at home and abroad have carried out a lot of exploration on the online damage detection methods of wind turbine blades. At present, the mainstream detection methods include acoustic emission detection and vibration detection.

(1) Acoustic emission detection

Acoustic emission testing refers to a testing method that evaluates the performance or structural integrity of wind turbine blades by receiving and analyzing the acoustic emission signals of materials. For example, patent CN103389341A discloses a wind turbine blade crack detection method, which is characterized in that: an acoustic emission sensor is installed on the wind turbine blade, and the received acoustic emission signal is transmitted to the acoustic emission acquisition system to determine the sampling frequency of the signal, Sampling length, filtering frequency; optimize the bandwidth parameter of Morlet wavelet basis function based on Shannon wavelet entropy, obtain the Morlet wavelet basis function matching the characteristics of the acoustic emission signal of propagating crack and crack initiation, and then calculate the redistribution scale spectrum of the acoustic emission signal to judge the crack state ; Then according to the time-frequency characteristic parameters of the extracted crack acoustic emission signal to determine the expansion state of the crack fault. Patent CN107657110A discloses a fatigue damage evaluation method for large wind turbine blades, which is characterized in that: an acoustic emission sensor is installed on the wind turbine blade, and the received acoustic emission signal is transmitted to the acoustic emission acquisition system, and the acoustic emission signal is analyzed. The evaluation is performed, and the fatigue level is evaluated according to the evaluation set, so as to achieve the evaluation of the fatigue damage state of the wind turbine blade and determine the real-time state of the blade fatigue damage.

(2) Vibration signal detection

Vibration signal detection refers to the detection of structural vibration signals to reflect the health status of wind turbine blades. For example, patent CN110568074A discloses a method for locating cracks in wind turbine blades based on non-contact multi-point vibration measurement and Hilbert transformation. The blade is excited, and the nonlinear vibration signal of the blade is collected under the random signal excitation condition, and the input and output position vibration signals are excited according to the random signal excitation condition, and the Hilbert transform is performed on the vibration signal to determine the crack position. Patent CN109541028A discloses a wind turbine blade crack position location detection method and system, which is characterized in that: the vibration response signals when the wind turbine blade is cracked or not are collected respectively, and the vibration response signals before and after crack damage are calculated according to the vibration response signals when there are cracks. Mutual information entropy and vibration response nonlinearity change, and the crack location is determined according to the change.

However, in actual operation, the above detection method still has several insurmountable shortcomings: 1. The operating conditions of wind turbine blades are complex and changeable, the sensor signal is easily interfered by a large amount of noise, and the fault information is easily submerged, resulting in judgment errors. It is difficult to extract robust fault features. 2. It is difficult to detect early damage of blades using methods based on acoustic or vibration signals. For example, when the crack of the blade is small or the crack is located close to the tip of the blade, the change in natural frequency and vibration response is small, which is difficult to be detected by the sensor. 3. During the operation of the wind turbine, different signal data are obtained through the sensors arranged on the wind turbine blades. The arrangement of the sensors, the service life and the accuracy of the collected signals will greatly affect the reliability of the wind turbine blade fault detection method. . For example, a patch-type optical fiber load sensor is installed on the wind turbine blade, and a certain method is used to monitor the blade crack by collecting the signal during the operation of the wind turbine. However, this method ignores the sensor's own performance and is easily affected by environmental factors. , Under the complex operating conditions of the wind turbine, the sensors installed on the blades are prone to damage, which affects the accuracy of the detection results and increases the detection cost.

In recent years, by using drones and surveillance cameras to capture a large number of high-resolution images of wind turbine blades, wind farms have realized remote real-time monitoring of the operating conditions of wind turbines, especially the surface damage of wind turbine blades, which greatly improves the efficiency of wind turbines. maintenance and operational efficiency. Compared with detection methods based on acoustic emission signals and vibration signals, wind turbine blade damage detection using images or videos is clear and intuitive, and is more sensitive to small defects that do not cause significant signal changes, and does not rely on sensors to obtain signal data. various disadvantages of the above method. However, for the large number of images captured by drones and surveillance cameras, the current processing method is still to manually screen the surface damage of wind turbine blades. In this case, it is of great significance to automatically process images captured by drones and surveillance cameras and generate analysis results by computers, which can save labor costs and eliminate human errors.

In the past decade, the tremendous development of computer vision and deep learning has facilitated the application of image processing and object detection in industrial scenarios. Abroad, Hutchinson et al. proposed a statistical-based image evaluation method based on Bayesian decision theory to detect the damage of concrete structures. Cha et al. used deep learning neural networks to detect concrete cracks. Wang et al. proposed a data-driven wind turbine blade damage detection framework for automatic crack detection based on images captured by drones by using an extended cascade classifier. Wang et al. proposed a crack detection method using unsupervised learning with deep features. Using computer vision and deep learning methods to automatically process wind turbine blade images and achieve damage detection and identification, the challenges are: 1. Wind turbine blade images have more complex background information, such as the sky around the wind turbine, forests and other wind power equipment; 2. The size, shape and texture of blade damage are different; 3. Compared with the size of the blade (tens of meters), the size of the defect is extremely small, and it is difficult for a general detector to accurately locate it. For example, early blade cracks can be a few centimeters or less. Based on the above factors, the current detection methods perform poorly in practical applications in terms of detection accuracy (high false early warning rate and missed detection rate) and recognition effect (either damage localization cannot be achieved, or specific damage types cannot be classified).

Therefore, it is necessary to develop a new method of wind turbine blade fault detection, through remote online detection of wind turbine blade damage, early warning before blade failure, effectively avoid accidents, and reduce maintenance and operation costs.

Invention content:

The technical problem to be solved by the present invention is that my country is a big country in wind power generation industry and equipment manufacturing, but not a strong country, and the possession of technologies and invention patents in related fields is at a disadvantage. Moreover, my country's major wind farms will gradually enter a period of high accident incidence, and the safety of wind power equipment in service has become a bottleneck for wind power development. The fault warning and safety guarantee of the healthy operation of wind turbines urgently need breakthroughs in theory and technology. The newly installed wind turbine blades lack detailed characteristic data, expert-level diagnosis and maintenance information under complex loads and harsh natural conditions, and it is difficult to accurately express them with accurate theoretical models. New theories and methods for forecasting and safety assurance.

In order to solve the above problems, the present invention develops an image recognition algorithm and an effective feature extraction algorithm reflecting the operating state of the wind turbine blades by introducing a deep learning algorithm, detects the damage of the wind turbine blades online remotely, and provides an early warning before the blades fail. Effectively avoid accidents, reduce maintenance and operation costs, improve the reliability of wind turbine blades under complex and harsh geographical and meteorological conditions, ensure long-term stable and reliable operation of wind turbines, reduce wind turbine operation and maintenance costs, and improve wind turbines. Economic benefits and market competitiveness.

In order to achieve the above object, the present invention is achieved through the following technical solutions: a wind turbine blade image damage detection and localization method, based on a deep convolutional neural network, includes two processes of model training and damage detection and localization;

Among them, the model training process includes:

S101. Collect surface images of wind turbine blades captured by drones or surveillance cameras;

S102. Manually mark the damaged position and type information in the wind turbine blade image, and cut out an image sample containing the damaged area from the image according to the manual marking, that is, a positive sample; and an image sample of a normal blade surface, that is, a negative sample ; Establish a sample database;

S103, using the sample database established in step S102, according to the classical AdaBoost Haar-like algorithm to train the cascade strong classifier;

S104, building a deep convolutional neural network damage identification model, and using the sample database established in step S102 to train, adjust parameters and verify the convolutional neural network model to obtain a trained damage identification model;

The damage detection and localization process includes:

S201, traverse the wind turbine blade image through the sliding window method, and identify whether the image may contain a damaged area through the identification of the cascade strong classifier trained in step S103;

S202: Scale the damage target area detected in step S201 to a uniform size, input the damage identification model trained in step S104, and after model identification, determine which damage type the area belongs to; due to the suspected damage detected in the previous step Damage target areas have different scales and sizes, and for the next step of identification, these areas need to be scaled to a uniform size. The suspected area scaled to a uniform size is sequentially input into the damage identification model trained in step S104, and after the model is identified, it is determined which damage type the suspected area belongs to. If it is judged that the suspected area does not belong to any kind of injury, the area is excluded from the suspected area (ie, it is regarded as a normal area);

S203. Output a result file, including information such as the position and type of the identified damage target area, and mark the damaged area and type with a box and a number in the corresponding position of the original wind turbine blade image.

Further, the manually marked damage in S102 includes four types of glass fiber damage, crack, skin damage and corrosion.

Further, the training process of the mid-body algorithm in S103 is as follows:

(1) First, convert the image samples into grayscale images, and then calculate their Haar-like features for each image;

(2) Train a strong classifier of image samples through the AdaBoost algorithm; suppose that the training data set {(x ₁ , y ₁ ), (x ₂ , y ₂ ), ..., (x _N , y _N )} (where N is number of training samples) and

y∈[-1, 1]. x _i is a training sample, and y _i =1 indicates that the training image sample contains a damaged area. Assuming that hi is one of the _Haar -like features calculated by the training sample x, the calculation formula of the single-feature weak classifier f(x) is:

where t _k is the segmentation threshold, t _k =0.5×(h _k +h _k+1 ), _wi is the weight of the training samples; for each iteration step m=1,2,...,M, by the following equation Compute the optimal weak classifier:

At the end of each iteration, the strong classifier F(x) and _wi weight values are updated by

F(x)←F(x)+f _m (x) (3)

w _i ←w _i exp(-y _i f _m (x _i )) (4)

At the beginning of training, F(x)=0, the weight distribution of the training data is given by

get initialized. After performing the required iterations, the final strong classifier C(x) can be expressed as:

Further, the deep convolutional neural network damage identification model built in S104 includes an input layer, a convolutional layer, a pooling layer and an output layer, without a fully connected layer. The model greatly reduces the number of training parameters, while maintaining the model's excellent recognition ability, which enables the model to have higher training efficiency and better generalization ability.

Further, the deep convolutional neural network damage identification model built in S104 includes four groups of feature extraction modules including convolutional layers and pooling layers, with a total of 8 convolutional layers and 4 pooling layers. The multi-layered convolution and pooling operations enable the model to capture abstract and robust features that vary greatly between categories in small-sized image samples, resulting in better damage recognition performance.

Further, the training step in S104 includes: using the gradient descent method to iteratively update the weights of the convolutional neural network model for multiple times, so that the output of the model loss function is reduced, and the detection value of the model and the real value of the data are getting closer and closer. , until the loss function converges to 0, the training and optimization of the model are completed, and the trained convolutional neural network model is obtained.

Further, the parameter adjustment step in S104 includes: using a cross-validation method to determine parameters such as the optimal training batch size and learning rate of the model; and the validation step includes: the performance result of the model on the validation data, that is, obtaining the model validation accuracy rate.

Further, step S201 is specifically as follows: zooming the image of the wind turbine blade to be detected multiple times according to a certain proportion, and on a specific zoom scale, panning the sliding window in the horizontal or vertical direction according to a fixed step size; sliding the window after each translation. The covered image area is used as the detection window, and the Haar-like features in the detection window are extracted, and are identified by the cascade strong classifier trained in step S103 to determine whether the area may contain damaged areas. The image is first based on Haar-like features and shallow images. The layer classifier filters out the suspected damage target area, which greatly reduces the computational cost and lays the foundation for the deployment of deep learning network identification in the next step; if the model judges that the detection window may contain damage areas, the area is marked with a box It is the suspected damage target area, otherwise skip this area to determine the next area; use the above method to traverse the entire image to obtain a series of suspected damage target areas.

Aiming at the lack of mature technology for wind turbine blade damage detection and localization that can be used in engineering practice, the invention discloses a wind turbine blade image damage detection method based on a deep convolutional neural network, which can shoot unmanned aerial vehicles and monitoring cameras. The automatic interpretation of wind turbine blade images based on the automatic interpretation of wind turbine blade images enables efficient and accurate identification and localization of various types of wind turbine blade damage. Achieve blade damage assessment and early warning, reduce the number of unexpected shutdowns of wind turbines caused by wind turbine blade failures, and reduce wind turbine operation and maintenance costs. This solution has the advantages of fast recognition speed, high accuracy, fully automated process, and low operating threshold, which makes up for the shortcomings of traditional methods that rely on manual completion, low efficiency, high misjudgment rate, and time-consuming and labor-intensive. At present, domestic research results in related aspects are rarely reported. The invention fills the blank of the domestic wind turbine blade image damage detection and identification method, and is of great significance to ensuring the safe production and development of the national wind power industry. The beneficial effects of the present invention are specifically:

(1) The method can automatically interpret and process the surface images of wind turbine blades captured by unmanned aerial vehicles and monitoring cameras, so as to realize efficient detection and early warning of surface damage of wind turbine blades. Compared with the traditional method, there is no need to consider the signal interference during the operation of the blade, and there is no need to deploy sensors, which can realize the early damage detection of the blade;

(2) This method uses computer vision and deep learning algorithms to automatically identify the type of damage on the surface of wind turbine blades, and locate the damage location in the image without manual assistance, which greatly saves manpower and material resources, saves time, and reduces the operation of wind farms. maintenance costs;

(3) This method implements a new deep convolutional neural network damage recognition model, which simplifies the traditional convolutional neural network model structure, maintains the model's excellent recognition ability, and enables the model to have higher training efficiency and Better generalization ability. The experimental results show that the recognition and classification accuracy of this model is significantly higher than that of traditional models such as VGG16, which lays a foundation for the feasibility of this method to recognize various types of damage on the surface of wind turbine blades.

(4) This method designs a new damage detection process, which includes a two-stage detection process. First, the suspected damage target area is screened through Haar-like features and shallow classifiers, and then through the deep convolutional neural network. The damage identification model identifies the suspected damage target area. Using this method can avoid the recognition model from wasting too much time in a large number of areas that obviously do not contain damage, so that more detection time can be concentrated on processing difficult-to-recognize areas, which greatly improves the detection efficiency of a single image. It can lay a foundation for the rapid identification of surface damage of wind turbine blades.

(5) This method uses Haar-like features and shallow classifiers to perform multi-scale detection on the image to be tested, thereby realizing damage localization of various sizes from large to small.

Description of drawings

Figure 1 is a schematic diagram of the model training process;

Fig. 2 is an image proof of the surface of the wind turbine blade;

Figure 3 is a schematic diagram of the structure of the damage recognition model of the deep convolutional neural network.

Among them, 1 convolution layer, 2 pooling layer, 3 global large pooling layer, 4 Dropout operation, 5 output layer;

Fig. 4 is the confusion matrix of the verification result;

Figure 5 is a schematic diagram of the damage image detection process;

Fig. 6 is the surface image proof sheet of the wind turbine blade to be detected;

Fig. 7 is the detection result of the suspected damaged area of the image to be detected;

FIG. 8 is the final damage area detection result of the image to be detected.

Detailed ways:

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Example. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1:

A wind turbine blade image damage detection and localization method, based on a deep convolutional neural network, includes two processes: model training and damage detection and localization:

Among them, as shown in Figure 1, model training includes the following steps:

In step S101, a monitoring camera is used to collect the surface image of the wind turbine blade. The image data used in the embodiments of the present invention are all from a wind farm in eastern China, and include a total of 725 wind turbine blade surface images captured by high-resolution cameras. Figure 2 is a sample image of the surface of the wind turbine blade collected at the wind farm site.

In step S102, manually mark the damaged position and type information on the wind turbine blade image, and cut out image samples (positive samples) containing damaged areas and image samples (negative samples) of the normal blade surface from the image according to the manual marking. sample) to establish a sample database. Manually marked damage includes four types of glass fiber damage, cracks, skin damage and corrosion. Among them, glass fiber damage refers to severe damage to the blade resulting in delamination of fibers and laminates, crack refers to the rupture or degumming of the blade gel coat, and skin damage is Refers to blade gel coat delamination, breakage and damage, and corrosion refers to blade leading edge corrosion.

In step S103, using the sample database established in step S102, the cascading strong classifier is trained according to the classical AdaBoost Haar-like algorithm. The specific algorithm training process is as follows:

(1) First, convert the image samples into grayscale images, and then calculate their Haar-like features for each image.

(2) A strong classifier of image samples is trained by the AdaBoost algorithm. Suppose a training dataset {(x ₁ , y ₁ ), (x ₂ , y ₂ ), ..., (x _N , y _N )} (where N is the number of training samples) and

where t _k is the segmentation threshold, t _k =0.5×(h _k +h _k+1 ), and _wi is the weight of the training samples. For each iteration step m = 1, 2, ..., M, the optimal weak classifier is calculated by:

F(x)←F(x)+f _m (x) (3)

w _i ←w _i exp(-y _i f _m (x _i )) (4)

In step S104, this example designs a deep convolutional neural network damage identification model, the input of the deep convolutional neural network model is a three-channel 50×50 image, and the schematic diagram of the model structure is shown in FIG. 3 . The established deep convolutional neural network model structure includes 4 feature extraction modules. The first feature extraction module consists of 2 convolutional layers, each of which includes 64 convolutional kernels of size 3 × 3. The second feature extraction module consists of 2 convolutional layers, each of which includes 128 convolutional kernels of size 3 × 3. The third feature extraction module consists of 3 convolutional layers, each of which includes 256 convolution kernels of size 3 × 3. The fourth feature extraction module consists of 1 convolutional layer, which includes 512 convolution kernels of size 3 × 3. The convolution stride of the convolutional layers is all 1. Each feature extraction module is linked by a maximum pooling layer, the pooling size of the pooling layer is 2×2, and the pooling operation step size is 2. The image is output to the global maximum pooling layer after four feature extraction modules, and then output to the output layer after a layer of dropout. The output layer is a one-dimensional vector containing 5 elements, and the values of the 5 elements respectively represent the confidence of the input image to be judged as five types of normal blade, glass fiber damage, crack, skin damage and corrosion. The output layer uses the Softmax function for classification. The activation function of the full model adopts the LReLU function, the Leaky Rate is set to 0.1, and the Dropout rate is set to 0.5. The training process adopts Adam optimizer, the training batch is between 6 and 24, the learning rate is 10 ^-3 -10 ^-4 , and the training batch size and learning rate are determined according to the cross-validation results.

The features of this model are:

(1) The convolutional neural network model specifically includes an input layer, a convolutional layer, a pooling layer and an output layer, but does not include a fully connected layer. The advantage is that the model greatly reduces the number of training parameters, while maintaining the model's excellent recognition ability, which enables the model to have higher training efficiency and better generalization ability.

(2) The convolutional neural network model includes four groups of feature extraction modules including convolutional layers and pooling layers, with a total of 8 convolutional layers and 4 pooling layers. The advantage is that the multi-layered convolution and pooling operations enable the model to capture robust features that are abstract and greatly different between classes in small-sized image samples, thus enabling the model to have better damage recognition performance.

In step S104, the convolutional neural network model is trained, adjusted and verified by using the sample database established in step S102 to obtain a convolutional neural network model with recognition ability. Among them, training refers to: using the gradient descent method to iteratively update the weights of the convolutional neural network model, so that the output of the model loss function is reduced, and the detection value of the model and the real value of the data are getting closer and closer, until the loss The function converges to 0, the training and optimization of the model are completed, and the trained convolutional neural network model is obtained. Parameter adjustment refers to: using the cross-validation method to determine the parameters such as the optimal training batch size and learning rate of the model. Validation refers to the performance of the model on the validation data, that is, the accuracy of model validation.

The random sampling method is used to extract verification data from the sample database generated in step S2, and the trained recognition model is verified, and compared with the classic convolutional neural network model VGG16 and the traditional classification model SVM, and the comparison results are shown in Table 1. . According to Table 1, the recognition accuracy of the instance recognition model of the present invention reaches 97%, which is significantly improved compared to SVM (88%) and VGG16 (91%).

Table 1 Comparison of model validation results

FIG. 4 is a schematic diagram of a confusion matrix showing the verification results in an embodiment of the present invention. The image proves that the recognition model of the present invention can effectively distinguish between normal leaves and damaged leaves, and this model can also correctly distinguish all four types in most image samples. type of damage. The above experimental results show that the trained convolutional neural network model can be used to identify and judge the surface damage of wind turbine blades.

As shown in Figure 5, damage detection and localization includes the following steps:

In step S201, the image of the wind turbine blade to be detected is scaled multiple times according to a certain scale, and on a specific scale, the sliding window is moved horizontally or vertically according to a fixed step size (in this example, the step size is set to 10). The image area covered by the sliding window after each translation is used as the detection window, and the Haar-like features in the detection window are extracted, and identified by the cascade strong classifier trained in step S103 to determine whether the area may contain a damaged area. If the model judges that the detection window may contain a damaged area, mark the area as a suspected damage target area with a box, otherwise skip this area to judge the next area. The above method is used to traverse the entire image to obtain a series of suspected damaged target areas. The detection result of the image proof after step S201 is shown in FIG. 7 , in which the detected suspected damage target area is marked with a box.

In step S201, the image is firstly screened for suspected damage target areas based on Haar-like features and shallow classifiers, which greatly reduces the computational cost and lays a foundation for deploying deep learning network identification in the next step.

In step S202, since the suspected damaged target areas detected in the previous step have different scales and sizes, these areas need to be scaled to a uniform size for the next step of identification (in this example, the suspected damaged target areas are all scaled to 50× 50 size for identification). The suspected area scaled to a uniform size is sequentially input into the damage identification model trained in step S104, and after the model is identified, it is determined which damage type the suspected area belongs to. If it is judged that the suspected area does not belong to any kind of injury, the area is excluded from the suspected area (ie, it is regarded as a normal area).

In step S203, the detection results after the first two steps are sorted out, and the result file is output, including information such as the position and type of the identified damage target area, and the corresponding position of the original wind turbine blade image is displayed with boxes and numbers. The number identifies the damage area and category. The detection and recognition results of the image proofs after the above steps are shown in Figure 8, in which the identified damage target area is marked with a box, and the damage category is marked with text in the upper left corner of the box. Compared with Fig. 7, it can be seen that after the damage identification model, the normal leaf area detected as a suspected area is basically excluded, and only the real damage area is identified.

Claims

A wind turbine blade image damage detection and localization method, characterized in that: based on a deep convolutional neural network, it includes two processes: model training and damage detection and localization;

Among them, the model training process includes:

S101. Collect surface images of wind turbine blades captured by drones or surveillance cameras;

S102. Manually mark the damaged position and type information in the wind turbine blade image, and cut out an image sample containing the damaged area from the image according to the manual marking, that is, a positive sample; and an image sample of a normal blade surface, that is, a negative sample ; Establish a sample database;

S103, using the sample database established in step S102, according to the classical AdaBoost Haar-like algorithm to train the cascade strong classifier;

S104, building a deep convolutional neural network damage identification model, and using the sample database established in step S102 to train, adjust parameters and verify the convolutional neural network model to obtain a trained damage identification model;

The damage detection and localization process includes:

S201, traverse the wind turbine blade image through the sliding window method, and identify whether the image may contain a damaged area through the identification of the cascade strong classifier trained in step S103;

S202, scaling the damage target area detected in step S201 to a uniform size, inputting the damage identification model trained in step S104, and after being identified by the model, determine which damage type the area belongs to;

S203. Output a result file, including information such as the position and type of the identified damage target area, and mark the damaged area and type with a box and a number in the corresponding position of the original wind turbine blade image.
The method for detecting and locating damage in an image of a wind turbine blade according to claim 1, wherein the damage manually marked in S102 includes four types of glass fiber damage, cracks, skin damage and corrosion.
The wind turbine blade image damage detection and localization method according to claim 1, wherein: S103, the mid-body algorithm training process is:

(1) First, convert the image samples into grayscale images, and then calculate their Haar-like features for each image;

(2) Train a strong classifier of image samples through the AdaBoost algorithm; suppose that the training data set {(x 1 , y 1 ), (x 2 , y 2 ), ..., (x N , y N )} (where N is number of training samples) and
y∈[-1, 1]. x i is a training sample, and y i =1 indicates that the training image sample contains a damaged area. Assuming that hi is one of the Haar -like features calculated by the training sample x, the calculation formula of the single-feature weak classifier f(x) is:

where t k is the segmentation threshold, t k =0.5×(h k +h k+1 ), wi is the weight of the training samples; for each iteration step m=1,2,...,M, by the following equation Compute the optimal weak classifier:

At the end of each iteration, the strong classifier F(x) and wi weight values are updated by

F(x)←F(x)+f m (x) (3)

w i ←w i exp(-y i f m (x i )) (4)

At the beginning of training, F(x)=0, the weight distribution of the training data is given by
get initialized. After performing the required iterations, the final strong classifier C(x) can be expressed as:
The wind turbine blade image damage detection and localization method according to claim 1, wherein: the deep convolutional neural network damage identification model built in S104 includes an input layer, a convolutional layer, a pooling layer and an output layer. connection layer.
The wind turbine blade image damage detection and localization method according to claim 1 or 4, characterized in that: the deep convolutional neural network damage identification model built in S104 includes four groups of feature extraction modules including convolutional layers and pooling layers , with a total of 8 convolutional layers and 4 pooling layers.
The wind turbine blade image damage detection and location method according to claim 1, wherein the training step in S104 comprises: using a gradient descent method to iteratively update the weights of the convolutional neural network model multiple times to make the model loss function As the output decreases, the detection value of the model and the real value of the data are getting closer and closer until the loss function converges to 0, the training and optimization of the model are completed, and the trained convolutional neural network model is obtained.
The wind turbine blade image damage detection and localization method according to claim 1, wherein the training step in S104 comprises: using a gradient descent method to iteratively update the weights of the convolutional neural network model for multiple times, so that the model loss function As the output decreases, the detection value of the model and the real value of the data are getting closer and closer until the loss function converges to 0, the training and optimization of the model are completed, and the trained convolutional neural network model is obtained.
The wind turbine blade image damage detection and location method according to claim 1, wherein the parameter adjustment step in S104 comprises: using a cross-validation method to determine parameters such as the optimal training batch size and learning rate of the model; the validation step comprises: : The performance result of the model on the validation data, that is, the accuracy of the model validation.
The wind turbine blade image damage detection and localization method according to claim 1, wherein the step S201 is specifically: scaling the wind turbine blade image to be detected multiple times according to a certain ratio, and on a specific scaling scale, along the horizontal Or move the sliding window vertically according to a fixed step size; take the image area covered by the sliding window after each translation as the detection window, extract the Haar-like features in the detection window, and identify it by the cascade strong classifier trained in step S103, Determine whether the area may contain a damaged area; if the model judges that the detection window may contain a damaged area, mark the area as a suspected damage target area with a box, otherwise skip this area to determine the next area; use the above method to traverse the entire image, A series of suspected damaged target areas are acquired.