CN115187878A - Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device - Google Patents
Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device Download PDFInfo
- Publication number
- CN115187878A CN115187878A CN202210710296.0A CN202210710296A CN115187878A CN 115187878 A CN115187878 A CN 115187878A CN 202210710296 A CN202210710296 A CN 202210710296A CN 115187878 A CN115187878 A CN 115187878A
- Authority
- CN
- China
- Prior art keywords
- defect
- power generation
- wind power
- lbp
- svm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E10/00—Energy generation through renewable energy sources
- Y02E10/70—Wind energy
- Y02E10/72—Wind turbines with rotation axis in wind direction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of nondestructive testing of wind power generation devices, and particularly relates to a method for detecting defects of blades of a wind power generation device based on unmanned aerial vehicle image analysis, which comprises the following steps: making a training sample set; preprocessing a training sample set by using a self-adaptive threshold segmentation algorithm to obtain a binarization sample set; extracting LBP characteristics from the binarization sample set, and establishing an LBP characteristic set; training an SVM classifier by using an LBP feature set, and simultaneously optimizing SVM penalty factors and kernel function parameters by using a sparrow search algorithm to obtain an LBP-SVM model file; and loading an LBP-SVM model file, and carrying out pyramid multi-scale sliding frame detection on the unmanned aerial vehicle image to be detected to finally obtain the type and the position of the defect. The invention can quickly and accurately detect the defects of the blades of the wind power generation device from the unmanned aerial vehicle image analysis, thereby improving the detection efficiency and the detection precision of the defects of the blades of the wind power generation device.
Description
Technical Field
The invention belongs to the technical field of nondestructive testing of wind power generation devices, and particularly relates to a method for detecting defects of blades of a wind power generation device based on unmanned aerial vehicle image analysis.
Background
The inspection data acquisition means of the wind power generation equipment generally adopts a nondestructive testing technology, namely a method for measuring under the premise of not causing damage to an object to be measured. It is generally determined by using information on abnormality or change in response to light, heat, sound, electricity, or the like at a defective portion. The acoustic emission technology or the analysis and detection technology based on the vibration signal needs to additionally install corresponding sensors, data acquisition equipment and the like, so that the fault sources of the system are increased, the cost of wind power equipment is increased, the damage cycle of blades of a wind driven generator is long, and unnecessary waste is caused by fixing a detection device on the equipment.
In the prior art, a wind power generation equipment inspection mode based on unmanned aerial vehicle image acquisition appears, namely equipment such as a camera mounted on an unmanned aerial vehicle is close to the wind power generation equipment to acquire images, the acquired images are analyzed, and the damage of the equipment is found out. Because wind power generation set is fixed mounting in the position that open-air wind-force is sufficient usually, when patrolling and examining the image based on unmanned aerial vehicle collection, receive influences such as outdoor illumination inequality, reflection of light easily and lead to the image brightness distribution who gathers to be uneven, make the target defect distinguish the difficulty, and the texture characteristic difference of all kinds of defects of aerogenerator blade is great, leads to defect classification inconvenient.
The defect detection of the blades of the wind driven generator based on unmanned aerial vehicle image analysis belongs to the category of target detection, and the current target detection technology mainly comprises two types: the method comprises a traditional target detection algorithm based on manual feature extraction and a target detection algorithm based on deep learning. The traditional target detection and calculation method based on the manually extracted features mostly adopts an artificial experience value method with low reliability, and has the defects of low classification performance and difficulty in quickly and accurately judging which type of target defect belongs to. The target detection algorithm based on deep learning is a mainstream method at present, but the target detection algorithm also has some technical problems which are difficult to solve, and is not suitable for all application scenarios. The target detection algorithm of two-stage represented by R-CNN, fast-RCNN and Fast-RCNN firstly adopts a selective search algorithm or RPN to generate a candidate region, and then carries out target classification and position regression on the candidate region, and the algorithm model is large and complex, slow in reasoning speed and poor in detection effect on small targets. The target detection algorithm of one-stage represented by SSD, YOLO, etc. does not generate candidate regions, and directly uses the CNN network to perform classification and regression, and this algorithm sacrifices a part of accuracy to improve inference speed, but has some distance from real-time detection. In addition, the target detection algorithms based on deep learning aim at network models designed by a universal data set, the network models are high in complexity and poor in pertinence, and the modifying effect is not obvious.
Therefore, a new method for detecting the blade defect of the wind power generation device based on unmanned aerial vehicle image analysis is needed to be provided so as to quickly and accurately detect the defect of the blade of the wind power generation device from the unmanned aerial vehicle image analysis, thereby improving the detection efficiency and the detection accuracy of the blade defect of the wind power generation device.
Disclosure of Invention
In view of the above situation, the present invention provides a new method for detecting blade defects of a wind turbine generator based on unmanned aerial vehicle image analysis, which can quickly and accurately detect blade defects of a wind turbine generator from unmanned aerial vehicle image analysis, thereby improving detection efficiency and detection accuracy of blade defects of a wind turbine generator.
In order to achieve the purpose, the invention is realized by the following technical scheme:
a wind power generation device blade defect detection method based on unmanned aerial vehicle image analysis comprises the following steps:
s1, manufacturing a training sample set, and establishing a positive sample set and a negative sample set according to the type and probability distribution of defects possibly appearing on the surface of a wind driven generator blade; the positive samples in the positive sample set are images of the wind power generation equipment, and the images of the wind power generation equipment comprise defect areas and non-defect areas; the negative sample in the negative sample set is a background image without wind power generation equipment in the picture;
s2, preprocessing the training sample set manufactured in the S1 by using a self-adaptive threshold segmentation algorithm to weaken the influence of unstable light on site, further better highlighting the texture characteristics of defects, and obtaining a preprocessed binary sample set;
s3, extracting LBP characteristics from the positive samples in the preprocessed binary sample set, establishing an LBP characteristic set, and endowing different label codes to each positive sample according to different types of defects and background LBP characteristics;
s4, training an SVM classifier by using an LBP feature set, optimizing SVM penalty factors and kernel function parameters by using a sparrow search algorithm, and performing iterative computation by using a classification error rate as a target function; stopping training until the classification accuracy is converged to obtain an LBP-SVM model file;
s5, loading an LBP-SVM model file, carrying out pyramid multi-scale sliding frame detection on the unmanned aerial vehicle image to be detected, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect.
Further, in step S1, the number ratio of positive and negative samples in the positive and negative sample sets is 1.
Further, step S2 includes:
s21, acquiring a feature set 1: the value range of each channel of RGB of each positive sample and each negative sample is 0-255, the value of each channel is equally divided into 8 parts to obtain the measurement grade 1-8 of each channel, a 8-8 multi-dimensional array is obtained after the histogram of each channel is calculated and is flattened, so that the RGB color histogram characteristics of each positive sample and each negative sample in the positive sample and the negative sample are obtained, label codes 1 and 0 are respectively assigned to the positive sample and the negative sample according to the RGB color histogram characteristics of each positive sample and each negative sample, and then the characteristic set 1 is obtained;
s22, acquiring a feature set of the positive sample: performing threshold segmentation processing on each positive sample in the feature set 1 by adopting an improved threshold segmentation algorithm to obtain a feature set of the positive sample; the improved threshold segmentation algorithm is obtained by carrying out non-local mean operation on the gray image before converting the RGB image into the gray image in the standard self-adaptive threshold segmentation algorithm.
Further, step S3 includes: and extracting LBP (local binary Pattern) characteristics of each positive sample in the characteristic set of the positive samples, wherein the label code number of the positive sample which does not contain the defects in the characteristic set of the positive samples is 2, the label code number of the positive sample containing the crack defects is 3, and the label code number of the positive sample containing the coating falling defects is 4, so that a characteristic set 2 is obtained.
Furthermore, in step S4, two SVM classifiers are provided, which are respectively referred to as an SVM _1 classifier and an SVM _2 classifier, and when the SVM classifier is trained by using the LBP feature set, the feature set 1 is used to drive the training of the SVM _1 classifier, and the model 1 is obtained after the training; and driving training of the SVM _2 classifier by using the feature set 2 to obtain a model 2.
Further, step S5 includes: predicting the image to be detected by using the trained model, firstly, judging whether the image to be detected is the image of the wind power generation equipment by using the model 1, and if the returned prediction label is 0, indicating that the image to be detected only contains the background; if the return prediction label is 1, performing threshold segmentation processing in step S22 on the image to be detected, extracting LBP features of each window from the processed image by using a multi-scale sliding window method based on the model 2, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect.
Further, the improved threshold segmentation algorithm in step S22 includes the following steps:
s221, dividing the image into n × n small windows, and numbering the small windows from top left to bottom right for subsequent retrieval;
s222, calculating the gray level mean value of each small window, and thus solving a one-dimensional histogram about the gray level mean value;
s223, by using the bimodal distribution presented by the one-dimensional histogram obtained in the step S222, taking the gray average corresponding to the valley between the two peaks as a threshold, thereby dividing the small window into two types, wherein one type is an interested area window, namely a non-defect area window; the other is a region-of-non-interest window, namely a defect region window;
s224, finding out small windows corresponding to the two peak points of the one-dimensional histogram in the step S223, and solving the entropies of the two small windows, wherein the small window with larger entropy has more complex texture and is a non-interest area window, and the small window with smaller entropy is an interest area window;
s225, using the gray average value of each small window obtained in the step S222, performing the following processing on the gray value of each pixel point in the small window by using the corresponding gray average value of the window in the region of interest: and replacing the gray value of the pixel point which is greater than the gray average value by using the gray average value, and keeping the gray value of the pixel point which is less than or equal to the gray average value unchanged.
Further, the sparrow search algorithm in step S4 is an improved sparrow search algorithm based on random walk, and the improved sparrow search algorithm perturbs the optimal sparrow by using random walk after sparrow search, so as to improve the search performance, and the improved sparrow search algorithm flow includes:
s41: initializing population, iteration times and the proportion of predators and participants;
s42: calculating and sequencing fitness;
s43: sparrows update predator positions;
s44: sparrow update joiner position;
s45: the sparrow updates the position of the alerter;
s46: calculating a fitness value and updating the position of the sparrow;
s47: updating the optimal sparrows by using random walk;
s48: calculating a fitness value and updating the position of the sparrow;
s49: and (4) judging whether the stop condition is met, if so, exiting and outputting the result, otherwise, repeating the steps S42-S48.
The working principle of the invention is that,
(1) in the image shot by the unmanned aerial vehicle inspection, the picture with the higher percentage of the wind power generation device in the picture is called a positive sample, and the picture with the lower percentage of the wind power generation device without the wind power generation device in the picture is called a negative sample. Wind power generation set is single white generally, and the background presents as complicated changeable colour piece such as blue sky, mountain range, land along with unmanned aerial vehicle shoots the angle difference. Therefore, there is a more significant difference in the RGB color histograms of the positive and negative samples. The present invention relies on this feature to use the RGB color histogram feature for distinguishing between positive and negative examples. Then, according to the characteristic that the difference of the texture characteristics of various defects on the surface of the blade of the wind power generation device is large, the LBP (Local Binary Pattern) characteristics are used for judging whether the defects exist in the positive sample and which type the defects belong to. The present invention proposes an RGB-LBP-SVM model in accordance with the above discussion. The model adopts a two-stage SVM (support vector machine) classifier, an SVM _1 classifier is trained by using RGB color histogram features in the first stage, and a positive sample and a negative sample are screened out; and in the second stage, the LBP texture characteristics of the positive sample screened in the first stage are used for training an SVM _2 classifier, so that whether the positive sample has defects or not and the defect type are distinguished. Compared with the traditional method for performing primary classification by adopting LBP (local binary pattern) characteristics or a plurality of characteristic value combinations, the method adopts the two-stage SVM (support vector machine) classifier, and retains the spatial information of texture characteristics while integrating color characteristics. Through practical effect verification, the surface defects of the wind driven generator blade can be rapidly and accurately detected, and the feasibility and the advancement of the RGB-LBP-SVM model are shown.
(2) Wind power generation set fixed mounting is usually in the position that open-air wind-force is sufficient, when patrolling and examining the image based on unmanned aerial vehicle gathers, receives outdoor illumination inequality, reflection of light etc. to influence easily and leads to the image brightness distribution who gathers to obtain uneven. Next, as described in (1), although the wind turbine generator is high in the screen, it is difficult to avoid that some regions of no interest exist in the target sample. In order to better extract the texture features of the defect target which we are interested in the second stage, the invention adds an improved adaptive threshold segmentation process before extracting the texture features of the positive sample. The method not only eliminates the interference of irrelevant textures, noise points and the like of the non-defect area in the positive sample, but also better shows the texture information of the target area.
(3) The selection of the penalty factor and the kernel function parameter in the SVM classifier directly influences the classification performance of the SVM, so that the invention utilizes a sparrow search algorithm improved based on random walk to find the global optimal combination of the penalty factor and the kernel function parameter of the SVM, replaces the traditional artificial experience value method with low reliability, and improves the classification performance of the SVM.
Compared with the prior art, the invention has the following beneficial effects:
the method is used for manually extracting the characteristics of the specific object, namely the surface defect of the wind driven generator blade according to actual needs, has the advantages of strong pertinence, flexibility, simplicity and the like, and has obvious advantages in reasoning speed compared with a target detection algorithm based on deep learning. Therefore, the method and the device can quickly and accurately detect the defects of the blades of the wind power generation device from the unmanned aerial vehicle image analysis, so that the detection efficiency and the detection precision of the defects of the blades of the wind power generation device are improved.
Drawings
FIG. 1 is a diagram of a defect detection model in an embodiment.
FIG. 2 is an exemplary diagram of a positive sample in the example;
FIG. 3 is a schematic view of a negative example in the example;
FIG. 4 is a diagram illustrating an embodiment of dividing an image into small windows;
FIG. 5 is a comparison chart of the processing of an image in the embodiment, where FIG. 5a is an unprocessed original image, FIG. 5b is an image after standard adaptive threshold segmentation, and FIG. 5c is an image after adaptive threshold segmentation improved by the present invention;
FIG. 6 is a comparison diagram of the processing of another image in the embodiment, where FIG. 6a shows an unprocessed original image, FIG. 6b shows an image after standard adaptive threshold segmentation, and FIG. 6c shows an image after adaptive threshold segmentation improved by the present invention;
FIG. 7 is a schematic diagram of an RGB-LBP-SVM model in an embodiment;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, belong to the scope of the present invention.
Examples
With reference to the defect detection model diagram shown in fig. 1, the method for detecting the blade defect of the wind power generation device based on unmanned aerial vehicle image analysis provided by the application comprises the following steps:
s1, making a training sample set, and establishing a positive sample set and a negative sample set according to the number proportion of 1; as shown in fig. 2, the positive sample in the positive sample set is a wind power generation equipment image, and the wind power generation equipment image includes a defective region and a non-defective region; as shown in fig. 3, the negative sample in the negative sample set is a background view in which no wind turbine generator is present in the screen.
S2, preprocessing the training sample set manufactured in the S1 by using a self-adaptive threshold segmentation algorithm to weaken the influence of unstable light on the site, further better highlighting the textural features of defects, and obtaining a preprocessed binary sample set, wherein the method specifically comprises the following steps:
s21, acquiring a feature set 1: the value range of each channel of RGB of each positive sample and each negative sample is 0-255, the value of each channel is equally divided into 8 parts to obtain the measurement grade 1-8 of each channel, a 8-8 multi-dimensional array is obtained after the histogram of each channel is calculated and is flattened, so that the RGB color histogram characteristics of each positive sample and each negative sample in the positive sample and the negative sample are obtained, label codes 1 and 0 are respectively assigned to the positive sample and the negative sample according to the RGB color histogram characteristics of each positive sample and each negative sample, and then the characteristic set 1 is obtained;
s22, acquiring a feature set of the positive sample: performing threshold segmentation processing on each positive sample in the feature set 1 by adopting an improved threshold segmentation algorithm to obtain a feature set of the positive sample; the improved threshold segmentation algorithm is obtained by carrying out non-local mean operation on the gray image before converting the RGB image into the gray image in the standard self-adaptive threshold segmentation algorithm.
S3, extracting LBP characteristics from the positive samples in the preprocessed binary sample set, establishing an LBP characteristic set, and endowing different label codes to each positive sample according to different types of defects and background LBP characteristics; the specific method comprises the following steps: and extracting LBP (local binary Pattern) characteristics of each positive sample in the characteristic set of the positive samples, wherein the label code number of the positive sample which does not contain the defects in the characteristic set of the positive samples is 2, the label code number of the positive sample containing the crack defects is 3, and the label code number of the positive sample containing the coating falling defects is 4, so that a characteristic set 2 is obtained.
S4, training an SVM classifier by using an LBP feature set, optimizing SVM penalty factors and kernel function parameters by using a sparrow search algorithm, and performing iterative computation by using a classification error rate as a target function; stopping training until the classification accuracy is converged to obtain an LBP-SVM model file; the specific method for training the SVM classifier by using the LBP feature set is as follows: setting an SVM classifier which is respectively recorded as an SVM _1 classifier and an SVM _2 classifier, driving training of the SVM _1 classifier by using the feature set 1 when the SVM classifier is trained by using the LBP feature set, and obtaining a model 1 after training; and driving training of the SVM _2 classifier by using the feature set 2 to obtain a model 2.
S5, loading an LBP-SVM model file, carrying out pyramid multi-scale sliding frame detection on the unmanned aerial vehicle image to be detected, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect. The method comprises the following steps: predicting an image to be detected by using the trained model 1 and model 2, firstly, judging whether the image to be detected is an image of the wind power generation equipment by using the model 1, and if the returned prediction label is 0, indicating that the image to be detected only contains a background; if the return prediction label is 1, performing threshold segmentation processing in step S22 on the image to be detected, extracting LBP features of each window from the processed image by using a multi-scale sliding window method based on the model 2, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect.
In this embodiment, the improved threshold segmentation algorithm in step S22 includes the following steps:
s221, dividing the image into n × n small windows (as shown in FIG. 4), and numbering the small windows from top left to bottom right for subsequent retrieval;
s222, calculating the gray level mean value of each small window, and thus solving a one-dimensional histogram related to the gray level mean value;
s223, by using the bimodal distribution presented by the one-dimensional histogram obtained in the step S222, taking the gray average corresponding to the valley between the two peaks as a threshold, thereby dividing the small window into two types, wherein one type is an interested area window, namely a non-defect area window; the other type is a non-interested region window, namely a defect region window;
s224, finding out small windows corresponding to the double peak points of the one-dimensional histogram in the step S223, and solving the entropies of the two small windows, wherein the small window with the larger entropy is more complex in texture and is a window without an interested region, and the small window with the smaller entropy is the window with the interested region;
s225, using the gray mean value of each small window obtained in the step S222, carrying out the following processing on the gray value of each pixel point in the small window by using the corresponding gray mean value of the window in the region of interest: and replacing the gray value of the pixel point which is greater than the gray average value by using the gray average value, and keeping the gray value of the pixel point which is less than or equal to the gray average value unchanged.
Principle description of the improved thresholding algorithm:
the standard adaptive threshold segmentation algorithm is used for solving the problem of uneven brightness of different regions of an image. The processing steps are as follows:
step1: the RGB image is first converted into a gray scale image, and the size of the neighborhood blocks and the offset are set.
Step2: and calculating the mean value or Gaussian mean value of the gray value of the pixel points in the window by sliding the window according to the set size of the neighborhood block.
Step3: and subtracting the set offset from the mean value of each neighborhood block to obtain the local threshold of the neighborhood block.
Step4: and carrying out binarization processing on the gray value of each pixel point in each neighborhood block according to the corresponding threshold value of the gray value to obtain a segmented image.
When the method is used for threshold segmentation processing, the practical effect shows that the method cannot well inhibit irrelevant texture information, noise points and the like of a non-defect area. In order to solve the problem and better extract a defect region, the method carries out non-local mean operation on the gray image in Step1 of an adaptive threshold segmentation algorithm to obtain a processed gray image. The method has the idea that a statistical method is utilized to count an interested region (non-defect region) and carry out noise point removing operation on the interested region, so that the interference texture and noise points of the non-defect region are inhibited; meanwhile, the texture information of the defect area is reserved to the maximum extent. Provides better conditions for the subsequent LBP feature extraction.
Fig. 5 and 6 are graphs showing a comparison of processing effects of a group of images, fig. 5a and 6a are unprocessed original images, fig. 5b and 6b are images divided by a standard adaptive threshold, and fig. 5c and 6c are images divided by an adaptive threshold improved by the present application. As can be seen from fig. 5 and 6, in the image obtained by using the improved adaptive threshold segmentation algorithm, the interference noise and texture of the non-defective region are both suppressed to a great extent, so that the irrelevant texture feature points of the non-defective region can be reduced in the texture features extracted by the subsequent LBP operator.
FIG. 7 shows a schematic diagram of an RGB-LBP-SVM model; as can be known from fig. 7, the model drives the training of the SVM _1 classifier by the RGB color histogram features of the training sample set (including the positive and negative sample sets); meanwhile, the positive samples are processed by adopting an improved image segmentation algorithm, and training of the SVM _2 classifier is driven through LBP characteristics of the processed positive sample set. The method for extracting the features of the full set and the subset of the data set is a new idea of the feature extraction method, is a scheme specially designed according to the characteristics of the unmanned aerial vehicle inspection image, and has pertinence and advancement.
LBP (local binary Pattern) is a feature description operator for describing local texture, and has the advantages of gray scale invariance, rotation invariance and the like. The original LBP feature descriptor operator is defined within a 3 × 3 square neighborhood. The surrounding 8 points are compared to the center point grey value. If the gray value is larger than the gray value of the central point, the point is marked as 1; if less than the center point gray value, it is marked 0. Arranging these 8 points to form an 8-bit binary number, which is the LBP value at the 3 × 3 neighborhood center point. The texture features of different scales need to select neighborhoods of different scales to improve the receptive field of the template, and the original 3 × 3 square neighborhoods cannot be adjusted, so that the circular neighborhoods are adopted for calculation, and the calculation formula is as follows:
wherein P is the number of sampling points, and R is the radius of the circular neighborhood. x is the number of p And y p Are coordinate values of the sample points in the lateral and longitudinal directions. I (c) represents the gray value of the center point, and I (p) represents the gray value of the p-th sampling point.
The basic idea of a support vector machine is to map the input sample space to a high-dimensional feature space by means of a kernel function such that the input samples are separated by a hyperplane in the high-dimensional feature space. The hyperplane divided is:
f(x)＝w T Φ(x)+b(5)
where ω represents the weight vector and b represents the threshold. For a given nonlinear irretrievable data set, considering the existence of an error xi, the optimization problem under the constraint condition is as follows:
the optimization problem of the formula (6) can introduce lagrangian factors to convert the lagrangian factors into a dual problem, and the solution of the formula (5) obtained by the solution of the dual problem is as follows:
wherein alpha is i Is the Lagrange factor, l is the number of support vectors, κ (x, x) i ) Is a kernel function.
As known from the optimization problem, the selection of the penalty factor c and the kernel parameter g directly influences the classification performance of the SVM. In a conventional support vector machine model, a penalty function c, a kernel function parameter g and the like are selected empirically or by adopting K-fold cross validation to select appropriate parameters. In the classification process of the LBP characteristics of the wind power generation equipment blades, the input data has the characteristics of diversity and complexity. The empirical parameter selection is not only time consuming, but also gives some randomness to the calculation process. The K-turn cross validation depends on the value of the parameter range too much, and if the value range is not proper, the optimal parameter cannot be found. The sparrow search algorithm is a novel group intelligent algorithm evolved from the sparrow group foraging behavior, is simple and efficient, has few adjustable parameters, and can realize global convergence.
In the embodiment, in order to improve the local search performance of the optimal position of the sparrow search algorithm, the sparrow search algorithm in the step S4 is a sparrow search algorithm improved based on random walk, the improved sparrow search algorithm is used for disturbing the optimal sparrow by using random walk after sparrow search, the search performance is improved, the random walk boundary is larger at the beginning of iteration, the global search performance is improved, the walk boundary is smaller after iteration is carried out for multiple times, and the local search performance of the optimal position of the algorithm is improved.
The improved sparrow search algorithm takes the value ranges of kernel function parameters and penalty factors of a support vector machine as the search range of sparrows, the test error rate of the model is used as a target function for iteration, so that the optimal parameter combination of the kernel function parameters and the penalty factors is obtained, and the optimal parameter combination is used for training in a back-substitution support vector machine model.
The improved sparrow searching algorithm flow is as follows:
s41: initializing population, iteration times and proportion of predators and addicts;
s42: calculating and sequencing fitness;
s43: sparrows update predator positions;
s44: sparrow update joiner position;
s45: the sparrow updates the position of the alerter;
s46: calculating a fitness value and updating the position of the sparrow;
s47: updating the optimal sparrows by using random walk;
s48: calculating a fitness value and updating the position of the sparrow;
s49: and (4) judging whether the stop condition is met, if so, exiting and outputting the result, otherwise, repeating the steps S42-S48.
The working principle of the invention is as follows:
(1) in the image shot by the unmanned aerial vehicle inspection, the picture with the higher percentage of the wind power generation device in the picture is called a positive sample, and the picture with the lower percentage of the wind power generation device without the wind power generation device in the picture is called a negative sample. Wind power generation set is single white generally, and the background presents as complicated changeable colour piece such as blue sky, mountain range, land along with unmanned aerial vehicle shoots the angle difference. Therefore, there is a more significant difference in the RGB color histograms of the positive and negative samples. The present invention relies on this feature to use the RGB color histogram feature for distinguishing between positive and negative examples. Then, according to the characteristic that the difference of the texture characteristics of various defects on the surface of the blade of the wind power generation device is large, the LBP (Local Binary Pattern) characteristics are used for judging whether the defects exist in the positive sample and which type the defects belong to. The present invention proposes an RGB-LBP-SVM model in accordance with the above discussion. The model adopts a two-stage SVM (support vector machine) classifier, and in the first stage, an SVM _1 classifier is trained by using RGB (red, green and blue) color histogram features to screen out positive samples and negative samples; and in the second stage, the LBP texture characteristics of the positive samples screened in the first stage are used for training an SVM _2 classifier, so that whether defects exist in the positive samples or not and the defect types are distinguished. Compared with the traditional method for performing primary classification by adopting LBP (local binary pattern) characteristics or a plurality of characteristic value combinations, the method adopts the two-stage SVM (support vector machine) classifier, and retains the spatial information of texture characteristics while integrating color characteristics. Through practical effect verification, the surface defects of the wind driven generator blade can be quickly and accurately detected, and the feasibility and the advancement of the RGB-LBP-SVM model are shown.
(2) Wind power generation set fixed mounting usually is in the sufficient position of open-air wind-force, when patrolling and examining the image based on unmanned aerial vehicle collection, receives influences such as outdoor illumination inequality, reflection of light easily and leads to the image brightness distribution who gathers the acquisition uneven. Next, as described in (1), although the wind turbine generator is high in the screen, it is difficult to avoid that some regions of no interest exist in the target sample. In order to better extract the texture features of the defect target which we are interested in the second stage, the invention adds an improved adaptive threshold segmentation process before extracting the texture features of the positive sample. The method not only eliminates the interference of irrelevant textures, noise points and the like of non-defective areas in the positive sample, but also better shows the texture information of the target area.
(3) The selection of the penalty factor and the kernel function parameter in the SVM classifier directly influences the classification performance of the SVM, so that the invention utilizes a sparrow search algorithm improved based on random walk to find the global optimal combination of the penalty factor and the kernel function parameter of the SVM, replaces the traditional artificial experience value method with low reliability, and improves the classification performance of the SVM.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the illustrated embodiments.
Claims (8)
1. A wind power generation device blade defect detection method based on unmanned aerial vehicle image analysis is characterized by comprising the following steps:
s1, manufacturing a training sample set, and establishing a positive sample set and a negative sample set according to the type and probability distribution of defects possibly appearing on the surface of a wind driven generator blade; the positive sample in the positive sample set is a wind power generation equipment image, and the wind power generation equipment image comprises a defect area and a non-defect area; the negative sample in the negative sample set is a background image without wind power generation equipment in the picture;
s2, preprocessing the training sample set manufactured in the S1 by using a self-adaptive threshold segmentation algorithm to weaken the influence of unstable light on the site, and further highlighting the textural features of defects better to obtain a preprocessed binaryzation sample set;
s3, extracting LBP characteristics from the positive samples in the preprocessed binary sample set, establishing an LBP characteristic set, and endowing different label codes to each positive sample according to different types of defects and background LBP characteristics;
s4, training an SVM classifier by using an LBP feature set, optimizing SVM penalty factors and kernel function parameters by using a sparrow search algorithm, and performing iterative computation by using a classification error rate as a target function; stopping training until the classification accuracy is converged to obtain an LBP-SVM model file;
s5, loading an LBP-SVM model file, carrying out pyramid multi-scale sliding frame detection on the unmanned aerial vehicle image to be detected, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect.
2. The method for detecting the blade defect of the wind power generation device based on unmanned aerial vehicle image analysis according to claim 1, wherein in the step S1, the ratio of the number of the positive samples to the number of the negative samples in the positive and negative sample sets is 1.
3. The method for detecting the blade defect of the wind power generation device based on unmanned aerial vehicle image analysis according to claim 1, wherein the step S2 comprises the following steps:
s21, acquiring a feature set 1: dividing the values of the channels into 8 parts equally to obtain measurement grades 1-8 of the channels, calculating a histogram of each channel to obtain an 8 x 8 multi-dimensional array, flattening the array to obtain RGB color histogram features of each positive sample and each negative sample in the positive sample set and the negative sample set, and respectively assigning label codes 1 and 0 to the positive sample and the negative sample according to the RGB color histogram features of each positive sample and each negative sample to obtain a feature set 1;
s22, acquiring a feature set of the positive sample: performing threshold segmentation processing on each positive sample in the feature set 1 by adopting an improved threshold segmentation algorithm to obtain a feature set of the positive sample; the improved threshold segmentation algorithm is obtained by performing non-local mean operation on a gray image before converting an RGB image into the gray image in a standard adaptive threshold segmentation algorithm.
4. The unmanned aerial vehicle image analysis-based blade defect detection method for the wind power generation device according to claim 3, wherein the step S3 comprises: and extracting the LBP (local binary pattern) characteristic of each positive sample in the characteristic set of the positive samples, assigning a label code number of 2 to the positive sample which does not contain the defect in the characteristic set of the positive samples, assigning a label code number of 3 to the positive sample containing the crack defect, and assigning a label code number of 4 to the positive sample containing the coating falling defect, thereby obtaining a characteristic set 2.
5. The unmanned aerial vehicle image analysis-based wind power generation device blade defect detection method according to claim 4, wherein in step S4, two SVM classifiers are provided and respectively marked as an SVM _1 classifier and an SVM _2 classifier, when the SVM classifier is trained by using the LBP feature set, the feature set 1 is used for driving the training of the SVM _1 classifier, and a model 1 is obtained after the training; and driving training of an SVM _2 classifier by using the feature set 2 to obtain a model 2.
6. The method for detecting the blade defect of the wind power generation device based on the unmanned aerial vehicle image analysis according to claim 5, wherein the step S5 comprises the following steps: predicting an image to be detected by using the trained model 1 and model 2, firstly, judging whether the image to be detected is an image of the wind power generation equipment by using the model 1, and if the returned prediction label is 0, indicating that the image to be detected only contains a background; if the return prediction label is 1, performing threshold segmentation processing in step S22 on the image to be detected, extracting LBP features of each window from the processed image by using a multi-scale sliding window method based on the model 2, further judging whether the current window contains the defect to be detected and what kind of defect exists, and finally obtaining the type and position of the defect.
7. The method for detecting the blade defect of the wind power generation device based on unmanned aerial vehicle image analysis according to any one of claims 3-6, wherein the improved threshold segmentation algorithm in the step S22 comprises the following steps:
s221, dividing the image into n × n small windows, and numbering the coordinates of the small windows from top left to bottom right for subsequent retrieval;
s222, calculating the gray level mean value of each small window, and thus solving a one-dimensional histogram about the gray level mean value;
s223, using the one-dimensional histogram obtained in the step S222 to present bimodal distribution, and taking the gray average value corresponding to the valley between the bimodal as a threshold value, so as to divide the small window into two types, wherein one type is an interested area window, namely a non-defect area window; the other type is a non-interested region window, namely a defect region window;
s224, finding out small windows corresponding to the double peak points of the one-dimensional histogram in the step S223, and solving the entropies of the two small windows, wherein the small window with the larger entropy is more complex in texture and is a window without an interested region, and the small window with the smaller entropy is the window with the interested region;
s225, using the gray average value of each small window obtained in the step S222, performing the following processing on the gray value of each pixel point in the small window by using the corresponding gray average value of the window in the region of interest: and replacing the gray value of the pixel point which is greater than the gray average value by using the gray average value, and keeping the gray value of the pixel point which is less than or equal to the gray average value unchanged.
8. The method for detecting the blade defect of the wind power generation device based on the unmanned aerial vehicle image analysis according to claim 1, wherein the sparrow search algorithm in the step S4 is an improved sparrow search algorithm based on random walk, after the sparrow search, the improved sparrow search algorithm disturbs the optimal sparrow by using the random walk, so that the search performance is improved, and the improved sparrow search algorithm flow comprises the following steps:
s41: initializing population, iteration times and proportion of predators and addicts;
s42: calculating and sequencing the fitness;
s43: sparrows update predator positions;
s44: sparrow update joiner position;
s45: the sparrow updates the position of the alerter;
s46: calculating a fitness value and updating the position of the sparrow;
s47: updating the optimal sparrow by random walk;
s48: calculating a fitness value and updating the position of a sparrow;
s49: and (4) judging whether the stopping condition is met or not, if so, exiting and outputting a result, otherwise, repeating the steps S42-S48.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710296.0A CN115187878A (en) | 2022-06-22 | 2022-06-22 | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710296.0A CN115187878A (en) | 2022-06-22 | 2022-06-22 | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115187878A true CN115187878A (en) | 2022-10-14 |
Family
ID=83515667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210710296.0A Pending CN115187878A (en) | 2022-06-22 | 2022-06-22 | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115187878A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058396A (en) * | 2023-10-11 | 2023-11-14 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
-
2022
- 2022-06-22 CN CN202210710296.0A patent/CN115187878A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058396A (en) * | 2023-10-11 | 2023-11-14 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
| CN117058396B (en) * | 2023-10-11 | 2023-12-26 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhao et al. | Cloud shape classification system based on multi-channel cnn and improved fdm | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN111444821B (en) | Automatic identification method for urban road signs | |
| CN109977808B (en) | Wafer surface defect mode detection and analysis method | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
| CN111667455B (en) | AI detection method for brushing multiple defects | |
| CN110097053B (en) | Improved fast-RCNN-based electric power equipment appearance defect detection method | |
| CN107316036B (en) | Insect pest identification method based on cascade classifier | |
| CN112233073A (en) | Real-time detection method for infrared thermal imaging abnormity of power transformation equipment | |
| CN106934386B (en) | A kind of natural scene character detecting method and system based on from heuristic strategies | |
| CN111080691A (en) | Infrared hot spot detection method and device for photovoltaic module | |
| CN107871316B (en) | Automatic X-ray film hand bone interest area extraction method based on deep neural network | |
| CN110659550A (en) | Traffic sign recognition method, traffic sign recognition device, computer equipment and storage medium | |
| CN116188880B (en) | Cultivated land classification method and system based on remote sensing image and fuzzy recognition | |
| CN111539293A (en) | Fruit tree disease diagnosis method and system | |
| CN113256624A (en) | Continuous casting round billet defect detection method and device, electronic equipment and readable storage medium | |
| CN116030237A (en) | Industrial defect detection method and device, electronic equipment and storage medium | |
| CN115294033A (en) | Tire belt layer difference level and misalignment defect detection method based on semantic segmentation network | |
| CN117456358A (en) | Method for detecting plant diseases and insect pests based on YOLOv5 neural network | |
| CN115187878A (en) | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device | |
| CN111597875A (en) | Traffic sign identification method, device, equipment and storage medium | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| CN110738166A (en) | Fishing administration monitoring system infrared target identification method based on PCNN and PCANet and storage medium | |
| CN114529906A (en) | Method and system for detecting abnormity of digital instrument of power transmission equipment based on character recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |