CN114565571A - Fan blade defect detection method and device based on computer vision - Google Patents
Fan blade defect detection method and device based on computer vision Download PDFInfo
- Publication number
- CN114565571A CN114565571A CN202210158476.2A CN202210158476A CN114565571A CN 114565571 A CN114565571 A CN 114565571A CN 202210158476 A CN202210158476 A CN 202210158476A CN 114565571 A CN114565571 A CN 114565571A
- Authority
- CN
- China
- Prior art keywords
- fan blade
- module
- defect detection
- feature
- defect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 114
- 238000001514 detection method Methods 0.000 title claims abstract description 68
- 230000004927 fusion Effects 0.000 claims abstract description 42
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000011176 pooling Methods 0.000 claims abstract description 31
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims description 17
- 238000012549 training Methods 0.000 claims description 14
- 238000003860 storage Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 239000004576 sand Substances 0.000 abstract description 6
- 230000032798 delamination Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 6
- 239000013307 optical fiber Substances 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000001931 thermography Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 239000011152 fibreglass Substances 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009659 non-destructive testing Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0006—Industrial image inspection using a design-rule based approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E10/00—Energy generation through renewable energy sources
- Y02E10/70—Wind energy
- Y02E10/72—Wind turbines with rotation axis in wind direction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a fan blade defect detection method and equipment based on computer vision, wherein the method carries out fan blade defect detection by constructing a fan blade defect detection model comprising a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module based on an HnNet network; the method comprises the steps that a single CCD camera is used for longitudinally shifting a scanning cycle to shoot or a CCD array cycle shooting mode is used for obtaining local images of the fan blade, and adjacent local images have an overlapping area, so that image data are guaranteed not to be lost; by setting a characteristic layer decoupling mode, sufficient characteristic quantities are provided for different prediction items, and the prediction accuracy is improved; the invention can identify the defects of various fan blades with different forms, such as cracks, sand holes, delamination, debonding and the like.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a method and a device for detecting defects of a fan blade based on computer vision, computer equipment and a storage medium.
Background
With the increasing year by year of wind power output and capacity, many large and medium-sized wind power plants have been built at home and abroad. The fan blade is one of the most critical parts of the wind turbine generator system for obtaining wind energy, and along with the development of manufacturing technology, the size of the wind turbine blade is continuously increased, the length of the wind turbine blade exceeds 50 meters, and the weight of the wind turbine blade reaches more than ten tons. Accidents caused by falling of wind power blades are caused too much at home and abroad. Once the blade falls off in operation because of serious damage, very serious accidents are easily caused, possibly causing impact on other units and even causing personal and property loss to surrounding residents. Therefore, the state of the wind power blade in operation is regularly detected and evaluated through a certain nondestructive detection technology, and the wind power blade is timely maintained before being seriously damaged, so that the method has great significance.
The damage to the wind turbine blade is mainly caused by the erosion of the surface of the wind turbine blade by severe weather and the corrosion damage to the blade by media in the air. Most wind power blades are made of glass fiber reinforced plastic composite materials and are complex in structure. In the operation process of the wind power blade, the wind power blade is influenced by severe weather such as typhoon, thunder, snow, salt fog and the like, and is easy to have damages such as cracks, sand holes, layering, debonding and the like under the action of alternating load in the long-term use process.
Common wind power blade nondestructive testing schemes include visual methods, acoustic emission monitoring, optical fiber sensor monitoring, strain gauge monitoring, infrared thermal imaging and the like.
The wind power blade in operation is directly shot by a high power telescope or a high power camera through a visual method, and the surface defects of the wind power blade are observed. The requirements on related experiences of detection personnel are high, and the workload is large. Small defects such as cracks, blisters, etc. are difficult to detect.
The acoustic emission monitoring utilizes the principle that the internal state of an object can automatically emit transient elastic sound waves when changed to monitor the defects of the object in real time. Because the sound waves can be attenuated in the transmission process, particularly in a defect area, the sound waves have higher frequency and larger attenuation, the sound waves generated by different types of defects have small difference, and the signal analysis difficulty is very high.
The optical fiber sensor monitoring method is characterized in that the deformation of the shape of an optical fiber along with the change of stress is utilized, so that the internal light propagation characteristic of the optical fiber is changed, and the internal strain of the wind power blade is measured in real time. The optical fiber sensor needs to be installed inside the wind power blade in advance, the manufacturing cost is high, and the blade cannot be repaired and replaced after being molded.
The strain gauge monitoring method is characterized in that a method of sticking a strain gauge on the surface of a wind power blade is adopted to realize real-time detection of stress change in the operation process of the wind power blade, and then the collected strain value is analyzed to judge the operation state of the blade. However, the strain gauges are adhered to the surface of the wind turbine blade, and therefore the strain gauges are easily damaged and fall off, and the strain gauges are usually connected through copper wires, so that the strain gauges are also easily subjected to electromagnetic interference.
The infrared thermal imaging method is used for detecting the defects of the wind power blade by utilizing the temperature difference between the defect part and other parts of the object. The method can reflect the fuzzy area of the internal defect of the wind power blade, and cannot identify the type of the external defect.
Disclosure of Invention
The invention provides a method and a device for detecting defects of fan blades based on computer vision, computer equipment and a storage medium, aiming at identifying various defects of fan blades with different forms, such as cracks, sand holes, delamination, debonding and the like.
To this end, a first objective of the present invention is to provide a method for detecting a defect of a fan blade based on computer vision, which includes:
constructing a fan blade defect detection model, wherein the fan blade defect detection model comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module;
acquiring a fan blade surface image with defects, preprocessing the fan blade surface image to form a fan blade surface defect image data set, inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model, and performing model training;
and inputting the fan blade surface image acquired in real time into a trained fan blade defect detection model, and outputting a result through the model to detect the fan blade surface defect.
Wherein, in the fan blade defect detection model,
the basic feature extraction module is a feature extractor, and a depth convolution network is adopted to extract basic features of the surface image of the fan blade to obtain feature layers with different sizes and depths;
the multi-scale pooling fusion module is used for pooling fusion of effective feature layers with different sizes and depths so as to improve the detection rate of small defect targets;
the multi-scale feature layer fusion module is used for carrying out feature layer fusion on effective feature layers with different sizes and depths;
the characteristic decoupling module is used for decoupling effective characteristic layers with different sizes obtained after multi-scale pooling fusion so as to provide sufficient characteristic data for different prediction items;
the prediction module is used for outputting a prediction result.
The deep convolution network of the feature extractor is an HnNet network, a ResNet-101 network or a ResNet-152 network.
The minimum components of the HnNet network structure comprise convolutional layers, BN and Mish activation functions; wherein the content of the first and second substances,
the BN layer is used for realizing the regularization effect, and can improve the convergence speed of the model and prevent the overfitting of the model; the Mish activation function is used for improving the training stability and the average accuracy.
The multi-scale pooling fusion module is formed by connecting a plurality of pooling layers in parallel, and the characteristic layers extracted by the basic characteristic extraction module are input and then subjected to multi-scale pooling treatment through each pooling layer.
The multi-scale feature layer fusion module fuses feature layers with different sizes by using FPN and PAN network structures, and small target and multi-target detection rate is improved.
The characteristic decoupling module is used for predicting defect types, defect positions and defect confidence coefficients of the surface images of the fan blades by arranging a plurality of layers of basic convolution units; the characteristic decoupling module comprises an input end, the input end is connected with one ends of the first multilayer basic convolution unit and the second multilayer basic convolution unit, and the other end of the second multilayer basic convolution unit is connected with the third multilayer basic convolution unit and the fourth multilayer basic convolution unit.
The second purpose of the invention is to provide a fan blade defect detection device based on computer vision, comprising:
the system comprises a model construction module, a prediction module and a fault detection module, wherein the model construction module is used for constructing a fan blade fault detection model which comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and the prediction module;
the model training module is used for acquiring and preprocessing a fan blade surface image with defects to form a fan blade surface defect image data set, and inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model to perform model training;
and the defect detection module is used for inputting the fan blade surface image acquired in real time into the trained fan blade defect detection model, and outputting a result through the model to detect the fan blade surface defect.
A third object of the present invention is to provide a computer device, which includes a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method according to the foregoing technical solution.
A fourth object of the invention is to propose a non-transitory computer-readable storage medium on which a computer program is stored, which computer program, when executed by a processor, implements the method of the aforementioned technical solution.
Different from the prior art, the fan blade defect detection method based on computer vision provided by the invention carries out fan blade defect detection by constructing a fan blade defect detection model comprising a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module based on an HnNet network; the method comprises the steps that a single CCD camera is used for longitudinally shifting a scanning cycle to shoot or a CCD array cycle shooting mode is used for obtaining local images of the fan blade, and adjacent local images have an overlapping area, so that image data are guaranteed not to be lost; by setting a characteristic layer decoupling mode, sufficient characteristic quantities are provided for different prediction items, and the prediction accuracy is improved; the invention can identify the defects of various fan blades with different forms, such as cracks, sand holes, delamination, debonding and the like.
Drawings
The invention and/or additional aspects and advantages will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic flow chart of a fan blade defect detection method based on computer vision provided by the invention.
FIG. 2 is a schematic diagram of a basic feature extraction network structure in a computer vision-based wind turbine blade defect detection method provided by the invention.
Fig. 3 is a schematic structural diagram of a multi-scale pooling fusion module in the fan blade defect detection method based on computer vision.
Fig. 4 is a schematic structural diagram of a multi-dimensional feature layer fusion module based on FPN and PAN network structures in a computer vision-based fan blade defect detection method provided by the present invention.
FIG. 5 is a schematic structural diagram of a decoupling module in the fan blade defect detection method based on computer vision provided by the invention.
FIG. 6 is a schematic structural diagram of a fan blade defect detection device based on computer vision provided by the invention.
Fig. 7 is a schematic structural diagram of a non-transitory computer-readable storage medium according to the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
Fig. 1 is a method for detecting a defect of a fan blade based on computer vision according to an embodiment of the present invention. The method comprises the following steps:
101, constructing a fan blade defect detection model, wherein the fan blade defect detection model comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module.
Aiming at various fan blade surface defects such as cracks, sand holes, layering, debonding and the like, the invention constructs a defect detection model based on computer vision. Firstly, acquiring 101 data, and periodically shooting a fan blade by using a single/multiple CCS cameras to obtain a fan blade surface image; then, data enhancement 102 is carried out, the diversity of defect samples is increased, and the identification robustness is improved; then, using a depth full convolution network to extract 103 basic features of the image to obtain feature layers with different sizes and different depths; secondly, performing multi-scale pooling 104 on the effective characteristic layer to improve the detection rate of the small defect target; then, fusion 105 of effective feature layers with different sizes is carried out, and the detection rate of the small defect target is further improved; then, characteristic decoupling 106 is carried out to provide sufficient characteristic data for different prediction items; finally, the defect location category is predicted 107.
In the fan blade defect detection model, the defect detection model,
the basic feature extraction module is a feature extractor, and a depth convolution network is adopted to extract basic features of the surface image of the fan blade to obtain feature layers with different sizes and depths;
the deep convolution network of the feature extractor is an HnNet network, a ResNet-101 network or a ResNet-152 network.
As shown in FIG. 2, the HnNet network is selected as the deep convolution network in the invention, and the HnNet network, the ResNet-101 network and the ResNet-152 network are tested and compared, so that the HnNet has better performance than the ResNet-101 network, the speed is 1.5 times, and the performance is similar to the ResNet-152 network but the speed is almost 2 times. And the HnNet floating point calculation amount per second is the highest, the acceleration performance of the GPU can be better utilized, and the shorter algorithm execution time is ensured.
The minimum component of the HnNet network structure comprises a convolutional layer, a BN and a Mish activation function; wherein the content of the first and second substances,
the BN layer is used for realizing the regularization effect, and can improve the convergence speed of the model and prevent the overfitting of the model; compared with other activation functions such as Leaky _ ReLU and the like, the Mish activation function has better effects in the aspects of training stability, average accuracy and the like.
The multi-scale pooling fusion module is used for pooling fusion of effective feature layers with different sizes and depths so as to improve the detection rate of small defect targets; as shown in fig. 3, the multi-scale pooling fusion module is formed by connecting multiple pooling layers in parallel, and the feature layers extracted by the basic feature extraction module are input and then subjected to multi-scale pooling by each pooling layer.
The multi-scale feature layer fusion module is used for carrying out feature layer fusion on effective feature layers with different sizes and depths; as shown in fig. 4, the multi-scale feature layer fusion module fuses feature layers of different sizes by using FPN and PAN network structures, so as to improve the small target and multi-target detection rate.
The multi-scale feature layer fusion module firstly performs up-sampling on the image, inputs an up-sampling result into the FPN network for down-sampling after the sampling is completed, simultaneously inputs a down-sampling result into the PAN network for up-sampling operation, and outputs a result to complete feature fusion on the effective feature layer.
The characteristic decoupling module is used for decoupling effective characteristic layers with different sizes obtained after multi-scale pooling fusion so as to provide sufficient characteristic data for different prediction items; as shown in fig. 5, the feature decoupling module is configured with a plurality of layers of basic convolution units, and is configured to predict defect types, defect positions, and defect confidence levels of the surface image of the fan blade; the characteristic decoupling module comprises an input end, the input end is connected with one end of the first multilayer basic convolution unit and one end of the second multilayer basic convolution unit, and the other end of the second multilayer basic convolution unit is connected with the third multilayer basic convolution unit and the fourth multilayer basic convolution unit. The first multi-layer basic convolution unit is used for defect type prediction, the third multi-layer basic convolution unit is used for defect position prediction, and the fourth multi-layer basic convolution unit is used for defect confidence degree prediction.
The prediction module is used for outputting a prediction result.
102, acquiring a fan blade surface image with defects, preprocessing the fan blade surface image to form a fan blade surface defect image data set, inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model, and performing model training.
The fan blade is long and rotates periodically, the invention uses a single CCD camera to longitudinally shift and scan periodic shooting or uses a CCD array periodic shooting mode to obtain local images of the fan blade, and adjacent local images have an overlapping area, thereby ensuring that image data is not lost. And marking the types and positions of various fan blade surface defects such as cracks, sand holes, layering, debonding and the like after collection.
After the image acquisition is finished, the shooting samples are subjected to image enhancement by using a random scaling, random cutting and random arrangement mode or using a GAN network, so that the sample diversity is increased, and the model robustness is improved.
After image preprocessing, all the obtained images are used as a training set and input into a constructed fan blade defect detection model for model training, and functions and parameters of the model are adjusted based on a loss function, so that the detection efficiency of the network model is optimal. In the invention, CIOU is used for the position loss function, and cross entropy loss function is used for the category loss function.
And 103, inputting the fan blade surface image acquired in real time into the trained fan blade defect detection model, and outputting a result through the model to detect the fan blade surface defect.
In order to implement the embodiment, the present invention further provides a computer vision-based fan blade defect detection apparatus, as shown in fig. 6, including:
the model building module 310 is used for building a fan blade defect detection model, and the fan blade defect detection model comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module;
the model training module 320 is used for acquiring and preprocessing a fan blade surface image with a defect to form a fan blade surface defect image data set, and inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model to perform model training;
and the defect detection module 330 is configured to input the fan blade surface image acquired in real time into the trained fan blade defect detection model, and output a result through the model to detect the fan blade surface defect.
To implement the embodiments, the present invention also proposes another computer device, including: the wind generating set primary frequency modulation control system comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and when the processor executes the computer program, the primary frequency modulation control of the wind generating set according to the embodiment of the invention is realized.
As shown in fig. 7, the non-transitory computer readable storage medium includes a memory 810 of instructions executable by processor 820 to perform a method according to a wind turbine generator set primary frequency control device, and an interface 830. Alternatively, the storage medium may be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In order to implement the embodiment, the invention further proposes a non-transitory computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements primary frequency modulation control of a wind turbine generator set according to an embodiment of the invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, a schematic representation of the terms does not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the described embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
One of ordinary skill in the art will appreciate that all or part of the steps carried by the method implementing the embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The mentioned storage medium may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the embodiments are illustrative and not restrictive, and that those skilled in the art may make changes, modifications, substitutions and alterations to the embodiments described herein without departing from the scope of the invention.
Claims (10)
1. A fan blade defect detection method based on computer vision is characterized by comprising the following steps:
constructing a fan blade defect detection model, wherein the fan blade defect detection model comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and a prediction module;
acquiring a fan blade surface image with defects, preprocessing the fan blade surface image to form a fan blade surface defect image data set, inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model, and performing model training;
and inputting the fan blade surface image acquired in real time into a trained fan blade defect detection model, and outputting a result through the model to detect the fan blade surface defect.
2. The computer vision-based fan blade defect detection method of claim 1, wherein in the fan blade defect detection model,
the basic feature extraction module is a feature extractor, and a depth convolution network is adopted to extract basic features of the surface image of the fan blade to obtain feature layers with different sizes and depths;
the multi-scale pooling fusion module is used for pooling fusion of effective feature layers with different sizes and depths so as to improve the detection rate of small defect targets;
the multi-scale feature layer fusion module is used for carrying out feature layer fusion on effective feature layers with different sizes and depths;
the characteristic decoupling module is used for decoupling effective characteristic layers with different sizes obtained after multi-scale pooling fusion so as to provide sufficient characteristic data for different prediction items;
the prediction module is used for outputting a prediction result.
3. The computer vision-based fan blade defect detection method of claim 2, wherein the deep convolution network of the feature extractor is an HnNet network, a ResNet-101 network, or a ResNet-152 network.
4. The computer vision-based fan blade defect detection method of claim 3, wherein the smallest components of the HnNet network structure comprise convolutional layers, BN, Mish activation functions; wherein the content of the first and second substances,
the BN layer is used for realizing the regularization effect, and can improve the convergence speed of the model and prevent the overfitting of the model; the Mish activation function is used for improving the training stability and the average accuracy.
5. The computer vision-based fan blade defect detection method according to claim 2, wherein the multi-scale pooling fusion module is formed by connecting a plurality of pooling layers in parallel, and after the feature layers extracted by the basic feature extraction module are input, multi-scale pooling processing is performed on each pooling layer.
6. The computer vision-based fan blade defect detection method as claimed in claim 2, wherein the multi-scale feature layer fusion module fuses feature layers of different sizes by using FPN and PAN network structures, so as to improve small target and multi-target detection rate.
7. The computer vision-based fan blade defect detection method according to claim 2, wherein the feature decoupling module is used for predicting the defect type, defect position and defect confidence of the fan blade surface image by setting a multilayer basic convolution unit; the characteristic decoupling module comprises an input end, the input end is connected with one ends of the first multilayer basic convolution unit and the second multilayer basic convolution unit, and the other end of the second multilayer basic convolution unit is connected with the third multilayer basic convolution unit and the fourth multilayer basic convolution unit.
8. A fan blade defect detection device based on computer vision, characterized by comprising:
the system comprises a model construction module, a prediction module and a fault detection module, wherein the model construction module is used for constructing a fan blade fault detection model which comprises a basic feature extraction module, a multi-scale pooling fusion module, a multi-scale feature layer fusion module, a feature decoupling module and the prediction module;
the model training module is used for acquiring and preprocessing a fan blade surface image with defects to form a fan blade surface defect image data set, and inputting the preprocessed fan blade surface defect image data set into the fan blade defect detection model to perform model training;
and the defect detection module is used for inputting the fan blade surface image acquired in real time into the trained fan blade defect detection model, and outputting a result through the model to detect the fan blade surface defect.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1-7 when executing the computer program.
10. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158476.2A CN114565571A (en) | 2022-02-21 | 2022-02-21 | Fan blade defect detection method and device based on computer vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210158476.2A CN114565571A (en) | 2022-02-21 | 2022-02-21 | Fan blade defect detection method and device based on computer vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114565571A true CN114565571A (en) | 2022-05-31 |
Family
ID=81714199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210158476.2A Pending CN114565571A (en) | 2022-02-21 | 2022-02-21 | Fan blade defect detection method and device based on computer vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114565571A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058396A (en) * | 2023-10-11 | 2023-11-14 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
| CN117593294A (en) * | 2024-01-18 | 2024-02-23 | 中化学西部新材料有限公司 | Centrifugal fan blade crack visual detection method based on image feature analysis |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110610475A (en) * | 2019-07-07 | 2019-12-24 | 河北工业大学 | Visual defect detection method of deep convolutional neural network |
| WO2021000404A1 (en) * | 2019-07-03 | 2021-01-07 | 平安科技（深圳）有限公司 | Target detection method based on deep learning, and electronic apparatus |
| CN113077453A (en) * | 2021-04-15 | 2021-07-06 | 华南理工大学 | Circuit board component defect detection method based on deep learning |
-
2022
- 2022-02-21 CN CN202210158476.2A patent/CN114565571A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021000404A1 (en) * | 2019-07-03 | 2021-01-07 | 平安科技（深圳）有限公司 | Target detection method based on deep learning, and electronic apparatus |
| CN110610475A (en) * | 2019-07-07 | 2019-12-24 | 河北工业大学 | Visual defect detection method of deep convolutional neural network |
| CN113077453A (en) * | 2021-04-15 | 2021-07-06 | 华南理工大学 | Circuit board component defect detection method based on deep learning |
Non-Patent Citations (5)
| Title |
|---|
| 张新良;付鹏飞;赵运基;谢恒;王琬如;: "融合图卷积和差异性池化函数的点云数据分类分割模型", 中国图象图形学报, no. 06, 16 June 2020 (2020-06-16), pages 1201 - 1208 * |
| 张超;文传博;: "基于改进Mask R-CNN的风机叶片缺陷检测", 可再生能源, no. 09, 18 September 2020 (2020-09-18), pages 1181 - 1186 * |
| 张雪菲;程乐超;白升利;张繁;孙农亮;王章野;: "基于变分自编码器的人脸图像修复", 计算机辅助设计与图形学学报, vol. 32, no. 03, 31 March 2020 (2020-03-31), pages 401 - 409 * |
| 范晨亮;李国庆;马长啸;黄海;: "基于深度学习的风机叶片裂纹检测算法", 科学技术创新, no. 13, 5 May 2020 (2020-05-05), pages 72 - 74 * |
| 颜洵;吴正平;雷帮军;: "基于深度学习的视频火焰烟雾检测方法", 信息通信, no. 09, 15 September 2020 (2020-09-15), pages 70 - 72 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058396A (en) * | 2023-10-11 | 2023-11-14 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
| CN117058396B (en) * | 2023-10-11 | 2023-12-26 | 精效悬浮(苏州)科技有限公司 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
| CN117593294A (en) * | 2024-01-18 | 2024-02-23 | 中化学西部新材料有限公司 | Centrifugal fan blade crack visual detection method based on image feature analysis |
| CN117593294B (en) * | 2024-01-18 | 2024-04-09 | 中化学西部新材料有限公司 | Centrifugal fan blade crack visual detection method based on image feature analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Du et al. | Damage detection techniques for wind turbine blades: A review | |
| CN105122046B (en) | For the defects of detection object method | |
| CN112233091B (en) | Wind turbine blade image damage detection and positioning method | |
| Beganovic et al. | Structural health management utilization for lifetime prognosis and advanced control strategy deployment of wind turbines: An overview and outlook concerning actual methods, tools, and obtained results | |
| CN114565571A (en) | Fan blade defect detection method and device based on computer vision | |
| Rizk et al. | Hyperspectral imaging applied for the detection of wind turbine blade damage and icing | |
| Sun et al. | In-situ condition monitoring of wind turbine blades: A critical and systematic review of techniques, challenges, and futures | |
| CN111094956A (en) | Processing the thermographic image with a neural network to identify Corrosion Under Insulation (CUI) | |
| Wang et al. | Review of the typical damage and damage-detection methods of large wind turbine blades | |
| CN111400961B (en) | Wind generating set blade fault judging method and device | |
| CN110738753B (en) | System and method for predicting fatigue damage of ship power pipeline | |
| Awadallah et al. | Effect and detection of cracks on small wind turbine blade vibration using special Kriging analysis of spectral shifts | |
| CN104458911A (en) | Defect type determination method for ultrasonic testing of wind turbine blades | |
| NO20230279A1 (en) | Method and system for diagnosing structural damage of overwater building on basis of flow-state video monitoring | |
| Le et al. | Piezoelectric impedance-based structural health monitoring of wind turbine structures: Current status and future perspectives | |
| CN114183312A (en) | System and method for monitoring state of blades of wind turbine generator | |
| Zhang et al. | Surface defect detection of wind turbine based on lightweight YOLOv5s model | |
| Chen et al. | Detection of the pipeline elbow erosion by percussion and deep learning | |
| CN207366507U (en) | The in-service ultrasonic nondestructive testing device of wind turbine paddle | |
| Memari et al. | Review on the advancements in wind turbine blade inspection: Integrating drone and deep learning technologies for enhanced defect detection | |
| Chen et al. | AQUADA PLUS: Automated damage inspection of cyclic-loaded large-scale composite structures using thermal imagery and computer vision | |
| CN113296072B (en) | Method and system for automatically identifying thunderstorm strong wind based on YOLOv3 model | |
| CN116306262A (en) | Galvanized steel corrosion prediction method based on ensemble learning algorithm | |
| CN115758915A (en) | Heat exchange tube overhauling and intelligent operation and maintenance control method of energy pile active bridge deck deicing and snow melting system | |
| CN114549436A (en) | Multi-mode-based fan blade defect detection method, system, equipment and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |