CN115908354A

CN115908354A - Photovoltaic panel defect detection method based on double-scale strategy and improved YOLOV5 network

Info

Publication number: CN115908354A
Application number: CN202211551104.2A
Authority: CN
Inventors: 郑魁; 丁维龙
Original assignee: Shanghai Paiying Medical Technology Co ltd
Current assignee: Shanghai Paiying Medical Technology Co ltd
Priority date: 2022-12-05
Filing date: 2022-12-05
Publication date: 2023-04-04

Abstract

The invention discloses a photovoltaic panel defect detection method based on a double-scale strategy and an improved YOLOV5 network, which comprises the following steps: screening samples in the EL electroluminescence image, and labeling; filtering the screened sample image by adopting an Otsu binarization method and removing black background noise in a photovoltaic panel area; obtaining a trained weight file by training a two-scale MET _ YOLOV5 model; verifying the trained MET _ YOLOV5 network model; and predicting the collected photovoltaic panel EL image by using the model after the verification is passed to obtain a dual-scale prediction result. By applying the embodiment of the invention, not only can relatively obvious hidden crack defects be detected in a large-scale image, but also small-scale defect detection can be carried out on each grid unit in the photovoltaic panel, so that finer hidden cracks can be identified. And the fusion of the dual-scale defect prediction results can enable the final prediction result to be more accurate and have higher confidence.

Description

Photovoltaic panel defect detection method based on double-scale strategy and improved YOLOV5 network

Technical Field

The invention relates to the technical field of solar panel defect identification, in particular to a photovoltaic panel defect detection method and device based on a dual-scale strategy and an improved YOLOV5 network.

Background

In recent years, deep learning technology has numerous achievements and applications in machine vision, and deep learning has become an important technical means for image detection and image processing. Machine vision defect detection methods have gradually replaced manual vision detection methods.

The machine vision defect detection depends on the powerful calculation force of a computer, a great amount of positive and negative sample picture data of the defects are input for model learning, and then the trained model is directly utilized to detect whether the sample has the defects. The machine vision defect detection system generally comprises a hardware system and a software system, wherein the hardware system comprises an image acquisition device and a display device, and the software system mainly uses a deep neural network model and an image segmentation algorithm to complete the processing and analysis of image characteristics.

In research related to this field, demant et al have proposed a battery piece detection method based on a Support Vector Machine (SVM), which determines whether a battery piece exists based on defect features of extracted positive and negative samples. Wangxiang, etc. reconstructs cell images through unsupervised learning, trains Deep Belief Networks (DBNs) to compare and analyze reconstructed images with real images, and sequentially detects input data.

Although the efficiency of the defect detection method based on deep learning is greatly improved compared with that of a manual method, some problems still exist in an actual industrial scene. For the field of defect detection of photovoltaic panels, there are mainly: 1) The defect sample capacity is too small, so that the characteristic extraction of the network model is not facilitated, and the network model is easy to over-fit; 2) The scale and the form of the defect are different, the defect characteristics with fine granularity are easily lost in a network layer or covered by other noises, and the robustness and the generalization of the model are influenced; 3) With the increase of the depth and the width of the network, the parameters in the network increase in geometric multiples, the training time of the model is greatly increased, the detection efficiency is low, and the requirement on the storage performance is higher, so that the model parameters of the network need to be optimized; 4) The original size of the EL image is larger, the defect detection precision of the whole image from end to end is not high by the existing method, and meanwhile, the network has no adaptability to images with different scales. 5) In a task based on small-scale image sliding window detection, the existing method cannot well aggregate the same defect of adjacent grids, and the comprehension capability of global semantic information is lacked.

Disclosure of Invention

The invention aims to provide a photovoltaic panel defect detection method based on a double-scale strategy and an improved YOLOV5 network, aims to realize an end-to-end photovoltaic panel defect identification task, obtains better identification accuracy rate for tiny defects, and has stronger algorithm real-time performance and wide application scenes.

In order to achieve the above object, the present invention provides a method for detecting defects of a photovoltaic panel based on a dual-scale strategy and an improved YOLOV5 network, wherein the method comprises:

collecting an EL electroluminescence image of a photovoltaic panel;

screening a sample image from the collected EL electroluminescence image, and labeling the image with electroluminescence defects;

filtering the screened sample image by adopting an Otsu binarization method to obtain a filtered sample image, and removing black background noise in a photovoltaic panel area;

constructing a training sample set and a verification set based on the filtered sample images and the labeled sample images;

based on the training sample set, training by using a MET _ YOLOV5 network model and adopting a dual-scale strategy to obtain a trained MET _ YOLOV5 network model;

and verifying by adopting the trained dual-scale MET _ YOLOV5 network model, and predicting the collected photovoltaic panel EL image by using the MET _ YOLOV5 model after verification is passed to obtain a dual-scale prediction result.

In one implementation, the step of collecting an EL electroluminescence image of a photovoltaic panel includes:

shooting a near-infrared image of the photovoltaic panel assembly by using a high-resolution CCD camera;

and taking the near-infrared image as an EL electroluminescence image of the photovoltaic panel.

In one implementation manner, the step of filtering the screened sample image by using an Otsu binarization method to obtain a filtered sample image and removing black background noise in a photovoltaic panel region includes:

carrying out gray level processing on the screened sample image, obtaining a threshold value on a gray level image by using an Otsu algorithm, and carrying out image threshold value binarization by using the threshold value;

finding out a maximum connected domain by adopting closed operation to obtain a filtered image;

and (4) adopting a Canny operator to carry out edge detection, and extracting the original edge texture with interference effect on the characteristics in the filtered image.

In one implementation, the step of constructing a training sample set and a validation set based on the filtered sample images and the labeled sample images includes:

adopting a Mosaic data enhancement method, and realizing data set expansion by carrying out image splicing on the filtered sample image and the labeled sample image in the modes of random scaling, random cutting and random arrangement;

and sending the expanded data set into a network model for training.

In one implementation, the step of training the MET _ YOLOV5 network model based on the training sample set to obtain a trained weight file includes:

the MET _ YOLOV5 network model structure comprises:

input end: the method comprises the following steps of Mosaic data enhancement, self-adaptive anchor frame calculation and self-adaptive picture scaling;

backbone network: CSPNet, neck: SPP, FPN, PAN;

head prediction: yoloHead; wherein CSPNet is used for extracting features, SPP, FPN and PAN are used for enhancing the features, and GIOU _ Loss is adopted by a head Yolohead to be predicted as a Loss function for calculating a Boundingbox; its default input image size is 640 x 3.

In one implementation mode, MET _ YOLOV5 adds a Swin-Transformer module to a C3 module in a backbone network CSPNet to perform targeted optimization and improvement on a YoloV5 network, the improved C3 module is called C3STR, a self-attention structure is used for enhancing semantic information and feature extraction capability of a small target, cross-window information interaction is realized in a local window dividing mode, and the calculated amount is reduced;

adding an ECA attention module at the tail end of a neck SPP layer of the MET-YOLOV 5, and performing average pooling on all characteristic channels under the condition of not reducing dimensions;

the ECA module captures information among different channels by using one-dimensional convolution, multiplies the channel attention characteristic diagram and the input characteristic diagram channel by channel and outputs the multiplied channel attention characteristic diagram and the input characteristic diagram;

the ECA module can optimize a characteristic diagram, so that the network focuses on hidden crack defects of photovoltaic panels with different sizes and shapes.

In one implementation manner, the step of predicting the acquired photovoltaic panel EL image by using the MET _ YOLOV5 model after passing the verification to obtain the dual-scale prediction result includes:

for one EL image, the whole large-size EL image is sent into a MET _ YOLOV5 network for prediction after being preprocessed and data enhanced, and a large-size prediction result F1 is obtained; meanwhile, a single patch image obtained by segmenting the EL image according to the grid of the photovoltaic panel is sent to a MET _ YOLOV5 network for prediction, defect prediction frames meeting the neighbor relation are combined, the prediction result of the patch scale is mapped to the original scale to obtain a prediction result F2, and finally the F2 and the full-scale prediction result F1 are fused to obtain F.

In one implementation, the step of fusing the F2 and the full-scale prediction result F1 to obtain F includes:

for an EL panel image, a large-scale prediction result set F1 and a small-scale prediction result set F2 are given; f1 and F2 respectively comprise i and j prediction frames, and the single prediction frame is represented as F1 _i ，F2 _j Each prediction box information includes: predicted frame position Px, confidence Pc. Fusing in an anchor frame-by-anchor frame matching mode;

for any one prediction frame F1 _i And a prediction box F2 _j Judging whether the prediction frames have the same defect or not by calculating the intersection ratio IoU, wherein the IoU measures the relative overlapping size of the two boundary frames; for F1 _i And F2 _j The calculation formula is as follows:

if F1 _i And F2 _j If the threshold condition that the IoU is more than or equal to 0.5 is met, the same defect is judged;

setting the final prediction frame position Px as a prediction frame F1 _i And a prediction box F2 _j The prediction frame position with higher middle probability:

Px＝[max(Pc _i ,Pc _j ),Px]

and then, the final confidence Pc of the prediction frame is given by calculating the joint probability, and the calculation formula is as follows:

Pc _i and Pc _j Respectively represent prediction frames F1 _i And a prediction box F2 _j The confidence of Pc. The prediction result set F obtained by fusion comprises m prediction frames obtained by fusion, and a single prediction frame is represented as F _m Each prediction box information includes: the position Px of the prediction frame and the confidence Pc, i and j represent the sequence number in the prediction result set to which the prediction frame belongs.

In one implementation, the method further comprises: for the small-scale prediction result, merging the cross-grid defects by adopting a fusion algorithm based on a neighbor relation; fusing the prediction result by adopting a dual-scale image fusion strategy algorithm;

prediction result F2 of each grid in small-scale prediction result F2 _j Calculate the prediction result F2 in the grid adjacent to it _j+1 Minimum distance, use thresholdFiltering by a value judgment method (the shortest distance between the boundary points of any two prediction frames is less than 1/4 of the average value of the longest diagonal), and merging the prediction frames meeting the requirements; for the merged prediction box F2 _j ', setting the position Px of the prediction frame as the minimum circumscribed rectangle of the union set of the two prediction frames; and calculating the joint probability to obtain the final confidence Pc of the prediction frame, wherein the calculation formula is as follows:

and filtering by using a threshold judgment method until the shortest distance between any two boundary points of the prediction frames is less than 1/4 of the average value of the longest diagonal.

In one implementation, the step of verifying the trained MET _ YOLOV5 network model by using the verification set includes:

and setting the number of images and the iteration times of each training, adjusting the proportion of the test set and the verification set to carry out multiple times of verification, and selecting an optimal weight parameter file to predict defects. .

By applying the photovoltaic panel defect detection method and device based on the dual-scale strategy and the improved YOLOV5 network, according to the characteristics of the sample image, the original image is subjected to background filtering by adopting an Otsu binarization method, and meanwhile, the edge of the image is extracted by adopting a Canny operator, so that the extraction capability of the network on effective features is improved.

The improvement of the MET _ YOLOV5 network model in the invention is that: by adding Swin-Block, the semantic information and the feature extraction capability of a small target are enhanced by using multi-head self-attention, and cross-window information interaction is realized in a manner of dividing local windows; by adding the attention mechanism ECA module to the SPP layer for multi-scale feature weighting adjustment, the capability of extracting features of a network across channels is enhanced, and the precision of detecting the micro defects is improved.

The method realizes the task of identifying the defects of the photovoltaic panel from end to end, obtains better identification accuracy for the tiny defects, and has stronger algorithm real-time performance and wide application scenes.

Drawings

Fig. 1 is a schematic structural diagram of a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network according to an embodiment of the present invention.

Fig. 2 is a schematic processing result of the photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to the embodiment of the present invention.

Fig. 3 is a schematic processing result of the photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network according to the embodiment of the present invention.

Fig. 4 is a schematic processing result of a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network according to an embodiment of the present invention.

Fig. 5 is a schematic diagram of a double-scale processing flow of the photovoltaic panel defect detection method based on the double-scale strategy and the improved YOLOV5 network according to the embodiment of the present invention.

The reference numbers are: 1. a loading platform; 2. using an industrial detection phase from beginning to end; 3. a modularized air outlet; 4. industrial detection cameras for the process; 5. temperature sensor

Detailed Description

The following embodiments of the present invention are provided by way of specific examples, and other advantages and effects of the present invention will be readily apparent to those skilled in the art from the disclosure herein. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.

Please refer to fig. 1-5. It should be noted that the drawings provided in this embodiment are only for schematically illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings and not drawn according to the number, shape and size of the components in actual implementation, and the form, quantity and proportion of each component in actual implementation may be arbitrarily changed, and the component layout may be more complicated.

With the great increase of energy demand in the global scope, the development of new energy, especially clean energy, is also highly valued, and the photovoltaic power generation technology is widely applied at home and abroad. According to statistics, the photovoltaic installed scale in 2017 cumulatively reaches 400GW, and the installed capacity of the Chinese grid-connected photovoltaic exceeds 1.4 hundred million kW by 4 months in 2018. The application scenes and the places of the photovoltaic power generation technology are very widely distributed. The photovoltaic panel is used as a core element of photovoltaic power generation, and the photovoltaic panel can have serious faults due to problems and defects caused by environmental factors, physical factors, material functional characteristics and the like in all links of manufacturing, transportation, installation and use.

There are three main types of defects in photovoltaic power generation solar panels: contamination of silicon wafer raw materials, damage during cell production and defects generated during use. Methods for detecting these defects include: electrical parameter measurement, laser beam induced current, phase-locked infrared thermal imaging, electroluminescence and the like. Among these methods, the electrical parameter measurement method and the laser beam induced current method cannot directly obtain a defect image of a sample; the infrared thermal imaging method needs to apply a reverse bias voltage to the battery and then pick up an image of infrared rays emitted from the surface of the panel; the method has low detection efficiency and relatively high cost because the surface of the battery needs to be heated for a certain time and an infrared camera is used for detection. Electroluminescence (EL) is based on the principle that when a photovoltaic panel is forward biased, the intensity of light produced is proportional to the voltage, and thus electrically inactive shadow areas in the photovoltaic panel are detected. The EL technology can quickly and efficiently detect the defects of grid breakage, hidden crack, insufficient solder, fragments, concentric circles, electric leakage, pollution and the like in the photovoltaic panel. The EL technology is widely used at present, and the operation, maintenance and detection efficiency of the photovoltaic panel of the large power station is improved. However, the defect identification and the defect elimination of the EL image are mainly performed manually, the automation degree is not high, and due to the limitation of manual work, for some small defects, the possibility of missing detection and false detection to a great extent exists. In recent years, with the development of Deep learning and Convolutional Neural network technologies, methods such as Support Vector Machines (SVM), deep Belief Networks (DBNs), convolutional Neural Networks (CNN), and Generative Adaptive Networks (GAN) have been widely and deeply studied and applied in the fields of image classification, object recognition, semantic segmentation, and the like. The full-automatic defect detection of the photovoltaic panel EL image by using a computer method based on deep learning is also the direction of research of many scholars.

With the continuous expansion of the photovoltaic industry scale, the full-automatic defect detection of the photovoltaic panel EL image is realized by using a computer method based on deep learning, the detection efficiency can be improved, the false judgment rate and the omission factor are reduced, the labor cost is greatly reduced in the links of photovoltaic panel production, operation and maintenance and the like, and the method has higher practical value and significance.

In the prior art, in the field of target detection algorithm research based on deep learning, in the background art, a method for enhancing data of a trained image is generally adopted for problem 1) in the background art. Simony et al, for example, increase the diversity of training images by flipping, scaling, and randomly cropping the images. Srivastava et al also mitigated the overfitting of the network to some extent using Dropout and regularization methods. Adamu et al generated more training data by using a generation countermeasure network (GAN) and helped to alleviate model overfitting, but GAN still had the problems of unstable training, disappearance of gradient and the like.

For problem 2) in the background art, the feature extraction network is usually optimized based on different target detection tasks. For example, he et al proposed a Spatial Pyramid Pooling (SPP) structure in 2014, which realizes the fusion of multi-scale local features and global features, and enhances the expression capability of feature maps. How to achieve the purpose of the detection of the PCB in the PCB detection task is how to achieve the national loyalty and the like [ a global space attention module aiming at fine-grained features is introduced into a YOLOV4 network, the possibility that the fine-grained features disappear due to downsampling is reduced through a global connection channel, and the precision is improved in the PCB detection task. However, the method does not consider the difference of importance degree of each channel of the image, and still needs to be improved.

For problem 3) in the background art, the network model is usually lightened mainly by optimizing the computation method of the convolution layer and the full connection layer, and by compressing the overall network parameters, performing deep separable convolution, and the like. By replacing the backbone network of YOLO-V3 with simplified Darknet-53 as in official et al, only 10 convolutional layers and 6 pooling layers were reserved and no pooling layer was used. The network performance is not lost while the network parameters are reduced. However, the method is not universal enough, and more false detections are generated in image data with complex background.

The method has the advantages that: 1) Aiming at the problem that the generalization capability of the model is insufficient due to insufficient sample quantity, canny operator is adopted to carry out first-order gradient processing on the defect part, so that the defect is more obvious, and the processed characteristic diagram and the original image are used as multi-channel input of the model. The data enhancement method based on the bottom layer characteristic image optimization can improve the generalization capability of the model and relieve the problem of sample imbalance. 2) And for the problem that the sample image has an interference region, filtering background noise by using an Otsu threshold method. 3) The defect forms and scales of the EL images are greatly different, the image boundaries are often fuzzy and have noise, so that some fine defect features are easily ignored by the model, and the method combines a Transformer structure, a channel attention mechanism ECA module and a multi-scale feature fusion strategy on a Yolo-v5 network. A Swin-Transformer module is added into a C3 module in a Yolo-v5 backbone network, a self-attention structure of a Transformer is introduced to enhance the semantic information and feature extraction capability of a small target, and cross-window information interaction is realized in a manner of dividing a local window; a channel attention mechanism ECA is added at the end of an SPP (Spatial Pyramid) layer of a backbone network, weighting adjustment is carried out on the feature weights of different channels, the extraction efficiency of the network on cross-channel features is optimized, and the identification accuracy of the model on micro defects is improved. 4) Through a double-scale network prediction and prediction frame fusion method, a defect prediction frame based on different scale views is provided, and the defect detection precision and efficiency are enhanced. 5) Through a cross-grid defect prediction frame combination method based on the neighbor relation, the cross-grid defects in the small-scale prediction result are ablated, and the semantic accuracy of the defect prediction result is improved.

Compared with the existing photovoltaic panel defect detection method based on deep learning, the method fully considers the characteristics of the photovoltaic panel EL image in the aspect of data preprocessing, performs background noise filtering and defect significance enhancing operation on the input image, and improves the robustness of the model. On the network structure, according to the improved research on the Yolo target detection algorithm, a transform module, an attention mechanism and a multi-scale feature fusion strategy are added into the network, so that the extraction capability of the network on features of different scales and different image areas is improved, and the target detection precision is improved.

The invention provides a photovoltaic panel defect detection method based on a dual-scale strategy and an improved YOLOV5 network as shown in fig. 1, which comprises the following steps:

s101, collecting an EL electroluminescence image of the photovoltaic panel.

In the embodiment of the invention, professional equipment is used for collecting the EL electroluminescence image of the photovoltaic panel, and specifically, an EL detector is used for collecting the EL image of the photovoltaic panel. The instrument utilizes the electroluminescence principle of crystalline silicon and a high-resolution CCD camera to shoot near-infrared images of the photovoltaic panel assembly.

S102, screening a sample image from the collected EL electroluminescence image, and labeling the image with the electroluminescence defect.

Screening the collected photovoltaic panel EL images to obtain images with better quality; and labeling the image with defects, and storing the labeling information in an XML file form.

In one specific implementation, a sample image with high imaging quality is screened, and the screened image is finely labeled by an expert. The number of images screened out to contain fine annotations should not be less than 400.

And S103, filtering the screened sample image by adopting an Otsu binarization method to obtain a filtered sample image, and removing black background noise in a photovoltaic panel area.

And filtering the image screened by the S102 by adopting an Otsu binarization method, and removing black background noise around the photovoltaic panel area.

The specific processing steps may include:

and S3.1, carrying out gray level processing on the screened sample image, obtaining a threshold value on the gray level image by using an Otsu algorithm, and carrying out image threshold value binarization by using the threshold value. And (5) finding out the maximum connected domain by adopting closed operation to obtain the filtered image shown in the figure 2.

The Otsu algorithm firstly carries out gray processing on an input RBG channel image, calculates a threshold value according to the maximum inter-class variance of a gray image histogram, and finally separates a foreground and a background according to the threshold value.

S3.2: and (3) carrying out edge detection by adopting a Canny operator, filtering original edge textures which have interference on feature extraction in the image, and carrying out edge detection to obtain the image shown in the figure 3.

The edge detection algorithm based on the Canny operator comprises the steps of firstly carrying out Gaussian smoothing on an input image, calculating gradient change amplitude and direction, estimating edge strength and direction of each point, carrying out non-maximum suppression on the gradient amplitude according to the gradient direction, and finally connecting detected edges by utilizing double-threshold processing.

According to the invention, the original image is subjected to background filtering by adopting an Otsu binarization method, and meanwhile, the edge of the image is extracted by adopting a Canny operator, so that the extraction capability of a network on effective characteristics is improved.

And S104, constructing a training sample set and a verification set based on the filtered sample images and the labeled sample images.

And constructing a training sample set and a verification set by using the image after binarization filtering and the labeling data generated in S102. Due to the fact that the number of samples is limited, the method for enhancing the Mosaic data is adopted in the embodiment of the invention, the data set is expanded through methods of random scaling, random cutting, random arrangement and image splicing, the generalization capability of the model is enhanced, and the identification accuracy of the model is improved.

And S105, based on the training sample set, using a MET _ YOLOV5 network model and adopting a dual-scale strategy to train to obtain the trained MET _ YOLOV5 network model.

And training by using a MET _ YOLOV5 network to obtain a trained weight file. The following steps may be employed:

s5.1: training parameters were set, including epoch =100, and input image size was 1280 x 1280.

S5.2: the MET _ YOLOV5 backbone network slices the characteristic image by adopting a Focus structure and divides the characteristic image into two parts. And the feature maps are fused through the CSP structure, so that the calculation amount is reduced. And the Neck is used for generating a characteristic pyramid and enhancing the detection capability of the model on the characteristics of different scales.

S5.3: adding an ECA module at the end of an SPP layer of a MET _ YOLOV5 backbone network, capturing information among different channels by using 1-by-1 convolution, multiplying a channel attention feature graph and an input feature graph channel by channel and outputting, wherein for an aggregation feature y belonging to RC without dimension reduction, the channel attention feature graph is as follows:

ω＝σ(C1D _k (y))

wherein C1D represents a one-dimensional convolution; for the characteristics of different dimensions, the size k of one-dimensional convolution kernel is determined by an adaptive method:

the convolution kernel size is k, the channel dimension is C, γ and b are parameters of the ECA module, γ =2,b =1; | t odd represents the odd number closest to t. The ECA module can be concerned about hidden crack defects of photovoltaic panels with different sizes and shapes.

MET _ YOLOV5 adds an improved ECA module at the end of an SPP layer of a backbone network, and averagely pools all characteristic channels without dimension reduction. And the ECA module captures information among different channels by using one-dimensional convolution, and multiplies the channel attention feature map and the input feature map channel by channel and outputs the channel attention feature map and the input feature map. It is divided into two branches: branching a pair of input feature maps for global average pooling, and then calculating a channel attention Weight _avg (ii) a Performing global maximum pooling on the branch two-pair input feature map, and calculating to obtain a channel attention Weight _max . Adding the two attention weight tensors and inputting the sum into a sigmoid function for calculation to obtain [0,1]A weight value of a range.

S5.4: and adopting the adaptive anchor frame regression calculation at the output end of the Prediction, and adopting GIOU _ Loss to perform Bounding Box Loss calculation. And adopting a weighted nms non-maximum suppression method based on the IOU.

And setting the number of images and the iteration times of each training, and adjusting the proportion of the test set to the verification set to perform multiple experiments to obtain the optimal weight parameters.

S5.5: detecting global defects in a large-scale network branch by using a dual-scale defect detection strategy as shown in FIG. 5; detecting fine defects in grids of the photovoltaic panel in small-scale network branches, and merging small-scale network prediction frames by adopting a cross-grid defect prediction frame merging method based on a neighbor relation; and finally, fusing the prediction results of the two network branches by adopting a dual-scale prediction result fusion algorithm.

S106, verifying by adopting the trained MET _ YOLOV5 network model, and predicting the collected photovoltaic panel EL image by using the MET _ YOLOV5 network model after the verification is passed to obtain a double-scale prediction result.

And testing through the test set based on the trained weight file. And predicting the collected photovoltaic panel EL image by using the trained MET _ YOLOV5 network model to obtain a prediction result shown in figure 4.

The invention improves the MET _ YOLOV5 network model, namely, a Swin-transformer block is added into a backbone network feature extraction module C3 of the MET _ YOLOV5 network model, and the semantic information and the feature extraction capability of a small target are enhanced by using a self-attention structure; an attention mechanism ECA is added at the tail end of the backbone network to perform multi-scale feature weighting adjustment, so that the capability of extracting features across channels of the network is enhanced, and the precision of detecting the micro defects is improved. Based on an improved Yolo-v5 network model, an end-to-end photovoltaic power generation solar panel EL image detection task can be completed. Meanwhile, the network adopts a double-scale strategy to train and predict, and provides a small-scale cross-grid defect merging algorithm and a double-scale prediction result fusion algorithm.

Based on an improved network MET _ YOLOV5 network model, firstly, images are normalized at an input end, the size of the images is 1280 pixels by 1280 pixels, and the images are sent to a backbone network to extract features after being processed by a Mosaic data enhancement method and a self-adaptive anchor frame calculation method. And extracting features by adopting a Focus structure and a CSP-Net structure in a Backbone layer, wherein the Focus structure obtains a 320 by 32 feature map by carrying out convolution operation of slicing and 32 convolution kernels on the image. The MET-YOLOV 5 network model comprises two CSP structures, wherein the backbone network is a CSP 1-X structure, an ECA channel attention mechanism is added at the tail end of an SPP layer of the backbone network, and the characteristics of different channels are adjusted in a non-dimensionality reduction mode. And in the Neck layer, fusing image features by adopting a FPN + PAN structure to generate a feature pyramid. And the output end calculates the Loss of the bounding box by using the GIOU _ Loss, and calculates the Loss of the class probability and the target score by using the cross entropy and the logs Loss function.

In the photovoltaic panel EL image defect identification task, firstly, black background in an original picture and original texture features of a photovoltaic panel are filtered through Otsu binarization processing in a first stage, and efficiency and accuracy of network feature extraction are improved. And the generalization performance of the network MET _ YOLOV5 network model is improved under the condition of limited samples and marking information by a Mosaic data enhancement method. The method realizes the task of identifying the defects of the photovoltaic panel from end to end, obtains better identification accuracy for the tiny defects, and has stronger algorithm real-time performance and wide application scenes.

Secondly, swin-Transformarmerplock is added into a backbone network C3 structure, an advanced method based on multi-head self-attention of a sliding window is used for improving network feature extraction capacity, semantic information of small targets in the network is enhanced, and therefore performance baselines are improved.

And thirdly, a channel attention mechanism ECA is added at the tail end of the SPP layer of the backbone network, the dependency relationship among the characteristic channels is obtained, meanwhile, the dimension of the characteristic graph is kept unchanged, cross-channel interaction information is captured, the characteristics of defects are focused, and the detection precision of the MET _ YOLOV5 network model is improved. Through cooperation of the three stages, an end-to-end detection task of the EL image defects of the photovoltaic panel is realized, and possibility is provided for putting into practical production and application.

Finally, the invention enables the network to notice the global defect characteristics and the local defect characteristics by using a double-scale target detection strategy, combining a double-scale prediction result fusion algorithm and a cross-grid defect prediction frame fusion algorithm based on the neighbor relation, and simultaneously improves the semantic accuracy of the defects in the prediction result.

In the present application, aiming at the problems existing in the above researches, the present invention provides a targeted solution: 1) By preprocessing the image, the significance of the features is improved. And the sample diversity is improved through data enhancement operation. 2) Swin-TransformerBlock is added into a feature extraction module C3 of the backbone network, so that the feature extraction capability of the network is enhanced. 3) The global semantic information of the characteristic channel is fused by introducing the channel attention module ECA into the network, so that the performance of the network is further improved. 4) And acquiring and fusing defect prediction results under different detection fields by a double-scale target detection strategy, so that the confidence of the prediction results is improved. 5) Combining the cross-grid defects under small scale by calculating the cross-grid defect distance with the nearest neighbor relation; and the semantic accuracy of defect detection is improved.

In the research of computer vision, the quality of image quality directly influences the efficiency of the algorithm and the precision of the effect. Therefore, image preprocessing is very important to improve recognition accuracy and algorithm robustness. The main purpose of image preprocessing is to eliminate irrelevant information in an image, retain and restore useful real information, and simplify data to the maximum extent, and general image preprocessing operations include: graying, geometric transformation, image denoising, edge extraction and the like. According to the characteristics of an input image, the method filters the black background which does not contain semantic information in the image by adopting an Otsu threshold method. Meanwhile, in order to optimize the visual effect of the defect, the Canny operator edge detection algorithm is adopted to improve the significance of the defect.

The MET _ YOLOV5 network model adopted by the method expands the data set by using a Mosaic method, and simultaneously adopts operations such as turning, brightness adjustment, clipping and the like, so that the method has better generalization capability on the sample set with less data volume. The backbone network adopts a network structure combining CSP-Darknet (Cross Stage Partial Networks Darknet), swin-Transformer and Spatial Pyramid Pooling (SPP), and is based on a sliding window multi-head self-attention mechanism; the gradient change of the characteristic layer is integrated into the characteristic diagram, so that the operation amount of parameters is reduced to a large extent, and the model scale is reduced while the accuracy and the recognition speed are high. The method comprises the steps that firstly, a network divides each image in a training set into S multiplied by S (S =19, 38, 76) grids, each grid has candidate frames with different sizes after being calculated through a self-adaptive anchor frame, and the grid where the center of an object is located is responsible for detecting the object; then, extracting characteristics through the convolution layer of the backbone network; finally, the prediction layer is used for multi-scale prediction. The predicted characteristic maps have multiple scales, objects with different sizes can be predicted, and the predicted characteristic maps are obtained by fusing the characteristic pyramid structures in a multi-scale mode. Meanwhile, the invention adopts a double-scale target detection strategy, namely, the whole image and the small-scale grid image are respectively trained and predicted; and merging the defect frames of the prediction results of the small-scale network, and fusing the prediction frame result sets obtained by the double-scale network.

Specifically, for one EL image, the whole large-size EL image is sent into a MET _ YOLOV5 network for prediction after being preprocessed and data enhanced, and a large-size prediction result F1 is obtained; meanwhile, a single patch image obtained by segmenting the EL image according to the grid of the photovoltaic panel is sent to a MET _ YOLOV5 network for prediction, defect prediction frames meeting the neighbor relation are combined, the prediction result of the patch scale is mapped to the original scale to obtain a prediction result F2, and finally the F2 and the full-scale prediction result F1 are fused to obtain F.

And a step of fusing the F2 and the full-scale prediction result F1 to obtain F, wherein the step comprises the following steps:

for an EL panel image, a large-scale prediction result set F1 and a small-scale prediction result set F2 are given; f1 and F2 respectively comprise i and j prediction frames, and the single prediction frame is represented as F1 _i ，F2 _j Each prediction box information includes: predicting the frame position Px and the confidence coefficient Pc, and fusing in an anchor frame-by-anchor frame matching mode;

for any one prediction frame F1 _i And a prediction box F2 _j By calculating the cross-over ratio thereofThe IoU judges whether the prediction frames are the same defect or not, and measures the relative overlapping size of the two boundary frames; for F1 _i And F2 _j The calculation formula is as follows:

setting the final position Px of the prediction frame as a prediction frame F1 _i And a prediction box F2 _j The prediction frame position with higher middle probability:

Px＝[max(Pc _j ,Pc _j ),px]

Pc _i and Pc _j Respectively represent prediction frames F1 _i And a prediction box F2 _j The confidence Pc, the prediction result set F obtained by fusion contains m prediction frames obtained by fusion, and a single prediction frame is represented as F _m Each prediction box information includes: the position Px of the prediction frame and the confidence Pc, i and j represent the sequence number in the prediction result set to which the prediction frame belongs.

For the small-scale prediction result, merging the cross-grid defects by adopting a fusion algorithm based on a neighbor relation; fusing the prediction result by adopting a dual-scale image fusion strategy algorithm;

prediction result F2 of each grid in small-scale prediction result F2 _j Calculate its prediction result F2 in the grid adjacent to it _j+1 Filtering by using a threshold judgment method (the shortest distance between the boundary points of any two prediction frames is less than 1/4 of the average value of the longest diagonal), and merging the prediction frames meeting the requirements; for the merged prediction frame F2 _j ', setting its predicted frame positionPlacing Px as the minimum circumscribed rectangle of the union set of the two prediction frames; and calculating the joint probability to obtain the final confidence Pc of the prediction frame, wherein the calculation formula is as follows:

In the embodiment of the present invention, specifically, F formed by fusing the prediction results F1 and F2 is also a prediction result set, where F includes a plurality of prediction frames, and then the prediction frames in F1 and F2 are compared to remove repeated prediction frames, and the prediction frames with the same semantic information are retained to obtain a final fusion result, that is, a prediction result.

The Yolo series network model has the advantages of relatively high performance and obvious lightweight model size, and achieves excellent performance on target detection competitions of data sets disclosed at home and abroad. The performance of the method on the accuracy, the detection speed and the storage requirement can well meet the requirement of industrial product defect detection.

The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims

1. A photovoltaic panel defect detection method based on a double-scale strategy and an improved YOLOV5 network is characterized by comprising the following steps:

collecting an EL (electroluminescence) image of a photovoltaic panel;

based on the training sample set, training by using a MET _ YOLOV5 network model and adopting a double-scale strategy to obtain a trained MET _ YOLOV5 network model;

2. The method of claim 1, wherein the step of acquiring EL electroluminescence images of the photovoltaic panel comprises:

3. The photovoltaic panel defect detection method based on the dual-scale strategy and the improved YOLOV5 network as claimed in claim 1, wherein the step of filtering the screened sample image by Otsu binarization to obtain the filtered sample image and removing black background noise of the photovoltaic panel region comprises:

carrying out gray processing on the screened sample image, obtaining a threshold value on the gray image by using an Otsu algorithm, and carrying out image threshold value binaryzation by using the threshold value;

4. The method for detecting defects of a photovoltaic panel based on a double-scale strategy and an improved YOLOV5 network as claimed in claim 1, wherein the step of constructing a training sample set and a verification set based on the filtered sample images and the labeled sample images comprises:

and sending the expanded data set into a network model for training.

5. The method of claim 1, wherein the MET _ YOLOV5 network model structure comprises: input end: the method comprises the following steps of Mosaic data enhancement, self-adaptive anchor frame calculation and self-adaptive picture scaling;

backbone network: CSPNet, neck: SPP, FPN, PAN;

6. The method of claim 5 for detecting defects in photovoltaic panels based on a dual-scale strategy and a modified YOLOV5 network,

MET _ YOLOV5 adds a Swin-Transformer module to a C3 module in a backbone network CSPNet to carry out targeted optimization and improvement on a YooloV 5 network, wherein the improved C3 module is called C3STR, a self-attention structure is used for enhancing semantic information and feature extraction capability of a small target, cross-window information interaction is realized in a manner of dividing local windows, and the calculated amount is reduced;

adding an ECA attention module at the tail end of the neck SPP layer of the MET _ YOLOV5, and performing average pooling on all characteristic channels under the condition of not reducing dimension;

the ECA module captures information among different channels by using one-dimensional convolution, multiplies the channel attention characteristic diagram and the input characteristic diagram channel by channel and outputs the multiplied channel attention characteristic diagram and input characteristic diagram;

the ECA module can optimize the characteristic diagram, so that the network can pay attention to hidden crack defects of photovoltaic panels with different sizes and shapes.

7. The method for detecting defects of a photovoltaic panel based on a double-scale strategy and an improved YOLOV5 network as claimed in claim 1, wherein the step of predicting the collected EL image of the photovoltaic panel by using a MET _ YOLOV5 model after passing the verification to obtain a double-scale prediction result comprises:

8. The method for detecting defects of a photovoltaic panel based on a dual-scale strategy and an improved YOLOV5 network as claimed in claim 1, wherein the step of fusing F2 with the full-scale prediction result F1 to obtain F comprises:

for any one prediction frame F1 _i And a prediction box F2 _j Judging whether the prediction frames are the same defect or not by calculating the intersection ratio IoU, wherein the IoU measures the relative overlapping size of the two boundary frames; for F1 _i And F2 _j The calculation formula is as follows:

Px＝[max(Pc _i ,Pc _j ),Px]

Pc _i and Pc _j Respectively represent prediction frames F1 _i And a prediction box F2 _j The confidence Pc, the prediction result set F obtained by fusion contains m prediction frames obtained by fusion, and a single prediction frame is represented as F _m Each prediction box information includes: the position Px of the prediction frame and the confidence Pc, i and j of the prediction frame represent the sequence numbers in the prediction result set to which the prediction frame belongs.

9. The method of photovoltaic panel defect detection based on a dual-scale strategy and a modified YOLOV5 network of claim 8, wherein the method further comprises: for the small-scale prediction result, merging the cross-grid defects by adopting a fusion algorithm based on a neighbor relation; fusing the prediction result by adopting a dual-scale image fusion strategy algorithm;

prediction result F2 of each grid in small-scale prediction result F2 _j Calculate its prediction result F2 in the grid adjacent to it _j+1 Filtering by using a threshold judgment method (the shortest distance between the boundary points of any two prediction frames is less than 1/4 of the average value of the longest diagonal), and merging the prediction frames meeting the requirements; for is toIn the merged prediction box F2 _j ' setting the position Px of a prediction frame as the minimum circumscribed rectangle of the union set of the two prediction frames; and calculating the joint probability to obtain the final confidence Pc of the prediction frame, wherein the calculation formula is as follows:

10. The method for detecting defects of a photovoltaic panel based on a double-scale strategy and an improved YOLOV5 network as claimed in claim 1, wherein the step of verifying the trained MET _ YOLOV5 network model by using the verification set includes:

and setting the number of images and the iteration times of each training, adjusting the proportion of the test set and the verification set to carry out multiple times of verification, and selecting an optimal weight parameter file to predict defects.