CN114283137A - Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network - Google Patents
Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network Download PDFInfo
- Publication number
- CN114283137A CN114283137A CN202111596547.9A CN202111596547A CN114283137A CN 114283137 A CN114283137 A CN 114283137A CN 202111596547 A CN202111596547 A CN 202111596547A CN 114283137 A CN114283137 A CN 114283137A
- Authority
- CN
- China
- Prior art keywords
- hot spot
- spot defect
- detector
- network
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a photovoltaic module hot spot defect detection method based on a multi-scale characteristic diagram inference network, which is based on an improved YOLOv4 neural network model, adds a self-designed multi-scale characteristic diagram inference module suitable for photovoltaic power station hot spot defect detection in a characteristic fusion part in an original model, guides multi-scale characteristic fusion and highlights complex background characteristics in a defect position area inhibition image, and effectively improves the identification capability of a photovoltaic module hot spot defect. The detection method combines the deep learning technology and the image processing technology, so that the inefficiency and uncertainty of the traditional manual feature extraction are avoided, meanwhile, the detection process has stronger robustness, the detection precision is obviously improved, and the detection speed is improved.
Description
Technical Field
The invention relates to the field of defect detection, in particular to a visual detection method for hot spot defects of a photovoltaic power station, and specifically relates to a photovoltaic module hot spot defect detection method based on a multi-scale characteristic diagram inference network.
Background
The photovoltaic power generation system is widely applied due to the advantages of no pollution, reproducibility, resource universality, flexibility, good storage property, safety, reliability and the like. With the continuous expansion of the scale of the photovoltaic power station, the photovoltaic power generation system has wide application places, including large-scale ground photovoltaic power stations, roofs of houses and commercial buildings, photovoltaic street lamps and the like. In these places, the defects of blocking and hot spot formation of photovoltaic components caused by buildings, tree shadows, chimneys, dust, clouds and the like are inevitable. The generation of hot spot defects can reduce the efficiency of the photovoltaic system, shorten the service life of the photovoltaic module and even cause fire. Therefore, the hot spot defect can be identified timely and accurately, and the method is very important for improving the operation safety of the photovoltaic power plant and reducing the downtime.
Because photovoltaic power plant is located the scene that the topography is complicated, the environment is changeable mostly, the infrared image of hot spot who catches inevitably can receive illumination intensity, shoot the influence of angle change, complicated topography. And the hot spot defect size is small, it is challenging to train a robust artificial intelligence model in such a highly open environment. A deep learning network is designed and applied to real-time detection of hot spot defects of a solar photovoltaic module, so that safe and stable operation of a photovoltaic power station is guaranteed.
Disclosure of Invention
The problem of poor robustness of hot spot defect detectors of infrared images of different heights of a photovoltaic power station is solved. The invention provides a photovoltaic module hot spot defect detection method based on a multi-scale characteristic diagram inference network. The detection method designs a novel cross embedding mapping mechanism, and maps different scale features in a coordinate space into a graph space. Then, the cross embedded graph reasoning module is utilized to combine the strongest semantic information and rich spatial information, and meanwhile, the interaction of different scales and the reasoning of global information are completed. The model integrates spatial features and semantic features to obtain global relevant information. By adopting a multi-scale feature fusion and global reasoning strategy, common information of the depth convolution features is fully mined, and the method has remarkable robustness on photovoltaic infrared image data sets with different heights.
The technical scheme adopted by the invention for solving the problems is that a photovoltaic module hot spot defect detection method based on a multi-scale characteristic diagram reasoning network is designed, and the method is characterized by comprising the following specific steps:
first step acquisition of a data set
1.1 original image acquisition: carrying an infrared camera on the unmanned aerial vehicle, and shooting photos of the front side of the photovoltaic module when the unmanned aerial vehicle flies at different heights through the infrared camera;
1.2 data set preparation: carrying out size normalization processing on the picture obtained in the step 1.1, and screening out an image containing the hot spot defect to obtain a hot spot defect image library;
1.3 manually labeling the hot spot defect type label of the defect area of each image in the hot spot defect image library by using LabelImg to obtain a hot spot defect image data set; randomly extracting a hot spot defect image data set, taking not less than 60% of the hot spot defect image data set as a training sample set, and taking the rest part of the hot spot defect image data set as a verification sample set;
the second step is that: training of multi-scale feature inference network detectors
2.1 training sample set Pre-processing
Preprocessing a training sample set in a Mosaic data enhancement mode;
2.2 parameter settings
Initializing all weight values, bias values and batch normalization scale factor values, setting the initial learning rate and batch _ size of the detector, inputting initialized parameter data into the detector, and setting the initial learning rate of the detector to be 0.001;
2.3 Detector training
Inputting the preprocessed training sample set into a detector with set initialization parameters, wherein the detector is obtained by replacing a PANet module in a Yolov4 network model with a multi-scale feature map inference network module; firstly, extracting features and combining a multi-scale feature map inference network module to perform multi-scale fusion, then refining and fusing the feature maps obtained by extracting the last three layers by a convolution layer to reduce dimensionality, and predicting tensors by using a classification regression network to obtain predicted values of positions, defect types and confidence coefficients; comparing the generated predicted value with the marking information to generate a loss value, then performing reverse propagation, updating parameters of a backbone network and a classification regression network until the loss value accords with the preset value, and finishing the training of the detector parameters;
2.4 Detector testing
Inputting the verification sample set into the detector which completes parameter training in the step 2.3 to obtain a tensor prediction value of the verification sample set; comparing the tensor predicted value with the labeling information, verifying the reliability of the detector, and monitoring whether the detector is over-fitted or not so as to determine whether training needs to be stopped and parameters need to be readjusted or not; when the hot spot defect prediction accuracy rates of different heights in the sample set are verified to be more than 80%, the detector is verified to be reliable;
the third step: photovoltaic module hot spot defect detection
And (3) carrying out the same size normalization operation in the step (1.2) in the first step on the photo of the front side of the photovoltaic module to be detected shot by the infrared camera, and then inputting the photo into the detector verified to be reliable in the step (II) to obtain hotspot defect tensor information of the infrared image of the photovoltaic module to be detected, wherein the hotspot defect tensor information comprises a defect position, a defect type and a confidence coefficient.
Compared with the prior art, the invention has the beneficial effects that: the detection method is based on an improved YOLO v4 neural network model, a self-designed multi-scale feature map inference module suitable for detecting the hot spot defect of the photovoltaic power station is added to a feature fusion part in an original model, multi-scale feature fusion is guided, the complex background feature in a defect position area inhibition image is highlighted, and the identification capability of the hot spot defect of the photovoltaic module is effectively improved. The detection method combines the deep learning technology and the image processing technology, thereby not only avoiding the low efficiency and uncertainty of the traditional manual feature extraction, but also having stronger robustness in the detection process; the method adopts an original YOLOv4 network model to detect the hot spot defect of the photovoltaic assembly, the average recognition rate of the hot spot defect on an image shot at the height of 28 meters is 86.83%, and when the initial learning rate is 0.001, the average accuracy of the model reaches 91.08%, the detection precision is obviously improved, and the detection speed is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of an embodiment of the detection method of the present invention.
Fig. 2 is a schematic diagram of the structure and principle of a multi-scale feature inference network detector according to an embodiment of the detection method of the present invention.
FIG. 3 is a schematic diagram of the structure and principle of a cross-embedded graph inference module of a multi-scale feature graph inference network module of a multi-scale feature inference network detector according to an embodiment of the detection method of the present invention.
Fig. 4 is an infrared image of a photovoltaic module to be inspected.
FIG. 5 is a graph showing the results of the detection of the image of FIG. 4 using the detection method of the present invention.
Detailed Description
The present invention will be further explained with reference to the following examples and drawings.
The invention provides a photovoltaic module hot spot defect detection method (detection method for short) based on a multi-scale characteristic diagram inference network, which is characterized by comprising the following specific steps:
first step acquisition of a data set
1.1 original image acquisition: carrying an infrared camera on the unmanned aerial vehicle, and shooting photos of the front side of the photovoltaic module when the unmanned aerial vehicle flies at different heights through the infrared camera;
1.2 data set preparation: carrying out size normalization processing on the picture obtained in the step 1.1, and screening out an image containing the hot spot defect to obtain a hot spot defect image library;
1.3 manually labeling the hot spot defect type label of the defect area of each image in the hot spot defect image library by using LabelImg to obtain a hot spot defect image data set; and randomly extracting the hot spot defect image data set, taking not less than 60% of the hot spot defect image data set as a training sample set, and taking the rest part of the hot spot defect image data set as a verification sample set.
The second step is that: training of multiscale feature inference network detectors (detectors for short)
2.1 training sample set Pre-processing
Preprocessing a training sample set in a Mosaic data enhancement mode;
2.2 parameter settings
Initializing all weight values, bias values and batch normalization scale factor values, setting the initial learning rate and batch _ size of the detector, inputting initialized parameter data into the detector, and setting the initial learning rate of the detector to be 0.001;
2.3 Detector training
Inputting the preprocessed training sample set into a detector with set initialization parameters, wherein the detector is obtained by replacing a PANet module (multi-scale fusion module) in a Yolov4 network model with a multi-scale feature map inference network module; the method comprises the steps that a preprocessed training sample set is arranged in a detector, firstly, feature extraction is carried out, multi-scale fusion is carried out by combining a multi-scale feature map inference network module, then, feature maps obtained by the last three layers of extraction are subjected to thinning fusion by a convolution layer to reduce dimensionality, and a position, a defect type and a confidence coefficient predicted value are obtained by utilizing a classification regression network prediction tensor. And comparing the generated predicted value with the labeling information to generate a loss value, then performing reverse propagation, updating parameters of the backbone network and the classification regression network until the loss value is in accordance with the preset value, and finishing the training of the detector parameters.
2.4 Detector testing
Inputting the verification sample set into the detector which completes parameter training in the step 2.3 to obtain a tensor prediction value of the verification sample set; comparing the tensor predicted value with the labeling information, verifying the reliability of the detector, and monitoring whether the detector is over-fitted or not so as to determine whether training needs to be stopped and parameters need to be readjusted or not; when the hot spot defect prediction accuracy rates of different heights in the sample set are verified to be more than 80%, the detector is verified to be reliable.
The third step: photovoltaic module hot spot defect detection
And (3) carrying out the same size normalization operation in the step (1.2) in the first step on the photo of the front side of the photovoltaic module to be detected shot by the infrared camera, and then inputting the photo into the detector verified to be reliable in the step (II) to obtain hotspot defect tensor information of the infrared image of the photovoltaic module to be detected, wherein the hotspot defect tensor information comprises a defect position, a defect type and a confidence coefficient.
The multi-scale characteristic diagram reasoning network module working principle is as follows: the goal of the multi-scale feature map inference network module is to find a transfer function that can efficiently aggregate rich spatial and semantic features and infer global information. Formally given a backbone network multiscale feature list (C3, C4, C5), the definition of the feature fusion and inference process is:
P5＝dbl(spp(dbl(C3))) (1)
P4＝G(dbl(C4),up(dbl(P5))) (2)
P3＝G(dbl(C3),up(dbl(P5))) (3)
where DBL denotes the DBL (two-dimensional convolution contribution batch normalization-leakage relu, two-dimensional convolution operation-normalization-nonlinear activation function) operation, and spp denotes the spatial pyramid pool operation. up represents the upsampling operation and G (-) is the operation of the cross-embedding graph inference module.
The cross embedded graph reasoning module works according to the principle that: the cross embedded graph reasoning module aims at realizing complementary fusion and global relationship reasoning of semantic features and spatial features, and comprises the following specific processes:
first of all from the coordinate space will have detailed spatial features
And features of strong semantic information
And the data are interleaved from a coordinate space omega into the same graph space H, wherein L is W multiplied by H, W, H and C are the width, height and channel number of the feature graph respectively. Thereby obtaining a joint distribution f (X) of two scales1、X2) And (V, E) a structural diagram G ═ X fused to the structural diagram1And X2Two features, where the number of nodes V and the edge E representing the node and node relationship are defined.
Converting the features in the structural graph G ═ V, E into features in a new coordinate space, determining corresponding feature nodes by the learnable mapping weights, and generating each node in the nodes V of the features in the new coordinate space by the following steps:
with a learnable mapping matrix of
And
B1and B2Are each formed by X1And X2Through the training, the training can obtain the training result,
and
individual watchShow B1And B2The line vectors of (a) are,
and
are respectively row vectors
And
the jth element in (a); n is the dimension of the mapped node, equal to C/2. Wherein the content of the first and second substances,
are respectively an input feature X1And X2The column vector of (2). Two features X can be obtained by operation1And X2The ith node of
And
the resulting graph structure can update the relationship of nodes and edges by defining a single-layer graph convolution network GCN (-). In the definition of graph convolution, the way of updating the node graph is shown in formula (6), where AgThe adjacency matrix representing the edge relation is initialized randomly, and the learning weight W is trained through gradient descent during traininggAnd I represents a unit matrix.
Is a new node feature matrix in coordinate space, the convolution of the graph is expressed as:
Z＝((I-Ag)V)Wg (6)
for more convenient embedding of graph convolution network, a graph updating process implemented by one-dimensional convolution operation Conv1D (-) is designed as shown in formula (7):
Z＝GCN(V)＝Conv1D(Conv1D(V)T)T (7)
and finally, mapping the output characteristics of the single-layer graph convolution network GCN back to the original coordinate space, and obtaining a node characteristic matrix Z through updating, wherein the aim is to establish a mapping function to map the updated characteristics into the characteristics Y of the coordinate space:
Y＝g(Z) (8)
similar to the map space mapping function, the mapping relationship is implemented using a linear mapping g (-) as equation (9), where the inverse mapping matrix D is as equation (10):
wherein, d is usediA row vector, D, representing the inverse mapping matrix DijIs a row vector diThe jth element in (a). z is a radical ofjIs the column vector, y, of the node feature matrix ZiIs a row vector of feature Y mapped to coordinate space. According to the matrix operation, an inverse mapping relation is constructed by using an equation. After mapping the information back to the coordinate space, adding residual connection operation to improve the universality of the module, and finally outputting O by the cross embedding graph reasoning module as follows:
O＝Y+X1 (11)
example 1
The embodiment provides a photovoltaic module hot spot defect detection method based on a multi-scale characteristic diagram inference network, which is characterized by comprising the following specific steps:
first step acquisition of a data set
1.1 original image acquisition: carrying an infrared camera on the unmanned aerial vehicle, and shooting photos of the front side of the photovoltaic module when the unmanned aerial vehicle flies at different heights through the infrared camera; this example captures infrared images at four heights of 28 meters, 35 meters, 47 meters, and 60 meters.
1.2 data set preparation: carrying out size normalization processing on the photo obtained in the step 1.1, wherein the size of the infrared image is 640 multiplied by 512, 1500 defect images are selected at each height, and a hot spot defect image library with the number of 6000 is obtained;
1.3 manually labeling the hot spot defect type label of the defect area of each image in the hot spot defect image library by using LabelImg to obtain a hot spot defect image data set; and randomly extracting the hot spot defect image data set, wherein 80% of the hot spot defect image data set is used as a training sample set, and the rest 20% of the hot spot defect image data set is used as a verification sample set.
The second step is that: training of multi-scale feature inference network detectors
2.1 training sample set Pre-processing
Preprocessing a training sample set in a Mosaic data enhancement mode;
the Mosaic data enhancement adopts the modes of random zooming, random cutting and random arrangement to splice the images, and the detection effect on small targets is good. And then, adjusting the resolution of the original input image, and adaptively scaling the picture according to the different aspect ratio of the picture.
2.2 parameter settings
Initializing all weight values, bias values and batch normalization scale factor values, setting the initial learning rate and batch _ size of the detector, inputting initialized parameter data into the detector, and setting the initial learning rate of the detector to be 0.001;
the detector parameters are initialized as follows: the maximum number of iterations (epoch) is set to 200, the first 50 epoch learning rates are set to 0.001, the last 150 epoch learning rates are set to 0.0001, the descent factor of the learning rate is 0.1, and the weight decay of the regularization term is 0.0005.
2.3 Detector training
Inputting the preprocessed training sample set into a detector with set initialization parameters, wherein the detector is obtained by replacing a PANet module (multi-scale fusion module) in a Yolov4 network model with a multi-scale feature map inference network module; the method comprises the steps that a preprocessed training sample set is arranged in a detector, firstly, feature extraction is carried out, multi-scale fusion is carried out by combining a multi-scale feature map inference network module, then, feature maps obtained by the last three layers of extraction are subjected to thinning fusion by a convolution layer to reduce dimensionality, and a position, a defect type and a confidence coefficient predicted value are obtained by utilizing a classification regression network prediction tensor. And comparing the generated predicted value with the labeling information to generate a loss value, then performing reverse propagation, updating parameters of the backbone network and the classification regression network until the loss value is in accordance with the preset value, and finishing the training of the detector parameters.
2.4 Detector testing
Inputting the verification sample set into the detector which completes parameter training in the step 2.3 to obtain a tensor prediction value of the verification sample set; and comparing the tensor predicted value with the labeling information to obtain hot spot defects with different heights, wherein the accuracy rate of the hot spot defects is more than 85%.
The third step: photovoltaic module hot spot defect detection
And (3) carrying out the same size normalization operation in the step (1.2) in the first step on the photo of the front side of the photovoltaic module to be detected shot by the infrared camera, and then inputting the photo into the detector verified to be reliable in the step (II) to obtain hotspot defect tensor information of the infrared image of the photovoltaic module to be detected, wherein the hotspot defect tensor information comprises a defect position, a defect type and a confidence coefficient.
The multiscale feature inference network detector adopts an improved Yolov4 neural network model, has high detection precision and high identification speed: specifically, the detector model structure is mainly composed of a data enhancement part, a feature extraction part, a feature fusion part and a classification regression part, as shown in fig. 2. The data enhancement part adopts the Mosaic data enhancement, splices the infrared image of the photovoltaic power station through the modes of random zooming, random cutting and random arrangement, and improves the detection capability of small targets. In the aspect of feature extraction, the infrared image of the photovoltaic power station is input into a CSPDarknet53 backbone network, a feature map is extracted from the infrared image through operations such as convolution, pooling and the like, and the feature map can be shared for subsequent feature fusion. And a multi-scale characteristic diagram reasoning network module is adopted in the characteristic fusion part to generate pooling characteristic vectors with different fixed sizes, so that the expression capability of the characteristics is enhanced, and the method has a good effect on the detection of the same object in different sizes. The classification regression part uses GIOU _ Loss as a Loss function of the Bounding box, effectively solves the problem of non-coincidence of the boundary frames, and improves the speed and the precision of the regression of the prediction frame.
The feature extraction stage of the detector follows the CSPDarknet53 structure of a Yolov4 neural network model, and CSPDarknet53 is added with a CSP module (cross-stage local module) on the basis of Darket53, wherein the CSP module consists of a convolutional layer and a residual structure Concat in a Resnet network. Darket53 totals 53 layers of convolutions, with a total of 52 convolutions removed from the last FC (full connectivity layer, actually achieved by a 1x1 convolution) for use as the host network. The 52 convolutional layers are composed of: the method comprises the steps of firstly performing convolution kernels of 1 32 filters, then performing 5 repeated residual unit resblock _ body (each unit of the 5 residual units is composed of 1 independent convolution layer and a group of repeatedly executed convolution layers, and the repeatedly executed convolution layers are respectively repeated for 1 time, 2 times, 8 times and 4 times, wherein in each repeatedly executed convolution layer, the convolution operation of 1x1 is performed firstly, then the convolution operation of 3x3 is performed, the number of the filters is reduced by half and then recovered, and the total number of the filters is 52.
The multi-scale feature map inference network module integrates strong semantic features into spatial features using a cross-embedding map inference module. The multi-scale feature map inference network module can reserve the optimal space detail features of the small defects and introduce strong semantic information to realize global context inference. In the multi-scale feature map inference network, a cross embedded map inference module plays a key link in feature refinement. The multi-scale feature map inference network combines sufficient spatial and semantic information of small defects in the simplest and most effective mode, and eliminates unnecessary information fusion. Through the integration and global reasoning of the strongest semantic features and the spatial features of small defects, the performance of the multi-scale feature map reasoning network in the aspect of hot spot defect detection is superior to that of the PANet. To avoid inefficient network connections to improve the utilization of features, the small size characteristic of hot spot defects was analyzed. And integrating and deducing the strongest semantic information by using a cross-embedded graph reasoning module. The multi-scale feature map inference network effectively refines feature distribution and utilizes global features to enhance the robustness of the network.
The test result of the invention shows that the improved Yolov4 model is applied to the hot spot defect detection of the photovoltaic power station, and the quick intelligent inspection of the photovoltaic power station can be realized.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Nothing in this specification is said to apply to the prior art.
Claims (4)
1. The photovoltaic module hot spot defect detection method based on the multi-scale characteristic diagram inference network is characterized by comprising the following specific steps:
first step acquisition of a data set
1.1 original image acquisition: carrying an infrared camera on the unmanned aerial vehicle, and shooting photos of the front side of the photovoltaic module when the unmanned aerial vehicle flies at different heights through the infrared camera;
1.2 data set preparation: carrying out size normalization processing on the picture obtained in the step 1.1, and screening out an image containing the hot spot defect to obtain a hot spot defect image library;
1.3 manually labeling the hot spot defect type label of the defect area of each image in the hot spot defect image library by using LabelImg to obtain a hot spot defect image data set; randomly extracting a hot spot defect image data set, taking not less than 60% of the hot spot defect image data set as a training sample set, and taking the rest part of the hot spot defect image data set as a verification sample set;
the second step is that: training of multi-scale feature inference network detectors
2.1 training sample set Pre-processing
Preprocessing a training sample set in a Mosaic data enhancement mode;
2.2 parameter settings
Initializing all weight values, bias values and batch normalization scale factor values, setting the initial learning rate and batch _ size of the detector, inputting initialized parameter data into the detector, and setting the initial learning rate of the detector to be 0.001;
2.3 Detector training
Inputting the preprocessed training sample set into a detector with set initialization parameters, wherein the detector is obtained by replacing a PANet module in a Yolov4 network model with a multi-scale feature map inference network module; firstly, extracting features and combining a multi-scale feature map inference network module to perform multi-scale fusion, then refining and fusing the feature maps obtained by extracting the last three layers by a convolution layer to reduce dimensionality, and predicting tensors by using a classification regression network to obtain predicted values of positions, defect types and confidence coefficients; comparing the generated predicted value with the marking information to generate a loss value, then performing reverse propagation, updating parameters of a backbone network and a classification regression network until the loss value accords with the preset value, and finishing the training of the detector parameters;
2.4 Detector testing
Inputting the verification sample set into the detector which completes parameter training in the step 2.3 to obtain a tensor prediction value of the verification sample set; comparing the tensor predicted value with the labeling information, verifying the reliability of the detector, and monitoring whether the detector is over-fitted or not so as to determine whether training needs to be stopped and parameters need to be readjusted or not; when the hot spot defect prediction accuracy rates of different heights in the sample set are verified to be more than 80%, the detector is verified to be reliable;
the third step: photovoltaic module hot spot defect detection
Performing the same size normalization operation in the step 1.2 in the first step on the photo of the front side of the photovoltaic module to be detected shot by the infrared camera, and then inputting the photo into the detector verified as reliable in the step two to obtain hotspot defect tensor information of the infrared image of the photovoltaic module to be detected, wherein the hotspot defect tensor information comprises a defect position, a defect type and a confidence coefficient;
the multi-scale characteristic diagram reasoning network module working principle is as follows: the multi-scale feature map inference network module aims at finding a conversion function which can effectively aggregate abundant space and semantic features and infer global information; formally given a backbone network multiscale feature list (C3, C4, C5), the definition of the feature fusion and inference process is:
P5＝dbl(spp(dbl(C3))) (1)
P4＝G(dbl(C4),up(dbl(P5))) (2)
P3＝G(dbl(C3),up(dbl(P5))) (3)
wherein DBL represents DBL operation and spp represents spatial pyramid pool operation; up represents the upsampling operation, and G (-) is the operation of the cross-embedding graph inference module;
the cross embedded graph reasoning module works according to the principle that: the cross embedded graph reasoning module aims at realizing complementary fusion and global relationship reasoning of semantic features and spatial features, and comprises the following specific processes:
first of all from the coordinate space will have detailed spatial features
And features of strong semantic information
Embedding the feature map into the same map space H from a coordinate space omega in a crossed manner, wherein L is W multiplied by H, W, H and C are the width, height and channel number of the feature map respectively; thereby obtaining a joint distribution f (X) of two scales1、X2) And (V, E) a structural diagram G ═ X fused to the structural diagram1And X2Two features, where the number of nodes V and the edge E representing the node and node relationship are defined;
converting the features in the structural graph G ═ V, E into features in a new coordinate space, determining corresponding feature nodes by the learnable mapping weights, and generating each node in the nodes V of the features in the new coordinate space by the following steps:
with a learnable mapping matrix of
And
B1and B2Are each formed by X1And X2Through the training, the training can obtain the training result,
and
respectively represent B1And B2The line vectors of (a) are,
and
are respectively row vectors
And
the jth element in (a); n is the dimension of the mapped node, equal to C/2; wherein the content of the first and second substances,
are respectively an input feature X1And X2A column vector of (a); two features X can be obtained by operation1And X2The ith node of
And
the obtained graph structure updates the relationship between nodes and edges by defining a single-layer graph convolution network GCN (-); in the definition of graph convolution, the way of updating the node graph is shown in formula (6), where AgThe adjacency matrix representing the edge relation is initialized randomly, and the learning weight W is trained through gradient descent during traininggI represents a unit array;
is a new node feature matrix in coordinate space, the convolution of the graph is expressed as:
Z＝((I-Ag)V)Wg (6)
for more convenient embedding of graph convolution network, a graph updating process implemented by one-dimensional convolution operation Conv1D (-) is designed as shown in formula (7):
Z＝GCN(V)＝Conv1D(Conv1D(V)T)T (7)
and finally, mapping the output characteristics of the single-layer graph convolution network GCN back to the original coordinate space, and obtaining a node characteristic matrix Z through updating, wherein the aim is to establish a mapping function to map the updated characteristics into the characteristics Y of the coordinate space:
Y＝g(Z) (8)
similar to the map space mapping function, the mapping relationship is implemented using a linear mapping g (-) as equation (9), where the inverse mapping matrix D is as equation (10):
wherein, d is usediA row vector, D, representing the inverse mapping matrix DijIs a row vector diThe jth element in (a); z is a radical ofjIs the column vector, y, of the node feature matrix ZiA row vector that is a feature Y mapped to a coordinate space; constructing an inverse mapping relation by using an equation according to matrix operation; after mapping the information back to the coordinate space, adding residual connection operation to improve the universality of the module, and finally outputting O by the cross embedding graph reasoning module as follows:
O＝Y+X1 (11)
2. the photovoltaic module hot spot defect detection method based on the multi-scale feature map inference network of claim 1, wherein in the first step, photos of the front side of the photovoltaic module at four different flying heights of 28 meters, 35 meters, 47 meters and 60 meters of the unmanned aerial vehicle are shot through an infrared camera.
3. The photovoltaic module hot spot defect detection method based on the multi-scale feature map inference network of claim 1, wherein in the first step, the number of defect images in the hot spot defect image library is 6000.
4. The photovoltaic module hot spot defect detection method based on the multi-scale feature map inference network as claimed in claim 1, wherein in the first step, a hot spot defect image data set is randomly extracted, 80% is used as a training sample set, and the remaining 20% is used as a verification sample set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111596547.9A CN114283137A (en) | 2021-12-24 | 2021-12-24 | Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111596547.9A CN114283137A (en) | 2021-12-24 | 2021-12-24 | Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114283137A true CN114283137A (en) | 2022-04-05 |
Family
ID=80875494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111596547.9A Pending CN114283137A (en) | 2021-12-24 | 2021-12-24 | Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114283137A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116523888A (en) * | 2023-05-08 | 2023-08-01 | 北京天鼎殊同科技有限公司 | Pavement crack detection method, device, equipment and medium |
| CN116958713A (en) * | 2023-09-20 | 2023-10-27 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN117274616A (en) * | 2023-09-26 | 2023-12-22 | 南京信息工程大学 | Multi-feature fusion deep learning service QoS prediction system and prediction method |
-
2021
- 2021-12-24 CN CN202111596547.9A patent/CN114283137A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116523888A (en) * | 2023-05-08 | 2023-08-01 | 北京天鼎殊同科技有限公司 | Pavement crack detection method, device, equipment and medium |
| CN116523888B (en) * | 2023-05-08 | 2023-11-03 | 北京天鼎殊同科技有限公司 | Pavement crack detection method, device, equipment and medium |
| CN116958713A (en) * | 2023-09-20 | 2023-10-27 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN116958713B (en) * | 2023-09-20 | 2023-12-15 | 中航西安飞机工业集团股份有限公司 | Quick recognition and statistics method and system for surface fastener of aviation part |
| CN117274616A (en) * | 2023-09-26 | 2023-12-22 | 南京信息工程大学 | Multi-feature fusion deep learning service QoS prediction system and prediction method |
| CN117274616B (en) * | 2023-09-26 | 2024-03-29 | 南京信息工程大学 | Multi-feature fusion deep learning service QoS prediction system and prediction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241982B (en) | Target detection method based on deep and shallow layer convolutional neural network | |
| CN113538391B (en) | Photovoltaic defect detection method based on Yolov4 and thermal infrared image | |
| WO2022111219A1 (en) | Domain adaptation device operation and maintenance system and method | |
| CN114283137A (en) | Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| CN113076871B (en) | Fish shoal automatic detection method based on target shielding compensation | |
| CN109670405B (en) | Complex background pedestrian detection method based on deep learning | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN115331087B (en) | Remote sensing image change detection method and system fusing regional semantics and pixel characteristics | |
| CN109766873B (en) | Pedestrian re-identification method based on hybrid deformable convolution | |
| CN113420607A (en) | Multi-scale target detection and identification method for unmanned aerial vehicle | |
| CN112837315A (en) | Transmission line insulator defect detection method based on deep learning | |
| CN109635726B (en) | Landslide identification method based on combination of symmetric deep network and multi-scale pooling | |
| CN111178177A (en) | Cucumber disease identification method based on convolutional neural network | |
| CN113326735B (en) | YOLOv 5-based multi-mode small target detection method | |
| CN114973032A (en) | Photovoltaic panel hot spot detection method and device based on deep convolutional neural network | |
| CN114972312A (en) | Improved insulator defect detection method based on YOLOv4-Tiny | |
| CN115984543A (en) | Target detection algorithm based on infrared and visible light images | |
| CN115240069A (en) | Real-time obstacle detection method in full-fog scene | |
| CN115410087A (en) | Transmission line foreign matter detection method based on improved YOLOv4 | |
| CN115908354A (en) | Photovoltaic panel defect detection method based on double-scale strategy and improved YOLOV5 network | |
| Chen et al. | Object detection using deep learning: Single shot detector with a refined feature-fusion structure | |
| CN114897857A (en) | Solar cell defect detection method based on light neural network | |
| CN111461006A (en) | Optical remote sensing image tower position detection method based on deep migration learning | |
| CN116778346B (en) | Pipeline identification method and system based on improved self-attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |