CN117788436A - Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting - Google Patents
Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting Download PDFInfo
- Publication number
- CN117788436A CN117788436A CN202311841086.6A CN202311841086A CN117788436A CN 117788436 A CN117788436 A CN 117788436A CN 202311841086 A CN202311841086 A CN 202311841086A CN 117788436 A CN117788436 A CN 117788436A
- Authority
- CN
- China
- Prior art keywords
- rust
- detection
- defect
- module
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 88
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000001514 detection method Methods 0.000 claims abstract description 115
- 230000005540 biological transmission Effects 0.000 claims abstract description 30
- 230000001502 supplementing effect Effects 0.000 claims abstract description 6
- 230000004927 fusion Effects 0.000 claims description 24
- 238000000605 extraction Methods 0.000 claims description 22
- 230000008859 change Effects 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 17
- 238000007906 compression Methods 0.000 claims description 13
- 230000006835 compression Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 7
- 238000001228 spectrum Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 claims description 4
- 230000002040 relaxant effect Effects 0.000 claims description 4
- 241000282326 Felis catus Species 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000003993 interaction Effects 0.000 abstract description 4
- 238000012423 maintenance Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 9
- 230000007797 corrosion Effects 0.000 description 6
- 238000005260 corrosion Methods 0.000 description 6
- 238000012935 Averaging Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011897 real-time detection Methods 0.000 description 2
- 241000073677 Changea Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000011253 protective coating Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Landscapes
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a method, a system and a medium for detecting rust defects of a line hardware fitting based on cloud edge cooperation, and particularly relates to the technical field of defect detection; inputting the defect sample into an improved YOLOv8m model to train a new structure detection and identification model; updating the detection recognition model by using the new structure detection recognition model, and detecting rust defects of the line hardware fitting; aiming at the problems that the rust shape is changeable, the characteristics are not obvious, and key information is easy to lose in a deep network, the cloud edge cooperative technology is combined to provide a power transmission line hardware rust defect detection method which is used for supplementing rust characteristics, reducing information loss, improving the detection precision of rust, realizing the data interaction of an intelligent terminal and an edge computing device, and realizing the integrated operation and maintenance service of the cloud platform from the data acquisition of the power transmission equipment side.
Description
Technical Field
The invention relates to the technical field of defect detection, in particular to a method, a system and a medium for detecting rust defects of line hardware fittings based on cloud edge cooperation.
Background
The power transmission line hardware fitting is an important component of a power transmission line and has the main functions of connecting various devices, transmitting electric loads and the like. Because the transmission line hardware works outdoors for a long time, the protective coating is easy to fail and rust. The rust problem can directly affect the safe and stable operation of the power transmission line, so that the rust detection method has important significance for rust detection of the power transmission line hardware.
The traditional monitoring technology is to perform offline processing on equipment operation data acquired by unmanned aerial vehicle equipment and the like, so that great challenges exist in data storage and transmission, and real-time interaction cannot be completed. With the rise of technologies such as cloud computing and artificial intelligence, the cloud edge cooperative technology can be utilized to complete data acquisition, sensor data analysis and edge computing functions, and state evaluation is directly carried out on the power transmission equipment side to achieve. The data collected by the intelligent terminal at the power transmission equipment side is directly accessed to an edge computing device, and the edge computing device completes data processing according to a related model synchronized from the cloud platform, so that state evaluation, fault diagnosis and reasoning decision of the edge side are realized.
The rust has irregular and changeable shape, so the rust has unobvious characteristics; and secondly, the background of the electric power scene image is complex, the corrosion significance is low, key information is easy to lose in the downsampling process of feature fusion, and network detection is difficult.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: in the transmission line hardware rust defect detection process, rust features are not obvious due to irregular and changeable rust shapes; secondly, the background of the electric power scene image is complex, the corrosion significance is low, key information is easy to lose in the downsampling process of feature fusion, and network detection is difficult; the invention aims to provide a method, a system and a medium for detecting rust defects of a power transmission line fitting, which aim at the problems that the rust shape is changeable, the characteristics are not obvious, and key information is easy to lose in a deep network, and a cloud edge cooperative technology is combined to provide the method for detecting the rust defects of the power transmission line fitting, a frequency attention mechanism and an additional detection head are introduced to supplement rust characteristics, reduce information loss, improve the detection precision of rust, realize data interaction of an intelligent terminal and an edge computing device, and simultaneously realize data acquisition of a power transmission equipment side to integrated operation and maintenance service of a cloud platform.
The invention is realized by the following technical scheme:
the scheme provides a circuit fitting rust defect detection method based on cloud edge cooperation, which comprises the following steps:
step one: acquiring hardware rust defect data of a power transmission line, and acquiring a defect sample and a normal sample in the hardware rust defect data based on a detection and identification model;
step two: inputting the defect sample into an improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module at the joint of a feature extraction network and a feature fusion network of the YOLOv8m model, and adding an additional detection head in the feature fusion network;
step three: and updating the detection and identification model by using the new detection and identification model, and detecting rust defects of the line hardware.
In a further optimization scheme, the frequency attention module is added after the last two C2f modules and the SPPF module in the feature extraction network, and the additional detection head is added after the frequency attention module after the first two C2f modules and the SPPF module in the feature fusion network.
The training method of the new structure detection and identification model comprises the following steps:
pretreating a defect sample;
extracting rust features from the pre-processed defect samples based on the feature extraction network, and supplementing detail features of the deep network based on a frequency attention module, wherein the frequency attention module regards the channel representation problem as a compression process using frequency analysis;
the rust features are fused based on the feature fusion network, and additional features are extracted from a feature map generated by the main network based on the additional detection head, so that the condition of information loss caused by downsampling of a model convolution layer is relieved; wherein the additional detection head treats more candidate boxes as positive samples by relaxing the constraints on the positive sample allocation.
The further optimization scheme is that the feature fusion network is used for fusing the rust features, and the additional feature is extracted from the feature map generated by the main network based on the additional detection head, and the method comprises the following steps:
generating a coarse-to-fine hierarchical label by taking a Lead Head prediction result as a guide, wherein the hierarchical label is used for training an additional detection Head and a Lead Head respectively; and the Lead Head selects N prediction frames with highest weighted scores as positive samples.
The further optimization scheme is that the channel attention framework of the frequency attention module is as follows:
ms_att＝sigmoid(fc(Freq))
wherein ms_att represents a feature map output after frequency attention adjustment, sigmoid (x) represents an activation function, fc (x) represents a full-connection layer, and Freq represents feature map coupling after channel weight adjustment to obtain an entire compression vector.
The further optimization scheme is that the method for acquiring the channel attention framework of the frequency attention module comprises the following steps:
basis function of changing two-dimensional discrete cosine:written as:
where h.epsilon.0, 1, …, H-1, w.epsilon.0, 1, …, W-1,frequency spectrum representing two-dimensional discrete cosine change, +.>Representing the spectrum obtained by two-dimensional discrete cosine transform of a characteristic diagram with height of h and width of w, x 2d For inputting features +.> Representing an input profile of height i and width j, H representing the height of the input profile, W representing the width of the input profile, < >>Representing a discrete cosine transform formula;
then, there are:
let the h and w components be 0, then there are
Wherein the method comprises the steps ofA lowest frequency component representing a two-dimensional discrete cosine change; gap () represents global averaging pooling, i.e. averaging all pixel values of each channel map, and then yielding a new 1 x 1 channel map.
The further optimization scheme is that the method for acquiring the channel attention framework of the frequency attention module further comprises the following steps:
the frequency attention module uses multiple frequency components to divide the input X into n parts along the channel dimension, written as [ X ] 0 ,X 1 ,…,X n-1 ]Whereini∈{0,1,…,n-1}，/>C may be divided by n;
for each section, a corresponding two-dimensional discrete cosine change frequency component is assigned, and the two-dimensional discrete cosine change result is used as a compression result of the channel attention, the corresponding two-dimensional discrete cosine change frequency component is expressed as:
wherein Freq is i Representing the characteristic diagram after the i-th partial channel weight adjustment,representing two-dimensional discrete cosine change of the input ith channel part; x is X i I channel parts representing inputs; />I channel parts representing the input of a feature map with height h and width w are +.>Representing the two-dimensional discrete cosine change basis function of the input ith channel portion, i e {0,1, …, n-1}, [ u ] i ,v i ]Is corresponding to X i Index of frequency component, compressed +.>The vector is changed into a vector in a C' dimension, and the compressed vector can be obtained by splicing:
Freq＝compress(X)
＝cat([Freq 0 ,Freq 1 ,…,Freq n-1 ])
wherein the method comprises the steps ofFor the multispectral vector obtained, +.>Representing a compression method;
the whole channel attention framework is:
ms_att＝sigmoid(fc(Freq))。
the scheme also provides a circuit fitting rust defect detection system based on cloud-edge cooperation, which is used for realizing the circuit fitting rust defect detection method based on cloud-edge cooperation; comprising the following steps:
the acquisition and inspection module is used for acquiring hardware rust defect data of the power transmission line;
the defect detection module is used for acquiring a defect sample and a normal sample in the hardware rust defect data based on the detection and identification model;
the model training module is used for inputting the defect sample into the improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module in a feature extraction network of the YOLOv8m model, and adding an additional detection head in a feature fusion network;
the data transmission module is used for uploading the detection result to the cloud server and receiving the newly constructed detection model;
the defect detection module is also used for updating the detection recognition model by using the new structure detection recognition model to detect the rust defect of the line hardware.
The model training module is located in the cloud server; the defect detection module is positioned at the edge end;
the cloud server transmits the new structure detection and identification model to the edge end, and the edge end uploads the detection result of the rust defect of the circuit hardware fitting to the cloud server.
Firstly, the intelligent terminal collects the image data of the rust defect of the power transmission line, completes identification, then uploads the data to the cloud, the cloud trains an update model, and the edge end synchronizes the cloud model to achieve algorithm update.
The present solution also provides a computer readable medium having stored thereon a computer program to be executed by a processor to implement a line fitting rust defect detection method based on cloud-edge synergy as described above.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the invention aims to provide a method, a system and a medium for detecting rust defects of line hardware based on cloud edge cooperation, and aims to solve the problems that rust shapes are changeable, characteristics are not obvious, and key information is easy to lose in a deep network.
2. The invention aims to provide a method, a system and a medium for detecting rust defects of line hardware fittings based on cloud-edge cooperation, which utilize cloud-edge cooperation technology to complete information interaction between a cloud end and an edge end, so as to realize real-time detection of a power transmission line equipment side; meanwhile, aiming at the corrosion characteristics, the cloud training model is improved, and high-precision detection of the corrosion of the power transmission line hardware is realized; introducing frequency attention at the joint of the feature extraction network and the feature fusion network, reducing the loss of image feature information by introducing more frequency components, supplementing detail features such as textures and the like of a deep network which are less influenced by contour change, and optimizing the extraction of rust features by a model; then, an additional detection head is introduced, additional features are extracted from a feature map generated by the main network, the condition of information loss caused by downsampling of a model convolution layer is relieved, the network can learn some more complex and finer features, the discrimination capability of the network is enhanced, and further high-precision power transmission line hardware rust detection is realized.
Drawings
In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the drawings that are needed in the examples will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and that other related drawings may be obtained from these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a schematic flow chart of a method for detecting rust defects of line hardware based on cloud edge cooperation;
FIG. 2 is a schematic diagram of a cloud-edge cooperative work flow in a line hardware rust defect detection system based on cloud-edge cooperative work flow;
FIG. 3 is a schematic diagram of a new structure detection and identification model;
FIG. 4 is a flow chart of a frequency attention mechanism;
fig. 5 is a schematic view of an additional head structure.
Detailed Description
For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.
Example 1
The embodiment provides a method for detecting rust defects of line hardware based on cloud edge cooperation, as shown in fig. 1, which comprises the following steps:
step one: acquiring hardware rust defect data of a power transmission line, and acquiring a defect sample and a normal sample in the hardware rust defect data based on a detection and identification model;
step two: inputting the defect sample into an improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module at the joint of a feature extraction network and a feature fusion network of the YOLOv8m model, and adding an additional detection head in the feature fusion network;
the frequency attention module is added after the last two C2f modules and the SPPF module in the feature extraction network, and the additional detection head is added after the first two C2f modules and the frequency attention module after the SPPF module in the feature fusion network.
The training method of the new structure detection and identification model comprises the following steps:
pretreating a defect sample; after dividing the data set, the network pre-processes the sample picture data, the size of the input feature map of the network is 3×640×640, wherein 3 is the number of input picture channels, and 640×640 is the length and width of the picture.
Extracting rust features from the pre-processed defect samples based on the feature extraction network, and supplementing detail features of the deep network based on a frequency attention module, wherein the frequency attention module regards the channel representation problem as a compression process using frequency analysis;
and extracting rich features of the image through a feature extraction network to generate a feature map of the sample set. The 0 th layer and the 1 st layer of the feature extraction network are mainly Conv layers, and are used for expanding channels and shrinking feature graphs, and the obtained feature graphs have the size of 128×160×160, wherein 128 is the number of channels, and 160×160 is the size of a picture. Layer 2 is a C2f module, the input channel is 128, the tensor is changed to 64 in the channel dimension (Split operation), when passing through the bottleck module, the input feature and the output feature are added to realize residual connection through two convolution modules of k=3s=1p=1, wherein k is the convolution kernel size, s is the step size, and p is the filling, if residual connection (shortcut=true) is set and the channel number of the input feature is the same as the channel number of the output feature. Otherwise, the convolved features are directly output. At this time, the output of the butteleneck module is 64×160×160, then the butteleneck module is spliced to obtain two parts including the output of n butteleneck modules and Split operation, C/2× (n+2) =192×160×160, and finally the output is 128×160×160 through a Conv layer with k=1s=1p=0. The 3 rd to 9 th layers are composed of 3 convolution layers, 2C 2f modules and 1 SPPF module, wherein the SPPF structure is formed by three continuous maximum pooling with residual structure, convolution kernels are unified to 5*5, and the results before pooling and after each pooling are spliced.
As shown in fig. 4, a frequency attention module is inserted after the penultimate C2f module (layer 6) and after the SPPF module (layer 9) of the feature extraction network, and the frequency attention module refers to fig. 3, which treats the channel representation problem as a compression process using frequency analysis. On the basis of frequency domain analysis, it has been mathematically demonstrated that conventional global averaging pooling is a special case of feature decomposition in the frequency domain.
The channel attention framework of the frequency attention module is:
ms_att＝sigmoid(fc(Freq))
wherein ms_att represents a feature map output after frequency attention adjustment, sigmoid () represents an activation function, fc () represents a fully connected layer, and Freq represents feature map linkage after channel weight adjustment to obtain an entire compression vector.
The method for acquiring the channel attention framework of the frequency attention module comprises the following steps:
basis function of changing two-dimensional discrete cosine:written as:
where h.epsilon.0, 1, …, H-1, w.epsilon.0, 1, …, W-1,frequency spectrum representing two-dimensional discrete cosine change, +.>Representing the spectrum obtained by two-dimensional discrete cosine transform of a characteristic diagram with height of h and width of w, x 2d For inputting features +.> Representing an input profile of height i and width j, H representing the height of the input profile, W representing the width of the input profile, < >>The discrete cosine transform formula is then represented by:
let the h and w components be 0, then there are
Wherein the method comprises the steps ofMinimum frequency representing two-dimensional discrete cosine changeA rate component; gap represents global averaging pooling, i.e. averaging all pixel values of each channel map, and then obtaining a new 1 x 1 channel map.
The method for acquiring the channel attention framework of the frequency attention module further comprises the following steps:
the frequency attention module uses multiple frequency components to divide the input X into n parts along the channel dimension, written as [ X ] 0 ,X 1 ,…,X n-1 ]Whereini∈{0,1,…,n-1}，/>C may be divided by n;
for each section, a corresponding two-dimensional discrete cosine change frequency component is assigned, and the two-dimensional discrete cosine change result is used as a compression result of the channel attention, the corresponding two-dimensional discrete cosine change frequency component is expressed as:
wherein Freq is i Representing the characteristic diagram after the i-th partial channel weight adjustment,representing two-dimensional discrete cosine change of the input ith channel part; x is X i I channel parts representing inputs; />Representing i channel parts of the input of the feature map with height h and width w +.>Representing the two-dimensional discrete cosine change basis function of the input ith channel portion, i e {0,1, …, n-1}, [ u ] i ,v i ]Is corresponding to X i Frequency componentIndex, compressed->The vector is changed into a vector in a C' dimension, and the compressed vector can be obtained by splicing:
Freq＝compress(X)
＝cat([Freq 0 ,Freq 1 ,…,Freq n-1 ])
wherein the method comprises the steps ofFor the multispectral vector obtained, express:>representing a compression method;
the whole channel attention framework is:
ms_att＝sigmoid(fc(Freq))。
the rust features are fused based on the feature fusion network, and additional features are extracted from a feature map generated by the main network based on the additional detection head, so that the condition of information loss caused by downsampling of a model convolution layer is relieved; wherein the additional detection head treats more candidate boxes as positive samples by relaxing the constraints on the positive sample allocation.
Generating a coarse-to-fine hierarchical label by taking a Lead Head prediction result as a guide, wherein the hierarchical label is used for training an additional detection Head and a Lead Head respectively; and the Lead Head selects N prediction frames with highest weighted scores as positive samples.
The shallow feature extraction is input to an additional detection Head (Additional Detection Head, AD-Head), see specifically fig. 5, which is added after the first two C2f modules of the YOLOv8m feature fusion network and the frequency attention module after the SPPF module. Using the Lead Head prediction results as guidance, a coarse to fine hierarchical label is generated for training the additional detection Head and the Lead Head, respectively. YOLOv8m employs a Task-Aligned Assigner dynamic label assignment strategy to select positive samples based on weighted scores of classification and regression scores. The Lead Head selects the N (10 default) prediction frames with the highest weighted scores as positive samples. The additional detection head takes more candidate boxes as positive samples by relaxing the constraints on the positive sample allocation. And when in training, an additional detection head is added to improve the model identification capability, and when in reasoning, the model reasoning speed is not influenced by the removal of the additional detection head.
Step three: and updating the detection and identification model by using the new detection and identification model, and detecting rust defects of the line hardware.
Example 2
The embodiment provides a circuit fitting rust defect detection system based on cloud edge cooperation, which is used for realizing the circuit fitting rust defect detection method based on cloud edge cooperation described in embodiment 1; comprising the following steps:
the acquisition and inspection module is used for acquiring hardware rust defect data of the power transmission line; as shown in fig. 2, the acquisition and inspection module comprises a network camera and an AI edge computing device at the edge end, wherein the AI edge computing device performs real-time monitoring, result storage and algorithm management;
the defect detection module is used for acquiring a defect sample and a normal sample in the hardware rust defect data based on the detection and identification model;
the model training module is used for inputting the defect sample into the improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module in a feature extraction network of the YOLOv8m model, and adding an additional detection head in a feature fusion network;
the data transmission module is used for uploading the detection result to the cloud server and receiving the newly constructed detection model;
the defect detection module is also used for updating the detection recognition model by using the new structure detection recognition model to detect the rust defect of the line hardware.
The model training module is located at the cloud server; the defect detection module is positioned at the edge end;
the cloud server transmits the new structure detection and identification model to the edge end, and the edge end uploads the detection result of the rust defect of the circuit hardware fitting to the cloud server.
And acquiring rust defect data of the power transmission line hardware fitting through intelligent terminal equipment, and transmitting the data to an AI edge computing device.
The edge end uses a corresponding algorithm to detect and identify the data in real time, stores the detection result, and transmits the detection result to the cloud server through a network.
The cloud server processes the results transmitted by the edge end, and respectively sends the defect sample and the normal sample into a corresponding sample library and sends the defect sample to the training platform.
And retraining the defect sample by using the improved YOLOv8m model to generate a new model, and issuing the model to an AI edge computing device through a network. Improvement procedure for YOLOv8m model: inserting a frequency attention module (FcNet) after the last C2f module and the SPPF module of the Yolov8m backbone feature extraction network, and inputting the frequency information into a feature fusion network after supplementing the frequency information; shallow feature extraction is input to an additional detection Head (Additional Detection Head, AD-Head) added after the frequency attention module behind the first two C2f modules and SPPF modules of the Yolov8m feature fusion network.
Example 3
The present embodiment provides a computer-readable medium having stored thereon a computer program that is executed by a processor to implement the line fitting rust defect detection method based on cloud-edge synergy as described in embodiment 1.
The embodiment solves the problem that network identification is difficult because the rust features are not obvious and key information is easy to lose in a deep network, and realizes real-time detection of the equipment end of the power transmission line. The cloud edge cooperative technology is utilized to complete the functions of data acquisition, database construction, cloud training and edge calculation, and state evaluation is directly carried out on the power transmission equipment side to obtain the realization; the detection precision of the network to the corrosion is improved, more frequency information is supplemented by introducing frequency attention, the problem of weakening of the high-frequency information of the deep network is solved, and the extraction of the corrosion characteristics by the model is optimized; and an additional detection head is introduced, so that the condition of losing rust key information caused by downsampling of a model convolution layer is relieved.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The method for detecting the rust defect of the line hardware fitting based on cloud edge cooperation is characterized by comprising the following steps:
step one: acquiring hardware rust defect data of a power transmission line, and acquiring a defect sample and a normal sample in the hardware rust defect data based on a detection and identification model;
step two: inputting the defect sample into an improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module at the joint of a feature extraction network and a feature fusion network of the YOLOv8m model, and adding an additional detection head in the feature fusion network;
step three: and updating the detection and identification model by using the new detection and identification model, and detecting rust defects of the line hardware.
2. The cloud edge cooperation-based line hardware rust defect detection method according to claim 1, wherein the frequency attention module is added after a C2f module and an SPPF module in a feature extraction network, and the additional detection head is added after the C2f module and the SPPF module in a feature fusion network.
3. The cloud edge cooperation-based line fitting rust defect detection method according to claim 2, wherein the training method of the new structure detection and identification model comprises the following steps:
pretreating a defect sample;
extracting rust features from the pre-processed defect samples based on the feature extraction network, and supplementing detail features of the deep network based on a frequency attention module, wherein the frequency attention module regards the channel representation problem as a compression process using frequency analysis;
the rust features are fused based on a feature fusion network, and additional features are extracted from a feature map generated by a main network based on an additional detection head; wherein the additional detection head treats more candidate boxes as positive samples by relaxing the constraints on the positive sample allocation.
4. The cloud edge collaboration-based line fitting rust defect detection method according to claim 3, wherein the feature fusion network-based feature fusion is used for fusing rust features, and the additional feature extraction from the feature map generated by the main network based on the additional detection head comprises the following steps:
generating a coarse-to-fine hierarchical label by taking a Lead Head prediction result as a guide, wherein the hierarchical label is used for training an additional detection Head and a Lead Head respectively; and the Lead Head selects N prediction frames with highest weighted scores as positive samples.
5. The cloud edge cooperation-based line fitting rust defect detection method according to claim 3, wherein the channel attention frame of the frequency attention module is:
ms_att＝sigmoid(fc(Freq))
wherein ms_att represents a feature map output after frequency attention adjustment, sigmoid () represents an activation function, fc () represents a fully connected layer, and Freq represents feature map linkage after channel weight adjustment to obtain an entire compression vector.
6. The method for detecting rust defects of line hardware based on cloud edge cooperation according to claim 5, wherein the method for acquiring the channel attention frame of the frequency attention module comprises the following steps:
basis function of changing two-dimensional discrete cosine:written as:
where h.epsilon.0, 1, …, H-1, w.epsilon.0, 1, …, W-1,frequency spectrum representing two-dimensional discrete cosine change, +.>Representing the spectrum obtained by two-dimensional discrete cosine transform of a feature map having height h and width w, x 2d For inputting features +.> An input feature map having a height i and a width j is shown, H is the height of the input feature map, W is the width of the input feature map, < >>Representing a discrete cosine transform formula;
then, there are:
let the h and w components be 0, then there are
Wherein the method comprises the steps ofA lowest frequency component representing a two-dimensional discrete cosine change; gap represents global average pooling.
7. The cloud edge collaboration-based line hardware rust defect detection method of claim 6, wherein the method for acquiring the channel attention frame of the frequency attention module further comprises:
the frequency attention module uses multiple frequency components to divide the input X into n parts along the channel dimension, written as [ X ] 0 ,X 1 ,…,X n-1 ]WhereinC may be divided by n;
for each section, a corresponding two-dimensional discrete cosine change frequency component is assigned, and the two-dimensional discrete cosine change result is used as a compression result of the channel attention, the corresponding two-dimensional discrete cosine change frequency component is expressed as:
wherein Freq is i Representing the characteristic diagram after the i-th partial channel weight adjustment,representing two-dimensional discrete cosine change of the input ith channel part; x is X i I channel parts representing inputs; />I channel parts representing the input of a feature map with height h and width w are +.>Representing the two-dimensional discrete cosine change basis function of the input ith channel portion, i e {0,1, …, n-1}, [ u ] i ,v i ]Is corresponding to X i Index of frequency component, compressed +.>The vector is changed into a vector in a C' dimension, and the compressed vector can be obtained by splicing:
Freq＝compress(X)
＝cat([Freq 0 ,Freq 1 ,…,Freq n-1 ])
wherein the method comprises the steps ofFor the multispectral vector obtained, express:>representing a compression method;
the whole channel attention framework is:
8. the circuit fitting rust defect detection system based on cloud edge cooperation is characterized by being used for realizing the circuit fitting rust defect detection method based on cloud edge cooperation as claimed in any one of claims 1-7; comprising the following steps:
the acquisition and inspection module is used for acquiring hardware rust defect data of the power transmission line;
the defect detection module is used for acquiring a defect sample and a normal sample in the hardware rust defect data based on the detection and identification model;
the model training module is used for inputting the defect sample into the improved YOLOv8m model to train a new structure detection and identification model; the improved YOLOv8m model comprises: adding a frequency attention module in a feature extraction network of the YOLOv8m model, and adding an additional detection head in a feature fusion network;
the data transmission module is used for uploading the detection result to the cloud server and receiving the newly constructed detection model;
the defect detection module is also used for updating the detection recognition model by using the new structure detection recognition model to detect the rust defect of the line hardware.
9. The cloud-edge-collaboration-based line fitting rust defect detection system of claim 8, wherein the model training module is located at a cloud server; the defect detection module is positioned at the edge end;
the cloud server transmits the new structure detection and identification model to the edge end, and the edge end uploads the detection result of the rust defect of the circuit hardware fitting to the cloud server.
10. A computer readable medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the cloud-edge synergy-based line hardware rust defect detection method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311841086.6A CN117788436A (en) | 2023-12-27 | 2023-12-27 | Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311841086.6A CN117788436A (en) | 2023-12-27 | 2023-12-27 | Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117788436A true CN117788436A (en) | 2024-03-29 |
Family
ID=90398032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311841086.6A Pending CN117788436A (en) | 2023-12-27 | 2023-12-27 | Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117788436A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115829991A (en) * | 2022-12-15 | 2023-03-21 | 淮阴工学院 | Steel surface defect detection method based on improved YOLOv5s |
| CN116152171A (en) * | 2022-12-15 | 2023-05-23 | 重庆市西部水资源开发有限公司 | Intelligent building target counting method, electronic equipment and storage medium |
| CN116229278A (en) * | 2023-05-10 | 2023-06-06 | 广东电网有限责任公司珠海供电局 | Method and system for detecting rust defect of vibration damper of power transmission line |
| CN116597313A (en) * | 2023-05-25 | 2023-08-15 | 清华大学深圳国际研究生院 | Ship optical image wake detection method based on improved YOLOv7 |
| CN116934696A (en) * | 2023-07-12 | 2023-10-24 | 华东交通大学 | Industrial PCB defect detection method and device based on YOLOv7-Tiny model improvement |
-
2023
- 2023-12-27 CN CN202311841086.6A patent/CN117788436A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115829991A (en) * | 2022-12-15 | 2023-03-21 | 淮阴工学院 | Steel surface defect detection method based on improved YOLOv5s |
| CN116152171A (en) * | 2022-12-15 | 2023-05-23 | 重庆市西部水资源开发有限公司 | Intelligent building target counting method, electronic equipment and storage medium |
| CN116229278A (en) * | 2023-05-10 | 2023-06-06 | 广东电网有限责任公司珠海供电局 | Method and system for detecting rust defect of vibration damper of power transmission line |
| CN116597313A (en) * | 2023-05-25 | 2023-08-15 | 清华大学深圳国际研究生院 | Ship optical image wake detection method based on improved YOLOv7 |
| CN116934696A (en) * | 2023-07-12 | 2023-10-24 | 华东交通大学 | Industrial PCB defect detection method and device based on YOLOv7-Tiny model improvement |
Non-Patent Citations (2)
| Title |
|---|
| 王亮等: "智能光电感知", vol. 1, 31 July 2022, 中国青年出版社, pages: 527 * |
| 韩锐等: "云边协同大数据技术与应用", vol. 1, 28 February 2022, 机械工业出版社, pages: 228 - 230 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Biasutti et al. | Lu-net: An efficient network for 3d lidar point cloud semantic segmentation based on end-to-end-learned 3d features and u-net | |
| CN110390308B (en) | Video behavior identification method based on space-time confrontation generation network | |
| CN113534832B (en) | Unmanned aerial vehicle routing inspection tracking distribution network line flight method based on edge calculation | |
| CN116681885B (en) | Infrared image target identification method and system for power transmission and transformation equipment | |
| CN111199238A (en) | Behavior identification method and equipment based on double-current convolutional neural network | |
| CN114842363B (en) | Identification method and system for key power equipment in digital twin platform area | |
| CN114913396A (en) | Motor bearing fault diagnosis method | |
| CN114882494B (en) | Three-dimensional point cloud feature extraction method based on multi-modal attention driving | |
| CN117152459A (en) | Image detection method, device, computer readable medium and electronic equipment | |
| CN110533066B (en) | Image data set automatic construction method based on deep neural network | |
| CN113920498B (en) | Point cloud 3D object detection method based on multilayer feature pyramid | |
| CN114359167A (en) | Insulator defect detection method based on lightweight YOLOv4 in complex scene | |
| CN112016592B (en) | Domain adaptive semantic segmentation method and device based on cross domain category perception | |
| WO2024156172A1 (en) | Ar-based cable connection assembly quality analysis system | |
| CN110555406B (en) | Video moving target identification method based on Haar-like characteristics and CNN matching | |
| CN117079005A (en) | Optical cable fault monitoring method, system, device and readable storage medium | |
| CN117788436A (en) | Cloud edge cooperation-based method, system and medium for detecting rust defect of line hardware fitting | |
| CN116229315A (en) | Time sequence action detection method and device based on coarse-to-fine granularity information capture | |
| CN115661904A (en) | Data labeling and domain adaptation model training method, device, equipment and medium | |
| CN115601674A (en) | Power transmission project smoke hidden danger identification method and device and storage medium | |
| CN115661429A (en) | System and method for identifying defects of water wall tube of boiler and storage medium | |
| CN111488889B (en) | Intelligent image processor for extracting image edges | |
| CN114862796A (en) | A unmanned aerial vehicle for fan blade damage detects | |
| CN112036472A (en) | Visual image classification method and system for power system | |
| CN117173731B (en) | Model training method, image processing method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |