CN113515829B - Situation awareness method for transmission line hardware defects under extremely cold disasters - Google Patents
Situation awareness method for transmission line hardware defects under extremely cold disasters Download PDFInfo
- Publication number
- CN113515829B CN113515829B CN202110557643.6A CN202110557643A CN113515829B CN 113515829 B CN113515829 B CN 113515829B CN 202110557643 A CN202110557643 A CN 202110557643A CN 113515829 B CN113515829 B CN 113515829B
- Authority
- CN
- China
- Prior art keywords
- hardware
- model
- stage
- transmission line
- disasters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/18—Network design, e.g. design based on topological or interconnect aspects of utility systems, piping, heating ventilation air conditioning [HVAC] or cabling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/04—Constraint-based CAD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Hardware Design (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Medical Informatics (AREA)
- Pure & Applied Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a situation awareness method for power transmission line hardware defects under extremely cold disasters, which comprises the steps of establishing a first-stage detection model and a second-stage classification model, acquiring a hardware target data set and a defect data set under extremely cold disasters, training the first-stage detection model and the second-stage classification model through the hardware target data set and the defect data set under extremely cold disasters, and cascading the first-stage detection model and the second-stage classification model; inputting the aerial image of the power transmission line to be detected into a trained first-stage detection model to obtain a hardware fitting image and a label, and simultaneously inputting the hardware fitting image and the label into a trained second-stage classification model to obtain the defect condition of the hardware fitting. The situation awareness method for the power transmission line hardware defects under the extremely cold disasters realizes sustainable learning of the model, saves the occupied space of the model, ensures that the model cannot forget old classification tasks while learning new classification tasks, and improves the recognition capability of the model on hardware with different defect degrees.
Description
Technical Field
The invention relates to the technical field of transmission line defect detection, in particular to a situation awareness method for transmission line hardware defects under extremely cold disasters.
Background
Whether the hardware fitting on the transmission line has defects or not directly influences the operation safety of the transmission line, especially the possibility that the transmission line hardware fitting has defects under severe cold disasters is higher, and manual inspection is more difficult. The hardware fittings on the power transmission line are gradually converted from a normal state to a defect state, and meanwhile, the same defect has different defect degrees and is continuously evolving. For the same defect of the same hardware fitting, the influence degree of different defect states on the power transmission line is different. Therefore, in order to ensure the safe operation of the power transmission line under the severe cold disaster, situation awareness needs to be performed on a large number of hardware defects on the power transmission line so as to obtain what state the hardware is currently in and maintain the hardware in a targeted manner.
Situation awareness refers to the steps of awareness, understanding and prediction of future running states of elements in a system under a specific space-time background, wherein the three steps correspond to three stages of a situation awareness process respectively: situation element acquisition, real-time situation understanding and future situation prediction. The object of power transmission line situation perception is hardware fitting on the line, and the next conversion state of the hardware fitting is predicted after the state of the hardware fitting is perceived, so that the power transmission line situation perception is used for assisting patrol personnel in making decisions.
Because manual inspection operation is extremely difficult under extremely cold disasters, the power transmission line image is obtained through unmanned aerial vehicle aerial photography, and then the inspection mode of defect situation perception is carried out on the aerial image by using the classification model based on deep learning, so that the inspection workload of the operation and maintenance personnel on the pole can be reduced, the safety of the inspection personnel is ensured, and the defect state is rapidly and accurately judged. The work corresponds to a situation element acquisition stage in situation awareness, basic data information is provided for the next step of situation awareness, and the stage is a basic link of the situation awareness.
However, when the existing deep learning model is trained, all kinds of hardware fittings and defect images thereof are required to be used, the types of hardware fittings and defects which can be perceived by the trained model cannot be increased, and when new defects to be classified are required to be added, if the original model is directly trained by using a new data set to be classified, the processing capacity of the model on an old task is reduced, and the phenomenon is called catastrophic forgetting. Therefore, only the original model can be abandoned, and the new data set is mixed into the old data set to train the model from the new one, but a lot of time is consumed.
In order to allow models to reduce the impact of catastrophic forgetfulness without retraining, the concept of continuous learning is proposed. The continuous learning at the present stage is mainly realized by three modes: the first is based on regularization, preventing forgetting by limiting the model parameters to deviate too far from the previous solutions, but this also limits the model's ability to adapt to new tasks, resulting in the model failing to find the optimal solution; the second is a method based on data set playback, which makes a model memorize the characteristics of tasks that have been learned by playing back the data saved by the previous task a plurality of times, and while this method is effective in preventing forgetting, the performance of the playback-based method is highly dependent on the size of the memory buffer and the selection of the playback content, and in some strict settings, saving any data may not be an optimal option, and in a large-scale environment, the performance may decrease rapidly with the increase of the number of tasks; the third approach is a model augmentation-based approach that mitigates catastrophic forgetfulness by setting a growth model in each task and can easily add additional necessary capacity to accommodate new tasks, the ability to expand arbitrarily without saving any data provides the expansion approach with the possibility of success in a large-scale data environment setting. However, as the amount of tasks increases, how to keep the amount of increase in the number of model parameters within acceptable limits remains a matter of primary concern.
Because the hardware fittings and defects thereof on the transmission line are numerous, the distribution of the transmission lines is wide, the transmission lines in each provincial area are different, the materials of used parts, the shapes of the parts, the installation specifications and the geographical environment where the parts are positioned are extremely different, and the hardware fittings and defects are difficult to collect under extremely cold disasters, the data are less, and the great difficulty is faced in collecting all kinds of data at one time. The existing detection model is limited by the problem of unbalanced distribution of hardware defect data sets, has lower defect detection precision on some data sets, and does not have continuous learning capability.
Therefore, the resistance continuous learning is applied to situation awareness of hardware defects on the transmission line under extremely cold disasters, and the inspection efficiency is effectively improved. However, the current resistance continuous learning model can only be used for classification tasks, and cannot achieve good effects when classifying hardware defects with different degrees and smaller differences between classes. Therefore, it is necessary to design a situation awareness method for the transmission line hardware defects under severe cold disasters.
Disclosure of Invention
The invention aims to provide a situation awareness method for the hardware defects of a power transmission line under an extremely cold disaster, which realizes sustainable learning of a hardware defect detection model, can relieve the problem of low detection accuracy of certain hardware defects caused by unbalance of hardware defect data sets, improves the accuracy of hardware detection under the extremely cold disaster, saves the occupied space of the model, ensures that the model cannot forget old classification tasks while learning new classification tasks, and improves the recognition capability of the model on hardware with different defect degrees.
In order to achieve the above object, the present invention provides the following solutions:
a situation awareness method for power transmission line hardware defects under extremely cold disasters comprises the following steps:
step 1: establishing a first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, and training the first-stage detection model through the data set;
step 2: establishing a second-stage classification model, acquiring a hardware fitting defect data set under extremely cold disasters, classifying the hardware fitting defect data set, and training the second-stage classification model through the classified hardware fitting defect data set;
step 3: cascading the first-stage detection model and the second-stage classification model, and introducing an attention mechanism into the second-stage classification model;
step 4: inputting the aerial image of the power transmission line to be detected into the first-stage detection model trained in the step 1 to obtain a hardware fitting image and a label, and simultaneously inputting the hardware fitting image and the label into the second-stage classification model trained in the step 2 to obtain the defect condition of the hardware fitting.
Optionally, in step 1, a first-stage detection model is established, a hardware fitting target data set under an extremely cold disaster is obtained, and the first-stage detection model is trained through the data set, specifically:
establishing a YOLOv4 model as a cascaded first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, training the first-stage detection model through the data set to enable the hardware fitting target to be detected on an aerial image of a power transmission line, cutting off the hardware fitting image and a tag obtained through detection to serve as input of a second-stage classification model, wherein the input aerial image of the power transmission line is made to be x global The cut hardware image is x, the labels of various hardware are y, and a formula F for detecting the hardware by utilizing the YOLOv4 model is obtained yolov4 The method comprises the following steps:
F yolov4 (x global )＝(x,y) (1)
optionally, in step 2, a hardware defect dataset under an extremely cold disaster is obtained and classified, which specifically includes:
acquiring a hardware defect data set under an extremely cold disaster, dividing the hardware defect data set according to different types of hardware, dividing the hardware again by taking specific defects of the hardware as subclasses after dividing,
D＝{D 1 ,D 2 ,………,D T } (2)
wherein D is the set of all hardware defect data sets, D k For some hardware defect data, x i k To input an image, y i k To output labels, t i k For task labels, n k For the number of training setsAnd the quantity i is the hardware defect type.
Optionally, in the second-stage classification model described in step 2, the feature knowledge learned by the deep learning network is divided into a shared knowledge module and a specific knowledge module, where the shared knowledge module learns the common image features of all training sets in all classification tasks in a manner of antagonistic training, and the specific knowledge module learns only the specific features in the sequential classification tasks, and the orthogonal constraint method is adopted to make the feature knowledge learned by the shared knowledge module and the specific knowledge module different.
Optionally, the unique knowledge module only learns unique features in the sequence classification task, specifically:
learning dissimilar characteristics of different class images in the same sequence classification task, and storing a sub-model P for each sequence classification task separately, wherein for a sequence classification task comprising c class images, it is actually a class c classification task, and the loss function is:
wherein x is k Representing input data, y k For a true class label of input data, σ is a softmax function, f k A model that maps inputs to two separate potential spaces, one containing shared features for all tasks and the other containing features that are unique to each task.
Optionally, the shared knowledge module learns the common image features of all training sets in all classification tasks by adopting an antagonism training mode, specifically:
constructing a feature map Shared and a countermeasure discriminator D, and training the feature map Shared in a manner of training the resistance, wherein the loss function is as follows:
wherein t is k For task sequence number labels, when S can generate features that D can no longer predict the correct task labels, S and D training is completed, and T is the total number of sequence tasks.
Optionally, the orthogonal constraint method is adopted to make the feature knowledge learned by the shared knowledge module and the unique knowledge module different, which specifically comprises:
the feature knowledge of the shared knowledge module and the feature knowledge of the specific knowledge module are factorized by using an orthogonal constraint method, and the difference is calculated as follows:
in the method, in the process of the invention,is F-norm, P k Representing the kth P model.
Optionally, the final total loss of the second-stage classification model is the sum of the differences calculated by the loss function of the shared knowledge module, the loss function of the specific knowledge module and the orthogonal constraint method, namely:
L final ＝L adv +L task +L diff (8)
wherein L is final Is the final total loss.
Optionally, in step 3, attention mechanisms are introduced into the second-stage classification model, specifically:
introducing a channel dimension-based attention module in the base network of the second classification model, and adaptively recalibrating the characteristic response of the channel direction by modeling the relation between channels, wherein the formula is as follows:
X→u,X∈R W'×H'×C' ,u∈R W×H×C (9)
v C ＝ω C ×u C (13)
in the formula (9), X represents an input feature map, u represents a feature map after convolution, and the formula represents that the input feature map of W ' X H ' X C ' is convolved into W X H X C, so as to obtain C feature maps with the size W X H; in the formula (10), H and W represent the height and width of the input, u C Representing an input feature map, wherein C represents the number of channels of an input neuron in formula (11), C/r is the number of neurons in a second layer, delta represents a ReLU function, sigma represents a sigmoid function, the neurons in the first layer use the ReLU function to prevent gradient dispersion, the neurons in the second layer use the sigmoid function, and the output values are normalized in (0, 1) in formula (12)]Between the regions, equation (13) shows that the feature map v after prediction of the importance of each channel is obtained by multiplying w by u.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects: according to the situation sensing method for the hardware defects of the transmission line under the extremely cold disaster, the first-stage detection model and the second-stage classification model are cascaded, wherein the second-stage classification model is the opposite continuous learning classification model, sustainable learning of the hardware defect detection model is realized, the hardware targets under the extremely cold disaster are detected through the first-stage detection model, the problem that the detection accuracy of certain hardware defects is too low due to unbalance of hardware defect data sets can be relieved, meanwhile, the used hardware target data sets under the extremely cold disaster do not subdivide the defect hardware images any more, the hardware target data sets under the extremely cold disaster are amplified, and the accuracy of detection hardware is improved; the characteristics of different classification tasks learned by the second-stage classification model are divided into a shared knowledge module and a special knowledge module by the shared characteristics and the special characteristics, wherein the shared knowledge module learns the shared image characteristics of all training sets in all classification tasks in an opposite training mode, and the special knowledge module learns only the special characteristics in the sequence classification tasks, so that the occupied space of the model is saved, the model is ensured not to forget the old classification tasks when learning the new classification tasks, and the orthogonal constraint method is adopted to ensure that the feature knowledge learned by the shared knowledge module and the special knowledge module is as different as possible; attention mechanisms are introduced into the feature extraction network of the second-stage classification model, so that the defect recognition capability of the second-stage classification model on hardware fittings with different defect degrees is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a situation awareness method for power transmission line hardware defects under extremely cold disasters in an embodiment of the invention;
fig. 2 is a schematic diagram of a situation awareness flow of a power transmission line;
fig. 3 is a flow chart of a situation awareness method for power transmission line hardware defects under severe cold disasters according to an embodiment of the present invention;
FIG. 4 is a flowchart of continuous learning model training;
fig. 5 is a schematic diagram of an attention module.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a situation awareness method for the hardware defects of a power transmission line under an extremely cold disaster, which realizes sustainable learning of a hardware defect detection model under the extremely cold disaster, can relieve the problem of low detection accuracy of certain hardware defects caused by unbalance of hardware defect data sets, improves the precision of hardware detection, saves the occupied space of the model, ensures that the model cannot forget old classification tasks while learning new classification tasks, and improves the recognition capability of the model on hardware with different defect degrees.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
As shown in fig. 1-5, the situation awareness method for the transmission line hardware defects under the severe cold disaster provided by the embodiment of the invention, as shown in fig. 1, comprises the following steps:
step 1: establishing a first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, and training the first-stage detection model through the data set;
step 2: establishing a second-stage classification model, acquiring and classifying the hardware defect data set under the severe cold disaster, and training the second-stage classification model through the classified hardware defect data set under the severe cold disaster;
step 3: cascading the first-stage detection model and the second-stage classification model, and introducing an attention mechanism into the second-stage classification model;
step 4: inputting the aerial image of the power transmission line to be detected into the first-stage detection model trained in the step 1 to obtain a hardware fitting image and a label, and simultaneously inputting the hardware fitting image and the label into the second-stage classification model trained in the step 2 to obtain the defect condition of the hardware fitting.
Situation awareness refers to the steps of awareness, understanding and prediction of future running states of elements in a system under a specific space-time background, wherein the three steps correspond to three stages of a situation awareness process respectively: situation element acquisition, real-time situation understanding and future situation prediction. As shown in fig. 2, the object of power transmission line situation sensing is hardware on the line, and after sensing the state of the hardware, the next conversion state of the hardware is predicted for assisting the patrol personnel to make decisions.
In step 1, a first-stage detection model is established, a hardware fitting target data set under extremely cold disasters is obtained, and the first-stage detection model is trained through the data set, specifically:
establishing a YOLOv4 model as a cascaded first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, training the first-stage detection model through the data set to enable the hardware fitting target to be detected on an aerial image of a power transmission line, cutting off the hardware fitting image and a tag obtained through detection to serve as input of a second-stage classification model, wherein the input aerial image of the power transmission line is made to be x global The cut hardware image is x, the labels of various hardware are y, and a formula F for detecting the hardware by utilizing the YOLOv4 model is obtained yolov4 The method comprises the following steps:
F yolov4 (x global )＝(x,y) (1)
the reason why the YOLOv4 model is used as a cascade model is that the model is high in monitoring speed and high in detection accuracy.
In the step 2, acquiring a hardware defect data set under an extremely cold disaster, and classifying the hardware defect data set, specifically:
acquiring a hardware defect data set under an extremely cold disaster, dividing the hardware defect data set according to different types of hardware, such as a bag clamp defect data set, a heavy hammer defect data set, a grading ring defect data set and the like, dividing the hardware again by taking specific defects of the hardware as subclasses after the division is finished, wherein the bag clamp data set comprises the types of rusting, dirt and the like of a bag clamp,
D＝{D 1 ,D 2 ,…,D T } (2)
in the method, in the process of the invention,d is the collection of all hardware defect data sets, D k For some hardware defect data, x i k To input an image, y i k To output labels, t i k For task labels, n k For the number of training sets, i is the hardware defect class.
In the second-stage classification model described in step 2, the feature knowledge learned by the deep learning network is divided into a shared knowledge module and a unique knowledge module, wherein the shared knowledge module learns the common image features of all training sets in all classification tasks in a manner of resistance training, and the unique knowledge module learns only the unique features in the sequence classification tasks, so that the feature knowledge learned by the shared knowledge module and the unique knowledge module are different by adopting an orthogonal constraint method.
As shown in fig. 4, the unique knowledge module only learns the unique features in the sequence classification task, specifically:
learning dissimilar characteristics of different class images in the same sequence classification task, and storing a sub-model P for each sequence classification task separately, wherein for a sequence classification task comprising c class images, it is actually a class c classification task, and the loss function is:
wherein x is k Representing input data, y k For a true class label of input data, σ is a softmax function, ideal f k Is a model that maps inputs to two separate potential spaces, one of which contains shared features for all tasks and the other of which contains features that are unique to each task.
The shared knowledge module learns the common image characteristics of all training sets in all classification tasks in an opposite training mode, and specifically comprises the following steps:
constructing a feature map Shared and a countermeasure discriminator D, and training the feature map Shared in a manner of training the resistance, wherein the loss function is as follows:
wherein t is k For task sequence number labels, when S can generate features that D can no longer predict the correct task labels, S and D training is completed, and T is the total number of sequence tasks.
The orthogonal constraint method is adopted to enable the characteristic knowledge learned by the shared knowledge module and the characteristic knowledge module to be different, and the method specifically comprises the following steps:
the feature knowledge of the shared knowledge module and the feature knowledge of the specific knowledge module are factorized by using an orthogonal constraint method, and the difference is calculated as follows:
in the method, in the process of the invention,is F-norm, P k Representing the kth P model.
Optionally, the final total loss of the second-stage classification model is the sum of the differences calculated by the loss function of the shared knowledge module, the loss function of the specific knowledge module and the orthogonal constraint method, namely:
L final ＝L adv +L task +L diff (8)
wherein L is final Is the final total loss.
In step 3, attention mechanisms are introduced into the second-stage classification model, specifically:
as shown in fig. 5, a channel dimension-based attention module is introduced into a basic network of a second classification model, and by modeling the relationship between channels, the characteristic response of the channel direction is adaptively recalibrated, so that the capability of extracting the characteristics of the model is improved, and the recognition precision of the classification model on recognition tasks with small inter-class differences is improved, wherein the formula is as follows:
X→u,X∈R W'×H'×C' ,u∈R W×H×C (9)
v C ＝ω C ×u C (13)
in the formula (9), X represents an input feature map, u represents a feature map after convolution, and the formula represents that the input feature map of W ' X H ' X C ' is convolved into W X H X C, so as to obtain C feature maps with the size W X H; in the formula (10), H and W represent the height and width of the input, u C Representing an input feature map, wherein C represents the number of channels of an input neuron in formula (11), C/r is the number of neurons in a second layer, delta represents a ReLU function, sigma represents a sigmoid function, the neurons in the first layer use the ReLU function to prevent gradient dispersion, the neurons in the second layer use the sigmoid function, and the output values are normalized in (0, 1) in formula (12)]Between the regions, equation (13) shows that the feature map v after prediction of the importance of each channel is obtained by multiplying w by u.
The invention introduces a attention mechanism, improves the capability of the model for identifying hardware defects with different defect degrees, and improves the average detection precision of the original model on 20 types of hardware defect data sets from 85.5611% to 87.2923%.
One embodiment of the invention is: as shown in fig. 3, training of a first-stage detection model and a second-stage classification model are respectively completed by using a hardware target data set and a hardware defect data set under an extremely cold disaster, in an actual detection process, after an unmanned aerial vehicle transmits back an aerial image of a transmission line under the extremely cold disaster, the image is input into the first-stage detection model, a cut hardware image and a hardware tag are output and are used as input of the second-stage classification model, a corresponding hardware special knowledge module and a corresponding sharing module are started by the second-stage classification model according to the hardware tag, defect classification of hardware is completed, when new types of hardware and defects thereof occur, the new data set can be trained on an original model directly after being classified by using defects as tags, and then the classification model can be used directly, and the model retraining by mixing new data with all old data is not needed.
According to the situation sensing method for the hardware defects of the transmission line under the extremely cold disaster, the first-stage detection model and the second-stage classification model are cascaded, wherein the second-stage classification model is an opposite continuous learning classification model, so that sustainable learning of the hardware defect detection model under the extremely cold disaster is realized, the hardware targets under the extremely cold disaster are detected through the first-stage detection model, the problem that the detection accuracy of certain hardware defects is too low due to unbalance of hardware defect data sets under the extremely cold disaster can be relieved, meanwhile, the used hardware target data sets under the extremely cold disaster do not subdivide the defect hardware images any more, which is equivalent to amplifying the hardware target data sets under the extremely cold disaster, and the accuracy of the detection hardware is improved; the characteristics of different classification tasks learned by the second-stage classification model are divided into a shared knowledge module and a special knowledge module by the shared characteristics and the special characteristics, wherein the shared knowledge module learns the shared image characteristics of all training sets in all classification tasks in an opposite training mode, and the special knowledge module learns only the special characteristics in the sequence classification tasks, so that the occupied space of the model is saved, the model is ensured not to forget the old classification tasks when learning the new classification tasks, and the orthogonal constraint method is adopted to ensure that the feature knowledge learned by the shared knowledge module and the special knowledge module is as different as possible; attention mechanisms are introduced into the feature extraction network of the second-stage classification model, so that the defect recognition capability of the second-stage classification model on hardware fittings with different defect degrees is improved.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (5)
1. A situation awareness method for power transmission line hardware defects under extremely cold disasters is characterized by comprising the following steps:
step 1: establishing a first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, and training the first-stage detection model through the data set;
step 2: establishing a second-stage classification model, acquiring and classifying a hardware defect data set under an extremely cold disaster, training the second-stage classification model through the classified hardware defect data set, and dividing feature knowledge learned by a deep learning network into a shared knowledge module and a specific knowledge module in the second-stage classification model, wherein the shared knowledge module learns common image features of all training sets in all classification tasks in a manner of antagonistic training, and specifically comprises the following steps:
constructing a feature map S and a countermeasure discriminator D, and training the feature map S in a mode of training resistance, wherein a loss function is as follows:
wherein t is k For the task sequence number label, when S can generate the characteristic that D can not predict the correct task label any more, training of S and D is completed, and T is the total number of sequence tasks;
the special knowledge module only learns special features in the sequence classification task, and specifically comprises the following steps:
learning dissimilar characteristics of different class images in the same sequence classification task, and storing a sub-model P for each sequence classification task separately, wherein for a sequence classification task comprising c class images, it is actually a class c classification task, and the loss function is:
wherein x is k Representing input data, y k For a true class label of input data, σ is a softmax function, f k A model that maps inputs to two separate potential spaces, one space containing shared features for all tasks and the other space containing features that are unique to each task;
the orthogonal constraint method is adopted to enable the characteristic knowledge learned by the shared knowledge module and the characteristic knowledge module to be different, and the method specifically comprises the following steps:
the feature knowledge of the shared knowledge module and the feature knowledge of the specific knowledge module are factorized by using an orthogonal constraint method, and the difference is calculated as follows:
in the method, in the process of the invention,is F-norm, P k Representing a kth P model;
step 3: cascading the first-stage detection model and the second-stage classification model, and introducing an attention mechanism into the second-stage classification model;
step 4: inputting the aerial image of the power transmission line to be detected into the first-stage detection model trained in the step 1 to obtain a hardware fitting image and a label, and simultaneously inputting the hardware fitting image and the label into the second-stage classification model trained in the step 2 to obtain the defect condition of the hardware fitting.
2. The situation awareness method for power transmission line hardware defects under severe cold disasters according to claim 1, wherein in step 1, a first-stage detection model is established, a hardware target dataset under severe cold disasters is obtained, and the first-stage detection model is trained through the dataset, specifically:
establishing a YOLOv4 model as a cascaded first-stage detection model, acquiring a hardware fitting target data set under extremely cold disasters, training the first-stage detection model through the data set to enable the hardware fitting target to be detected on an aerial image of a power transmission line, cutting off the hardware fitting image and a tag obtained through detection to serve as input of a second-stage classification model, wherein the input aerial image of the power transmission line is made to be x global The cut hardware image is x, the labels of various hardware are y, and a formula F for detecting the hardware by utilizing the YOLOv4 model is obtained yolov4 The method comprises the following steps:
F yolov4 (x global )＝(x,y) (5)。
3. the situation awareness method for power transmission line hardware defects under severe cold disasters according to claim 1, wherein in step 2, a hardware defect dataset under severe cold disasters is obtained and classified, and specifically comprises:
acquiring a hardware defect data set under an extremely cold disaster, dividing the hardware defect data set according to different types of hardware, dividing the hardware again by taking specific defects of the hardware as subclasses after dividing,
D＝{D 1 ,D 2 ,···,D T } (6)
wherein D is the set of all hardware defect data sets, D k For some hardware defect data, x i k To input an image, y i k To output labels, t i k For task labels, n k For the number of training sets, i is the hardware defect class.
4. The situation awareness method for transmission line hardware defects under severe cold disasters according to claim 1, wherein the final total loss of the second-stage classification model is the sum of differences calculated by a loss function of a shared knowledge module, a loss function of a specific knowledge module and an orthogonal constraint method, namely:
L final ＝L adv +L task +L diff (8)
wherein L is final Is the final total loss.
5. The situation awareness method for transmission line hardware defects under severe cold disasters according to claim 1, wherein in step 3, attention mechanisms are introduced into a second-stage classification model, specifically:
introducing a channel dimension-based attention module in the base network of the second classification model, and adaptively recalibrating the characteristic response of the channel direction by modeling the relation between channels, wherein the formula is as follows:
X→u,X∈R W'×H'×C' ,u∈R W×H×C (9)
v C ＝ω C ×u C (13)
x in formula (9) represents an input feature map, and u represents a warpThe characteristic diagram after convolution is expressed by the formula, the input characteristic diagram of W ' x H ' x C ' is convolved into W x H x C, and C characteristic diagrams with the size W x H are obtained; in the formula (10), H and W represent the height and width of the input, u C Representing an input feature map, wherein C represents the number of channels of an input neuron in formula (11), C/r is the number of neurons in a second layer, delta represents a ReLU function, sigma represents a sigmoid function, the neurons in the first layer use the ReLU function to prevent gradient dispersion, the neurons in the second layer use the sigmoid function, and the output values are normalized in (0, 1) in formula (12)]Between the regions, equation (13) shows that the feature map v after prediction of the importance of each channel is obtained by multiplying w by u.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110557643.6A CN113515829B (en) | 2021-05-21 | 2021-05-21 | Situation awareness method for transmission line hardware defects under extremely cold disasters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110557643.6A CN113515829B (en) | 2021-05-21 | 2021-05-21 | Situation awareness method for transmission line hardware defects under extremely cold disasters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113515829A CN113515829A (en) | 2021-10-19 |
| CN113515829B true CN113515829B (en) | 2023-07-21 |
Family
ID=78064612
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110557643.6A Active CN113515829B (en) | 2021-05-21 | 2021-05-21 | Situation awareness method for transmission line hardware defects under extremely cold disasters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113515829B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784685A (en) * | 2020-07-17 | 2020-10-16 | 国网湖南省电力有限公司 | Power transmission line defect image identification method based on cloud edge cooperative detection |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3392780A3 (en) * | 2017-04-19 | 2018-11-07 | Tata Consultancy Services Limited | Systems and methods for classification of software defect reports |
| CN108171117B (en) * | 2017-12-05 | 2019-05-21 | 南京南瑞信息通信科技有限公司 | Electric power artificial intelligence visual analysis system based on multicore heterogeneous Computing |
| CN108734691A (en) * | 2018-03-22 | 2018-11-02 | 广东电网有限责任公司清远供电局 | A kind of transmission line of electricity defect image recognition methods |
| CN111881937A (en) * | 2020-06-22 | 2020-11-03 | 深圳金三立视频科技股份有限公司 | Transmission line hardware target detection and defect identification method and terminal |
| CN112070715A (en) * | 2020-07-30 | 2020-12-11 | 许继集团有限公司 | Transmission line small-size hardware defect detection method based on improved SSD model |
-
2021
- 2021-05-21 CN CN202110557643.6A patent/CN113515829B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784685A (en) * | 2020-07-17 | 2020-10-16 | 国网湖南省电力有限公司 | Power transmission line defect image identification method based on cloud edge cooperative detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113515829A (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106781669A (en) | Recommend method and device in a kind of parking stall | |
| CN109993138A (en) | A kind of car plate detection and recognition methods and device | |
| CN112883648B (en) | Training method and device for automobile fuel consumption prediction model and computer equipment | |
| CN110458047B (en) | Cross-country environment scene recognition method and system based on deep learning | |
| CN111295666A (en) | Lane line detection method, device, control equipment and storage medium | |
| Lam et al. | Real-time traffic status detection from on-line images using generic object detection system with deep learning | |
| CN115996249A (en) | Data transmission method and device based on grading | |
| CN115546671A (en) | Unmanned aerial vehicle change detection method and system based on multitask learning | |
| CN116579697A (en) | Cold chain full link data information management method, device, equipment and storage medium | |
| CN116824861A (en) | Method and system for scheduling sharing bicycle based on multidimensional data of urban brain platform | |
| CN113344470B (en) | Intelligent power supply system management method based on block chain | |
| CN113515829B (en) | Situation awareness method for transmission line hardware defects under extremely cold disasters | |
| CN113177528A (en) | License plate recognition method and system based on multi-task learning strategy training network model | |
| Xia et al. | A distributed EMDN-GRU model on Spark for passenger waiting time forecasting | |
| CN112052405A (en) | Passenger searching area recommendation method based on driver experience | |
| CN111830937B (en) | Vehicle fault identification model construction and identification method and device and management system | |
| Tang et al. | An Efficientnet Based Method for Autonomous Vehicles Trajectory Prediction | |
| Chen et al. | Research on pedestrian intrusion detection in static scenes | |
| CN113807704A (en) | Intelligent algorithm platform construction method for urban rail transit data | |
| CN113762021A (en) | Complex pavement crack detection method based on recurrent neural network and attention guidance mechanism | |
| Papanikolaou | Data-driven state space reconstruction of mobility on demand systems for sizing-rebalancing analysis | |
| CN115984077B (en) | Traffic abnormal flow causal detection method and equipment | |
| Wen et al. | A novel traffic optimization method using GRU based deep neural network for the IoV system | |
| CN117523173A (en) | Efficient training method for 3D target detection task model for service lighthouse cognition | |
| Castillo-Rojas et al. | Bibliographic Review on Data Mining Techniques Used with Weather Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Zhao Zhenbing Inventor after: Liu Ruoshi Inventor after: Zhang Shaojun Inventor after: Lou Jing Inventor after: Peng Bai Inventor after: Xiao Na Inventor after: Jiang Zhigang Inventor after: Liu Qingshi Inventor after: Wang Dongsheng Inventor after: Li Xin Inventor after: Xi Yanna Inventor after: Li Jian Inventor after: Wei Lingxiao Inventor after: Wang Shu Inventor before: Zhao Zhenbing Inventor before: Liu Ruoshi Inventor before: Zhang Shaojun Inventor before: Lou Jing Inventor before: Peng Bai Inventor before: Xiao Na Inventor before: Jiang Zhigang Inventor before: Liu Qingshi Inventor before: Wang Dongsheng Inventor before: Li Xin Inventor before: Xi Yanna Inventor before: Li Jian Inventor before: Wei Lingxiao Inventor before: Wang Shu |