CN116563243A - Foreign matter detection method and device for power transmission line, computer equipment and storage medium - Google Patents
Foreign matter detection method and device for power transmission line, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116563243A CN116563243A CN202310523683.8A CN202310523683A CN116563243A CN 116563243 A CN116563243 A CN 116563243A CN 202310523683 A CN202310523683 A CN 202310523683A CN 116563243 A CN116563243 A CN 116563243A
- Authority
- CN
- China
- Prior art keywords
- image
- low
- detected
- power transmission
- illumination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 73
- 238000001514 detection method Methods 0.000 title claims abstract description 68
- 238000005286 illumination Methods 0.000 claims abstract description 111
- 238000012549 training Methods 0.000 claims abstract description 104
- 239000013598 vector Substances 0.000 claims abstract description 72
- 238000005457 optimization Methods 0.000 claims abstract description 57
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 16
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 7
- 238000005096 rolling process Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 230000002779 inactivation Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a foreign matter detection method, a device, computer equipment and a storage medium of a power transmission line, wherein a low-illumination image optimization model is trained by acquiring a low-illumination image training set of the power transmission line, and the low-illumination image optimization model can improve each image in the low-illumination image training set to perform optimization processing, so that an acquired enhanced image to be detected is clearer, the enhanced image to be detected is acquired based on the low-illumination image optimization model, the enhanced image to be detected is input into the foreign matter detection model, and a target prediction vector of the foreign matter is acquired. The power transmission line can be monitored in real time, and when foreign objects invade, the target prediction vector of the foreign objects can be accurately obtained.
Description
Technical Field
The present invention relates to the field of foreign matter detection technology of power transmission lines, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for detecting foreign matter of a power transmission line.
Background
Foreign matter invasion of a power transmission line refers to the fact that some external objects (such as branches, birds, aircrafts and the like) are mistakenly input into the power transmission line, and accidents such as line short circuit, tripping, equipment damage and the like can be caused. These accidents not only bring economic loss to power supply enterprises, but also threaten life and property security of people. Therefore, research on the foreign matter intrusion detection technology of the power transmission line has important practical significance and application value for improving the safety, reliability and stability of the power transmission line.
The existing foreign matter intrusion detection of the power transmission line is to find and position any foreign matter, such as trees, cables, personnel and the like, entering a line forbidden zone through a manual or automatic algorithm by arranging a sensor or a camera around the power transmission line, and the problem of lower detection precision exists.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a foreign matter detection method, apparatus, computer device, computer-readable storage medium, and computer program product for a power transmission line that can improve detection accuracy.
In a first aspect, the present application provides a foreign object detection method for a power transmission line. The method comprises the following steps:
acquiring a low-illumination image training set of a power transmission line;
training a low-illumination image optimization model based on the low-illumination image training set, and acquiring an enhanced image to be detected based on the low-illumination image optimization model;
and inputting the enhanced image to be detected into a foreign object detection model to output a target prediction vector of the foreign object.
In one embodiment, the acquiring the low-illumination image training set of the power transmission line includes:
acquiring to-be-detected original images corresponding to a plurality of to-be-detected power transmission lines;
removing noise and background which are irrelevant to the image content in each original image to be detected to obtain an initial image set;
and performing data enhancement and contrast enhancement processing on the initial image set to obtain a low-illumination image data set, and extracting the low-illumination image training set from the low-illumination image data set.
In one embodiment, training a low-intensity image optimization model based on the low-intensity image dataset comprises:
extracting image features of each image in the low-illumination image training set through a convolutional neural network;
and inputting the low-illumination image training set and the image features into a transducer model for training so as to construct the low-illumination image optimization model.
In one embodiment, extracting image features of each image in the low-light image training set by the convolutional neural network comprises:
rolling and pooling each image in the low-illumination image training set to obtain the image characteristics of each image; the image features are high-level abstract features.
In one embodiment, the inputting the low-illumination image training set and the image features of each original image to be tested into a transducer model to construct the low-illumination image optimization model includes:
performing feature reconstruction on the image features to obtain a feature vector sequence;
inputting the feature vector sequence to a multi-head self-attention mechanism layer and a feedforward neural network layer of the transducer model so as to enable each image feature to interact with other image features in the feature vector sequence to obtain a result feature vector sequence;
and decoding the result feature vector sequence by a decoder of the transducer model to obtain high-quality images corresponding to the images in the low-illumination image training set.
In one embodiment, obtaining the enhanced image to be measured based on the low-illuminance image optimization model includes:
and adding all the images in the low-illumination image training set and the high-quality images corresponding to all the images through residual connection to obtain enhanced images to be detected corresponding to all the original images to be detected.
In one embodiment, the original image to be detected includes two images with different exposure times in the same transmission line to be detected; the training a low-intensity image optimization model based on the low-intensity image dataset further comprises:
and calculating a loss function of the original image to be detected with long exposure time in the enhanced image to be detected and two images with different exposure time in the same power transmission line to be detected by a mean square error method, so as to minimize the difference between the enhanced image to be detected and the high-quality image corresponding to the enhanced image to be detected according to the loss function.
In one embodiment, the inputting the enhanced image to be detected into the foreign object detection model to output the target prediction vector of the foreign object includes:
extracting a multi-scale feature pyramid from the enhanced image to be detected to obtain target features;
and inputting the target characteristics into a RetinaNet head network to obtain a target prediction vector of the foreign object.
In a second aspect, the present application further provides a foreign object detection device for a power transmission line. The device comprises:
the data acquisition module is used for acquiring a low-illumination image training set of the power transmission line;
the model training module is used for training a low-illumination image optimization model based on the low-illumination image training set and acquiring an enhanced image to be detected based on the low-illumination image optimization model;
and the foreign matter detection module is used for inputting the enhanced image to be detected into a foreign matter detection model to output a target prediction vector of the foreign matter.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the method steps of any of the embodiments described above when the processor executes the computer program.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method steps of any of the embodiments described above.
In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the method steps of any of the embodiments described above.
According to the foreign matter detection method, device, computer equipment, storage medium and computer program product of the power transmission line, the low-illumination image optimization model is trained by acquiring the low-illumination image training set of the power transmission line, and the low-illumination image optimization model can improve all images in the low-illumination image training set to perform optimization processing, so that the acquired enhanced image to be detected is clearer, the enhanced image to be detected is acquired based on the low-illumination image optimization model and the original image to be detected of the power transmission line, and the enhanced image to be detected is input into the foreign matter detection model, so that the target prediction vector of the foreign matter is acquired. The power transmission line can be monitored in real time, and when foreign objects invade, the target prediction vector of the foreign objects can be accurately obtained.
Drawings
Fig. 1 is a flow chart of a foreign matter detection method of a power transmission line in an embodiment;
fig. 2 is a schematic flow chart of acquiring a low-illumination image training set of a power transmission line in one embodiment;
FIG. 3 is a flow diagram of training a low-intensity image optimization model based on a low-intensity image dataset in one embodiment;
FIG. 4 is a schematic flow diagram of constructing a low-intensity image training optimization model in one embodiment;
FIG. 5 is a flow chart of a target prediction vector of an output foreign object according to an embodiment;
fig. 6 is a flowchart of a foreign matter detection method of a power transmission line according to another embodiment;
fig. 7 is a block diagram showing a structure of a foreign matter detection device of a power transmission line in one embodiment;
fig. 8 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In one embodiment, as shown in a flow chart of a foreign matter detection method of a power transmission line in fig. 1, the present application provides a foreign matter detection method of a power transmission line, including the following steps:
step 102, obtaining a low-illumination image training set of the power transmission line.
The low-illumination image training set is an image after preprocessing an original image to be detected.
And 104, training a low-illumination image optimization model based on the low-illumination image training set, and acquiring an enhanced image to be detected based on the low-illumination image optimization model.
The training of the low-illumination image optimization model based on the low-illumination image training set can be understood as a process of performing optimization training on images in the low-illumination image training set, and the whole optimization training process forms the low-illumination image optimization model. The enhanced image to be detected refers to an image obtained by optimizing each image in the low-illumination image training set.
Step 106, the enhanced image to be detected is input to the foreign object detection model to output the target prediction vector of the foreign object.
Wherein the target prediction vector includes a target location of the foreign object and a category of the foreign object. Specifically, the size of the target prediction vector may be used to represent the target location of the foreign object, and the direction of the target prediction vector may be used to represent the type of the foreign object.
According to the foreign matter detection method for the power transmission line, the low-illumination image optimization model is trained by acquiring the low-illumination image training set of the power transmission line, and the low-illumination image optimization model can improve each image in the low-illumination image training set to perform optimization processing, so that the acquired enhanced image to be detected is clearer, the enhanced image to be detected is acquired based on the low-illumination image optimization model and the original image to be detected of the power transmission line, and the enhanced image to be detected is input into the foreign matter detection model, so that the target prediction vector of the foreign matter is acquired. The power transmission line can be monitored in real time, and when foreign objects invade, the target prediction vector of the foreign objects can be accurately obtained.
In one embodiment, as shown in a flowchart of fig. 2, the obtaining the low-illumination image training set of the power transmission line includes:
step 202, obtaining to-be-detected original images corresponding to a plurality of to-be-detected power transmission lines.
The original image to be detected comprises scene pictures of various power transmission lines to be detected, which are shot by various different shooting devices under the condition of low illumination. At least two original images to be detected with different exposure time are shot by each power transmission line to be detected, and meanwhile, the original images to be detected with long exposure time are preprocessed to reduce noise and increase contrast, and preprocessing means comprise histogram equalization, gaussian blur, median filtering, brightness enhancement and the like.
And 204, removing noise and background which are irrelevant to the image content in each original image to be detected to acquire an initial image set.
Specifically, after the original images to be measured are acquired, the original image data to be measured is cleaned to ensure that each original image to be measured is valid and free of any noise or damage. In this process, processing and repair can be performed by various image editing tools to remove background and noise that is not related to the image content.
In step 206, data enhancement and contrast enhancement processing are performed on the initial image set to obtain a low-illumination image data set, and a low-illumination image training set is extracted from the low-illumination image data set.
Among other data enhancement techniques, clipping, rotation, scaling, translation, adding noise, and the like.
In this embodiment, for obtaining a plurality of to-be-detected original images corresponding to the power transmission lines to be detected, noise and background irrelevant to image content in the to-be-detected original image data are removed to obtain an initial image set, and further, data enhancement and contrast enhancement processing are performed on the initial image set, so as to achieve the purposes of increasing the image data amount and enhancing generalization capability of a low-illumination image optimization model trained based on a low-illumination image training set. Therefore, the processed image (i.e., the low-illuminance image data set) can obtain a higher-definition image, and a higher-quality image can be obtained by the low-illuminance image optimization model trained by the low-illuminance image training set extracted from the low-illuminance image data set.
In one embodiment, as shown in the flow diagram of fig. 3, training a low-intensity image optimization model based on a low-intensity image dataset includes:
step 302, extracting image features of each image in the low-illumination image training set through a convolutional neural network.
Among them, convolutional neural networks (Convolutional Neural Networks, CNN) are a type of feedforward neural network (Feedforward Neural Networks) that includes convolutional calculation and has a deep structure, and are one of representative algorithms of deep learning (deep learning). Convolutional neural networks have the capability of token learning (representation learning) and are capable of performing a shift-invariant classification (shift-invariant classification) on input information in their hierarchical structure. The convolutional neural network comprises an input layer, a pooling layer and an output layer. Wherein the output layer outputs the class labels using a logic function or a normalized exponential function (softmax function). Specifically, in this embodiment, the above function may be implemented by using a Resnet50 (Residual Network), and the image features of each image in the low-light image training set are extracted by using the Resnet 50.
And 304, inputting the low-illumination image training set and the image features into a transducer model for training to construct a low-illumination image optimization model.
In this embodiment, a convolutional neural network is used to extract image features of each image in the low-illumination image training set, and a low-illumination image optimization model capable of enhancing image definition can be trained by using a transducer model based on the image features of each image and the low-illumination image training set.
In one embodiment, extracting image features of each image in the low-light image training set by a convolutional neural network comprises: rolling and pooling each image in the low-illumination image training set to obtain the image characteristics of each image; image features are high-level abstract features.
In this embodiment, specifically, the convolutional neural network may be a res net50, and the performing rolling and pooling processing on each image in the low-illumination image training set through the res net50 to obtain high-level abstract features of each image, so as to obtain a feature map with a fixed size (i.e., an image feature), where the fixed size refers to a size after downsampling at a fixed magnification based on a size of each image in the low-illumination image training set. The acquired image features facilitate subsequent further feature reconstruction of the image to acquire a higher definition image.
In one embodiment, a flow diagram of constructing a low-light image training optimization model is shown in FIG. 4; inputting the low-illumination image training set and the image features of each original image to be detected into a transducer model to construct a low-illumination image optimization model, wherein the method comprises the following steps of:
at step 402, image features are subjected to feature reconstruction to obtain a feature vector sequence.
Specifically, image features are expanded into a sequence of feature vectors in rows or columns, each vector representing a row or column in the feature map, with a spatial sequence relationship.
Step 404, inputting the feature vector sequence to a multi-head self-attention mechanism layer and a feedforward neural network layer of the transducer model to enable each image feature to interact with other image features in the feature vector sequence to obtain a result feature vector sequence.
Specifically, the sequence of feature vectors is encoded using an encoder of a transducer model. In the encoder of the transducer model, each feature vector is fed into a multi-headed self-attention mechanism layer and a feedforward neural network layer for processing, each vector interacting with other vectors in the sequence.
In step 406, the decoder of the transform model decodes the resulting feature vector sequence to obtain high quality images corresponding to each image in the training set of low-light images.
In this embodiment, feature reconstruction is performed on the image features to obtain a feature vector sequence, the feature vector sequence is input to a multi-head self-attention mechanism layer and a feedforward neural network layer of a transform model, so that each image feature interacts with other image features in the feature vector sequence, and a result feature vector sequence is obtained.
In one embodiment, obtaining the enhanced image to be measured based on the low-light image optimization model includes: and adding each image in the low-illumination image training set and the high-quality image corresponding to each image through residual connection to obtain the enhanced image to be detected corresponding to each original image to be detected.
In this embodiment, the images in the low-illuminance image training set and the high-quality images corresponding to the images are added through residual connection, so that the obtained enhanced images to be measured corresponding to the original images to be measured have higher similarity and are clearer than the high-quality images and the original images to be measured which are originally obtained.
In one embodiment, the original image to be detected comprises two images of different exposure times in the same transmission line to be detected; training the low-intensity image optimization model based on the low-intensity image dataset further comprises: and carrying out loss function calculation on the to-be-detected original image with long exposure time in the to-be-detected enhanced image and two images with different exposure times in the same to-be-detected transmission line by a mean square error method so as to minimize the difference between the to-be-detected enhanced image and the high-quality image corresponding to the to-be-detected enhanced image according to the loss function.
Where mean-square error (MSE) is a measure reflecting the degree of difference between an estimated quantity and an estimated quantity.
In this embodiment, the loss function calculation is performed on the to-be-detected enhanced image and the to-be-detected original image with long exposure time in two images with different exposure times in the same to-be-detected power transmission line by using the mean square error method, and the low-illumination image optimization model can be optimized based on the obtained loss function, so that the difference between the to-be-detected enhanced image and the high-quality image corresponding to the to-be-detected enhanced image is reduced, the finally obtained enhanced image to be detected is more similar to the corresponding high-quality image, namely, is more similar to the to-be-detected original image, and provides a more accurate detection scene for the foreign matter detection of the power transmission line based on the enhanced image to be detected.
In one embodiment, a low-light image verification set low-light image test set may also be extracted based on the low-light image dataset; the low-illumination image verification set is used for adjusting the super parameters of the low-illumination image optimization model, and the low-illumination image test set is used for evaluating the performance of the low-illumination image optimization model.
In this embodiment, a low-illuminance image optimization model with a higher optimization effect may be obtained by the low-illuminance image verification set and the low-illuminance image test set.
In one embodiment, a flow chart of outputting a target prediction vector of a foreign object as shown in fig. 5; inputting the enhanced image to be detected into a foreign object detection model to output a target prediction vector of the foreign object, comprising:
step 502, multi-scale feature pyramid extraction is performed on the enhanced image to be detected to obtain target features.
Among other things, multi-scale feature pyramids can be categorized into Gaussian pyramids (Gaussian pyramids) and laplacian pyramids (Laplacian pyramid). The Gaussian pyramid is a series of images obtained by gradually downsampling the bottom maximum resolution image, the lowest image resolution is highest, and the higher the image resolution is, the lower the image resolution is; the laplacian pyramid can be considered as a residual pyramid to store the difference of the downsampled picture from the original picture.
The main network of the foreign matter detection model has the same structure as the low-illumination image optimization model, but specific parameters are different.
In step 504, the target feature is input to the RetinaNet header network to obtain a target prediction vector of the foreign object.
In this embodiment, the foreign object detection model performs multi-scale feature pyramid extraction on the enhanced image to be detected, and inputs the extracted target features to the RetinaNet head network, so as to output a target prediction vector of the detected foreign object, specifically, the size of the target prediction vector of the foreign object represents the target location (coordinate) where the foreign object is located, and the direction of the target prediction vector of the foreign object represents the category of the foreign object. Therefore, based on the method, the accurate foreign matter detection can be carried out on the power transmission line to be detected, and the target positioning and the type of the foreign matter can be output.
In one embodiment, in training the foreign object detection model, the gap between the target prediction vector and the true value vector can also be measured by a loss function (focal loss) and a piecewise function (smooth-l 1 loss), and the model optimization is performed as a total loss together with the image-enhanced loss function. The truth vector refers to information of a target detection position in the low-illumination image data set, and the information of the target detection position exists in the form of a manual label when a foreign object detection model is trained. Meanwhile, parameters of the foreign object detection model are optimized using a back propagation algorithm to minimize the value of the loss function. To avoid overfitting, it is also generally necessary to employ some regularization technique, such as random inactivation (dropout) or L2 regularization. Therefore, the foreign matter detection model can detect the foreign matters on the power transmission line more accurately.
In one embodiment, a flow chart of a foreign matter detection method of another power transmission line as shown in fig. 6 is shown; the foreign matter detection method of the power transmission line comprises the following steps:
step 602, obtaining to-be-detected original images corresponding to a plurality of to-be-detected power transmission lines.
Step 604, removing noise and background irrelevant to the image content in each original image to be detected to obtain an initial image set.
In step 606, data enhancement and contrast enhancement processing is performed on the initial image set to obtain a low-illumination image data set, and a low-illumination image training set is extracted from the low-illumination image data set.
Step 608, rolling and pooling each image in the low-illumination image training set to obtain the image characteristics of each image; image features are high-level abstract features.
At step 610, feature reconstruction is performed on the image features to obtain a sequence of feature vectors.
Step 612, inputting the feature vector sequence to a multi-head self-attention mechanism layer and a feedforward neural network layer of the transducer model to enable each image feature to interact with other image features in the feature vector sequence to obtain a result feature vector sequence.
In step 614, the decoder of the transform model decodes the resulting feature vector sequence to obtain high quality images corresponding to each image in the training set of low-light images.
Step 616, adding each image in the low-illumination image training set and the high-quality image corresponding to each image through residual connection to obtain the enhanced image to be detected corresponding to each original image to be detected.
Step 618, performing multi-scale feature pyramid extraction on the enhanced image to be detected to obtain target features.
In step 620, the target feature is input to the RetinaNet header network to obtain a target prediction vector of the foreign object.
In this embodiment, to-be-detected original images corresponding to a plurality of to-be-detected power transmission lines are obtained, and preliminary processing is performed on the to-be-detected original images: removing noise and background which are irrelevant to the image content in each original image to be detected to obtain an initial image set; and performing data enhancement and contrast enhancement processing on the initial image set to obtain a low-illumination image data set, and extracting a low-illumination image training set from the low-illumination image data set. Therefore, the definition and brightness of the original image to be detected can be improved through the processing, so that the picture shot by the monitoring camera is brighter and clearer, the condition of missing detection is reduced, and the detection efficiency is improved. Further, the image features of the images in the low-illumination image training set are obtained by rolling and pooling, the image features are subjected to feature reconstruction to obtain a feature vector sequence, the feature vector sequence is input to a multi-head self-attention mechanism layer and a feedforward neural network layer of the Transformer model so that the image features interact with other image features in the feature vector sequence to obtain a result feature vector sequence, a decoder of the Transformer model decodes the result feature vector sequence to obtain high-quality images corresponding to the images in the low-illumination image training set, the images in the low-illumination image training set and the high-quality images corresponding to the images are added through residual connection to obtain to-be-detected enhanced images corresponding to the to-be-detected original images, and the contrast and detail of the images in the low-illumination image training set can be improved through the method, so that the obtained target features extracted based on the to-be-detected enhanced images and the target predicted vectors of the foreign matters obtained based on the target features are more accurate, the situations of foreign matter detection misjudgment are reduced, and the detection accuracy is improved.
It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a foreign matter detection device of the power transmission line for realizing the foreign matter detection method of the power transmission line. The implementation scheme of the device for solving the problem is similar to that described in the above method, so the specific limitation in the embodiments of the foreign matter detection device for one or more transmission lines provided below may refer to the limitation of the foreign matter detection method for a transmission line hereinabove, and will not be repeated herein.
In one embodiment, as shown in a block diagram of a foreign object detection device of a power transmission line in fig. 7, a foreign object detection device 700 provided in the present application includes: a data acquisition module 710, a model training module 720, and a foreign object detection module 730, wherein:
the data acquisition module 710 is configured to acquire a low-illumination image training set of the power transmission line.
The model training module 720 is configured to train a low-illumination image optimization model based on the low-illumination image training set, and obtain an enhanced image to be detected based on the low-illumination image optimization model.
The foreign object detection module 730 is configured to input the enhanced image to be detected to the foreign object detection model to output a target prediction vector of the foreign object.
In one embodiment, the data acquisition module is further configured to acquire to-be-detected original images corresponding to the plurality of to-be-detected power transmission lines; removing noise and background which are irrelevant to the image content in each original image to be detected to obtain an initial image set; and performing data enhancement and contrast enhancement processing on the initial image set to obtain a low-illumination image data set, and extracting a low-illumination image training set from the low-illumination image data set.
In one embodiment, the model training module is further configured to extract image features of each image in the low-light image training set through a convolutional neural network; and inputting the low-illumination image training set and the image features into a transducer model for training to construct a low-illumination image optimization model.
In one embodiment, the model training module is further configured to volume and pool each image in the low-light image training set to obtain an image feature of each image; image features are high-level abstract features.
In one embodiment, the model training module is further configured to perform feature reconstruction on the image features to obtain a feature vector sequence; inputting the feature vector sequence to a multi-head self-attention mechanism layer and a feedforward neural network layer of a transducer model so as to enable each image feature to interact with other image features in the feature vector sequence to obtain a result feature vector sequence; the decoder of the Transformer model decodes the resulting feature vector sequence to obtain high quality images corresponding to each image in the training set of low-light images.
In one embodiment, the model training module is further configured to add each image in the low-illumination image training set and the high-quality image corresponding to each image through residual connection to obtain an enhanced image to be measured corresponding to each original image to be measured.
In one embodiment, the original image to be detected comprises two images of different exposure times in the same transmission line to be detected; the model training module is also used for carrying out loss function calculation on the to-be-detected original image with long exposure time in the to-be-detected enhanced image and two images with different exposure times in the same to-be-detected transmission line by a mean square error method so as to minimize the difference between the to-be-detected enhanced image and the high-quality image corresponding to the to-be-detected enhanced image according to the loss function.
In one embodiment, the foreign object detection module is used for performing multi-scale feature pyramid extraction on the enhanced image to be detected to obtain target features; the target feature is input to the RetinaNet header network to obtain a target prediction vector of the foreign object.
All or part of the modules in the foreign matter detection device of the power transmission line can be realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 8. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a foreign object detection method for a power transmission line. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (12)
1. A foreign matter detection method for a power transmission line, the method comprising:
acquiring a low-illumination image training set of a power transmission line;
training a low-illumination image optimization model based on the low-illumination image training set, and acquiring an enhanced image to be detected based on the low-illumination image optimization model;
and inputting the enhanced image to be detected into a foreign object detection model to output a target prediction vector of the foreign object.
2. The method for detecting foreign objects on a power transmission line according to claim 1, wherein the acquiring the training set of low-illuminance images of the power transmission line includes:
acquiring to-be-detected original images corresponding to a plurality of to-be-detected power transmission lines;
removing noise and background which are irrelevant to the image content in each original image to be detected to obtain an initial image set;
and performing data enhancement and contrast enhancement processing on the initial image set to obtain a low-illumination image data set, and extracting the low-illumination image training set from the low-illumination image data set.
3. The foreign object detection method of claim 1, wherein training a low-illuminance image optimization model based on the low-illuminance image dataset includes:
extracting image features of each image in the low-illumination image training set through a convolutional neural network;
and inputting the low-illumination image training set and the image features into a transducer model for training so as to construct the low-illumination image optimization model.
4. The foreign object detection method of claim 3, wherein extracting image features of each image in the low-light image training set by the convolutional neural network includes:
rolling and pooling each image in the low-illumination image training set to obtain the image characteristics of each image; the image features are high-level abstract features.
5. The foreign object detection method of claim 3, wherein the inputting the low-illuminance image training set and the image features of each original image to be detected into a transducer model to construct the low-illuminance image optimization model includes:
performing feature reconstruction on the image features to obtain a feature vector sequence;
inputting the feature vector sequence to a multi-head self-attention mechanism layer and a feedforward neural network layer of the transducer model so as to enable each image feature to interact with other image features in the feature vector sequence to obtain a result feature vector sequence;
and decoding the result feature vector sequence by a decoder of the transducer model to obtain high-quality images corresponding to the images in the low-illumination image training set.
6. The foreign object detection method of claim 5, wherein obtaining an enhanced image to be detected based on the low-illuminance image optimization model includes:
and adding all the images in the low-illumination image training set and the high-quality images corresponding to all the images through residual connection to obtain enhanced images to be detected corresponding to all the original images to be detected.
7. The foreign object detection method of a power transmission line according to claim 5, wherein the original image to be detected includes two images of different exposure times in the same power transmission line to be detected; the training a low-intensity image optimization model based on the low-intensity image dataset further comprises:
and calculating a loss function of the original image to be detected with long exposure time in the enhanced image to be detected and two images with different exposure time in the same power transmission line to be detected by a mean square error method, so as to minimize the difference between the enhanced image to be detected and the high-quality image corresponding to the enhanced image to be detected according to the loss function.
8. The foreign object detection method of claim 1, wherein inputting the enhanced image to be detected into a foreign object detection model to output a target prediction vector of a foreign object, comprises:
extracting a multi-scale feature pyramid from the enhanced image to be detected to obtain target features;
and inputting the target characteristics into a RetinaNet head network to obtain a target prediction vector of the foreign object.
9. A foreign matter detection device for a power transmission line, the device comprising:
the data acquisition module is used for acquiring a low-illumination image training set of the power transmission line;
the model training module is used for training a low-illumination image optimization model based on the low-illumination image training set and acquiring an enhanced image to be detected based on the low-illumination image optimization model;
and the foreign matter detection module is used for inputting the enhanced image to be detected into a foreign matter detection model to output a target prediction vector of the foreign matter.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 8 when the computer program is executed.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310523683.8A CN116563243A (en) | 2023-05-10 | 2023-05-10 | Foreign matter detection method and device for power transmission line, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310523683.8A CN116563243A (en) | 2023-05-10 | 2023-05-10 | Foreign matter detection method and device for power transmission line, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563243A true CN116563243A (en) | 2023-08-08 |
Family
ID=87485631
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310523683.8A Pending CN116563243A (en) | 2023-05-10 | 2023-05-10 | Foreign matter detection method and device for power transmission line, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563243A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117036828A (en) * | 2023-09-19 | 2023-11-10 | 南方电网数字电网研究院有限公司 | Fast-growing tree monitoring method, device, equipment and medium for protecting power transmission line |
-
2023
- 2023-05-10 CN CN202310523683.8A patent/CN116563243A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117036828A (en) * | 2023-09-19 | 2023-11-10 | 南方电网数字电网研究院有限公司 | Fast-growing tree monitoring method, device, equipment and medium for protecting power transmission line |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111080628B (en) | Image tampering detection method, apparatus, computer device and storage medium | |
| CN108805015B (en) | Crowd abnormity detection method for weighted convolution self-coding long-short term memory network | |
| Huang et al. | Abnormal event detection using deep contrastive learning for intelligent video surveillance system | |
| Chang et al. | Mau: A motion-aware unit for video prediction and beyond | |
| CN110598558B (en) | Crowd density estimation method, device, electronic equipment and medium | |
| CN109522828B (en) | Abnormal event detection method and system, storage medium and terminal | |
| CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| CN111738054B (en) | Behavior anomaly detection method based on space-time self-encoder network and space-time CNN | |
| KR20200039049A (en) | Inspection method for appearance badness and inspection system for appearance badness | |
| Khan et al. | Revisiting crowd counting: State-of-the-art, trends, and future perspectives | |
| CN110826429A (en) | Scenic spot video-based method and system for automatically monitoring travel emergency | |
| Yang et al. | Anomaly detection in moving crowds through spatiotemporal autoencoding and additional attention | |
| CN110766007A (en) | Certificate shielding detection method, device and equipment and readable storage medium | |
| CN116563243A (en) | Foreign matter detection method and device for power transmission line, computer equipment and storage medium | |
| Onim et al. | Blpnet: A new dnn model and bengali ocr engine for automatic licence plate recognition | |
| CN112802076A (en) | Reflection image generation model and training method of reflection removal model | |
| Li et al. | An improved YOLOv3 for foreign objects detection of transmission lines | |
| CN115984226A (en) | Insulator defect detection method, device, medium, and program product | |
| Sun et al. | A priori-guided multi-layer rain-aware network for single image deraining | |
| CN115131695A (en) | Training method of video detection model, video detection method and device | |
| Wagner et al. | River water segmentation in surveillance camera images: A comparative study of offline and online augmentation using 32 CNNs | |
| Shit et al. | An encoder‐decoder based CNN architecture using end to end dehaze and detection network for proper image visualization and detection | |
| Liu et al. | Exploring simple and transferable recognition-aware image processing | |
| CN112132867B (en) | Remote sensing image change detection method and device | |
| CN112149596A (en) | Abnormal behavior detection method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |