CN114331837A - Method for processing and storing panoramic monitoring image of protection system of extra-high voltage converter station - Google Patents

Abstract

A panoramic monitoring image processing and storing method for an extra-high voltage converter station protection system belongs to the technical field of panoramic monitoring of extra-high voltage converter stations and aims to solve the problem that a large amount of cloud resources are occupied as panoramic monitoring image data are directly uploaded to a cloud; the method comprises the steps that a multi-scale convolution block is adopted in a depth multi-scale residual error network model to construct low-order and high-order features of multiple scales of extracted images, the incomplete extraction phenomenon of image details is avoided, a residual error learning mechanism is adopted to retain low-order rough features, and the reconstruction capability of the images is improved; the topology optimization sequentially constructs the network according to predefined topological rules by constructing a topological structure of a heterogeneous network, a framework combining deep reinforcement learning and Monte Carlo tree search; the search result of the Monte Carlo tree strengthens the learning of the deep convolutional neural network so as to obtain more accurate prediction in the next iteration; the data are subjected to lightweight processing at the edge side and then are input to the cloud for storage, so that the cloud storage space and the transmission bandwidth are saved.

Description

Method for processing and storing panoramic monitoring image of protection system of extra-high voltage converter station

Technical Field

The invention belongs to the technical field of panoramic monitoring of an extra-high voltage converter station, and relates to a method for processing and storing a panoramic monitoring image of a protection system of the extra-high voltage converter station.

Background

With the development of power grids, the interconnection scale of the power grids is continuously increased, the electrical connection in the power grids is tighter, the safety and stability problems of large power grids are more and more prominent, and the difficulty and safety risk of operation management technology are obviously increased. The safe and reliable operation of the extra-high voltage converter station plays a self-evident important role in the safe and stable operation of a power grid, so that the equipment fault needs to be manually inspected in the daily operation and maintenance process of the extra-high voltage so as to ensure the safety and stability of the system. However, the manual inspection mode has high working strength, and the inspection performance is easily influenced by experience responsibility of personnel. In order to improve the efficiency of operation and maintenance management of the extra-high voltage converter station, a panoramic monitoring system is widely deployed in the extra-high voltage converter station and used for monitoring the running state of equipment in each link.

The state signal parameters of the extra-high voltage direct current protection core link needing to be monitored by the extra-high voltage converter station protection device are as follows: A. monitoring the state of the outlet pressure plate; B. measuring the temperature of the terminal row in the screen cabinet; C. monitoring a front panel of secondary equipment in the screen cabinet; D. the working temperature of secondary equipment in the screen cabinet; E. working voltage of secondary equipment in the screen cabinet; F. monitoring the light intensity of the optical fiber; G. detecting the insulation of the cable; H. detecting an outlet loop; I. the position of the auxiliary contact; J. detecting the state of the cable; K. detection of parameters of the environment, such as temperature, humidity, etc.; and L, corrosion state of the wiring terminal.

However, the operating environment and equipment of the monitoring system are used for a long time, shaking caused by vibration cannot be avoided, and the interference of dust deposition, spider web and the like on a lens can be avoided, so that a video image is blurred, and the acquisition of panoramic monitoring data is not accurate.

The traditional image enhancement reconstruction method generally utilizes the method of improving the image contrast to highlight the target scenery, and mainly comprises the methods of histogram equalization, logarithmic transformation, sharpening, wavelet transformation, Retinex with different scales and the like. The method has low computing resources and strong portability, but has limited enhancement effect as a general algorithm, and the processed image is difficult to meet the requirements of panoramic monitoring in a specific scene. Image-enhanced reconstruction is a classic research topic in computer vision, and Single Image Super Resolution (SISR) is an important component of Image-enhanced reconstruction. SISR utilizes a group of low-quality and low-resolution images to generate a single high-quality and high-resolution image, obtains a region of interest with higher spatial resolution, realizes the concentration analysis of a target object, and enables the image to realize the conversion from a detection level to an identification level or further realizes the conversion to a fine resolution level so as to improve the identification capability and the identification precision of a panoramic monitoring image of the converter station.

The current SISR algorithm can be roughly divided into three types, namely interpolation-based algorithm, reconstruction-based algorithm and deep learning-based algorithm. The interpolation algorithm has low calculation amount and high real-time performance, but lacks the characteristics of external information, so that the high-frequency characteristics are lost after the image is degraded, and the generated image has obvious blurring and ringing effects. Compared with an interpolation algorithm, the effect based on the reconstruction algorithm is more obvious, but the problem of smooth and fuzzy image high-frequency characteristics is solved along with the increase of the reconstruction multiple. In recent years, depth learning-based methods have become mainstream, and HR images with more High-frequency details are learned by using a mapping relationship between an observed Low Resolution (LR) image and an original High Resolution (HR) image and a large number of training samples, but reconstructed images still have the defects of detail feature distortion and High computational complexity. Convolutional neural networks are widely used for visual analysis due to their powerful image feature learning capabilities. In recent years, SISR algorithms based on convolutional neural networks have been proposed and achieve significant performance gains. Document "Image Super-Resolution Using Deep relational Networks" (c.dong, IEEE Transactions on Pattern Analysis and Machine Analysis, published as 2016), proposes a CNN model named SRCNN, which replaces dictionary modeling with automatic adjustment of hidden layer parameters, learns a non-linear mapping relationship between low-Resolution input and high-Resolution output, improves reconstruction accuracy, and reduces calculation time. However, there are some disadvantages in the SRCNN, such as that bicubic interpolation may cause edge blurring and jagged edges in the image, and in the case of the constant model parameter quantity, the larger the super-division multiple indicates the larger the resolution of the input, the higher the calculation quantity of the model. The document "accumulating the Super-Resolution comprehensive Neural Network" (Chao D, European Conference on Computer Vision, 2016) proposes an improved algorithm FSRCNN for the defect of slow training of the SRCNN, and performs up-sampling by deconvolution, and simultaneously performs dimension reduction by using 1 × 1 convolution, and reduces the calculation amount of the model to accelerate the training speed. The core of ResNet is to add a jump connection between the convolutional layer output and its previous convolutional layer input to solve the problem of gradient disappearance. H (x) represents the base layer map fitted by several superimposed convolutional layers, the input of the first convolutional layer being x, x being connected to the output of the last convolutional layer. The stacked layers only need to learn the mapping f (x) h (x) -x, if f (x) is zero, the residual unit can fit the identity mapping.

Due to the fact that multiple network communication and service data in panoramic monitoring of the extra-high voltage direct current converter station are different in concurrence, if topological connection of a heterogeneous network is unreasonable, dynamic unbalance of data flow access to the network can be caused, overall performance of data transmission is not high, data jam can be caused in serious situations, and reliability of the network is affected. Therefore, topology optimization of the heterogeneous network is required, the topology structure of the network is improved, and the transmission performance of the network is improved.

In recent years, the topology optimization problem of a heterogeneous network is widely concerned, and a document 'logic topology design of a multi-interface multi-channel wireless Mesh network' (bag study, small microcomputer system, etc.), the disclosure date of which is 2015, proposes a logic topology design method under the constraint of topology reliability for the topology optimization problem. A new self-adaptive distributed topology control algorithm is provided in a document 'distribution network self-adaptive protection and self-healing control method based on dynamic topology analysis' (Zhang Anlong, protection and control of a power system), the publication date of which is 2019, and the connectivity of a network when a node fails is ensured by adjusting the transmission capability of the node in different states. Aiming at the problem of data transmission, the tree topology can better transmit the acquired data than other network topologies, and has strong anti-interference capability. A tree-Based algorithm is proposed in a document of improvement of Capacity of a Mesh LoRa Network by Spreading-Factor-Based Network aggregation (Zhu G, Liao C H, Sakdejayant T, et al. IEEE Access) with a publication date of 2019, has a set of heuristic rules to construct a tree topology in a multi-hop wireless Network, and has the advantages of the tree topology, namely efficient data transmission and data aggregation are carried out through non-leaf nodes in the tree. The throughput of the heterogeneous network is a main standard for measuring the advantages and disadvantages of the established network model, and currently, most of the heterogeneous networks have network topologies constructed by well transmitting data, such as a cluster-based topology and a tree-based topology. The performance of the topology in the heterogeneous network indicates that the quality of the topology has a great influence on data transmission, and therefore, the construction of the network topology should be based on specific network requirements, and it is required to meet various network types and network transmission performance as much as possible.

The network topology optimization method considers the quality and the safety of data transmission, but the optimization process takes longer time when the topology changes, and the performance requirement of the power industry data transmission network is difficult to meet. Finding a topological structure with optimal reliability in a heterogeneous network of an extra-high voltage direct current transmission system is a combination problem essentially. A minimum spanning tree topology optimization method is provided in a literature 'minimum spanning tree algorithm research of a ventilation network based on a weight matrix' (Roc, railway science and engineering reports, etc.) with a publication date of 2018, a heuristic rule is utilized to reduce the number of candidate searches, so that a certain sub-optimal solution is obtained, but the requirements of a communication network for fast reconstruction of an extra-high voltage direct current converter station on the real-time performance and reliability of solution when the network fails are still difficult to meet, and due to the fact that the search space of all possible topology configurations is very large, the complexity of the optimal network configuration is exponentially increased through exhaustive search.

The screen cabinet and the outdoor terminal box belong to the extra-high voltage converter station protection device, and most core protection information is fed back to the indoor screen cabinet and the outdoor terminal box, so that special attention needs to be paid to the protection states of the screen cabinet and the outdoor terminal box. The terminal box erected outside the small chamber and the wiring terminals inside the terminal box can be rusted due to the moist, dusty and closed environment, so that the normal operation of the converter station is threatened, and the safety of the whole power system is damaged.

In the prior art, a document "corosion identification of matching based computer vision" (Zhiren Tian, 2019International Conference on technical intelligent and Advanced Manufacturing (AIAM), published in 2019, performs identification, segmentation and detection of a corrosion region through an HSI space and an RGB model, respectively, for color characteristics of a corrosion fault. A document, quality of quality for correlation of correlation in images (wil Nash, Research Gate), published in 7.2018, segments and extracts a rust scene; in a document, namely a cable tunnel rust identification algorithm based on a migration learning convolutional neural network (Zhouyi, China power) published in 4 months 2019, the problem of small data samples is solved by introducing migration learning, and the rust detection effect is improved to a certain extent. The target detection algorithms of the documents are all dependent on a large convolutional neural network, and the algorithm model has the problems of overlarge parameter quantity, too low detection speed and the like, so that the real-time response requirement of the corrosion detection of the extra-high voltage converter station protection system cannot be met.

Therefore, the corrosion state of the terminal box outside the small chamber needs to be monitored in real time through the monitoring system, and after the camera is determined, the upper computer always collects the corrosion data of the wiring terminal under the same condition, so that the monitoring data in the small chamber of the ultra-high voltage converter station has repeatability, the uploading significance of a large amount of repeated data is not great, and a large amount of storage space and transmission bandwidth of a cloud end are occupied; therefore, how to upload a large amount of repeated wiring terminal corrosion picture data to the cloud after carrying out lightweight processing avoids a large amount of repeated data to occupy cloud resources, and is the problem that needs to be solved urgently in the current extra-high voltage converter station protection system panoramic monitoring system.

Sometimes, the extra-high voltage converter station protection device needs to be operated, but the device does not operate, under the condition that a monitoring method is not available in the prior art, field operators do not know the situation, after monitoring facilities are installed subsequently, due to subjective factors such as experience and responsibility of the operators, the situation of missing detection can occur, and therefore the operators are assisted to judge the operation of the protection device through the technology of video analysis.

The state signal parameters of the extra-high voltage direct current protection core link needing to be monitored by the extra-high voltage converter station protection device are as follows: A. monitoring the state of the outlet pressure plate; B. measuring the temperature of the terminal row in the screen cabinet; C. monitoring a front panel of secondary equipment in the screen cabinet; D. the working temperature of secondary equipment in the screen cabinet; E. working voltage of secondary equipment in the screen cabinet; F. monitoring the light intensity of the optical fiber; G. detecting the insulation of the cable; H. detecting an outlet loop; I. the position of the auxiliary contact; J. detecting the state of the cable; K. detection of parameters of the environment, such as temperature, humidity, etc.; and L, corrosion state of the wiring terminal. The screen cabinet and the outdoor terminal box belong to the extra-high voltage converter station protection device, most core protection information is fed back to the indoor screen cabinet and the outdoor terminal box, so that the protection states of the screen cabinet and the outdoor terminal box need to be paid special attention, and in the state signal parameter of the extra-high voltage direct current protection core link, the state monitoring of an outlet pressing plate and the monitoring of a front panel of secondary equipment in the screen cabinet are both in the screen cabinet, so that the screen cabinet needs to be subjected to image monitoring.

The screen cabinet is subjected to image monitoring mainly by adopting video monitoring which is widely applied to an extra-high voltage direct current converter station to monitor the running state of equipment in each link. Because the information bandwidth of the image is limited, the requirement on the storage capacity and the capacity of the cloud disk is high when the video is transmitted to the cloud. After the camera and the illumination condition are determined, the upper computer always collects data under the same condition, so that the monitoring data in the small chamber of the extra-high voltage converter station has repeatability, the significance of uploading a large amount of repeated data is not very large, and a large amount of storage space and transmission bandwidth of a cloud end are occupied. The distributed data processing and fault analysis technology research of the edge side of the ultra-high voltage direct current protection system based on the light artificial intelligence puts a large amount of data processed at the cloud end at the edge end for processing, so that the realization of light weight is a problem which needs to be solved urgently at present in an ultra-high voltage converter station.

The outer side of the screen cabinet of the extra-high voltage converter station is provided with a glass cover, the whole screen cabinet is arranged in the glass cover, the screen cabinet is provided with a pressing plate, a test data display window, a switch, a handle, a state indicator lamp and the like, and the condition of the screen cabinet of the extra-high voltage converter station can be observed by shooting pictures of the screen cabinet of the extra-high voltage converter station through a camera arranged in a small chamber of the extra-high voltage converter station. The traditional power equipment mainly uses manpower as main information processing of monitoring images, the whole efficiency is not high, and the accuracy of distinguishing the running state of the equipment varies from person to person. The electric power vision technology concept, the current research situation and the prospect [ J ] are provided in the literature [ Zhao Zhang Wei, Zhai Yongjie, and the like ]. electric power science and engineering, 2020,36(01):1-8 ], a bridge is established among the fields of electric power systems, computer vision, artificial intelligence and the like, along with the development of artificial intelligence and computer vision technology, a large amount of machine vision-based ultrahigh voltage direct current converter station equipment panoramic monitoring and research work has appeared, the running state characteristic information and the judgment of equipment can be efficiently and accurately acquired, further, the effective video inspection work is completed, a large amount of manpower is saved, and the inspection efficiency is improved.

However, in the video inspection process of the converter station equipment, certain specific scenes are influenced by reflected light, and the collected images can reflect light in a large area, so that the image information processing effect is influenced. Although some scenes can be solved by changing the directions of the cameras, in the practical process of monitoring the screen cabinet of the secondary equipment, the problem that the partial region of the acquired image is difficult to monitor due to the fact that the surface glass of the screen cabinet reflects light always exists, and therefore subsequent machine vision tasks such as target detection and semantic segmentation are influenced to a great extent. In severe cases, the reflection interference area even covers all areas to be detected, so that tasks cannot be completed or interrupted, which is an unsolved problem in the panoramic monitoring research of the extra-high voltage direct current converter station equipment.

Disclosure of Invention

The invention aims to design a method for processing and storing a panoramic monitoring image of an extra-high voltage converter station protection system, so as to solve the problem that the panoramic monitoring image data of the existing extra-high voltage converter station protection system is directly uploaded to a cloud end and occupies a large amount of cloud end resources.

The invention solves the technical problems through the following technical scheme:

the method for processing and storing the panoramic monitoring image of the protection system of the extra-high voltage converter station comprises the following steps:

s1, performing super-resolution reconstruction on the panoramic monitoring image, wherein the reconstruction method comprises the following steps:

s11, establishing a depth multi-scale residual error network model on the edge side;

s12, inputting a sample data set, and training a depth multi-scale residual error network model;

s13, testing the peak signal-to-noise ratio and the structural similarity index of the network by adopting a standard data set for the trained depth multi-scale residual error network model;

s14, inputting the panoramic monitoring image of the extra-high voltage converter station into the trained depth multi-scale residual error network model to complete super-resolution reconstruction;

s2, optimizing ubiquitous heterogeneous network transmission topology monitored by the ultra-high voltage converter station panorama, wherein the optimizing method comprises the following steps:

s21, modeling the heterogeneous network of the extra-high voltage converter station into a tree structure, wherein the tree structure is provided with a main station v₀And N-1 data transfer nodes v₁,v₂,...,v_N-1V to the primary station, each data transfer node having a node v to the primary station₀The unique path of (a);

s22, setting the master station v₀As a root node of the tree structure, carrying out a Monte Carlo tree recursive search on each state by taking the root node as an initial state to obtain a training data set;

s23, inputting the training data set obtained by searching into a deep convolutional neural network for training to obtain a value function and a strategy function, and guiding a Monte Carlo tree to recursively search a state with expected reward and updating and collecting the training data set of the deep convolutional neural network in return;

s24, after training, starting from the initial state S₀Starting with 0, by sequentially selecting a in a strategy for deep convolutional neural network prediction_t～π(s_t) And update the state s_t+1＝T(s_t,a_t) Until the complete tree is reached, so as to obtain a heterogeneous network topology;

and S3, transmitting the panoramic monitoring data to the edge side by optimizing the heterogeneous network transmission topology, and transmitting the panoramic monitoring data to a cloud storage after carrying out lightweight processing on the panoramic monitoring data at the edge side.

Further, the depth multi-scale residual error network model comprises: an input convolutional layer, an output convolutional layer, and k multi-scale convolutional blocks; the input end convolution layer is used as an encoder to extract original low-order characteristics of a low-resolution image; the output end convolution layer is used for fusing multi-scale detail features to reconstruct a high-resolution image; the input end convolution layer and the output end convolution layer are in jump connection, and an identity mapping from a low-resolution image to a high-resolution image is established so as to carry out global residual learning; the k multi-scale volume blocks are sequentially stacked and connected and are used for obtaining the depth of the network model; the original low-order features are correspondingly connected with the k multi-scale volume blocks through k paths, and the ability of a network model to learn complex features is enhanced through local residual error learning;

furthermore, the input convolution layer and the output convolution layer both adopt convolution kernels with the step length of 1, and the input convolution layer is activated by Relu; the multi-scale convolution block respectively uses convolution kernels of 3 x 3, 3 x 2, 2 x 3 and 2 x 2 scales to extract multi-level detail features from an input image, then the feature maps of the four scales are spliced pairwise on an appointed dimension through a cross mechanism, then the feature maps are sent into a convolution layer with the scale of 3 x 3 to be subjected to feature mapping, and a new feature map with the same input size is generated and sent into the next multi-scale convolution block.

Further, the local residual learning is defined as follows: h^k＝G^k(H^k-1) + F; wherein G is^kFeature mapping learned for the kth multi-scale volume block, H^kFor the output of the kth multi-scale volume block, H^k-1For the output of the k-1 th multi-scale convolution block, F is extracted for the input convolution layerOriginal low-order features;

the k multi-scale convolution block maps obtained by global residual and local residual learning are expressed as:

wherein, F₀() The mapping to be learned for the input convolutional layer, F_-1() Mapping to be learned for the output convolutional layer, wherein I^HR、I^LRRespectively representing a high-resolution image and a low-resolution image, G^k-1Feature mapping learned for the k-1 th multi-scale volume block, G^kR () is a mapping operation for the feature map learned for the 1 st multi-scale volume block.

Further, the loss function of the depth multi-scale residual error network model is as follows:

wherein, for the parameter of the depth multi-scale residual error network, an Adam optimizer is adopted to minimize a loss function; x⁽ⁱ⁾Is a sample data set

The ith sub-image of (1), Y⁽ⁱ⁾And N is a positive integer for the corresponding label.

Further, the panoramic monitoring image includes: secondary equipment, a hard pressing plate and a terminal corrosion image; the standard data set comprises: set5, Set14 and Urban 100.

Further, the formula for calculating the signal-to-noise ratio of the test peak is as follows:

wherein MSE is the mean square error, MAX, of the original image and the processed image_IA maximum value representing a color of the image; the calculation formula of the structural similarity index is as follows:

SSIM (X, Y) ═ L (X, Y) × C (X, Y) × S (X, Y); wherein u is_X、u_Y、σ_XAnd σ_YMeans and standard deviations, σ, of the images X and Y, respectively_XYRepresenting the covariance of images X and Y, C₁、C₂And C₃Is constant, usually take C₁＝(K₁*L)²，C₂＝(K₂*L)²，C₃＝C₂/2，K₁＝0.01，K₂L is the range of pixel values 0.03.

Further, the method for modeling the heterogeneous network of the extra-high voltage converter station into the tree structure specifically comprises the following steps: in each round of data acquisition, node v_iWill be provided with

Bit data is forwarded to its parent node, where i ∈ {1, 2.., N-1 };

is composed of v_iSelf-generated data, data sets

Is from v_iA () function is an aggregation function; using transmission models

The transmission flow of the nodes related to the topology in the transmission model consists of two parts, namely data processing and transmission time consumption;

and

respectively at node v_iThe time consumption of each bit of data processing and the time consumption of each bit of data transmission are carried out, the time consumption of each bit of data transmission depends on the distance from the parent node, and the calculation formula is as follows:

wherein the content of the first and second substances,

is node v_iWith its parent node v_jAnd p is a power amplification constant in the link budget, taking into account the shadow fading effect.

Further, the method of the monte carlo tree recursive search is as follows: each node on the Monte Carlo Tree represents 5-tuple data (s, a, M (s, a), π(s), Q^π(s, a)); at each search step t<N, an action is selected that maximizes the confidence ceiling, when the search reaches the termination state t-N, a reward is earned and propagated back along the search path to the root states of all accessed states and actions taken, Q on the path^πThe values are updated accordingly by the mean values on the nodes; wherein s is the state of the heterogeneous network; a is an operation in this state; m (s, a) is the total number of accesses (s, a) on the search tree; π(s) is the prior probability of a valid action predicted by the deep convolutional neural network; q^π(s, a) is a state action value, representing an expected reward for starting from state s and taking action a; the calculation formula of the action of maximizing the confidence upper limit is as follows:

wherein the content of the first and second substances,

is the access count of state s and, regardless of the action, c is a hyper-parameter that controls the search level.

Further, the deep convolutional neural network comprises a deep Vgg16 module, a full-link layer with softmax for policy, and a full-link layer with ReLU activation for value function; the deep Vgg16 module is composed of 2 convolutional layers with 64 convolutional filters, 2 convolutional layers with 128 convolutional filters, 3 convolutional layers with 256 convolutional filters, and 6 convolutional layers with 512 convolutional filters, wherein each convolutional filter has a 3 × 3 kernel and a maximum pooling layer.

Further, the value functionThe number satisfies the Bellman equation, indicating that the value of the current state is the reward for that state plus the expected return for the next state, the formula of the value function being:

the strategy function formula is as follows:

the invention has the advantages that:

according to the method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system, the low-order and high-order characteristics of the multi-scale extracted image are constructed by adopting the multi-scale convolution block in the deep multi-scale residual error network model, the phenomenon that the detail extraction of the image is incomplete is avoided, the low-order rough characteristics are reserved by adopting a residual error learning mechanism in the network model, the training difficulty is reduced, the reutilization of the characteristics is promoted, and the reconstruction capability of the image is improved; the reconstructed image has better structural similarity and peak signal-to-noise performance; the standard data set and the extra-high voltage converter station panoramic monitoring image data set are adopted successively to carry out image super-resolution reconstruction and target identification experiments, clearer edges and more details of the reference data set and the extra-high voltage panoramic monitoring image set are recovered, and experimental results show that the high-resolution image reconstructed by the method can meet the requirements of panoramic monitoring of inspection personnel. Sequentially constructing a topological structure of the heterogeneous network based on a topological control algorithm of deep reinforcement learning; adopting a framework combining deep reinforcement learning and Monte Carlo tree search, and sequentially constructing a network according to a predefined topological rule; the deep convolutional neural network is trained to predict the transmission flow of the partially established topology and guide the Monte Carlo tree to perform search in a more promising area in a search space; the search result of the Monte Carlo tree strengthens the learning of the deep convolutional neural network so as to obtain more accurate prediction in the next iteration; the data are subjected to lightweight processing at the edge side and then are input to the cloud for storage, so that the cloud storage space and the transmission bandwidth are saved.

Drawings

FIG. 1 is an architecture diagram of a depth multi-scale residual network model of an embodiment of the present invention;

FIG. 2 is a block diagram of a multi-scale volume block of an embodiment of the present invention;

FIG. 3 is a graph of PSNR performance at different network model depths according to an embodiment of the present invention;

FIGS. 4,5 and 6 are graphs comparing the reconstruction effect of the secondary device monitoring image, the hard pressure plate image and the terminal corrosion image of the method and other algorithms according to the embodiment of the invention;

FIG. 7 is a heterogeneous network model of an embodiment of the present invention;

FIG. 8 is a tree structure of a heterogeneous network according to an embodiment of the present invention;

FIG. 9 is a depiction of two steps of a finite time domain Markov decision process of an embodiment of the present invention;

FIG. 10 is a process for Monte Carlo tree searching according to an embodiment of the present invention;

FIG. 11 is a structure of a deep convolutional neural network of an embodiment of the present invention;

FIG. 12 is the convergence and performance of the proposed DRL-TC algorithm of an embodiment of the present invention;

FIG. 13 is an evolution of the training process of an embodiment of the present invention.

FIG. 14 is a flow chart of a light-weight detection method for rusted edge sides of connecting terminals of the extra-high voltage converter station protection device according to the embodiment of the invention;

FIG. 15 is a network structure diagram of a lightweight corrosion detection model based on dual attention MobileNet according to an embodiment of the present invention;

FIG. 16 is a structural comparison graph of a standard convolution and a depth separable convolution of a dual attention MobileNet based lightweight rust detection model of an embodiment of the present invention;

FIG. 17 is a flowchart of the operation of the cascaded attention model of the dual attention MobileNet based lightweight corrosion detection model according to an embodiment of the present invention;

fig. 18 is a detection result diagram of a light-weight detection method for the rusted edge side of the connecting terminal of the extra-high voltage converter station protection device in the embodiment of the invention.

Fig. 19 is a schematic structural diagram of a screen cabinet pressure plate monitoring and interference removing network for an extra-high voltage converter station according to an embodiment of the present invention;

fig. 20 is a schematic structural diagram of a residual block in a screen cabinet pressure plate monitoring and interference removing network of an extra-high voltage converter station according to an embodiment of the present invention;

fig. 21 is a multi-stage connection series ablation experimental result of a public data set (PSNR1, SSIM1) and a screen cabinet pressing plate state image data set (PSNR2, SSIM2) in a screen cabinet pressing plate monitoring and reflection interference removing network of an extra-high voltage converter station according to an embodiment of the present invention;

FIG. 22 is a de-reflection visual processing result of a real natural landscape image in a screen cabinet pressure plate monitoring de-reflection interference network of an extra-high voltage converter station according to an embodiment of the present invention;

fig. 23 is a reflection removing visual processing result of a converter station cabinet pressing plate image in an extra-high voltage converter station cabinet pressing plate monitoring reflection removing interference network according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The technical scheme of the invention is further described by combining the drawings and the specific embodiments in the specification:

example one

As shown in fig. 1, the super-resolution reconstruction method for the panoramic monitoring image of the extra-high voltage converter station protection system comprises the following steps:

super-resolution reconstruction of panoramic monitoring image

1.1, establishing a depth multi-scale residual error network model at the edge side

1.1.1, depth Multi-scale residual network (DMRN)

Fig. 1 is a depth Multi-scale residual network architecture, which consists of a convolutional layer, k Multi-scale convolution blocks (MC blocks) and hopping connections. The stacking of the k multi-scale convolution blocks is used for obtaining larger depth, and meanwhile, the convolution operation of the convolution blocks with different scales and small kernels is improved so as to extract detail features on different scales of the image for fusion, so that the reconstruction capability of the network on the micro texture and the macro geometric features of the input panoramic monitoring image is improved, and the HR image with more vivid detail information is generated. And a residual error structure is added in the training process of the network, so that the characteristic multiplexing is realized, the network redundancy is reduced, the network convergence speed is increased, and the problem of vanishing gradient is solved.

1.1.2, Multi-Scale volume Block

The DMRN uses a multi-scale volume block architecture to perform super-resolution tasks. The convolutional layers with different scales form a multi-scale convolutional block, and different levels of detail features can be generated and combined.

Fig. 2 is a block diagram of a single multi-scale volume block, where x represents the input of the multi-scale volume block and y is the output of the convolution block. The convolution blocks with different scales can extract details with different frequencies, in each multi-scale convolution block, convolution kernels with four scales of 3 x 3, 3 x 2, 2 x 3 and 2 x 2 are respectively used for extracting multi-level detail features from an input image, then feature maps with four scales are spliced two by two on an appointed dimension through a cross mechanism, then the feature maps with the four scales are sent into the convolution layer with the scale of 3 x 3 for feature mapping, a new feature map with the same input size is generated, and the new feature map is sent into the next multi-scale convolution block. The multi-scale convolution block better reserves the edge information of the image and increases the detail information of the reconstructed high-resolution image.

1.1.3 residual learning mechanism

The DMRN network architecture introduces a global residual learning mechanism and a local residual learning mechanism to carry out network training; due to the similarity between the low-resolution image and the high-resolution image, the DMRN establishes an identity mapping from the low-resolution image to the high-resolution image through a jump connection between input and output so as to perform global residual learning.

The reasons for using local residual learning are two: first, the detail required in high resolution reconstruction is the sum of high frequency features and low order features, the first convolutional layer in fig. 1 as an encoder extracts the original low order features of the low resolution image, and local residual learning can preserve the low order features. Secondly, multiple paths exist between the low-order features and the multi-scale volume blocks, and the capability of network learning of more complex features can be enhanced through local residual learning.

Local residual learning is defined as follows:

H^k＝G^k(H^k-1)+F (1)

wherein G is^kFeature mapping learned for the kth multi-scale volume block, H^kF is the original low-order features extracted from the first convolutional layer, which is the output of the kth multi-scale convolutional block.

Let F₀The mapping to be learned for the first convolutional layer (with ReLU), F_-1The mapping that needs to be learned for the last convolutional layer (without ReLU), then the k multi-scale convolutional block mappings learned based on the global and local residuals can be expressed as

I^HR＝R(I^LR)＝I^LR+F_-1(G^k(G^k-1(…(G¹(F)+F)…)+F)+F) (2)

Wherein F ═ F₀(I^LR) Are the original low-level features.

1.1.4 DMRN network details

The body structure of the DMRN in fig. 1 is different from that of ResNet, and the DMRN removes the pooling layer and the batch normalization layer. This is because the goal of SISR is to achieve accurate pixel prediction, and removing the pooling layer is beneficial to preserve more image detail. The batch normalization layer normalizes the features, which eliminates the range flexibility of the network and is not beneficial to image reconstruction, so that the features are also removed. The DMRN uses a convolution kernel with step size 1 and is activated using ReLU, so that any size image can be accepted as input. In addition, the DMRN uses two convolutional layers of size 5 × 5 in the first and last layers to extract coarse features and fuse the multi-scale detail features to reconstruct the HR image.

1.2, inputting sample data, and training the depth multi-scale residual error network model

And selecting 800 monitoring images collected by the panoramic monitoring system of the extra-high voltage converter station, wherein the resolution of the images is 1600 x 1200. The high resolution image is first reduced to 1/3 with the original resolution using a bicubic difference algorithm, and then resized to the original image size. 24000 sub-images with the size of 32 x 32 are selected as a data set from the adjusted image according to the step size of 32

Wherein N is 24000, X⁽ⁱ⁾Is the ith sub-image, Y⁽ⁱ⁾Is the corresponding label. Randomly choose 80% of the images as training set and the rest 20% as test set. Using Mean Square Error (MSE) as a loss function of the network:

and theta is a parameter of the DMRN, and an Adam optimizer is adopted to minimize a loss function.

1.3, testing and analyzing the trained deep multi-scale residual error network model by adopting a standard data set

After the training of the DMRN network is completed, the test is firstly carried out by using three standard data sets: set5, Set14, and Urban 100. Since human vision is more sensitive to brightness variation, the image is converted into YCbCr space, and Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) on the Y channel are used to evaluate the performance of super-resolution reconstruction.

PSNR is defined as the ratio of the maximum power of a signal to the noise power, in decibels (dB), and is often used to evaluate the quality of image compression, with larger values indicating more realism in the resulting image. The PSNR is calculated as follows:

wherein MSE is the mean square error, MAX, of the original image and the processed image_IRepresenting the maximum value of the image color.

The SSIM can evaluate the similarity between an original image and a processed image, the value range is [0, 1], the larger the numerical value is, the smaller the image distortion is, and the calculation formula of the SSIM is as follows:

SSIM(X,Y)＝L(X,Y)*C(X,Y)*S(X,Y) (8)

wherein u is_X、u_Y、σ_XAnd σ_YMeans and standard deviations, σ, of the images X and Y, respectively_XYRepresenting the covariance of images X and Y, C₁、C₂And C₃Is constant, usually take C₁＝(K₁*L)²，C₂＝(K₂*L)²，C₃＝C₂/2，K₁＝0.01，K₂L is the range of pixel values 0.03.

The depth of the DMRN is determined by the number of the multi-scale volume blocks, and a model with different numbers of the multi-scale volume blocks (k ═ {8,10,12,14}) is selected, as shown in fig. 3, the average PSNR and SSIM performance of 50 randomly selected images in the Set5, Set14 and Urban100 test data sets is given, and as the number of the multi-scale volume blocks increases, the PSNR performance of the DMRN on the Set5, Set14 and Urban100 is steadily improved, which indicates that the method of the present invention achieves the expected target of "deeper is better". However, too deep a network also causes a problem of increasing computational complexity, and k-14 has a limited performance improvement compared with k-12, so that the parameter setting of k-12 is adopted in subsequent experiments.

The values of SSIM and PSNR tested for the standard data sets Set5, Set14 and Urban100 are shown in tables 1-2, respectively. The table is also compared to other methods, including Bicubic (Bicubic), SRCNN, and FSRCNN.

TABLE 1 Set5, Set14, and Urban100 data Set Structure similarity indices

TABLE 2 Set5, Set14 and Urban100 data sets Peak SNR

Here, a DMRN with k being 12 is selected as a comparison model. As can be seen from the table, the average SSIM values of the SRCNN, FSRCNN and DMRN algorithms are 0.7784, 0.7827 and 0.8082, respectively, and the structural similarity of the algorithm of the present invention is increased by 0.0043 and 0.0298, respectively. The PSNR of SRCNN, FSRCNN and DMRN are respectively 27.50dB, 27.67dB and 28.33dB on average, and the algorithm of the invention is respectively improved by 0.17dB and 0.83 dB. The result shows that the algorithm can establish the nonlinear mapping relation from LR to HR by fusing low-order and high-order characteristics and adopting a mode of combining global residual errors and local residual errors.

1.4, inputting the panoramic monitoring image of the extra-high voltage converter station into a trained depth multi-scale residual error network model to complete super-resolution reconstruction

Fig. 4, fig. 5 and fig. 6 respectively show super-resolution reconstruction effect diagrams of the panoramic monitoring image of the extra-high voltage converter station, including secondary equipment, a hard pressing plate and a terminal corrosion image. The method of the present invention was compared to Bicubic, SRCNN and FSRNN, and quantitative experimental results are given in tables 3 and 4. The images before and after reconstruction are respectively input into a YOLOV3 recognition model used in the extra-high voltage converter station, and the obtained recognition results are shown in Table 5, and experimental results show that compared with other methods, the DMRN has better SSIM and PSNR performance, and recovers clearer edges and more details, such as an indicator light and corresponding fuzzy character information in a first image, a hard pressing plate switch state and character display in a second image, and a terminal corrosion state in a third image, so that the method can better help an inspector to perform panoramic monitoring.

TABLE 3 structural similarity index of monitoring image of ultra-high voltage converter station

TABLE 4 Surveillance image peak SNR for UHV converter station

TABLE 5 Extra-high voltage converter station monitoring image recognition results

The invention provides a depth multi-scale residual error network to realize super-resolution rapid reconstruction of a panoramic monitoring image of an extra-high voltage converter station protection system so as to meet the requirement of panoramic monitoring of inspection personnel. In the DMRN, a multi-scale rolling block is adopted to construct low-order and high-order characteristics of a multi-scale extracted image, so that the problem of incomplete extraction of image details is solved. The network keeps low-order rough features by residual learning, reduces training difficulty, promotes reuse of the features, and further improves reconstruction capability of images. Experimental results show that compared with other methods, the DMRN has better SSIM and PSNR performances, the clearer edges and more details of the standard data set and the extra-high voltage panoramic monitoring image set are recovered, the quality of high-resolution image reconstruction is improved, and the requirements of inspection personnel on the panoramic monitoring of the extra-high voltage converter station protection system are met.

Second, optimizing ubiquitous heterogeneous network transmission topology for panoramic monitoring of extra-high voltage converter station

2.1, establishing a heterogeneous network model of the extra-high voltage converter station

In order to ensure the stable operation of the extra-high voltage converter station, panoramic monitoring needs to be performed on a plurality of devices of the converter station, but networks adopted by different devices for data transmission are different, so that the whole network is heterogeneous, as shown in fig. 1. Aiming at the phenomenon of dynamic imbalance of a data stream access network caused by unreasonable topological connection of a heterogeneous network, topological optimization needs to be carried out on the heterogeneous network so as to meet the requirement of network communication performance.

TABLE 6 notation used in the network model

In the embodiment, a ± 1100 kv converter station is taken as an example, and a heterogeneous network is modeled into a tree structure for the heterogeneous network model shown in fig. 7, as shown in fig. 8, the structure has a main station v₀And N-1 data transfer nodes v₁,v₂,...,v_N-1Where each node has a master station v to₀Is determined. V ═ V₀,v₁,...,v_N-1As a set of all vertices and E as a set of directed edges.

In each round of data acquisition, node v_iNeed to be provided with

The bit data is forwarded to its parent node,

calculated from equation (9):

wherein the content of the first and second substances,

is composed of v_iSelf-generated data, data sets

Is from v_iA () function is an aggregation function, i ∈ {1, 2., N-1 }.

The embodiment adopts the transmission model transmission flow shown in the formula (10), wherein the node transmission flow related to the topology mainly comprises two parts of data processing (including data receiving) and transmission time consumption. The model is shown in equation (10):

wherein the content of the first and second substances,

and

respectively at node v_iThe transmission time per bit at which data processing and transmission takes place. The time consumption per bit of data transmission depends on the distance to the parent node, and is further modeled as shown in equation (11):

wherein the content of the first and second substances,

is node v_iWith its parent node (or master station) v_iAnd p is a power amplification constant in the link budget, taking into account the shadow fading effect.

To apply reinforcement learning to the present embodimentIn the method, a primary station v is first constructed₀Is an efficient tree structure for the root node. In each step, a node that is not yet connected is selected and connected to a node on the tree or the master station until all nodes have been connected. As shown in FIG. 9, this process can be described by a 4-tuple { S, A, T, R } fully observable finite time domain Markov Decision Process (MDP). At each step t ∈ [0, N ∈ [ ]]State s of the system_tE S is the current adjacency matrix of the network

At a_tThe action at e A is to select the next node to be connected to the tree, or equivalently

Wherein node v_iNode v to be connected to the tree_j(or primary station). The system then evolves to the next state s_t+1In this case with a deterministic transfer matrix T (s, a). At terminal-reached state s_NThe reward at step t is not certain (i.e. all nodes are connected to the tree). Then the life cycle of the heterogeneous network

The return is as a reward for each action along the state track.

The energy efficiency topology optimization framework proposed by the present invention follows the following overall setup:

(1) node v_iGenerated by

The data size is a random number extracted from a certain specific distribution in the DRL-TC algorithm;

(2) the aggregation function a () may be any deterministic function, and a summation method is used in the present invention;

(3) the topology control algorithm designed should be suitable for other network objectives, such as minimizing overall network time consumption or maximizing network throughput.

Will be able to be at node v_iDelivery deviceThe total flow of the fluid is expressed as

Since it is assumed that the master station v₀Without limitation, therefore

The lifetime of the heterogeneous network is defined as the minimum transmission flow of all nodes according to the total transmission round number, and the lifetime maximization of the heterogeneous network can be expressed as:

where δ (S) is the set of edges

If v is_iIs v_jA subset of (v) then_iOtherwise, it is 0. The constraint (12b) ensures that all nodes are connected, and the constraint (12c) ensures that each node can only transmit to one parent node at a time. To approximate the complexity of the problem, if the topology is considered as an undirected spanning tree, then the number of all possible spanning trees in the network is N according to Cayley's formula^N-2. While heuristic rules may reduce the number of search candidates, it is still not feasible to enumerate all possible solutions for reasonable values of N. The invention provides a random DRL-TC algorithm which focuses on a more promising area in a search space under limited computing resources and approaches to an optimal solution under the condition of increasing computing power.

2.2 topological optimization algorithm based on deep reinforcement learning

2.2.1 reinforcement learning

Reinforcement learning is learning which actions to take in a dynamic environment to maximize the reward signal. In step t, the agent performs an action in the environment and through an instant reward r_tAn observation of an environmental state is received. The action to be taken is determined by the policy. The policy may be dependent on s_tOr it may be a random strategy using a set of action probabilities.

Prize r_tInforming the agent of the degree of demand of the current environmental state on the target is given by a reward function which may depend on s_t、a_tAnd s_t+1When the target is implemented, it will produce a high value, otherwise it will produce a low value. A series of states and actions is called track movement τ, and the discount sum of all prize values collected on a track is called reward, as shown in equation (13):

where γ is a discount factor that reduces the value of future rewards. When γ <1, the currently earned prize is more valuable than the prizes earned later. The return may be a limited level return collected over a maximum number of time steps and if desired, γ — 1 may be used. Or the reward may be an unlimited level reward, in which case γ <1 is required.

The value function satisfies the Bellman equation, which expresses that the value of the current state is the reward for that state plus the expected return for the next state. The value function and the policy function are shown in equation (14) and equation (15):

the main problem of reinforcement learning is to find a strategy that maximizes this expected return, the algorithm of which usually uses an approximation function.

2.2.2 Monte Carlo Tree search

The present embodiment approximates a policy function and a value function using the deep convolutional neural network described above, which requires a training data set of states, policies, and values in order to fit the deep convolutional neural network to a function approximator. One approach is to enumerate and collect all states and their values as a training data set. However, when the state space is large, this approach may over-fit the deep convolutional neural network, and become infeasible. This embodiment does not use heuristic rules to reduce the number of search candidates, but instead uses MCTS to efficiently collect training data sets in more promising regions of the search space. Each node on the search tree represents 5-tuple data (s, a, M (s, a), π(s), Q^π(s, a)), where s is the state of the heterogeneous network, a is the action in that state, M (s, a) is the total number of visits (s, a) on the search tree, π(s) is the prior probability of a valid action predicted by the deep convolutional neural network, and Q^π(s, a) is a state action value, which is defined as the expected reward for starting with state s and taking action a, calculated using equation (14). At each search step t<At N, an action is selected that maximizes the confidence ceiling (UCB), as shown in equation (16):

intuitively, the selection strategy initially tends to have actions with higher a priori probabilities π, but asymptotically tends to have higher state-action values Q^πThe method can be performed. As shown in fig. 10, when the search reaches the termination state (i.e., t-N), the reward is earned and propagated back along the search path to the root state of all accessed states and actions taken, Q on the path^πThe values are updated accordingly by the mean values on the nodes. The details of MCTS are described in algorithm 1, as shown in table 7.

Table 7 sets forth the MCTS subroutine for the DRL-TC algorithm

Each search begins with a particular state and recursively searches for the next state until a new leaf state or terminal state is reached. A posteriori access counts m(s) are collected by performing multiple MCTSs at each state as part of a training data set for updating the deep convolutional neural network in the next iteration.

2.2.3 deep convolutional neural network

The random strategy π(s) defines the distribution of valid actions in a state under which the system follows state s_tUp to the terminal state s_NGenerating a state and a motion trajectory h(s)_t)＝{s_t,a_t,...,s_N-1,a_N-1,s_N}. Value function V^π(s) is defined as the expected reward for all possible tracks starting from state s, as calculated by equation (7).

The present embodiment approximates the policy function and the value function f using a deep convolutional neural network_Θ(s) (parameterized by Θ) approximates an optimum function V^*(s)＝max_πV^π(s) and optimal strategy π^*(s). As shown in FIG. 11, the input to the deep convolutional neural network is the training data set { (s, π(s), V^π(s)) }. To maintain the feasibility of multi-layer neural network training while significantly improving the representation capability of deep convolutional neural networks, the present embodiment employs a deep Vgg16 module consisting of 2 convolutional layers with 64 convolutional filters, 2 convolutional layers with 128 convolutional filters, 3 convolutional layers with 256 convolutional filters, 6 convolutional layers with 512 convolutional filters, each convolutional filter having a 3 × 3 kernel, followed by a maximum pooling layer. The deep convolutional neural network is then split into two branches of convolutional layers, followed by fully-connected layers with softmax and ReLU activation for policy and value functions, respectively. By deep convolutional neural networks (pi(s), V)^π(s))＝f_Θ(s) the predicted strategy and value for each state contain a priori information that guides MCTS searchThe training data set with the high rewarding state is collected in turn for the deep convolutional neural network.

Once the deep convolutional neural network (π(s), V) is trained^π(s))＝f_Θ(s), to obtain the tree topology of the heterogeneous network, the present embodiment derives from the root state s₀Starting at 0, then sequentially selecting a from the strategy of deep convolutional neural network prediction_t～π(s_t) And update the state s_t+1＝T(s_t,a_t) Until the complete tree is reached. This embodiment notes that this topology construction is a random process that converges to a solution once the deep convolutional neural network is trained for a sufficient number of iterations.

2.2.4 self-configuring DRL-TC Algorithm

Self-configuration and self-optimization features, known as son (self organizing network), are well suited to network fabric flattening and flexibility, and therefore are of great interest. Briefly, the DRL-TC proposed in this embodiment alternates between training of a deep convolutional neural network, which provides an a priori strategy to guide the MCTS, and MCTS, which then returns a posteriori access count and state values that are used to update the deep convolutional neural network. In this way, under limited computing resources, the proposed DRL-TC algorithm will focus more on the promising search area and converge to a solution with higher rewards.

The proposed DRL-TC algorithm can also adapt to dynamic changes of the environment. For example, when a node is suddenly added or deleted, the topology rules may cause certain actions to become valid or invalid. In a new run of MCTS, the strategy pi returned by the deep convolutional neural network for this state will be renormalized for all valid actions. Thus, the new a priori policy π(s) reflects changes in the network, but still correlates with historical data. The MCTS will collect a new training data set and use it to update the deep convolutional neural network. Assuming that the network changes slower than the training time (depending on the available computing resources), the proposed DRL-TC algorithm is able to track the dynamic changes of the network and reconfigure the network topology accordingly. Algorithm 2 describes the complete algorithm of the proposed DRL-TC, as shown in Table 8.

DRL-TC Algorithm as set forth in Table 8

2.3 simulation results and analysis

2.3.1 simulation setup

In order to evaluate the performance of the DRL-TC algorithm, simulation tests are carried out on a heterogeneous network of a certain +/-1100 kV converter station. The heterogeneous network comprises a master node and 12 nodes distributed in a circular area with a radius of 1000m, and uniformly generates 500 to 1000 bits of sensing data in each round of transmission. This embodiment assumes that all nodes have sufficient time to transmit data in each round. The data transmission flow of each unit of all nodes is set as

The power amplification constant is set to ρ 1.

In each iteration of the algorithm, from N in each state_mCollecting N in MCTS for 100 searches _e10 training sample sets. The batch size B is 16 and the learning rate α is 10^-6. In the embodiment, an ADAM optimizer is used to train a deep convolutional neural network, after each iterative training, 100 network topologies are constructed by using the deep convolutional neural network, and the results are averaged, so that the performance of the algorithm is evaluated.

2.3.2 analysis of results

First, this example demonstrates the convergence and performance of the proposed DRL-TC algorithm. The solid line in fig. 12 shows the network delay time for the return of 100 realizations of the deep convolutional neural network after each training iteration, and the algorithm converges after approximately 50 iterations. Table 9 compares the performance of the proposed DRL-TC algorithm with three heuristic methods: a star topology (all nodes connected to the master), a random topology (each node randomly selects one node to connect to), and a Minimum Spanning Tree (MST) topology, where MST is weighted by the euclidean distance between nodes. The star topology has the longest network delay time due to the higher traffic at the edge nodes that are far from the master. The random topology shows a short average network delay time but varies widely. The MST topology further reduces network latency time by shortening the overall transmission distance. The DRL-TC algorithm proposed by the embodiment greatly exceeds the performance of the heuristic methods, and the convergence time of the algorithm is short.

TABLE 9 Performance comparison of DRL-TC Algorithm with three heuristic methods

FIG. 13 illustrates the proposed ability of DRL-TC to adapt to sudden changes in heterogeneous networks, showing the average network delay time after each training iteration, where points A to D of the curve in FIG. 13 show 100 topological overlays given by the DRL-TC algorithm after 1 st, 62 nd, 63 th, and 100 th iterations; point a represents that in the first iteration, the DRL-TC randomly explores a search space, because the deep convolutional neural network does not have any prior information about the state value, the network delay is relatively high; point B represents that after a plurality of iterations, the network gradually converges; when the point C represents that the heterogeneous network structure changes, the method can adapt to the heterogeneous network structure quickly; point D indicates that the algorithm converged to the optimal solution after 100 iterations.

The invention provides a novel and unified heterogeneous network topology optimization algorithm based on deep reinforcement learning. The DRL-TC algorithm can adapt to the change of the environment, and the data transmission performance better than that of other heuristic algorithms is shown to a great extent, so that the DRL-TC algorithm has good adaptability when the network topology changes, and the reliability of the network is enhanced. The DRL-MCTS framework has great potential in heterogeneous networks, and online training can be realized without intervening network services. Furthermore, with the increasing increase of computing power, the present invention anticipates that in the 5G era, DRL-MCTS will emerge other promising topology control applications in ad hoc and fully automated networks of the internet of things.

Thirdly, transmitting the panoramic monitoring data to the edge side by optimizing the heterogeneous network transmission topology, and carrying out light weight processing on the panoramic monitoring data at the edge side

3.1, as shown in fig. 14, the light weight detection method for the rusted edge side of the wiring terminal comprises the following steps:

3.1.1, collecting the rust sample data of the wiring terminal, and preprocessing the collected rust sample data

(1) The camera acquires a terminal rust sample picture from the extra-high voltage converter station, and ensures that the terminal sample with rust defects covers various types of power equipment as much as possible;

(2) standardizing collected corrosion sample data to obtain a terminal corrosion sample set X, and carrying out 7-part treatment on the sample set X: 3, dividing the ratio into a training data set and a testing data set, wherein the training data set and the testing data set are mutually independent;

(3) normalizing the normalized sample data, wherein the formula of the normalization process is as follows:

wherein, a and b are two constants respectively, and a is 0.1, and b is 0.8, which are respectively the maximum value and the minimum value of each group of factor variables; x is the number of_i，x'_iRespectively before and after normalization; x is the number of_max、x_minThe maximum and minimum values in the sample data, respectively.

3.1.2, constructing a lightweight corrosion detection model based on double attention MobileNet

As shown in fig. 15, the present invention adopts a Dual-attentive MobileNet (Dual-Att MobileNet) as a basic network, selects an SSD (single Shot multi box detector) as a basic network frame, and adopts an improved Dual-attentive MobileNet to replace a feature extraction network VGG16 of the SSD as a feature extraction network, so as to improve the operation speed on the premise of ensuring the accuracy, and simultaneously greatly reduce the calculated amount and the parameter amount, compared with VGG16, the accuracy is reduced by 0.9%, but the model parameter is only 1/32 of VGG.

An SSD (Single Shot Multi Box Detector) target detection algorithm is a deep learning one-stage target detection algorithm proposed by Liu W et al (SSD: Single Shot Multi Box Detector. European Coniference Computer Vision. Amsterdan, The Netherlands.2016.21-37.) in 2016, and The target detection capability under different scales is improved by adding a multi-scale detection mode. The SSD target identification algorithm adopts VGG-16 as a feature extraction network, removes 2 full connection layers at the tail end, uses 3 convolution layers to further extract features, and reduces the size of a feature map. In order to improve the generalization capability of the target with large scale change, the SSD uses feature maps with 6 different scales for detection. In the strategy of generating the preselection frame (prior box), the SSD uses the anchor strategy of Faster R-CNN for reference, and generates 4 to 6 anchor frames with different sizes and different length-width ratios on feature maps with different scales as preselection frames of frame regression, so that the SSD is well suitable for target objects with different length-width ratios, and the detection effect is effectively improved.

The lightweight corrosion detection model based on the dual attention MobileNet adopts feature maps with 6 scales in total, namely 38 × 38, 19 × 19, 10 × 10, 5 × 5, 3 × 3, 1 × 1, for frame prediction and target classification, wherein the feature map with the larger size in the shallow layer can be used for detecting small targets, and the feature map with the smaller size in the deep layer can be used for detecting prominent targets. Wherein the feature maps of 38, 19, 10, 5 and 5 each employ six preselected frames of different sizes and aspect ratios, and the feature maps of 3, 3 and 1 each employ four preselected frames of different sizes and aspect ratios, for a total of 11620 preselected frames; the model carries out target classification and border regression on the extracted feature maps of 6 scales, wherein the classification network outputs probability values of each class, the regression network obtains coordinate values of each prediction frame, and then non-maximum value suppression is adopted when the positions of the candidate frames are corrected.

As shown in fig. 16, MobileNet adopts a convolution method of deep separable convolution, in which a convolution kernel with a size of 1 × 1 is first used to perform convolution operation on each channel, and then a convolution kernel with a size of 3 × 3 is used to perform information exchange between channels. A large number of parameters are effectively reduced by decomposing multiplication in standard convolution into addition without losing precision, and meanwhile, an activation function is replaced by an h-swish function with more excellent performance from ReLU. The basic unit of MobileNet is depth-level separable convolution (depthwise separable convolution), and indeed this structure has been used in the inclusion model before. Depth-level separable convolution is actually a type of decomposable convolution operation (factored convolution), which can be decomposed into two smaller operations: depthwise restriction and pointwise restriction.

Let MobileNet input feature graph size be D_FThe size of the convolution kernel is D_KWhen M is the number of channels of the input feature matrix and N is the number of channels of the output matrix, the size of the standard convolution is D_F×D_KX M, then the calculated quantitative ratio of the depth separable convolution to the standard convolution is:

wherein the standard convolution is calculated by D_K×D_K×M×N×D_F×D_FThe amount of computation of the deep convolution is D_K×D_K×M×D_F×D_FThe amount of calculation of the dot convolution is 1 × 1 × M × N × D_F×D_F. Since the value of N is generally large, the ratio of the above formula depends mainly on D_KSince the present invention employs a convolution kernel size of 3 x 3, the computation of the depth separable convolution is only one ninth of the standard convolution.

As shown in fig. 17, the Dual-Att MobileNet of the present invention employs a cascade Dual-attention model to construct an attention feature for each position in each channel by simultaneous calibration of a spatial domain and a channel domain, and then employs a cascade spatial attention and channel attention module to enhance the detection effect; the cascade double attention model is formed by splicing a space attention module and a channel attention module, wherein the space attention module firstly flattens an original Feature map with the size of C multiplied by H multiplied by W into C multiplied by N by taking a channel as a unit, then transposes the C multiplied by N to obtain a Feature calibration Matrix with the size of N multiplied by N by carrying out Matrix multiplication on the two Feature matrices, each position of the Matrix represents the relation between each pixel point of the original Feature and other pixel points, at the moment, the Feature calibration Matrix is normalized and calibrated by utilizing a two-dimensional softmax function to obtain a weight Mask Matrix FFM (FMM), the value of each position in the FMM is the information content ratio occupied by each pixel point in the original Feature map, and the original Feature map can be re-calibrated by carrying out Matrix multiplication on the weight Mask Matrix and the expanded original Feature map C multiplied by N, and finally adding the same residual error structure back to the original characteristic information, wherein the main expression of the residual error structure is as follows:

in the above formula, E_cFor calibrated characteristic maps, D_iFor feature maps before transformation, A_jThe original feature map added for the residual structure,^S _ijthe weight value for the (i, j) th position is obtained by the softmax function:

in the above formula, B_iIs a feature map of N × C after expansion, C_jThe expanded C × N feature map is shown.

Then, the channel attention Squeeze operation is carried out, and the Squeeze operation enables the feature map to be reduced to a feature vector with the size of 1 multiplied by 1 after the feature map is subjected to an Average Pooling operation (Average Pooling), which is equivalent to the fact that all information quantity of the feature map is integrated on the pixel point, so that the feature map can be used as a main basis for judging feature importance; the formula of the Squeeze operation is as follows:

wherein E is a feature graph, E (i, j) is a pixel point in the feature graph, H and W are feature graph sizes:

then, performing an Excitation operation, wherein the Excitation operation is densely connected with the preceding feature vector by establishing a full connection layer, and aims to form a learnable and trainable small network for distinguishing the importance of the feature vector and providing a back propagation path, then normalizing all channel information quantity to be between 0 and 1 through a sigmoid function, and simultaneously, explicitly reflecting the information quantity occupied by each channel and forming a mask vector; the formula of the specification operation is as follows:

S＝F_ex(z,w)＝σ(w₂δ(w₁z))

wherein W is an adjustable parameter and δ is an activation function;

finally, carrying out Reweight operation, wherein the Reweight operation multiplies the mask of each channel as a weight by each pixel point of the characteristic diagram, so that the proportion of channel information quantity is weighted to each characteristic diagram, the characteristic re-calibration of the channel level is completed, and the obtained characteristic diagram is the characteristic diagram which is calibrated by space and channel double attention; the formula of the Reweight operation is as follows:

x_c＝F_re(E_c,S_c)＝S_c·E_c

3.1.3, inputting the training data set and the labels thereof into a lightweight corrosion detection model based on the double attention MobileNet for training, and inputting the test data set after training to obtain a detection result

(1) Inputting the preprocessed sample data into a basic model, training a data set in batches according to the batch size, reversely transmitting an updating parameter by adopting a random gradient descent (SDG) method and storing a weight, and stopping training after the training times reach a set iteration time to obtain a trained model; the training parameters in this embodiment are set as: the training batch size is 20, and the iteration number is 1000;

(2) inputting a test data set to the trained model, and detecting a picture or a photo stream to be detected;

(3) and judging whether the detection frames marked in the detection result are intersected or not, combining all the intersected detection frames to obtain the minimum external matrix of all the intersected detection frames, and combining the minimum external matrix to obtain the final detection result.

As shown in fig. 18, for further verifying the advantages of the algorithm provided herein in terms of model size, detection speed and detection accuracy, the detection result of the lightweight detection method for the rusted edge side of the connection terminal of the extra-high voltage converter station protection device is compared with a standard SSD model using VGG-16 and ResNet-50 as the backbone networks and a lightweight SSD model provided herein based on an attention up-sampling strategy. The criteria herein are mainly composed of accuracy (Precision), Recall (Recall), weight size and detection time, wherein the accuracy and Recall are calculated as follows:

in the formula, TP represents the number of positive sample determination errors, FP represents the number of positive sample determination errors, and FN represents the number of negative sample determination errors.

Table 10 shows the comparison of the detection effect of different algorithm models under the power equipment corrosion data set in the present text as follows:

TABLE 10 comparison of the test results of different network models

Algorithm model	Recall(％)	Precision(％)	Weight (MB)	Detection time(s)
					SSD(VGGbase)	78.04	86.49	90.58	1.84
SSD(ResNetbase)	75.61	93.94	97.02	1.24
					SSD(MobileNetbase)	63.41	83.87	15.34	0.50
The method of the invention	78.05	95.89	42.36	1.08

As can be seen from table 1, if the lightweight MobileNet structure is adopted to perform lightweight processing on the SSD model, the detection effect is deteriorated due to parameter loss, and the method of the present invention can effectively improve the detection effect even exceeding the original standard SSD algorithm by adding the upsampling and feature fusion module. In summary, compared with the method that only a lightweight MobileNet SSD model is adopted to expand the network structure on the up-sampling network, the method of the present invention increases the parameter amount by 63.7%, but compared with a standard SSD model with a huge parameter amount and using VGG-16 as a backbone network, the method increases the accuracy by 9.4% when the parameter amount is reduced by 53.23% and the speed is increased by 41.3%, and compared with a standard SSD with ResNet-50 as a backbone network, the method can also increase the accuracy by 1.95% when the parameter amount is reduced by 56.34%.

3.2 monitoring of Screen Cabinet pressure plates to remove interference from reflections

A reflected light source for monitoring the screen cabinet of the extra-high voltage direct current converter station mainly takes daylight lamp light as a main light and has the characteristics of indoor light, unnatural light, scattered light spots, high illumination intensity and the like. For example, as shown in fig. 19, the scattered light spots are covered on the display screen and the monitoring area waiting for the pressing plate at a high probability, and the mountable position of the monitoring camera in the small chamber is restricted by environmental factors such as waterproof measures and cable routing, and the monitoring camera can only be mounted above the top of the screen cabinet to monitor at a overlooking observation angle, so that the equipment state information cannot be correctly read due to the excessively high illumination intensity, the image information processing effect is affected, and great interference is brought to subsequent machine vision tasks such as target detection and semantic segmentation.

Most of existing reflection light removing deep neural networks aim at natural scenes of the real world, and are usually in outdoor and natural light environments, the illumination intensity of reflection light is small, light spots are uniformly distributed, and the difference between the illumination intensity of reflection light and the illumination characteristics of a screen cabinet of an extra-high voltage direct current converter station is large. Therefore, when constructing the reflection-removing deep neural network of the screen cabinet of the extra-high voltage direct current converter station, the operating environment and the light characteristics of the equipment in the station need to be fully considered, so that reflection interference can be efficiently and accurately removed, and the characteristic information of the operating state of the equipment can be obtained.

Aiming at the requirement of monitoring image reflection removal, the invention provides a screen cabinet pressure plate monitoring reflection removal interference network of an extra-high voltage converter station, which is shown in fig. 19 and is hereinafter referred to as an MA-Net network. The MA-Net network consists of an encoder and a decoder, the first three stages constituting the encoder part and the remaining four stages constituting the decoder part. The levels are divided according to the size of the feature map, and one chunk is defined as one phase. By connecting between the stages, the MA-Net can connect all outputs of the encoder to all inputs of the decoder, thereby allowing different sized features to be used simultaneously during image restoration. Continuing to refer to fig. 19, the MA-Net network includes 3 sequentially cascaded encoders and 4 sequentially cascaded decoders, the encoder includes two residual blocks and one WRNL block connected in sequence, the WRNL block output result of the current encoder is down-sampled and input to the residual block of the next-stage encoder, the decoder includes a convolutional layer, two residual blocks and one WRNL block connected in sequence, the convolutional layer is connected to one SE block, the SE block in the current decoder is input after the last-stage output result fused at the input end of the current decoder and the output result of each encoder, the SE block adjusts the number of channels through the convolutional layer and inputs the result to the residual block in the current decoder, the image with reflection interference is input to the first-stage encoder, and the last-stage decoder outputs the image without reflection interference. The multi-level connection mechanism, WRNL blocks, network training process and loss functions related to training in the network structure are introduced in a sub-way, and finally the effect of the method is verified through simulation.

3.2.1, Multi-level connection mechanism

In the network structure similar to U-Net, the structure of feature connection between the same levels can alleviate the defect that the low-level features in the decoder cannot utilize multi-scale information. Image de-reflection, however, is a low-level visual task that requires more dimensionally rich features to restore detail in an image. The present invention constructs a multi-level connection structure, each level is composed of two closely connected residual (DCR) blocks (as shown in fig. 20) WRNL blocks. The residual block comprises a first residual structure to a third residual structure which are sequentially numbered, the first residual structure to the third residual structure are sequentially cascaded, each residual structure comprises a 3 x 3 convolutional layer and a PReLU layer, the input end of the second residual structure receives input data of the first residual structure and an output result thereof, the input end of the third residual structure receives input data of the second residual structure and an output result thereof as well as input data of the first residual structure, and output data of the third residual structure is fused with input data of the first residual structure to serve as output of the whole residual block. In the up-sampling part of the network, feature information from all scales in down-sampling can be aggregated through a multi-level connection. Since the features of different levels have different scales, in order to be able to adaptively adjust channel characteristics after multi-level connection, a compressed excitation (SE) block is added at each decoder stage, and the number of channels after the excitation block is adjusted by 1 × 1 convolutional layer. Wherein, the SE block adopts a compressed incentive block in the prior art, and reference can be made to the related description of the compressed incentive block in the document "SE module detailed description" published on CSDN blog.

Is provided with

As the output characteristics of the level l (l ═ 1,2,3) in the encoder, the input characteristics of each level l (l ═ 4,5,6,7) in the decoder

Comprises the following steps:

wherein the content of the first and second substances,

wherein the content of the first and second substances,

denotes cascade operation, H_up(. cndot.) denotes an up-sampling operation,

decoder output characteristics, W, representing level l_1×1Denotes a 1X 1 convolutional layer, f_SE(. cndot.) represents a block of SE,

indicating a sampling operation from level i to l, i.e. at l>i, l ═ i and l<i-time l-i downsampling and i-l upsampling operations.

The multi-level connections may be used to high-level features when processing low-level features, helping the network to utilize multiple scale representations when restoring large-scale objects, and vice versa. The invention adopts discrete wavelet transform to carry out up-down sampling operation so as to search the mapping relation between characteristic shapes on different scales. In addition, in consideration of the information loss problem, the invention selects two-dimensional Haar wavelets to perform sampling operation.

3.2.2 WRNL Block (Wide area non-local Block)

The illumination characteristics in the converter station cells make conventional blocks impractical. Thus, the WRNL block is first defined and then analyzed for its effectiveness using statistical knowledge.

Dividing WRNL input features X into a b feature map { X^kH, (k ═ 1.., ab), where k is the number of feature maps, by the formula

Generating an output characteristic, wherein,

representing the output characteristic of the kth profile at position i,

representing the ith row of feature points in the kth feature map,

represents the j-th row of feature points in the k-th feature map, ()^TA transposed matrix, W, representing a matrix_θ、

And W_gAll the weight matrixes have dimensions of C × L, C × L and C × C respectively, and L is C/2;

to represent

And a set of region locations S_iEach of which

The correlation between them; γ (·) represents a relationship function and γ (·) 1/(·) + 1). If a is>b, the grid is wider than a ═ b. Therefore, when a>b. a ═ b and a<And b, respectively calling the wide area rectangular block, the square block and the high area rectangular block.

Given that non-local blocks are based on information of other pixels in the blob to recover a particular pixel, enough background information is needed in each blob. However, due to the uneven distribution of the reflective layer, the regional non-local blocks are difficult to fully utilize the background information, and the wide-region rectangular patches have richer background information than the square and high-region rectangular patches. The image is divided into 16 × 4, 8 × 8 and 4 × 16 grids, respectively, and wide-area rectangular, square and high-area rectangular blocks are obtained. A pixel in the input reflected image and the corresponding de-reflected image is considered to belong to the reflective layer if the difference between the pixel exceeds a certain threshold. As shown in table 11, compared to the square and high-area rectangular patches, the wide-area rectangular patch has better peak signal to noise ratio (PSNR) and Structural Similarity (SSIM), and the distribution of the reflective layer is uniform over all patches, so that the image can be better restored.

TABLE 11 regional non-local Block regional type ablation experiments

3.2.3 network training Process

Constructing a data set; and training the reflection interference removing network by using a data set, obtaining the trained reflection interference removing network when a convergence condition is met, inputting the image with the reflection interference collected in real time into the trained reflection interference removing network, and outputting a picture with the reflection interference removed. The data set comprises a public data set and a screen cabinet pressing plate state image data set in a protection cell of an extra-high voltage converter station, and the data set is as follows: the ratio of 3 is randomly divided into a training set and a test set.

The convergence condition is that a loss function reaches a minimum value, and the loss function comprises:

L₁＝||x_gt-f(x_input)||₁+||x_gt-f(x_input)||₂

wherein x is_inputRepresenting the input reflected image, x_gtRepresenting the corresponding de-reflection image, f representing the output of the de-reflection interference network, | | | | | calculation_FRepresenting the F norm calculation and F taking 1, 2.

Furthermore, to better separate the reflective and transmissive layers, gradient domain based repulsion losses are defined. By analyzing the relationship between the edges of the two layers, it can be found that the transmissive layer and the reflective layer do not substantially overlap at the edges, and the edges in the image I should be caused by the transmissive layer or the reflective layer, rather than by the interaction of the two layers. Thus, minimizing the correlation between the predicted transmissive and reflective layers in the gradient domain and expressing the repulsive losses as the product of two layers normalizing the gradient field at multiple spatial resolutions, constructs the loss function as follows:

wherein the content of the first and second substances,

theta represents the weight of the network,_Drepresenting a data set, I representing an input image, n being an image down-sampling factor,

indicating a transport layer pass factor of 2ⁿ-1 down-sampling of the bilinear difference,

representing a reflection layer passing factor of 2ⁿ-1 down-sampling of the bilinear difference, T representing the transmission layer of the image I, R representing the reflection layer of the image I, λ_TAnd λ_RAre all the normalization factors, and the normalization factors,

the lines indicate that multiplication is performed in the order of cells,

a gradient map representing the transport layer of the image I,

a gradient map of the reflective layer of image I is shown,

to represent

The die of (a) is used,

to represent

N equals 3; the total loss function is L ═ L₁+L_excl(θ)。

3.2.4 simulation verification

(1) Selection of data sets

In order to verify the feasibility and effectiveness of the method of the invention, data set SIR disclosed in documents Renjie Wan, Boxin Shi, Lingyu Duan, et al, benchmark single-image reflection removal implementation algorithms IEEE International Conference on Computer Vision,3922-²And on a screen cabinet pressing plate state image data set in a protection cell of an extra-high voltage converter station, the method is verified. The present invention uses the Pressure-plate (1400), Object (1500), Postcard (560) and Zhang et al (800) datasets for a total of 4260 images for training, the remaining Pressure-plate (600), Object (640), Postcard (240) and Zhang et al (340) datasets for a total of 1820 images for quantitative evaluation, 4 datasets being evaluated as 7: the ratio of 3 is randomly divided into a training set and a test set.

The data set Pressure-plate is an image data set shot by using a Canon EOS 750D camera according to the state of a cabinet Pressure plate in an indoor environment, and comprises 220 real image pairs, namely, an image with reflection and a corresponding reference transmission layer. In order to simulate different imaging conditions, the following factors are taken into account when capturing images: 1) environment: indoor; 2) the illumination condition is as follows: an incandescent lamp; 3) thickness of the glass plate: 3mm and 8 mm; 4) distance between glass and camera: 3-15 cm; 5) the camera view angle: emmetropia and strabismus; 6) camera exposure value: 8.0-16.0; 7) camera aperture (affecting reflection blur): f/4.0-f/16.

(2) Results and analysis of the experiments

Feature information from all scales in the down-sampling part can be aggregated in the up-sampling part of the network through multi-level connection, but too deep levels can reduce the weight of key information, and too few levels can make the effect of feature information extraction not obvious, so that how to select proper multi-level connection levels is very important. Fig. 21 shows the results of a multi-level connected series ablation experiment for the public data set (PSNR1, SSIM1) and the cabinet platen state image data set (PSNR2, SSIM 2). As can be seen from fig. 21, as the number of levels of the multi-level connection increases, the PSNR and SSIM indexes gradually increase and reach a maximum value at level 4, and as the number of levels continues to increase, the PSNR and SSIM gradually decrease, which indicates that the aggregation capability of the constructed deep neural network for information of each scale gradually decreases. The number of levels of the subsequent multilevel connection is thus selected to be 4 levels.

Under the condition of selecting 4-layer connection series, the method MA-Net of the present invention is compared with other methods, including the methods mentioned in the publications "Qiangnan Fan, Jiang Yang, Gang Huang, et al. A genetic removal architecture for single image removal and image smoothing. IEEE International Conference component Vision, 3238. 3247,2017. RNCEILNet, E Yang G, Dong Gong, Lingqiao Liu, et al. Seeiodense and binary evaluation: A genetic adaptation for single image removal and reflection evaluation, European Conference component, 9, 11 J.8, E.E.E. simulation distribution, III, IV, III, IV, III, IV, III, IV, III, IV, III, IV, III. For peer-to-peer comparisons, the same public data set training samples and the cubicle pressboard state image data set training samples were used, the parameter fine tuning was performed for each model, and the best results were given for the fine tuned version (indicated with suffix '-F')

As shown in fig. 22 and 23, the present invention shows the results of de-reflection visual processing of real nature landscape images and converter station cabinet platen images, wherein the input images (column 1), ceirnet (column 2), BDN (column 3), ERRNet (column 4), the method of the present invention (column 5). Compared with other methods, the method disclosed by the invention has the advantages that the visual effect is more accurate, most of unwanted reflections are deleted, the advantages of the processing effect on the characteristics of indoor and unnatural light, scattered light spots and high illumination intensity are obvious, and the problems of unobvious reflection removal effect, high noise and the like generally exist in other methods. Table 12 summarizes the experimental results of the different methods on the four real datasets, including Pressure-plate, Object, Postcard, and Zhang et al. The number of the test images in each data set is displayed behind the name, and PSNR and SSIM measurement indexes are adopted, so that the larger the values of PSNR and SSIM are, the better the performance is.

TABLE 12 quantitative comparison of different methods on four real data sets

It can be seen from table 2 that, in addition to the Zhang et. data sets disclosed in the documents Xuaner Zhang, Ren Ng, Qifeng chemical image reflection separation with performance errors, ieee Conference on Computer Vision and Pattern Recognition,4786 and 20194, 2018, the MA-Net of the present invention achieves the best performance on all data sets, since ERRNet is built on the basis of Zhang et. model, whose network model has better generalization capability on the data set, the algorithm has better performance on Zhang et. data set. MA-Net outperforms other methods on average performance across all test data sets.

Table 13, for the existing platen state identification method (cluster matching method, improved BOF method, OpenVINO method, transfer learning method, and improved SSD method), the platen state data set Pressure-plate of the screen cabinet is used as an object, and the influence of the reflection removal network on the platen state identification result is contrastively analyzed.

TABLE 13 influence of the De-reflection network on the platen State identification

As can be seen from table 3, in the case of reflection, the recognition accuracy rates of the five platen state recognition methods are: 78.22%, 83.55%, 92.90%, 89.63% and 84.55%. After reflection interference is removed through the reflection removing network, the identification accuracy of the five methods is improved to different degrees. The recognition accuracy rates of the cluster matching method and the improved BOF method are respectively improved by 6.28% and 3.66%, and are obviously higher than those of the OpenVINO method, the transfer learning method and the improved SSD method by 0.45%, 1.57% and 1.87%, because the traditional image recognition method depends on the information of the original image, the anti-interference capability is relatively poor, and the effect generated by the reflection removing network is better. The invention provides a dereflection deep neural network based on multilevel connection and adaptive regional attention to remove reflection interference in an image, which is researched for solving the reflection problem in the monitoring of the pressing plate state of an extra-high voltage direct current converter station. The MA-Net network can adaptively aggregate features through multi-level connections and compressed excitation blocks, and fully utilize rich remote non-reflection background information based on wide regional non-local blocks. Experiments show that the MA-Net can not only recover the details of an input image, but also almost completely eliminate the reflection interference on a real image data set and a screen cabinet pressing plate state image data set, and can effectively improve the detection effect of the pressing plate state.

Claims (11)

1. The method for processing and storing the panoramic monitoring image of the protection system of the extra-high voltage converter station is characterized by comprising the following steps of:

s1, performing super-resolution reconstruction on the panoramic monitoring image, wherein the reconstruction method comprises the following steps:

s11, establishing a depth multi-scale residual error network model on the edge side;

s12, inputting a sample data set, and training a depth multi-scale residual error network model;

s13, testing the peak signal-to-noise ratio and the structural similarity index of the network by adopting a standard data set for the trained depth multi-scale residual error network model;

s14, inputting the panoramic monitoring image of the extra-high voltage converter station into the trained depth multi-scale residual error network model to complete super-resolution reconstruction;

s2, optimizing ubiquitous heterogeneous network transmission topology monitored by the ultra-high voltage converter station panorama, wherein the optimizing method comprises the following steps:

s21, modeling the heterogeneous network of the extra-high voltage converter station into a tree structure, wherein the tree structure is provided with a main station v₀And N-1 data transfer nodes v₁,v₂,...,v_N-1V to the primary station, each data transfer node having a node v to the primary station₀The unique path of (a);

s22, setting the master station v₀As a root node of the tree structure, carrying out a Monte Carlo tree recursive search on each state by taking the root node as an initial state to obtain a training data set;

s23, inputting the training data set obtained by searching into a deep convolutional neural network for training to obtain a value function and a strategy function, and guiding a Monte Carlo tree to recursively search a state with expected reward and updating and collecting the training data set of the deep convolutional neural network in return;

s24, after training, starting from the initial state S₀Starting with 0, by sequentially selecting a in a strategy for deep convolutional neural network prediction_t～π(s_t) And update the state s_t+1＝T(s_t,a_t) Until the complete tree is reached, so as to obtain a heterogeneous network topology;

and S3, transmitting the panoramic monitoring data to the edge side by optimizing the heterogeneous network transmission topology, and transmitting the panoramic monitoring data to a cloud storage after carrying out lightweight processing on the panoramic monitoring data at the edge side.

2. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 1, wherein the depth multi-scale residual error network model comprises: an input convolutional layer, an output convolutional layer, and k multi-scale convolutional blocks; the input end convolution layer is used as an encoder to extract original low-order characteristics of a low-resolution image; the output end convolution layer is used for fusing multi-scale detail features to reconstruct a high-resolution image; the input end convolution layer and the output end convolution layer are in jump connection, and an identity mapping from a low-resolution image to a high-resolution image is established so as to carry out global residual learning; the k multi-scale volume blocks are sequentially stacked and connected and are used for obtaining the depth of the network model; the original low-order features are correspondingly connected with the k multi-scale volume blocks through k paths, and the ability of a network model to learn complex features is enhanced through local residual error learning;

3. the method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 1, wherein the input convolution layer and the output convolution layer both adopt convolution kernels with the step length of 1, and the input convolution layer is activated by Relu; the multi-scale convolution block respectively uses convolution kernels of 3 x 3, 3 x 2, 2 x 3 and 2 x 2 scales to extract multi-level detail features from an input image, then the feature maps of the four scales are spliced pairwise on an appointed dimension through a cross mechanism, then the feature maps are sent into a convolution layer with the scale of 3 x 3 to be subjected to feature mapping, and a new feature map with the same input size is generated and sent into the next multi-scale convolution block.

4. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 3, wherein the local residual learning is defined as follows: h^k＝G^k(H^k-1) + F; wherein G is^kFeature mapping learned for the kth multi-scale volume block, H^kFor the output of the kth multi-scale volume block, H^k-1Outputting the (k-1) th multi-scale convolution blocks, and F being the original low-order features extracted from the input-end convolution layers; the k multi-scale convolution block maps obtained by global residual and local residual learning are expressed as: i is^HR＝R(I^LR)＝I^LR+F_-1(G^k(G^k-1(…(G¹(F) + F) …) + F) + F); wherein, F₀() For the input convolution layer, F is equal to F₀(I^LR)

Mapping to learn, F_-1() Mapping to be learned for the output convolutional layer, wherein I^HR、I^LRRespectively representing a high resolution image and a low resolution image,

feature mapping learned for the k-1 th multi-scale volume block, G^kR () is a mapping operation for the feature map learned for the 1 st multi-scale volume block.

5. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 4, wherein the loss function of the depth multi-scale residual error network model is as follows:

wherein theta is a parameter of the depth multi-scale residual error network, and an Adam optimizer is adopted to minimize a loss function; x⁽ⁱ⁾Is a sample data set

The ith sub-image of (1), Y⁽ⁱ⁾And N is a positive integer for the corresponding label.

6. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 1, wherein the panoramic monitoring image comprises: secondary equipment, a hard pressing plate and a terminal corrosion image; the standard data set comprises: set5, Set14 and Urban 100.

7. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 5, wherein the calculation formula of the test peak signal-to-noise ratio is as follows:

wherein MSE is the mean square error, MAX, of the original image and the processed image_IA maximum value representing a color of the image; the calculation formula of the structural similarity index is as follows:

SSIM (X, Y) ═ L (X, Y) × C (X, Y) × S (X, Y); wherein u is_X、u_Y、σ_XAnd σ_YMeans and standard deviations, σ, of the images X and Y, respectively_XYRepresenting the covariance of images X and Y, C₁、C₂And C₃Is constant, usually take C₁＝(K₁*L)²，C₂＝(K₂*L)²，C₃＝C₂/2，K₁＝0.01，K₂L is the range of pixel values 0.03.

8. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 1, wherein the method for modeling the heterogeneous network of the extra-high voltage converter station into a tree structure specifically comprises the following steps: in each round of data acquisition, node v_iWill be provided with

Bit data is forwarded to its parent node, where i ∈ {1, 2.., N-1 };

is composed of v_iSelf-generated data, data sets

Is from v_iA () function is an aggregation function; using transmission models

The transmission flow of the nodes related to the topology in the transmission model consists of two parts, namely data processing and transmission time consumption;

and

respectively, at node viThe time consumption of data processing and the time consumption of data transmission of each bit, the time consumption of data transmission of each bit depends on the distance from a parent node, and the calculation formula is as follows:

wherein the content of the first and second substances,

is node v_iWith its parent node v_jAnd p is a power amplification constant in the link budget, taking into account the shadow fading effect.

9. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 8, wherein the method for the recursive search of the Monte Carlo tree is as follows: each node on the Monte Carlo Tree represents 5-tuple data (s, a, M (s, a), π(s), Q^π(s, a)); at each search step t<N, an action is selected that maximizes the confidence ceiling, when the search reaches the termination state t-N, a reward is earned and propagated back along the search path to the root states of all accessed states and actions taken, Q on the path^πThe values are updated accordingly by the mean values on the nodes; wherein s is the state of the heterogeneous network; a is an operation in this state; m (s, a) is the total number of accesses (s, a) on the search tree; π(s) is the prior probability of a valid action predicted by the deep convolutional neural network; q^π(s, a) is a state action value, representing an expected reward for starting from state s and taking action a; the calculation formula of the action of maximizing the confidence upper limit is as follows:

wherein the content of the first and second substances,

is the access count of state s and, regardless of the action, c is a hyper-parameter that controls the search level.

10. The ultra-high voltage converter station protection system panoramic monitoring image processing and storing method of claim 9, wherein the deep convolutional neural network comprises a deep Vgg16 module, a full-link layer with softmax for policy, and a full-link layer with ReLU activation for value function; the deep Vgg16 module is composed of 2 convolutional layers with 64 convolutional filters, 2 convolutional layers with 128 convolutional filters, 3 convolutional layers with 256 convolutional filters, and 6 convolutional layers with 512 convolutional filters, wherein each convolutional filter has a 3 × 3 kernel and a maximum pooling layer.

11. The method for processing and storing the panoramic monitoring image of the extra-high voltage converter station protection system according to claim 10, wherein the value function satisfies Bellman's equation, the value representing the current state is the reward of the state plus the expected reward of the next state, and the formula of the value function is as follows:

the strategy function formula is as follows:

Images