CN110084165B - Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation - Google Patents

Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation Download PDF

Info

Publication number
CN110084165B
CN110084165B CN201910319835.6A CN201910319835A CN110084165B CN 110084165 B CN110084165 B CN 110084165B CN 201910319835 A CN201910319835 A CN 201910319835A CN 110084165 B CN110084165 B CN 110084165B
Authority
CN
China
Prior art keywords
bounding box
early warning
abnormal events
intelligent identification
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910319835.6A
Other languages
Chinese (zh)
Other versions
CN110084165A (en
Inventor
聂礼强
宋雪萌
孙腾
许克
姚一杨
宿仕华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
State Grid Zhejiang Electric Power Co Ltd
Zhiyang Innovation Technology Co Ltd
Original Assignee
Shandong University
State Grid Zhejiang Electric Power Co Ltd
Zhiyang Innovation Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University, State Grid Zhejiang Electric Power Co Ltd, Zhiyang Innovation Technology Co Ltd filed Critical Shandong University
Priority to CN201910319835.6A priority Critical patent/CN110084165B/en
Publication of CN110084165A publication Critical patent/CN110084165A/en
Application granted granted Critical
Publication of CN110084165B publication Critical patent/CN110084165B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection

Abstract

According to the intelligent identification and early warning method for the abnormal events in the open scene of the power field based on the edge calculation, an improved SSD target detection model is compressed and transplanted to a mobile terminal, the advantages of the edge calculation are fully played, and through experiments, an android terminal is used as an optimal scheme; the Conv4_ x characteristic layer and the Conv5_ x characteristic layer in the VGG16 network are fused, and the fused characteristic layer is directly applied to the last prediction layer, so that the accuracy of small target detection is improved; meanwhile, the invention summarizes a plurality of basic weather conditions: the method is characterized by comprising the following steps of increasing training data under different scenes by using an image enhancement technology on sunny days, cloudy days, rainy days, foggy days and the like so as to improve the generalization capability of the model.

Description

Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation
Technical Field
The invention discloses an intelligent identification and early warning method for abnormal events in an open scene of an electric power field based on edge calculation, and belongs to the technical field of intelligent identification of electric power abnormal events.
Background
With the continuous development and construction of electric power engineering, the scale of a power transmission network is larger and larger, more and more equipment is provided, and the inspection workload for power transmission equipment and lines is also continuously increased. In addition, in some remote areas, such as mountainous areas, snowfields and the like, related workers are difficult to reach, and the difficulty of patrol work is increased. Meanwhile, the power grid management level is continuously improved, and an operation and maintenance unit also continuously explores the operation management mode of the power transmission line. At present, mainstream solutions for identification and early warning of abnormal events include an online monitoring system, an unmanned aerial vehicle automatic inspection and the like, but all of the solutions have disadvantages.
In the online monitoring system, a monitor installed on a transmission tower (pole) shoots a video or an image, and then the video or the image is transmitted to a server, and the server identifies and processes an abnormality. The disadvantage of this scheme is the delay and instability of anomaly detection. It takes a necessary time to transmit the picture photographed by the terminal monitor to the server, and the picture cannot be transmitted to the server without fail in its entirety, possibly due to a problem of a transmission line. The server needs to process data transmitted from hundreds of remote monitors, which often have a large amount of redundant information, and the server has no ability to identify useful and useless information and cannot set processing priority for the transmitted data to be detected. The automatic patrol of the unmanned aerial vehicle has the defect that the unmanned aerial vehicle is not suitable for long-distance patrol work, and people are required to replace batteries and copy data for the unmanned aerial vehicle.
The invention aims to identify and early warn abnormal events in an open scene of the power field by utilizing an improved SSD target detection model based on edge calculation. By identifying and early warning at the edge, the communication bandwidth between the central control system and the data acquisition terminal is reduced. Because the detection scene is larger, the proportion of the abnormal object in the picture is smaller, and in addition, the invention performs targeted improvement on the original SSD target detection model in consideration of the scene variability, so that the detection of the small object still has higher accuracy and stronger generalization capability on different scenes.
In order to solve the problems of complex background, much noise, easy exposure of pictures and the like of a high-speed rail monitoring video, chinese patent document CN109165575A researches a smoke and fire recognition algorithm based on an image deep learning SSD framework, wherein a detection model training network is a reconstructed VGG16 network, and 6 convolutional layers and 1 pooling layer are added to the reconstructed detection model training network on the basis of VGG 16.
The invention belongs to the technical field of target detection, and particularly relates to a smoke discharge video detection method based on an improved VGG16 convolutional network, which comprises the following steps: step 1: generating a chimney emission image dataset; and 2, sending the training set into an improved VGG16 convolutional network for training to obtain a plurality of weight models. The main network structure of the improved VGG16 convolutional network is VGG16, the last two full-connection layers are changed into two convolutional layers and used for extracting image features of the chimney in a multi-scale mode, the convolutional layers are connected with the global mean pooling layer, the generated matrix is used for outputting results, and finally the matrices are input into a loss function for classification to construct a complete network structure.
Compared with the above-mentioned comparison documents and the prior art, the invention is mainly applied to the open scene in the power field, adopts a plurality of image enhancement methods to process data according to different weather conditions, and researches an intelligent detection algorithm of abnormal events based on an SSD frame, wherein a training network of a detection model is an improved VGG16 network. The Conv4_ x characteristic layer and the Conv5_ x characteristic layer of the VGG16 network are fused by the improved detection model training network, and the fused characteristics are directly acted on the final prediction layer, so that the accuracy of small target detection is improved.
Disclosure of Invention
The invention discloses an intelligent identification and early warning method of abnormal events in an open scene in the power field based on edge calculation.
Summary of the invention:
the improved SSD target detection model is compressed and transplanted to the mobile end, the edge calculation advantage is fully played, and through experiments, the android end is used as an optimal scheme; the Conv4_ x characteristic layer and the Conv5_ x characteristic layer in the VGG16 network are fused, and the fused characteristic layer is directly applied to the last prediction layer, so that the accuracy of small target detection is improved; meanwhile, the invention summarizes a plurality of basic weather conditions: the method is characterized by comprising the following steps of increasing training data under different scenes by using an image enhancement technology on sunny days, cloudy days, rainy days, foggy days and the like so as to improve the generalization capability of the model.
The technical scheme of the invention is as follows:
an intelligent identification and early warning method for abnormal events in an open scene of an electric power field based on edge calculation is characterized by comprising the following steps:
s1: performing image enhancement processing on training data under different scenes, and labeling the training data by using a labeling tool to obtain an xml file;
s2: extracting features of an original picture by using a VGG16 network as a basic network, performing feature fusion on a Conv4_ x feature layer and a Conv5_ x feature layer in the VGG16 network, and applying the fused features to a prediction layer of a final model;
s3: adding different network layers behind a basic network, predicting targets and scores of the attribution categories of the targets on the network layers, and meanwhile, using a small convolution kernel on a characteristic layer to regress accurate positions of a series of bounding boxes;
s4: aiming at a large number of boundary frames generated on the same target position, using non-maximum value to inhibit and find out the optimal target boundary frame, eliminating redundant boundary frames and training a model;
s5: and applying the trained model at the mobile terminal to perform intelligent identification and early warning of the abnormal event. Preferably, the mobile terminal is an android terminal.
Preferably, according to the present invention, the method for enhancing the image in step S1 is:
s11: setting different brightness and/or contrast for an original picture, and performing Gaussian blur processing to simulate different scenes;
s12: the labeling tool is used to label the picture after the processing of step S11,
s13: for each picture processed in step S11, the following operations are randomly performed:
1) using the original picture;
2) sampling a region according to a sampling rule, wherein the sampling rule is as follows: randomly selecting a region between 0 and 1 with the smallest intersection with the object;
3) randomly sampling a block of area;
s14: the sampled area is [0.1, 1] of the original image size ratio, and the aspect ratio is between 0.5 and 2; when the center of the label box is in the sampled area, the overlapping part is reserved;
s15: after the sampling step, each sampled region is adjusted to a fixed size and flipped at a random level with a probability of 0.5.
Preferably, in step S2, the extracting the features of the original picture includes:
s21: collecting images, sending the collected images into a VGG16 network, sequentially passing through 5 layers of convolution layers, a pooling layer and 2 layers of full-connection layers, and extracting the characteristics of the images;
s22: the Conv4_ x feature layer and the Conv5_ x feature layer are fused together, and the fused features are directly applied to the last prediction layer, namely the fused features are input as part of the last prediction layer.
According to the invention, the Conv4_ x feature layer and the Conv5_ x feature layer are preferably fused in a vector splicing mode or a corresponding element addition mode.
The method for determining the accurate position of the bounding box in the step S3 includes:
s31: the picture feature layer extracted through the step S2 is sliced into 8 × 8 or 4 × 4 grids;
s32: generating a series of fixed-size bounding boxes for each mesh on the feature layer, each bounding box including at least 5 prediction parameters: x, y, w, h, conf, wherein (x, y) represents the center coordinates of the bounding box with respect to the grid, (w, h) represents the width and height predicted with respect to the entire image, and (conf) represents the IOU value of the bounding box with any one of the label boxes;
s33: at each feature level, a series of convolution kernels are used to produce a series of fixed-size predictors.
Preferably, in step S33, the size of the convolution kernel used is 3 × 3 × p for an m × n p-channel feature layer.
According to a further embodiment of the invention, the predictor is a confidence score of the attribution category or a position offset value of the bounding box.
Preferably, the method for finding the optimal target bounding box in step S4 using non-maximum suppression includes:
s41: pairing each bounding box with all the annotation boxes: when the intersection ratio between the two is larger than a threshold value, the two are combined into a sample; preferably, the threshold is set between 0.5 and 0.9;
s42: sorting the bounding boxes of which the corresponding prediction results are negative samples at the positions of each object in the original image from large to small according to the confidence degrees, and selecting a plurality of positive and negative samples, wherein the number ratio of the positive and negative samples is about 1: 3;
s43: by using
Figure BDA0002034297190000051
Indicating that the ith bounding box is matched with the jth label box of the category k; otherwise, the data is not matched,
Figure BDA0002034297190000052
s44: the total target loss function is obtained by weighted summation of the position loss (loc) and the confidence loss (conf):
Figure BDA0002034297190000053
in the above formula, x represents whether the bounding box and the labeled box are matched, if so, x is 1, otherwise, x is 0, c represents the attribution type score of the bounding box, l represents the bounding box, g represents the labeled box, N is the number of matched bounding boxes, and if N is 0, the loss value of the objective function is set to be 0; the position loss (loc) is the smooth between the parameters of the bounding box (l) and the label box (g) after the step of S33L1Loss;
in the above formula, d represents that the S32 step is passed but the S33 step is not passedThe bounding box of (1). (cx, cy) represents the center coordinate of d, (w, h) represents the width and length of d,
Figure BDA0002034297190000054
and representing the label box after log processing.
The confidence loss (conf) is obtained by performing softmax operation on the confidence of a plurality of classes:
Figure BDA0002034297190000055
in the above-mentioned formula,
Figure BDA0002034297190000056
a attribution category score indicating that the ith bounding box belongs to the kth category,
Figure BDA0002034297190000057
indicating after softmax processing
Figure BDA0002034297190000058
S45: and (5) storing the weight parameters of the characteristic layer through iterative training until the model converges.
According to the preferable embodiment of the present invention, the process of performing intelligent identification and early warning of the abnormal event in the android application model in step S5 includes:
s51: acquiring an original image by using a high-definition camera of the android device, adjusting the brightness and contrast reading of the image to appropriate values, and denoising and enhancing the image;
s52: inputting the image into a model, and utilizing the extracted features of the trained VGG16 network to obtain the image features;
s53: generating a series of boundary frames on the extracted picture feature layer, returning all the boundary frames to correct positions through a trained SSD network, and predicting the correct classification of each boundary frame; the confidence score of each bounding box is determined by the classification information Pr (Class) of each gridi| Object) and classification confidence information in bounding box
Figure BDA0002034297190000061
The multiplication results in that:
Figure BDA0002034297190000062
s54: utilizing a non-maximum value to inhibit and remove redundant bounding boxes, and displaying a detection result on an original image; if an anomaly is detected, an alarm is triggered.
The invention has the advantages of
According to the invention, the model is compressed and transplanted to the mobile terminal, so that the model can autonomously perform intelligent identification and early warning of the abnormity at the front terminal, the advantages of edge calculation are fully exerted, and the time of data transmission and the calculation resources of a back-end server are saved. By fusing the picture feature layer in the VGG16 network, the invention enables the model to make full use of the peripheral information of the small target, thereby improving the accuracy of the small target detection. Meanwhile, the invention uses a series of small convolution kernels to regress the accurate position of the bounding box on the picture characteristic layer, so that the detection speed is further improved. In addition, the invention eliminates redundant bounding boxes by using a non-maximum value inhibition method, so that the detection result is more concise. Finally, the invention adjusts different brightness and contrast of the picture and adopts a Gaussian blur processing method to simulate the picture effect under different scenes by a data enhancement technology, thereby enhancing the generalization capability of the model.
Drawings
FIG. 1 is a flow diagram of the model framework of the present invention;
FIG. 2 is a frame flowchart illustrating that the Conv4_ x and Conv5_ x fusion mode is a vector splicing mode according to the present invention;
FIG. 3 is a block diagram of the Conv4_ x and Conv5_ x fusion method of the present invention as the corresponding element addition method;
FIG. 4 is a schematic diagram of application example 1 of the present invention;
FIG. 5 is a schematic diagram of application example 2 of the present invention;
fig. 6 is a schematic diagram of application example 3 of the present invention.
Detailed Description
The invention is described in detail below with reference to the following examples and the accompanying drawings of the specification, but is not limited thereto.
Examples of the following,
An intelligent identification and early warning method for abnormal events in an open scene of an electric power field based on edge calculation is characterized by comprising the following steps:
s1: performing image enhancement processing on training data under different scenes, and labeling the training data by using a labeling tool to obtain an xml file; the marking refers to artificially determining the position of a target to be detected (such as a crane, construction machinery, a tower crane and the like) in each training picture, using a marking tool to frame the targets respectively by using one rectangular frame, and setting an attribute value for each rectangular frame to indicate the category of the target in the rectangular frame. Therefore, when the model is trained in the subsequent step S4, the model can identify which position in which picture has which kind of object, and train the model according to this principle;
s2: extracting features of an original picture by using a VGG16 network as a basic network, performing feature fusion on a Conv4_ x feature layer and a Conv5_ x feature layer in the VGG16 network, and applying the fused features to a prediction layer of a final model; the step enables the model obtained by training after the steps of S2, S3 and S4 to fully utilize the peripheral information of the small target;
s3: adding different network layers behind a basic network, predicting targets and scores of the attribution categories of the targets on the network layers, and meanwhile, using a small convolution kernel on a characteristic layer to regress accurate positions of a series of bounding boxes; the attribution categories refer to those four categories set in advance: 1) normal conditions; 2) a crane; 3) a construction machine; 4) tower crane; for example, if the score for predicting the target to which the crane category belongs is 0.7, the probability that the model considers the target to be the crane is 70%;
s4: aiming at a large number of boundary frames generated on the same target position, using non-maximum value to inhibit and find out the optimal target boundary frame, eliminating redundant boundary frames and training a model; here, the target position is a target area on the picture after the original captured picture is processed in step S1;
s5: and applying the trained model at the mobile terminal to perform intelligent identification and early warning of the abnormal event. Preferably, the mobile terminal is an android terminal.
The method for enhancing the image in step S1 includes:
s11: setting different brightness and/or contrast for an original picture, and performing Gaussian blur processing to simulate different scenes;
s12: the labeling tool is used to label the picture processed in step S11, and the present invention uses 5 categories: 1) normal conditions; 2) a crane; 3) a construction machine; 4) tower crane;
s13: for each picture processed in step S11, the following operations are randomly performed:
1) using the original picture;
2) sampling a region according to a sampling rule, wherein the sampling rule is as follows: randomly selecting a region between 0 and 1 with the smallest intersection with the object;
3) randomly sampling a block of area;
s14: the sampled area is [0.1, 1] of the original image size ratio, and the aspect ratio is between 0.5 and 2; when the center of the label box is in the sampled area, the overlapping part is reserved;
s15: after the sampling step, each sampled region is adjusted to a fixed size and flipped at a random level with a probability of 0.5.
In step S2, the extracting features of the original picture includes:
s21: collecting images, sending the collected images into a VGG16 network, sequentially passing through 5 layers of convolution layers, a pooling layer and 2 layers of full-connection layers, and extracting the characteristics of the images;
s22: the Conv4_ x feature layer and the Conv5_ x feature layer are fused together, and the fused features are directly applied to the last prediction layer, namely the fused features are input as part of the last prediction layer.
The Conv4_ x feature layer and the Conv5_ x feature layer are fused in a vector splicing mode or a corresponding element addition mode. As shown in fig. 2 and 3.
The method for determining the accurate position of the bounding box in the step S3 includes:
s31: the picture feature layer extracted through the step S2 is sliced into 8 × 8 or 4 × 4 grids;
s32: generating a series of fixed-size bounding boxes for each mesh on the feature layer, each bounding box including at least 5 prediction parameters: x, y, w, h, conf, wherein (x, y) represents the center coordinates of the bounding box with respect to the grid, (w, h) represents the width and height predicted with respect to the entire image, and (conf) represents the IOU value of the bounding box with any one of the label boxes;
s33: at each feature level, a series of convolution kernels are used to produce a series of fixed-size predictors. This function is to calculate the parameters x, y, w, h, conf of the bounding box generated in step S32, that is, to move the bounding box to an appropriate position and calculate the attribution classification score of the bounding box. The most prominent advantage is high speed.
In step S33, for a m × n feature layer with p channels, the size of the convolution kernel used is 3 × 3 × p.
The predicted value is a confidence score for the attribution category or a position offset value for the bounding box.
The method for finding the optimal target bounding box by using non-maximum suppression in step S4 includes:
s41: pairing each bounding box with all the annotation boxes: when the intersection ratio between the two is larger than a threshold value, the two are combined into a sample; preferably, the threshold is set between 0.5 and 0.9;
s42: sorting the bounding boxes of which the corresponding prediction results are negative samples at the positions of each object in the original image from large confidence degrees to small confidence degrees, selecting a plurality of positive and negative samples, and then selecting the plurality of positive and negative samples with the maximum confidence degrees to ensure that the number ratio of the positive and negative samples is about 1: 3; wherein, a series of bounding boxes are generated by the step S32, and then the bounding boxes are moved to proper positions by the step S33, and the respective attribution classification scores are calculated; finally, these bounding boxes are the prediction results. Definition of negative examples: comparing the bounding box after the step of S33 with the labeling box, if the intersection ratio between the bounding box and the labeling box is more than 0.5, considering the bounding box as a positive sample, otherwise, considering the bounding box as a negative sample;
s43: by using
Figure BDA0002034297190000091
Indicating that the ith bounding box is matched with the jth label box of the category k; otherwise, the data is not matched,
s44: the total target loss function is obtained by weighted summation of the position loss (loc) and the confidence loss (conf):
Figure BDA0002034297190000101
in the above formula, x represents whether the bounding box and the labeled box are matched, if so, x is 1, otherwise, x is 0, c represents the attribution type score of the bounding box, l represents the bounding box, g represents the labeled box, N is the number of matched bounding boxes, and if N is 0, the loss value of the objective function is set to be 0; the position loss (loc) is the smooth between the parameters of the bounding box (l) and the label box (g) after the step of S33L1Loss;
in the above formula, d represents a bounding box that passes through the step S32 but has not yet passed through the step S33. (cx, cy) represents the center coordinate of d, (w, h) represents the width and length of d,
Figure BDA0002034297190000102
and representing the label box after log processing.
The confidence loss (conf) is obtained by performing softmax operation on the confidence of a plurality of classes:
Figure BDA0002034297190000103
in the above-mentioned formula,
Figure BDA0002034297190000104
a attribution category score indicating that the ith bounding box belongs to the kth category,
Figure BDA0002034297190000105
indicating after softmax processing
Figure BDA0002034297190000106
Weight coefficient
Figure BDA0002034297190000107
A suitable value can be determined by cross validation, generally between 0 and 1;
s45: and (5) storing the weight parameters of the characteristic layer through iterative training until the model converges. Namely, the method can be used for detecting the front-end equipment model.
The process of performing intelligent identification and early warning of the abnormal event on the android application model in the step S5 includes:
s51: acquiring an original image by using a high-definition camera of the android device, adjusting the brightness and contrast reading of the image to appropriate values, and performing denoising and enhancement processing on the image to improve the image quality and further improve the detection effect;
s52: inputting the image into a model, and utilizing the extracted features of the trained VGG16 network to obtain the image features;
s53: generating a series of boundary frames on the extracted picture feature layer, returning all the boundary frames to correct positions through a trained SSD network, and predicting the correct classification of each boundary frame; the confidence score of each bounding box is determined by the classification information Pr (Class) of each gridi| Object) and classification confidence information in bounding box
Figure BDA0002034297190000111
The multiplication results in that:
s54: utilizing a non-maximum value to inhibit and remove redundant bounding boxes, and displaying a detection result on an original image; if an anomaly is detected, such as the detection of a crane, construction machine, tower crane, etc., an alarm is triggered.
Application examples 1,
The specific application of the recognition and early warning method of the present invention is as follows, as shown in fig. 4.
By fusing the feature extraction layers, the model can make full use of information around the small target, thereby improving the calibration effect of the small target, as shown in the square in fig. 4.
Application examples 2,
The specific application of the recognition and early warning method of the present invention is as follows, as shown in fig. 5.
By means of the image enhancement technology, the model can still achieve good detection effect in foggy or rainy scenes.
Application examples 3,
The specific application of the recognition and early warning method of the present invention is as follows, as shown in fig. 6.
By means of the image enhancement technology, the model can accurately detect the abnormal target under the condition of dark light.

Claims (10)

1. An intelligent identification and early warning method for abnormal events in an open scene of an electric power field based on edge calculation is characterized by comprising the following steps:
s1: performing image enhancement processing on training data under different scenes, and labeling the training data by using a labeling tool to obtain an xml file;
s2: extracting features of an original picture by using a VGG16 network as a basic network, performing feature fusion on a Conv4_ x feature layer and a Conv5_ x feature layer in the VGG16 network, and applying the fused features to a prediction layer of a final model;
s3: adding different network layers behind a basic network, predicting targets and scores of the attribution categories of the targets on the network layers, and meanwhile, using a small convolution kernel on a characteristic layer to regress accurate positions of a series of bounding boxes;
s4: aiming at a large number of boundary frames generated on the same target position, using non-maximum value to inhibit and find out the optimal target boundary frame, eliminating redundant boundary frames and training a model;
s5: applying a trained model at a mobile terminal to intelligently identify and early warn abnormal events;
the method for enhancing the image in step S1 includes:
s11: setting different brightness and/or contrast for an original picture, and performing Gaussian blur processing to simulate different scenes;
s12: the labeling tool is used to label the picture after the processing of step S11,
s13: for each picture processed in step S11, the following operations are randomly performed:
1) using the original picture;
2) sampling a region according to a sampling rule, wherein the sampling rule is as follows: randomly selecting a region between 0 and 1 with the smallest intersection with the object;
3) randomly sampling a block of area;
s14: the sampled area is [0.1, 1] of the original image size ratio, and the aspect ratio is between 0.5 and 2; when the center of the label box is in the sampled area, the overlapping part is reserved;
s15: after the sampling step, each sampled region is adjusted to a fixed size and flipped at a random level with a probability of 0.5.
2. The intelligent identification and early warning method for the abnormal events in the open scene of the power field based on the edge computing as claimed in claim 1, wherein in the step S2, the feature extraction of the original picture comprises:
s21: collecting images, sending the collected images into a VGG16 network, sequentially passing through 5 layers of convolution layers, a pooling layer and 2 layers of full-connection layers, and extracting the characteristics of the images;
s22: the Conv4_ x feature layer and the Conv5_ x feature layer are fused together, and the fused features are directly applied to the last prediction layer, namely the fused features are input as part of the last prediction layer.
3. The intelligent identification and early warning method for the abnormal events under the open scene of the power field based on the edge computing as claimed in claim 2, wherein the Conv4_ x feature layer and the Conv5_ x feature layer are fused in a vector splicing manner or a corresponding element addition manner.
4. The intelligent identification and early warning method for the abnormal events in the open scene of the power field based on the edge computing as claimed in claim 1, wherein the method for determining the accurate position of the bounding box in the step S3 comprises:
s31: the picture feature layer extracted through the step S2 is sliced into 8 × 8 or 4 × 4 grids;
s32: generating a series of fixed-size bounding boxes for each mesh on the feature layer, each bounding box including at least 5 prediction parameters: x, y, w, h, conf, wherein (x, y) represents the center coordinates of the bounding box with respect to the grid, (w, h) represents the width and height predicted with respect to the entire image, and (conf) represents the IOU value of the bounding box with any one of the label boxes;
s33: at each feature level, a series of convolution kernels are used to produce a series of fixed-size predictors.
5. The intelligent identification and early warning method for abnormal events in the open scene of power field based on edge computing as claimed in claim 4, wherein in step S33, for a m × n feature layer with p channels, the size of the convolution kernel used is 3 × 3 × p.
6. The intelligent identification and early warning method for abnormal events in power domain open scenes based on edge computing as claimed in claim 4, wherein the predicted value is a confidence score of a belonging category or a position deviation value of a bounding box.
7. The intelligent identification and early warning method for abnormal events in the power field open scene based on edge computing as claimed in claim 1, wherein the method for finding the optimal target bounding box by using non-maximum suppression in step S4 includes:
s41: pairing each bounding box with all the annotation boxes: when the intersection ratio between the two is larger than a threshold value, the two are combined into a sample;
s42: sorting the bounding boxes of which the corresponding prediction results are negative samples at the positions of each object in the original image from large to small according to the confidence degrees, and selecting a plurality of positive and negative samples, wherein the number ratio of the positive and negative samples is about 1: 3;
s43: by using
Figure FDA0002276356460000031
Indicating that the ith bounding box is matched with the jth label box of the category k; otherwise, the data is not matched,
Figure FDA0002276356460000032
s44: the total target loss function is obtained by weighted summation of the position loss (loc) and the confidence loss (conf):
in the above formula, x represents whether the bounding box and the labeled box are matched, if so, x is 1, otherwise, x is 0, c represents the attribution type score of the bounding box, l represents the bounding box, g represents the labeled box, N is the number of matched bounding boxes, and if N is 0, the loss value of the objective function is set to be 0; the position loss (loc) is the smooth between the parameters of the bounding box (l) and the label box (g) after the step of S33L1Loss;
in the above formula, d represents an edge passing through the step S32 but not passing through the step S33Bounding box, (cx, cy) denotes the center coordinate of d, (w, h) denotes the width and length of d,
Figure FDA0002276356460000034
representing a label box after log processing;
the confidence loss (conf) is obtained by performing softmax operation on the confidence of a plurality of classes:
Figure FDA0002276356460000035
in the above-mentioned formula,
Figure FDA0002276356460000036
a attribution category score indicating that the ith bounding box belongs to the kth category,
Figure FDA0002276356460000037
indicating after softmax processing
Figure FDA0002276356460000038
S45: and (5) storing the weight parameters of the characteristic layer through iterative training until the model converges.
8. The intelligent identification and early warning method for the abnormal events under the open scene of the power field based on the edge computing as claimed in claim 1, wherein the process of performing the intelligent identification and early warning of the abnormal events in the android application model in the step S5 includes:
s51: acquiring an original image by using a high-definition camera of the android device, adjusting the brightness and contrast reading of the image to appropriate values, and denoising and enhancing the image;
s52: inputting the image into a model, and utilizing the extracted features of the trained VGG16 network to obtain the image features;
s53: generating a series of bounding boxes on the extracted picture feature layer, and returning all bounding boxes to correct positions through a trained SSD networkAnd predicting the correct classification of each bounding box; the confidence score of each bounding box is determined by the classification information Pr (Class) of each gridi| Object) and classification confidence information in bounding box
Figure FDA0002276356460000041
The multiplication results in that:
Figure FDA0002276356460000042
s54: utilizing a non-maximum value to inhibit and remove redundant bounding boxes, and displaying a detection result on an original image; if an anomaly is detected, an alarm is triggered.
9. The intelligent identification and early warning method for the abnormal events in the open scene of the power field based on the edge computing as claimed in claim 1, wherein the mobile terminal is an android terminal.
10. The intelligent identification and early warning method for abnormal events in an open scene of power field based on edge computing as claimed in claim 7, wherein the threshold is set between 0.5 and 0.9.
CN201910319835.6A 2019-04-19 2019-04-19 Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation Active CN110084165B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910319835.6A CN110084165B (en) 2019-04-19 2019-04-19 Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910319835.6A CN110084165B (en) 2019-04-19 2019-04-19 Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation

Publications (2)

Publication Number Publication Date
CN110084165A CN110084165A (en) 2019-08-02
CN110084165B true CN110084165B (en) 2020-02-07

Family

ID=67415764

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910319835.6A Active CN110084165B (en) 2019-04-19 2019-04-19 Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation

Country Status (1)

Country Link
CN (1) CN110084165B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110536299B (en) * 2019-09-04 2020-04-14 电子科技大学 Data packet validity confirmation method based on edge calculation and discrete random convolution
CN110766214A (en) * 2019-10-17 2020-02-07 中南林业科技大学 Real-time flood control monitoring system based on edge calculation and big data
CN111160084A (en) * 2019-11-19 2020-05-15 广州民航职业技术学院 Dangerous article detection method, dangerous article detection device and storage medium
CN111046822A (en) * 2019-12-19 2020-04-21 山东财经大学 Large vehicle anti-theft method based on artificial intelligence video identification
CN111311775A (en) * 2020-01-20 2020-06-19 深圳供电局有限公司 Electric robot inspection system and method
CN111204452B (en) * 2020-02-10 2021-07-16 北京建筑大学 Target detection system based on miniature aircraft
CN111582016A (en) * 2020-03-18 2020-08-25 宁波送变电建设有限公司永耀科技分公司 Intelligent maintenance-free power grid monitoring method and system based on cloud edge collaborative deep learning
CN111414969B (en) * 2020-03-26 2022-08-16 西安交通大学 Smoke detection method in foggy environment
CN112949849B (en) * 2021-01-27 2024-03-26 武汉星巡智能科技有限公司 Method and device for optimizing intelligent camera detection model by adopting edge calculation
CN114445661B (en) * 2022-01-24 2023-08-18 电子科技大学 Embedded image recognition method based on edge calculation
CN115237606B (en) * 2022-09-21 2023-04-07 亿咖通(北京)科技有限公司 Edge calculation control system

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10198671B1 (en) * 2016-11-10 2019-02-05 Snap Inc. Dense captioning with joint interference and visual context
CN108664981B (en) * 2017-03-30 2021-10-26 北京航空航天大学 Salient image extraction method and device
CN108564097B (en) * 2017-12-05 2020-09-22 华南理工大学 Multi-scale target detection method based on deep convolutional neural network
CN108334830B (en) * 2018-01-25 2022-10-04 南京邮电大学 Scene recognition method based on target semantic and depth appearance characteristic fusion
CN108734717B (en) * 2018-04-17 2021-11-23 西北工业大学 Single-frame star map background dark and weak target extraction method based on deep learning
CN108875595A (en) * 2018-05-29 2018-11-23 重庆大学 A kind of Driving Scene object detection method merged based on deep learning and multilayer feature
CN109165583B (en) * 2018-08-09 2021-01-05 苏州飞搜科技有限公司 Multi-size fusion face detection method and device and storage medium
CN109344878B (en) * 2018-09-06 2021-03-30 北京航空航天大学 Eagle brain-like feature integration small target recognition method based on ResNet

Also Published As

Publication number Publication date
CN110084165A (en) 2019-08-02

Similar Documents

Publication Publication Date Title
CN110084165B (en) Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation
CN111353413B (en) Low-missing-report-rate defect identification method for power transmission equipment
CN110059694B (en) Intelligent identification method for character data in complex scene of power industry
CN111563557B (en) Method for detecting target in power cable tunnel
CN110633610B (en) Student state detection method based on YOLO
CN111754498A (en) Conveyor belt carrier roller detection method based on YOLOv3
CN111401418A (en) Employee dressing specification detection method based on improved Faster r-cnn
CN111222478A (en) Construction site safety protection detection method and system
CN103246896A (en) Robust real-time vehicle detection and tracking method
CN106815576B (en) Target tracking method based on continuous space-time confidence map and semi-supervised extreme learning machine
CN111680705A (en) MB-SSD method and MB-SSD feature extraction network suitable for target detection
CN111815576B (en) Method, device, equipment and storage medium for detecting corrosion condition of metal part
CN113569981A (en) Power inspection bird nest detection method based on single-stage target detection network
CN116052082A (en) Power distribution station room anomaly detection method and device based on deep learning algorithm
CN112419261A (en) Visual acquisition method and device with abnormal point removing function
CN115861210A (en) Transformer substation equipment abnormity detection method and system based on twin network
CN115131747A (en) Knowledge distillation-based power transmission channel engineering vehicle target detection method and system
CN110837760A (en) Target detection method, training method and device for target detection
CN117351271A (en) Fault monitoring method and system for high-voltage distribution line monitoring equipment and storage medium thereof
CN117423157A (en) Mine abnormal video action understanding method combining migration learning and regional invasion
CN114596244A (en) Infrared image identification method and system based on visual processing and multi-feature fusion
CN116994161A (en) Insulator defect detection method based on improved YOLOv5
CN116543333A (en) Target recognition method, training method, device, equipment and medium of power system
CN114037834B (en) Semantic segmentation method and device based on fusion of vibration signal and RGB image
CN116309270A (en) Binocular image-based transmission line typical defect identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20190729

Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant after: SHANDONG University

Applicant after: SHANDONG ZHIYANG ELECTRIC Co.,Ltd.

Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant before: Shandong University

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant after: SHANDONG University

Applicant after: Zhiyang Innovation Technology Co.,Ltd.

Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant before: Shandong University

Applicant before: SHANDONG ZHIYANG ELECTRIC Co.,Ltd.

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20200108

Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant after: SHANDONG University

Applicant after: Zhiyang Innovation Technology Co.,Ltd.

Applicant after: STATE GRID ZHEJIANG ELECTRIC POWER Co.,Ltd.

Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27

Applicant before: Shandong University

Applicant before: Zhiyang Innovation Technology Co.,Ltd.

GR01 Patent grant
GR01 Patent grant