CN110084165A - The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations - Google Patents
The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations Download PDFInfo
- Publication number
- CN110084165A CN110084165A CN201910319835.6A CN201910319835A CN110084165A CN 110084165 A CN110084165 A CN 110084165A CN 201910319835 A CN201910319835 A CN 201910319835A CN 110084165 A CN110084165 A CN 110084165A
- Authority
- CN
- China
- Prior art keywords
- bounding box
- early warning
- image
- anomalous event
- edge calculations
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/44—Event detection
Abstract
A kind of power domain based on edge calculations opens the intelligent recognition and method for early warning of anomalous event under scene, the present invention is by improved SSD target detection model compression and is transplanted to mobile terminal, the advantage for giving full play to edge calculations, by experiment, the present invention is using Android end as a preferred embodiment；The present invention by VGG16 network Conv4_x characteristic layer and Conv5_x characteristic layer blend, then the characteristic layer after fusion is applied directly to last prediction interval, the accuracy rate of small target deteection is improved with this；Meanwhile the present invention sums up a variety of basic weather conditions: fine day, cloudy day, rainy day, greasy weather etc., and increases the training data under different scenes using image enhancement technique, so as to improve the generalization ability of model.
Description
Technical field
The present invention discloses the intelligent recognition of anomalous event and the pre- police under the open scene of the power domain based on edge calculations
Method belongs to the technical field of electricity exception event intelligent recognition.
Background technique
With the continuous development and construction of power engineering, power transmission network scale is increasing, and equipment is more and more, for transmission of electricity
Equipment and the tour workload of route are also being constantly increasing.In addition in some more remote areas, such as mountain area, snowfield
Deng, relevant staff is difficult to reach, more increase make an inspection tour work difficulty.At the same time, administration of power networks is horizontal constantly mentions
Height, operation and maintenance unit also constantly explore to the management mode of transmission line of electricity.At present for anomalous event
The mainstream solution of identification and early warning has on-line monitoring system, unmanned plane to make an inspection tour automatically, but they all respectively have disadvantage.
On-line monitoring system is the monitor shooting video or image by being mounted on power transmission tower (bar), then is transmitted to
Server is carried out abnormal identification and processing by server.The drawbacks of this scheme, is the retardance and not of abnormality detection
Stability.The picture transmission that terminal monitor is shot to be allowed the necessary time for server, and may be due to transmission line
The problem of road, errorless cannot be transferred to server for picture is complete.Server needs to handle by hundreds and thousands of long-range monitoring
The data that device transmits, wherein often have a large amount of redundancy, and server has no ability to identify useful and useless letter
Breath, also can not be to the data to be tested setting processing priority transmitted.The shortcomings that unmanned plane is maked an inspection tour automatically be not suitable for it is long away from
From tour work, need people to come for its replace battery and copy data.
The present invention is directed to be carried out under the open scene of power domain based on edge calculations using improved SSD target detection model
The identification and early warning of anomalous event.By carrying out identification and early warning at edge, between central control system and data collection station
Communication bandwidth be reduced.Since detection scene is bigger, aspect ratio shared by abnormal object is often smaller, in addition considers
To the variability of scene, the present invention is targetedly improved on original SSD target detection model, makes it to wisp
Detection still have higher accuracy rate, and have stronger generalization ability to different scenes.
Chinese patent literature CN109165575A is complicated for high-speed rail monitor video background, and noise is more, and picture easily exposes
Problem studies a kind of pyrotechnics recognizer based on picture depth study SSD frame, and wherein detection model training network is attached most importance to structure
VGG16 network afterwards, the detection model training network after reconstruct increase 6 convolutional layers and 1 pond on the basis of VGG16
Layer.
The invention belongs to target detection technique fields by Chinese patent literature CN109034033A, and in particular to one kind is based on changing
Into the smoke evacuation video detecting method of VGG16 convolutional network, steps are as follows: step 1: generating smoke stack emission image data set；Step
2, training set is sent in improvement VGG16 convolutional network and is trained, multiple weight models are obtained.The improvement VGG16
The master network structure of convolutional network is VGG16, and most latter two full articulamentum is changed to two layers of convolutional layer, is used for multiple dimensioned extraction chimney
Characteristics of image, reconnects global mean value pond layer, and the matrix of generation exports for result, recently enters in loss function and carry out
Classification, constructs complete network structure.
Compared to above-mentioned documents and the prior art, present invention is mainly applied to power domains to open under scene, for
Different weather conditions are handled data using a variety of image enchancing methods, study a kind of exception based on SSD frame
The intelligent measurement algorithm of event, wherein the training network of detection model is VGG16 network after improving.Improved detection model
Training network the Conv4_x characteristic layer of VGG16 network and Conv5_x characteristic layer are blended, then by the feature after fusion
Once last prediction interval was directly acted on, so as to improve the accuracy rate of small target deteection.
Summary of the invention
The present invention discloses the intelligent recognition of anomalous event and the pre- police under the open scene of the power domain based on edge calculations
Method.
Summary of the invention:
The present invention is by improved SSD target detection model compression and is transplanted to mobile terminal, gives full play to the excellent of edge calculations
Gesture, by experiment, the present invention is using Android end as a preferred embodiment；The present invention is by the Conv4_x characteristic layer in VGG16 network
It is blended with Conv5_x characteristic layer, then the characteristic layer after fusion is applied directly to last prediction interval, it is small to improve with this
The accuracy rate of target detection；Meanwhile the present invention sums up a variety of basic weather conditions: fine day, cloudy day, rainy day, greasy weather etc., and
Increase the training data under different scenes, using image enhancement technique so as to improve the generalization ability of model.
Technical scheme is as follows:
A kind of power domain based on edge calculations opens the intelligent recognition and method for early warning of anomalous event under scene, special
Sign is, comprising the following steps:
S1: carrying out image enhancement processing to the training data under different scenes, is carried out using annotation tool to training data
Mark, obtains .xml file；
S2: using VGG16 network as basic network, carries out feature extraction to original image, and will be in VGG16 network
Conv4_x characteristic layer and Conv5_x characteristic layer progress Fusion Features, and the model by the characteristic action after fusion to the end
Prediction interval；
S3: adding different network layers after basic network, and target and its ownership are then predicted in these network layers
The score of classification, meanwhile, small convolution kernel is used on characteristic layer, for returning a series of accurate location of bounding boxes；
S4: for a large amount of bounding boxes generated on same target position, optimal mesh is found using non-maxima suppression
Bounding box is marked, the bounding box of redundancy, and training pattern are eliminated；
S5: the good model of application training in mobile terminal carries out the intelligent recognition and early warning of anomalous event.Preferably, described
Mobile terminal is Android end.
It is preferred according to the present invention, the method for image enhancement in the step S1 are as follows:
S11: different brightness and/or contrast are set to original image, and carry out Gaussian Blur processing, to simulate difference
Scene；
S12: being labeled the picture after step S11 processing using annotation tool,
S13: it to each picture after step S11 processing, randomly proceeds as follows:
1) original picture is used；
2) by sampling one piece of region of rule sampling, sampling rule are as follows: the smallest friendship and ratio 0 to 1 between object at random
Between selection region；
3) one piece of region is randomly sampled；
S14: the region of sampling is original image size ratio [0.1,1], and depth-width ratio is between 0.5 to 2；Work as callout box
Center in the region of sampling when, retain lap；
S15: after sampling step, the region of each sampling is adjusted to fixed size, and with 0.5 probability with
The flip horizontal of machine.
Preferred according to the present invention, in the step S2, carrying out feature extraction to original image includes:
S21: Image Acquisition is carried out, then acquired image is sent into VGG16 network, successively passes through 5 layers of convolutional layer and pond
Change layer and 2 layers of full articulamentum, extracts the feature of image；
S22: by Conv4_x characteristic layer together with Conv5_x Feature-level fusion, fused feature is applied directly to
Last prediction interval refers to using the feature after fusion as a part input of last prediction interval.
Preferred according to the present invention, the Conv4_x characteristic layer and Conv5_x Feature-level fusion mode are vector splicing side
Formula or corresponding element phase add mode.
The method of determining bounding box accurate location includes: in the step S3
S31: the grid that picture feature layer is cut into 8 × 8 or 4 × 4 will be extracted by step S2；
S32: a series of bounding box of fixed sizes, each bounding box packet are generated for each grid on the characteristic layer
At least five Prediction Parameters: x, y, w, h, conf are included, wherein (x, y) indicates centre coordinate of the bounding box relative to grid, institute
Stating (w, h) indicates the width predicted relative to whole image and height, and the conf indicates bounding box and any one callout box
IOU value；
S33: on each characteristic layer, a series of predicted value for generating fixed sizes is gone using a series of convolution kernels.
It is preferred according to the present invention, in step S33, for a m × n and there are the characteristic layer of p-channel, the convolution used
The size of core is 3 × 3 × p.
It is preferred according to the present invention, the predicted value be belonging kinds a confidence score or bounding box position it is inclined
Shifting value.
It is preferred according to the present invention, the method that the step S4 finds optimal object boundary frame using non-maxima suppression
Include:
S41: each bounding box and all callout box are matched: when friendship between the two and when than being greater than a threshold value,
Then formed a sample；Preferably, the threshold value is set between 0.5 to 0.9；
S42: will correspond to prediction result in original image on each object space is the bounding box of negative sample according to confidence
Degree is ranked up from big to small and selects multiple positive and negative samples, makes its positive and negative samples ratio of number in 1:3 or so；
S43: it usesIndicate that i-th of bounding box and j-th of callout box of classification k match；Otherwise, it mismatches,
S44: total target loss function is just obtained by position loss (loc) and confidence level loss (conf) weighted sum:
In above-mentioned formula, x indicates whether bounding box matches with callout box, if matching, x=1, otherwise x=0, c indicate side
The belonging kinds score of boundary's frame, l indicate that bounding box, g indicate callout box, and N is the quantity of matched bounding box, if N=0, just
If objective function penalty values are 0；Position loss (loc) is the parameter of the bounding box (l) and callout box (g) after S33 step
Between smoothL1Loss；
In above-mentioned formula, d is indicated by S32 step but the also not bounding box Jing Guo S33 step.(cx, cy) is indicated in d
Heart coordinate, (w, h) indicate the width and length of d,It indicates by log treated callout box.
Confidence level loss (conf) is softmax by the confidence level of multiple classes and operates to obtain:
In above-mentioned formula,Indicate that i-th of bounding box belongs to the belonging kinds score of k-th of classification,It indicates to carry out
Treated by softmax
S45: by repetitive exercise until model convergence, the weight parameter of characteristic layer is saved.
Preferred according to the present invention, the step S5 carries out the intelligent recognition of anomalous event and pre- in Android end application model
Alert process includes:
S51: original image is obtained using the high-definition camera of Android device, the brightness of image, comparison are read to adjust to arrive and closed
Suitable numerical value, and image is denoised, enhancing processing；
S52: by image input model, the feature for extracting image of trained VGG16 network is utilized；
S53: generating a series of bounding boxes on the picture feature layer extracted, and passes through trained SSD network for institute
Bounding box revert to correct position, and predicts the correct classification of each bounding box；The confidence score of each bounding box
By the classification information Pr (Class of each gridi| Object) and bounding box in classification confidence informationMultiplication obtains, it may be assumed that
S54: using the bounding box of non-maxima suppression removal redundancy, and testing result is shown on the original image；If
Exception is detected, then triggers alarm.
Beneficial effects of the present invention
For the present invention by compact model and by model transplantations to mobile terminal, the carry out for allowing model autonomous in front end is different
Normal intelligent recognition and early warning, has given full play to the advantage of edge calculations, saves time and the back-end server of data transmission
Computing resource.The present invention enables model to make full use of the week of Small object by the picture feature layer in fusion VGG16 network
Side information, to improve the accuracy rate of small target deteection.Meanwhile the present invention uses a series of small volumes on picture feature layer
Product core goes to return the accurate location of bounding box, so that detection speed further increases.In addition, the present invention uses non-maxima suppression
Method eliminates the bounding box of redundancy, keeps testing result more succinct.Finally, the present invention enhances technology by data, picture is adjusted
Different brightness, contrast and use Gaussian Blur processing methods, simulates the picture effect under different scenes, enhances model
Generalization ability.
Detailed description of the invention
Fig. 1 is model framework flow chart of the present invention；
Fig. 2 is the block flow diagram that Conv4_x and Conv5_x amalgamation mode of the present invention is vector connecting method；
Fig. 3 is the block flow diagram that Conv4_x and Conv5_x amalgamation mode of the present invention is corresponding element phase add mode；
Fig. 4 is the schematic diagram of application examples 1 of the present invention；
Fig. 5 is the schematic diagram of application examples 2 of the present invention；
Fig. 6 is the schematic diagram of application examples 3 of the present invention.
Specific embodiment
The present invention is described in detail below with reference to embodiment and Figure of description, but not limited to this.
Embodiment,
A kind of power domain based on edge calculations opens the intelligent recognition and method for early warning of anomalous event under scene, special
Sign is, comprising the following steps:
S1: carrying out image enhancement processing to the training data under different scenes, is carried out using annotation tool to training data
Mark, obtains .xml file；Mark refers to each Zhang Xunlian picture herein, the artificial target to be detected (ratio determined in picture
Such as crane, construction machinery, tower crane) position, reuse annotation tool for these targets and use rectangle frame one by one respectively
It is framed, and sets an attribute value for each rectangle frame, show that the target in this rectangle frame belongs to any classification.By
This, in subsequent S4 step when training pattern, model can identify which position in which picture has which kind of classification
Target, by this principle training pattern；
S2: using VGG16 network as basic network, carries out feature extraction to original image, and will be in VGG16 network
Conv4_x characteristic layer and Conv5_x characteristic layer progress Fusion Features, and the model by the characteristic action after fusion to the end
Prediction interval；This step enables the model that training obtains after S2, S3, S4 step to make full use of the peripheral information of Small object；
S3: adding different network layers after basic network, and target and its ownership are then predicted in these network layers
The score of classification, meanwhile, small convolution kernel is used on characteristic layer, for returning a series of accurate location of bounding boxes；It is described
Belonging kinds refer to that four kinds of classifications being set in advance: 1) normal condition；2) crane；3) construction machinery；4) tower crane；Such as it is pre-
Survey some target ownership crane classification score be 0.7, then it represents that model think this target be crane probability be 70%；
S4: for a large amount of bounding boxes generated on same target position, optimal mesh is found using non-maxima suppression
Bounding box is marked, the bounding box of redundancy, and training pattern are eliminated；Target position described herein is acquired original picture by S1 step
The target area on picture after processing；
S5: the good model of application training in mobile terminal carries out the intelligent recognition and early warning of anomalous event.Preferably, described
Mobile terminal is Android end.
The method of image enhancement in the step S1 are as follows:
S11: different brightness and/or contrast are set to original image, and carry out Gaussian Blur processing, to simulate difference
Scene；
S12: the picture after step S11 processing is labeled using annotation tool, uses 5 in the present invention at present
A classification: 1) normal condition；2) crane；3) construction machinery；4) tower crane；
S13: it to each picture after step S11 processing, randomly proceeds as follows:
1) original picture is used；
2) by sampling one piece of region of rule sampling, sampling rule are as follows: the smallest friendship and ratio 0 to 1 between object at random
Between selection region；
3) one piece of region is randomly sampled；
S14: the region of sampling is original image size ratio [0.1,1], and depth-width ratio is between 0.5 to 2；Work as callout box
Center in the region of sampling when, retain lap；
S15: after sampling step, the region of each sampling is adjusted to fixed size, and with 0.5 probability with
The flip horizontal of machine.
In the step S2, carrying out feature extraction to original image includes:
S21: Image Acquisition is carried out, then acquired image is sent into VGG16 network, successively passes through 5 layers of convolutional layer and pond
Change layer and 2 layers of full articulamentum, extracts the feature of image；
S22: by Conv4_x characteristic layer together with Conv5_x Feature-level fusion, fused feature is applied directly to
Last prediction interval refers to using the feature after fusion as a part input of last prediction interval.
The Conv4_x characteristic layer is that vector connecting method or corresponding element are added with Conv5_x Feature-level fusion mode
Mode.As shown in Figure 2,3.
The method of determining bounding box accurate location includes: in the step S3
S31: the grid that picture feature layer is cut into 8 × 8 or 4 × 4 will be extracted by step S2；
S32: a series of bounding box of fixed sizes, each bounding box packet are generated for each grid on the characteristic layer
At least five Prediction Parameters: x, y, w, h, conf are included, wherein (x, y) indicates centre coordinate of the bounding box relative to grid, institute
Stating (w, h) indicates the width predicted relative to whole image and height, and the conf indicates bounding box and any one callout box
IOU value；
S33: on each characteristic layer, a series of predicted value for generating fixed sizes is gone using a series of convolution kernels.This
Effect is to calculate the parameter of the bounding box generated in S32 step, i.e. x, y, w, h, conf, that is, bounding box is moved to conjunction
Suitable position, and calculate the belonging kinds score of bounding box.Its most prominent advantage is that speed is fast.
In step S33, for a m × n and there is the characteristic layer of p-channel, the size of the convolution kernel used is 3 × 3 ×
p。
The predicted value is a confidence score of belonging kinds or the positional shift value of bounding box.
The step S4 includes: using the method that non-maxima suppression finds optimal object boundary frame
S41: each bounding box and all callout box are matched: when friendship between the two and when than being greater than a threshold value,
Then formed a sample；Preferably, the threshold value is set between 0.5 to 0.9；
S42: will correspond to prediction result in original image on each object space is the bounding box of negative sample according to confidence
Degree is ranked up from big to small and selects multiple positive and negative samples, then selects confidence level maximum multiple, makes its positive and negative samples
Ratio of number is in 1:3 or so；Wherein, a series of bounding box is generated by step S32, then these boundaries is frameed shift by step S33
Suitable position is moved, and calculates their own belonging kinds score；Finally, these bounding boxes are exactly prediction result.It is negative
The definition of sample: by the bounding box after process S33 step compared with callout box, if friendship and ratio between the two is greater than 0.5,
Then think that this bounding box is positive sample, is otherwise negative sample；
S43: it usesIndicate that i-th of bounding box and j-th of callout box of classification k match；Otherwise, it mismatches,
S44: total target loss function is just obtained by position loss (loc) and confidence level loss (conf) weighted sum:
In above-mentioned formula, x indicates whether bounding box matches with callout box, if matching, x=1, otherwise x=0, c indicate side
The belonging kinds score of boundary's frame, l indicate that bounding box, g indicate callout box, and N is the quantity of matched bounding box, if N=0, just
If objective function penalty values are 0；Position loss (loc) is the parameter of the bounding box (l) and callout box (g) after S33 step
Between smoothL1Loss；
In above-mentioned formula, d is indicated by S32 step but the also not bounding box Jing Guo S33 step.(cx, cy) is indicated in d
Heart coordinate, (w, h) indicate the width and length of d,It indicates by log treated callout box.
Confidence level loss (conf) is softmax by the confidence level of multiple classes and operates to obtain:
In above-mentioned formula,Indicate that i-th of bounding box belongs to the belonging kinds score of k-th of classification,It indicates to carry out
Treated by softmax
Weight coefficientSuitable value can be determined by cross validation, generally between 0 to 1；
S45: by repetitive exercise until model convergence, the weight parameter of characteristic layer is saved.It can be used to headend equipment mould
The detection of type.
The step S5 carries out the intelligent recognition of anomalous event in Android end application model and the process of early warning includes:
S51: original image is obtained using the high-definition camera of Android device, the brightness of image, comparison are read to adjust to arrive and closed
Suitable numerical value, and image is denoised, enhancing processing, to improve picture quality, further promote detection effect；
S52: by image input model, the feature for extracting image of trained VGG16 network is utilized；
S53: generating a series of bounding boxes on the picture feature layer extracted, and passes through trained SSD network for institute
Bounding box revert to correct position, and predicts the correct classification of each bounding box；The confidence score of each bounding box
By the classification information Pr (Class of each gridi| Object) and bounding box in classification confidence informationMultiplication obtains, it may be assumed that
S54: using the bounding box of non-maxima suppression removal redundancy, and testing result is shown on the original image；If
Detect exception, for example detected crane, construction machinery, tower crane etc., then trigger alarm.
Application examples 1,
It is of the present invention identification and method for early warning concrete application for example under, as shown in Fig. 4.
By fusion feature extract layer, model is enable to make full use of the information around Small object, to improve Small object
School survey effect, as shown in box in Fig. 4.
Application examples 2,
It is of the present invention identification and method for early warning concrete application for example under, as shown in Fig. 5.
Enhance technology by picture, model can still reach good detection effect under greasy weather or rainy day scene.
Application examples 3,
It is of the present invention identification and method for early warning concrete application for example under, as shown in Fig. 6.
Enhance technology by picture, model can accurately detect abnormal object in the case where dark.
Claims (9)
1. the intelligent recognition and method for early warning of anomalous event, feature under a kind of open scene of the power domain based on edge calculations
It is, comprising the following steps:
S1: being carried out image enhancement processing to the training data under different scenes, be labeled using annotation tool to training data,
Obtain .xml file；
S2: using VGG16 network as basic network, carries out feature extraction to original image, and will be in VGG16 network
Conv4_x characteristic layer and Conv5_x characteristic layer progress Fusion Features, and the model by the characteristic action after fusion to the end
Prediction interval；
S3: adding different network layers after basic network, and target and its belonging kinds are then predicted in these network layers
Score, meanwhile, on characteristic layer use small convolution kernel, for returning a series of accurate location of bounding boxes；
S4: for a large amount of bounding boxes generated on same target position, optimal target side is found using non-maxima suppression
Boundary's frame eliminates the bounding box of redundancy, and training pattern；
S5: the good model of application training in mobile terminal carries out the intelligent recognition and early warning of anomalous event；Preferably, the movement
End is Android end.
2. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 1
Not and method for early warning, which is characterized in that the method for image enhancement in the step S1 are as follows:
S11: different brightness and/or contrast being arranged to original image, and carries out Gaussian Blur processing, to simulate different fields
Scape；
S12: being labeled the picture after step S11 processing using annotation tool,
S13: it to each picture after step S11 processing, randomly proceeds as follows:
1) original picture is used；
2) sampling one piece of region of rule sampling, sampling rule are pressed are as follows: at random between object between the smallest friendship and ratio 0 to 1
Selection region；
3) one piece of region is randomly sampled；
S14: the region of sampling is original image size ratio [0.1,1], and depth-width ratio is between 0.5 to 2；When in callout box
When the heart is in the region of sampling, retain lap；
S15: after sampling step, the region of each sampling is adjusted to fixed size, and random with 0.5 probability
Flip horizontal.
3. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 1
Not and method for early warning, which is characterized in that in the step S2, carrying out feature extraction to original image includes:
S21: Image Acquisition is carried out, then acquired image is sent into VGG16 network, successively passes through 5 layers of convolutional layer and pond layer
And 2 layers of full articulamentum, extract the feature of image；
S22: by Conv4_x characteristic layer together with Conv5_x Feature-level fusion, fused feature is applied directly to finally
Prediction interval, that is, refer to using the feature after fusion as last prediction interval a part input.
4. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 3
Not and method for early warning, which is characterized in that the Conv4_x characteristic layer and Conv5_x Feature-level fusion mode are vector splicing side
Formula or corresponding element phase add mode.
5. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 1
Not and method for early warning, which is characterized in that the method that bounding box accurate location is determined in the step S3 includes:
S31: the grid that picture feature layer is cut into 8 × 8 or 4 × 4 will be extracted by step S2；
S32: generating a series of bounding box of fixed sizes for each grid on the characteristic layer, and each bounding box includes extremely
Few 5 Prediction Parameters: x, y, w, h, conf, wherein (x, y) indicates centre coordinate of the bounding box relative to grid, it is described
(w, h) indicates the width predicted relative to whole image and height, and the conf indicates bounding box and any one callout box
IOU value；
S33: on each characteristic layer, a series of predicted value for generating fixed sizes is gone using a series of convolution kernels.
6. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 5
Not and method for early warning, which is characterized in that in step S33, for a m × n and have the characteristic layer of p-channel, the convolution used
The size of core is 3 × 3 × p.
7. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 5
Not and method for early warning, which is characterized in that the predicted value be belonging kinds a confidence score or bounding box position it is inclined
Shifting value.
8. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 1
Not and method for early warning, which is characterized in that the method that the step S4 finds optimal object boundary frame using non-maxima suppression
Include:
S41: each bounding box and all callout box being matched: when friendship between the two and when than being greater than a threshold value, then will
It forms a sample；Preferably, the threshold value is set between 0.5 to 0.9；
S42: by corresponded on each object space in original image prediction result be negative sample bounding box according to confidence level from
Arrive greatly it is small be ranked up and select multiple positive and negative samples, make its positive and negative samples ratio of number in 1:3 or so；
S43: it usesIndicate that i-th of bounding box and j-th of callout box of classification k match；Otherwise, it mismatches,
S44: total target loss function is just obtained by position loss (loc) and confidence level loss (conf) weighted sum:
In above-mentioned formula, x indicates whether bounding box matches with callout box, if matching, x=1, otherwise x=0, c indicate bounding box
Belonging kinds score, l indicates that bounding box, g indicate callout box, and N is that the quantity of matched bounding box if N=0 just sets mesh
Scalar functions penalty values are 0；It is between the bounding box (l) after S33 step and the parameter of callout box (g) that (loc) is lost in position
SmoothL1Loss；
In above-mentioned formula, d indicates to pass through S32 step but also the bounding box Jing Guo S33 step, (cx, cy) do not indicate that the center of d is sat
Mark, (w, h) indicate the width and length of d,It indicates by log treated callout box；
Confidence level loss (conf) is softmax by the confidence level of multiple classes and operates to obtain:
In above-mentioned formula,Indicate that i-th of bounding box belongs to the belonging kinds score of k-th of classification,It indicates to carry out softmax
Treated
S45: by repetitive exercise until model convergence, the weight parameter of characteristic layer is saved.
9. the intelligence of anomalous event is known under the open scene of a kind of power domain based on edge calculations according to claim 1
Not and method for early warning, which is characterized in that the step S5 carries out the intelligent recognition of anomalous event and pre- in Android end application model
Alert process includes:
S51: obtaining original image using the high-definition camera of Android device, and the brightness of image, comparison are read to adjust to suitable
Numerical value, and image is denoised, enhancing processing；
S52: by image input model, the feature for extracting image of trained VGG16 network is utilized；
S53: generating a series of bounding boxes on the picture feature layer extracted, and passes through trained SSD network for all sides
Boundary's frame revert to correct position, and predicts the correct classification of each bounding box；The confidence score of each bounding box is by every
Classification information Pr (the Class of a gridi| Object) and bounding box in classification confidence informationPhase
It is multiplied to arrive, it may be assumed that
S54: using the bounding box of non-maxima suppression removal redundancy, and testing result is shown on the original image；If detection
Exception has been arrived, then has triggered alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910319835.6A CN110084165B (en) | 2019-04-19 | 2019-04-19 | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910319835.6A CN110084165B (en) | 2019-04-19 | 2019-04-19 | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110084165A true CN110084165A (en) | 2019-08-02 |
| CN110084165B CN110084165B (en) | 2020-02-07 |
Family
ID=67415764
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910319835.6A Active CN110084165B (en) | 2019-04-19 | 2019-04-19 | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110084165B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110766214A (en) * | 2019-10-17 | 2020-02-07 | 中南林业科技大学 | Real-time flood control monitoring system based on edge calculation and big data |
| CN111046822A (en) * | 2019-12-19 | 2020-04-21 | 山东财经大学 | Large vehicle anti-theft method based on artificial intelligence video identification |
| CN111160084A (en) * | 2019-11-19 | 2020-05-15 | 广州民航职业技术学院 | Dangerous article detection method, dangerous article detection device and storage medium |
| CN111204452A (en) * | 2020-02-10 | 2020-05-29 | 北京建筑大学 | Target detection system based on miniature aircraft |
| CN111311775A (en) * | 2020-01-20 | 2020-06-19 | 深圳供电局有限公司 | Electric robot inspection system and method |
| CN111414969A (en) * | 2020-03-26 | 2020-07-14 | 西安交通大学 | Smoke detection method in foggy environment |
| CN111582016A (en) * | 2020-03-18 | 2020-08-25 | 宁波送变电建设有限公司永耀科技分公司 | Intelligent maintenance-free power grid monitoring method and system based on cloud edge collaborative deep learning |
| WO2021042639A1 (en) * | 2019-09-04 | 2021-03-11 | 电子科技大学 | Data packet validity confirmation method based on edge computing and discrete random convolution |
| CN112949849A (en) * | 2021-01-27 | 2021-06-11 | 武汉星巡智能科技有限公司 | Method and device for optimizing intelligent camera detection model by adopting edge calculation |
| CN114445661A (en) * | 2022-01-24 | 2022-05-06 | 电子科技大学 | Embedded image identification method based on edge calculation |
| CN115237606A (en) * | 2022-09-21 | 2022-10-25 | 亿咖通(北京)科技有限公司 | Edge calculation control system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108334830A (en) * | 2018-01-25 | 2018-07-27 | 南京邮电大学 | A kind of scene recognition method based on target semanteme and appearance of depth Fusion Features |
| CN108564097A (en) * | 2017-12-05 | 2018-09-21 | 华南理工大学 | A kind of multiscale target detection method based on depth convolutional neural networks |
| US20180285683A1 (en) * | 2017-03-30 | 2018-10-04 | Beihang University | Methods and apparatus for image salient object detection |
| CN108734717A (en) * | 2018-04-17 | 2018-11-02 | 西北工业大学 | The dark weak signal target extracting method of single frames star chart background based on deep learning |
| CN108875595A (en) * | 2018-05-29 | 2018-11-23 | 重庆大学 | A kind of Driving Scene object detection method merged based on deep learning and multilayer feature |
| CN109165583A (en) * | 2018-08-09 | 2019-01-08 | 北京飞搜科技有限公司 | More size fusion method for detecting human face, device and storage medium |
| US10198671B1 (en) * | 2016-11-10 | 2019-02-05 | Snap Inc. | Dense captioning with joint interference and visual context |
| CN109344878A (en) * | 2018-09-06 | 2019-02-15 | 北京航空航天大学 | A kind of imitative hawk brain feature integration Small object recognition methods based on ResNet |
-
2019
- 2019-04-19 CN CN201910319835.6A patent/CN110084165B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10198671B1 (en) * | 2016-11-10 | 2019-02-05 | Snap Inc. | Dense captioning with joint interference and visual context |
| US20180285683A1 (en) * | 2017-03-30 | 2018-10-04 | Beihang University | Methods and apparatus for image salient object detection |
| CN108564097A (en) * | 2017-12-05 | 2018-09-21 | 华南理工大学 | A kind of multiscale target detection method based on depth convolutional neural networks |
| CN108334830A (en) * | 2018-01-25 | 2018-07-27 | 南京邮电大学 | A kind of scene recognition method based on target semanteme and appearance of depth Fusion Features |
| CN108734717A (en) * | 2018-04-17 | 2018-11-02 | 西北工业大学 | The dark weak signal target extracting method of single frames star chart background based on deep learning |
| CN108875595A (en) * | 2018-05-29 | 2018-11-23 | 重庆大学 | A kind of Driving Scene object detection method merged based on deep learning and multilayer feature |
| CN109165583A (en) * | 2018-08-09 | 2019-01-08 | 北京飞搜科技有限公司 | More size fusion method for detecting human face, device and storage medium |
| CN109344878A (en) * | 2018-09-06 | 2019-02-15 | 北京航空航天大学 | A kind of imitative hawk brain feature integration Small object recognition methods based on ResNet |
Non-Patent Citations (2)
| Title |
|---|
| 王哲峰: "移动端目标检测系统的设计与实现", 《万方数据 知识服务平台》 * |
| 郭之先: "基于深度卷积神经网络的小目标检测", 《万方数据 知识服务平台》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021042639A1 (en) * | 2019-09-04 | 2021-03-11 | 电子科技大学 | Data packet validity confirmation method based on edge computing and discrete random convolution |
| CN110766214A (en) * | 2019-10-17 | 2020-02-07 | 中南林业科技大学 | Real-time flood control monitoring system based on edge calculation and big data |
| CN111160084A (en) * | 2019-11-19 | 2020-05-15 | 广州民航职业技术学院 | Dangerous article detection method, dangerous article detection device and storage medium |
| CN111046822A (en) * | 2019-12-19 | 2020-04-21 | 山东财经大学 | Large vehicle anti-theft method based on artificial intelligence video identification |
| CN111311775A (en) * | 2020-01-20 | 2020-06-19 | 深圳供电局有限公司 | Electric robot inspection system and method |
| CN111204452B (en) * | 2020-02-10 | 2021-07-16 | 北京建筑大学 | Target detection system based on miniature aircraft |
| CN111204452A (en) * | 2020-02-10 | 2020-05-29 | 北京建筑大学 | Target detection system based on miniature aircraft |
| CN111582016A (en) * | 2020-03-18 | 2020-08-25 | 宁波送变电建设有限公司永耀科技分公司 | Intelligent maintenance-free power grid monitoring method and system based on cloud edge collaborative deep learning |
| CN111414969A (en) * | 2020-03-26 | 2020-07-14 | 西安交通大学 | Smoke detection method in foggy environment |
| CN112949849A (en) * | 2021-01-27 | 2021-06-11 | 武汉星巡智能科技有限公司 | Method and device for optimizing intelligent camera detection model by adopting edge calculation |
| CN112949849B (en) * | 2021-01-27 | 2024-03-26 | 武汉星巡智能科技有限公司 | Method and device for optimizing intelligent camera detection model by adopting edge calculation |
| CN114445661A (en) * | 2022-01-24 | 2022-05-06 | 电子科技大学 | Embedded image identification method based on edge calculation |
| CN114445661B (en) * | 2022-01-24 | 2023-08-18 | 电子科技大学 | Embedded image recognition method based on edge calculation |
| CN115237606A (en) * | 2022-09-21 | 2022-10-25 | 亿咖通(北京)科技有限公司 | Edge calculation control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110084165B (en) | 2020-02-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110084165A (en) | The intelligent recognition and method for early warning of anomalous event under the open scene of power domain based on edge calculations | |
| WO2021142902A1 (en) | Danet-based unmanned aerial vehicle coastline floating garbage inspection system | |
| CN104156734B (en) | A kind of complete autonomous on-line study method based on random fern grader | |
| CN103839065B (en) | Extraction method for dynamic crowd gathering characteristics | |
| CN107123131B (en) | Moving target detection method based on deep learning | |
| CN109214280B (en) | Shop identification method and device based on street view, electronic equipment and storage medium | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| CN110147743A (en) | Real-time online pedestrian analysis and number system and method under a kind of complex scene | |
| CN107392901A (en) | A kind of method for transmission line part intelligence automatic identification | |
| CN110135296A (en) | Airfield runway FOD detection method based on convolutional neural networks | |
| CN108764308A (en) | A kind of recognition methods again of the pedestrian based on convolution loop network | |
| CN106356757A (en) | Method for inspecting electric power lines by aid of unmanned aerial vehicle on basis of human vision characteristics | |
| CN107330357A (en) | Vision SLAM closed loop detection methods based on deep neural network | |
| CN108520114A (en) | A kind of textile cloth defect detection model and its training method and application | |
| CN111444939A (en) | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field | |
| CN109918971A (en) | Number detection method and device in monitor video | |
| CN103593679A (en) | Visual human-hand tracking method based on online machine learning | |
| Rong et al. | Pest identification and counting of yellow plate in field based on improved mask r-cnn | |
| Chen et al. | Building area estimation in drone aerial images based on mask R-CNN | |
| CN109800756A (en) | A kind of text detection recognition methods for the intensive text of Chinese historical document | |
| Zheng et al. | A review of remote sensing image object detection algorithms based on deep learning | |
| CN109376736A (en) | A kind of small video target detection method based on depth convolutional neural networks | |
| CN116385958A (en) | Edge intelligent detection method for power grid inspection and monitoring | |
| Tang et al. | Pest-YOLO: Deep image mining and multi-feature fusion for real-time agriculture pest detection | |
| Wang et al. | Citrus recognition based on YOLOv4 neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190729 Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant after: SHANDONG University Applicant after: SHANDONG ZHIYANG ELECTRIC Co.,Ltd. Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant before: Shandong University |
|
| TA01 | Transfer of patent application right | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant after: SHANDONG University Applicant after: Zhiyang Innovation Technology Co.,Ltd. Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant before: Shandong University Applicant before: SHANDONG ZHIYANG ELECTRIC Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200108 Address after: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant after: SHANDONG University Applicant after: Zhiyang Innovation Technology Co.,Ltd. Applicant after: STATE GRID ZHEJIANG ELECTRIC POWER Co.,Ltd. Address before: Licheng Alexander Road in Ji'nan City, Shandong province 250199 No. 27 Applicant before: Shandong University Applicant before: Zhiyang Innovation Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |