CN111091069A - Power grid target detection method and system guided by blind image quality evaluation - Google Patents
Power grid target detection method and system guided by blind image quality evaluation Download PDFInfo
- Publication number
- CN111091069A CN111091069A CN201911182943.XA CN201911182943A CN111091069A CN 111091069 A CN111091069 A CN 111091069A CN 201911182943 A CN201911182943 A CN 201911182943A CN 111091069 A CN111091069 A CN 111091069A
- Authority
- CN
- China
- Prior art keywords
- image quality
- quality evaluation
- target
- human body
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 97
- 238000001514 detection method Methods 0.000 title claims abstract description 86
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000005516 engineering process Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims description 50
- 238000012544 monitoring process Methods 0.000 claims description 26
- 238000013527 convolutional neural network Methods 0.000 claims description 19
- 238000012216 screening Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000013145 classification model Methods 0.000 claims description 6
- 230000006399 behavior Effects 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 4
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 abstract description 11
- 230000008859 change Effects 0.000 abstract description 9
- 238000012545 processing Methods 0.000 abstract description 6
- 238000005286 illumination Methods 0.000 abstract description 3
- 230000000007 visual effect Effects 0.000 abstract description 3
- 238000013507 mapping Methods 0.000 description 6
- 238000001228 spectrum Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The application belongs to the technical field of image data processing, and particularly relates to a power grid target detection method and system guided by blind image quality evaluation. The image recognition technology adopted in the power industry has the problems of low accuracy and poor algorithm robustness and expansibility. The application provides a power grid target detection method and system guided by blind image quality evaluation. The safety behavior of workers is analyzed through the detected characteristics of the relative position, the characteristic representation, the human body profile representation and the like of the safety appliance target and the human body target, and the management standard and the personnel safety are guaranteed. The method and the device filter the low-quality images, overcome the interference of visual angle change, illumination change, motion change and the like, and improve the detection precision; the method and the device have the advantages that any source image is preprocessed, the application range is wide, and the preprocessing time is shortened.
Description
Technical Field
The application relates to the technical field of image data processing, in particular to a power grid target detection method and system guided by blind image quality evaluation.
Background
In the power industry, due to the high voltage property of a power grid field, the operation safety is always the focus of much attention. In order to ensure that workers wear safe labor insurance and do work on duty, the workers generally carry out manual inspection before going to the field, and accidents caused by incomplete safety protection are prevented.
In busy and stressful actual work, the manual mode of self-checking and self-correcting or mutual checking is inevitable to avoid negligence or omission, and the phenomenon that the labor protection article is not worn completely happens, so that the high-risk characteristic is exposed. With the deep development of computer technology, the labor insurance wearing condition of field workers can be conveniently and efficiently identified through the computer vision identification technology, and further the computer vision identification technology becomes one of important ways for assisting manual inspection.
Existing common data sets for computer image recognition include: PASCAL VOC, ImageNet, MS COCO, and the like, mostly aim at higher quality images. These data sets are generally used for the performance test of target detection algorithms by researchers or for related competitions, and as the hot field of image processing, excellent algorithms such as R-CNN, Fast R-CNN, Faster R-CNN, YOLO and SSD are emerged in succession, and excellent performance is obtained on static data sets.
However, in a real environment, images obtained by a video often have phenomena of distortion and degradation, and the accuracy of the algorithm is also reduced. In the actual production of enterprises, in order to obtain a better identification effect, images from any source need to be screened, and the robustness and the expansibility of an image processing algorithm are maintained under the condition of improving the image identification accuracy, so that the actual objective requirement becomes a technical problem to be solved urgently.
Disclosure of Invention
The application provides a power grid target detection method and system guided by blind image quality evaluation, and aims to solve the problems of low target identification accuracy and poor algorithm robustness and expansibility in the current power industry.
The technical scheme adopted by the application is as follows:
in a first aspect of the present application, a power grid target detection method guided by blind image quality evaluation is provided, which includes the following steps:
screening an input monitoring video sequence by training a blind image quality evaluation network, and selecting a video frame with the quality meeting a preset requirement;
performing target detection on the video frames with the quality meeting the preset requirement by training a target detection network;
and analyzing the result of the target detection to obtain a conclusion of the safety state of the target human body.
Optionally, the step of screening the input surveillance video sequence and selecting a video frame with a quality meeting a preset requirement by training the blind image quality evaluation network includes:
the method comprises the steps of creating a blind image quality evaluation data set by using a monitoring video acquired on the site of a power grid and combining a data enhancement technology;
training a blind image quality evaluation network according to the blind image quality evaluation data set;
separating each frame of image of an input monitoring video sequence, inputting each frame of image into a blind image quality evaluation network for evaluation, setting a quality threshold value, extracting images with the quality value larger than or equal to the quality threshold value, and arranging the images into an image set according to the original time sequence.
Optionally, in the step of training the blind image quality evaluation network according to the blind image quality evaluation data set, the convolutional neural network is used to train the blind image quality evaluation data set, a VGG-F model with low computational complexity is used as a basis, a large-step convolutional kernel is selected, and all convolutional layers are filled with zero.
Optionally, in the step of creating an image quality evaluation data set by using the monitoring video acquired in the power grid field and combining with a data enhancement technology, the method includes:
creating a common image quality evaluation dedicated data set into an image quality evaluation data set, wherein the common image quality evaluation dedicated data set specifically comprises images of the data set and image scores, and the common image quality evaluation dedicated data set comprises a LIVE data set, a CSIQ data set and a TID2013 data set.
Optionally, in the step of performing target detection on the video frame with quality meeting the preset requirement by training the target detection network, the method includes:
constructing a target training set of a target human body, a safety helmet and a safety suit in a power grid field;
training a target detection network according to the target training set;
inputting the image sets arranged according to the original time sequence into a target detection network for target detection to obtain the confidence degrees and frame information of all targets, determining that the targets exist when the confidence degrees exceed a preset threshold value, setting the targets as quasi-targets, and reserving the frames of the targets;
and removing the redundant boxes by using a non-maximum suppression algorithm according to the quasi-target information to obtain a final detection result of the target information and the frame information.
Optionally, in the step of analyzing the result of the target detection to obtain the conclusion of the safety state of the target human body, the method includes:
and analyzing the safety state of the target human body according to the detection result of the final target information and the frame information to obtain the conclusion whether the posture of the target human body is safe or not and whether the wearing of the safety helmet and the safety clothes is safe or not.
Optionally, the step of analyzing the safety state of the target human body includes:
judging the relative relation between the safety helmet and the target human body, wherein the safety helmet is detected in the vertical or approximately vertical direction of the target human body, the target human body and the frame of the safety helmet are intersected, and otherwise, reporting that the safety behavior of the safety helmet is abnormal;
judging whether a target human body is wearing safety clothing, collecting samples of the human body wearing safety clothing by utilizing deep layer convolution characteristics, supporting a vector machine by using a machine learning method to train a two-classification model offline, extracting the deep layer convolution characteristics of the human body wearing safety clothing in a detected frame of the target human body during online monitoring, inputting the deep layer convolution characteristics into a two-classification model classifier trained in advance to classify, judging whether the target human body is wearing the safety clothing, and obtaining a conclusion whether the target human body wearing the safety clothing is safe;
and judging whether the target human body posture is safe, and when the width-height ratio of the human body frame is greater than 1 and the ratio is maintained for 10 seconds continuously, determining that a person falling accident occurs, and obtaining a conclusion that the target human body posture is unsafe.
Optionally, after the step of analyzing the result of the target detection to obtain the conclusion of the safety state of the target human body, the method further includes:
and triggering an alarm at the working site to give an alarm when the conclusion of the safety state of the target human body is negative.
In a second aspect of the application, a power grid target detection system guided by blind image quality evaluation is provided, which comprises a blind image quality evaluation module, a target detection module and a target human body safety analysis module;
the blind image quality evaluation module is used for screening the input monitoring video sequence by training a blind image quality evaluation network and selecting a video frame with the quality meeting the preset requirement;
the target detection module is used for carrying out target detection on the video frames with the quality meeting the preset requirement through training a target detection network;
and the target human body safety analysis module is used for analyzing the result of target detection to obtain the safety state conclusion of the target human body.
Optionally, the blind image quality evaluation module is configured to train a blind image quality evaluation network according to a blind image quality evaluation data set, where the blind image quality evaluation module is further configured to train the blind image quality evaluation data set by using a convolutional neural network, select a large-step convolutional kernel based on a VGG-F model with low computational complexity, and use zero padding for all convolutional layers.
The technical scheme of the application has the following beneficial effects:
according to the power grid target detection method guided by blind image quality evaluation, the blind image quality evaluation network is trained, the input monitoring video sequences are screened, and video frames with the quality meeting the preset requirements are selected. The method for preprocessing any image can construct a training set on real power grid image data, and learn characteristics by using a convolutional neural network technology, so that the characteristics can be more accurately represented and the application capability in a power grid environment is improved; the blind image quality evaluation network can filter low-quality images, overcome the interference of visual angle change, illumination change, motion change and the like, and improve the detection precision; the safety behavior of the staff is analyzed, management standards and personnel safety are guaranteed, and personnel safety accidents are timely handled. The method can be used for preprocessing any source image, and has the advantages of wider application range, shortened preprocessing time and saved time cost due to the improved preprocessing method.
Drawings
In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a block flow diagram of one embodiment provided by a first aspect of the present application;
FIG. 2 is a block flow diagram of another embodiment provided by the first aspect of the present application;
fig. 3 is a schematic structural diagram of an embodiment provided in the second aspect of the present application.
Detailed Description
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following examples do not represent all embodiments consistent with the present application. But merely as exemplifications of systems and methods consistent with certain aspects of the application, as recited in the claims.
Referring to fig. 1, for facilitating understanding of the first aspect of the present application, a blind image quality evaluation guided power grid target detection method is provided, which includes the following steps:
s101, screening an input monitoring video sequence by training a blind image quality evaluation network, and selecting a video frame with quality meeting a preset requirement;
s102, performing target detection on the video frame with the quality meeting the preset requirement through a training target detection network;
and S103, analyzing the target detection result to obtain a safety state conclusion of the target human body.
The images with the quality meeting the preset requirement are selected by preprocessing any source images, so that the method can be suitable for any field video images, and has wide compatibility with images shot by different places, different power grids or different monitoring devices.
Optionally, the step of screening the input surveillance video sequence and selecting a video frame with a quality meeting a preset requirement by training the blind image quality evaluation network includes:
s1001, creating a blind image quality evaluation data set by using a monitoring video acquired on the site of a power grid and combining a data enhancement technology;
s1002, training a blind image quality evaluation network according to the blind image quality evaluation data set;
s1003, separating each frame of image of the input monitoring video sequence, inputting each frame of image into a blind image quality evaluation network for evaluation, setting a quality threshold, extracting images with the quality value larger than or equal to the quality threshold, and arranging the images into an image set according to the original time sequence.
Referring to fig. 2, in the present embodiment, the utility model is combined with the grid site, so that the utility model is more practical, and the speciality of the utility model is also embodied. And setting a quality threshold value to screen each frame of image, preliminarily filtering out the image with poor quality, and extracting each frame of image of the monitoring video sequence which is beneficial to identification. And the accuracy of the target identification of the power grid site in the later period is improved.
Optionally, in the step of training the blind image quality evaluation network according to the blind image quality evaluation data set, the convolutional neural network is used to train the blind image quality evaluation data set, a VGG-F model with low computational complexity is used as a basis, a large-step convolutional kernel is selected, and all convolutional layers are filled with zero.
In the embodiment, the convolutional neural network is utilized, a VGG-F model with low operation complexity is adopted, and a large-step convolution kernel is selected, so that the resolution of a feature mapping spectrum is reduced, the operation complexity is reduced, and the aim of shortening the data processing process and time is fulfilled; in addition, the strategy of using zero padding for all convolutional layers can ensure that the resolution is not affected by convolution.
Optionally, in the step of creating an image quality evaluation data set by using the monitoring video acquired in the power grid field and combining with a data enhancement technology, the method includes:
creating a common image quality evaluation dedicated data set into an image quality evaluation data set, wherein the common image quality evaluation dedicated data set specifically comprises images of the data set and image scores, and the common image quality evaluation dedicated data set comprises a LIVE data set, a CSIQ data set and a TID2013 data set.
Optionally, in the step of performing target detection on the video frame with quality meeting the preset requirement by training the target detection network, the method includes:
constructing a target training set of a target human body, a safety helmet and a safety suit in a power grid field;
training a target detection network according to the target training set;
inputting the image sets arranged according to the original time sequence into a target detection network for target detection to obtain the confidence degrees and frame information of all targets, determining that the targets exist when the confidence degrees exceed a preset threshold value, setting the targets as quasi-targets, and reserving the frames of the targets;
and removing the redundant boxes by using a non-maximum suppression algorithm according to the quasi-target information to obtain a final detection result of the target information and the frame information.
In this embodiment, real monitoring video data in a power grid environment needs to be collected, a synthetic distortion operation of an image quality evaluation database is simulated, distortion processing is performed on a video frame, and a blind image quality evaluation training set is established; and (3) manually marking the video frame, distinguishing different objects such as a target human body and the like, and further establishing a target training set. Training a target detection network by constructing a target training set, and carrying out target detection on the screened image sets arranged according to the original time sequence to obtain confidence degrees and frame information of all targets; setting a confidence preset threshold value, and eliminating interference information, so that a target can be accurately identified; and a non-maximum value suppression algorithm is used for removing the redundant frame, so that the accuracy of the final target information and the frame information is further ensured.
Optionally, in the step of analyzing the result of the target detection to obtain the conclusion of the safety state of the target human body, the method includes:
and analyzing the safety state of the target human body according to the detection result of the final target information and the frame information to obtain the conclusion whether the posture of the target human body is safe or not and whether the wearing of the safety helmet and the safety clothes is safe or not.
The purpose of this application is in order to detect whether the staff of electric wire netting scene is the gesture safe, whether wear the safety helmet and whether wear the concrete state of safety clothes, therefore, also must draw corresponding target human posture safe, whether safe conclusion of wearing of safety helmet and safety clothes is safe to the pertinence ground.
Optionally, the step of analyzing the safety state of the target human body includes:
judging the relative relation between the safety helmet and the target human body, wherein the safety helmet is detected in the vertical or approximately vertical direction of the target human body, the target human body and the frame of the safety helmet are intersected, and otherwise, reporting that the safety behavior of the safety helmet is abnormal;
judging whether a target human body is wearing safety clothing, collecting samples of the human body wearing safety clothing by utilizing deep layer convolution characteristics, supporting a vector machine by using a machine learning method to train a two-classification model offline, extracting the deep layer convolution characteristics of the human body wearing safety clothing in a detected frame of the target human body during online monitoring, inputting the deep layer convolution characteristics into a two-classification model classifier trained in advance to classify, judging whether the target human body is wearing the safety clothing, and obtaining a conclusion whether the target human body wearing the safety clothing is safe;
and judging whether the target human body posture is safe, and when the width-height ratio of the human body frame is greater than 1 and the ratio is maintained for 10 seconds continuously, determining that a person falling accident occurs, and obtaining a conclusion that the target human body posture is unsafe.
In the present embodiment, how to judge the method and the determination criteria of the above three states are shown, and actually, those skilled in the art can easily conceive other setting and assignment methods, but all fall within the scope of the present application.
Optionally, after the step of analyzing the result of the target detection to obtain the conclusion of the safety state of the target human body, the method further includes:
and triggering an alarm at the working site to give an alarm when the conclusion of the safety state of the target human body is negative.
In the embodiment, when the conclusion of the safety state of the target human body is negative, an alarm of a working site is triggered to give an alarm. The system is beneficial to immediate reaction in the presence of unsafe conditions, so that the safety risk of field workers of the power grid is further reduced, and casualty accidents of the workers are reduced.
Referring to fig. 3, for convenience of understanding of the second aspect of the present application, the power grid target detection system guided by blind image quality evaluation provided includes a blind image quality evaluation module, a target detection module, and a target human body safety analysis module;
the blind image quality evaluation module is used for screening the input monitoring video sequence by training a blind image quality evaluation network and selecting a video frame with the quality meeting the preset requirement;
the target detection module is used for carrying out target detection on the video frames with the quality meeting the preset requirement through training a target detection network;
and the target human body safety analysis module is used for analyzing the result of target detection to obtain the safety state conclusion of the target human body.
Optionally, the blind image quality evaluation module is configured to train a blind image quality evaluation network according to a blind image quality evaluation data set, where the blind image quality evaluation module is further configured to train the blind image quality evaluation data set by using a convolutional neural network, select a large-step convolutional kernel based on a VGG-F model with low computational complexity, and use zero padding for all convolutional layers.
In the embodiment of the application, the applicant carries out more detailed practical verification on the power grid target detection method guided by blind image quality evaluation. The verification comprises the following specific contents:
in the training stage of the blind image quality evaluation network, in order to shorten the preprocessing time, the idea of the VGG-F model based on low operation complexity adopts a large-step convolution kernel to rapidly reduce the resolution of the feature mapping spectrum, thereby reducing the operation complexity. Specifically, when designing the CNN network structure, the input image size is predetermined to 224 × 224 × 3 based on the mainstream CNN network, and 8 convolution kernels with a step size of 7 × 7 are used in the first convolution layer to reduce the feature mapping spectrum from 224 × 224 to 56 × 56, and then one step size is usedThe maximum pooling layer of 2 further reduces it to 28 x 28. The second convolutional layer will use 16 convolutional kernels of size 3 x 3 with step size 2 to extract the more discriminatory features, where the resolution drops to 14 x 14. After a maximum pooling layer with step size of 2, a bilinear pooling layer is used to convert the 7 × 7 × 16 convolution response tensor into a 1 × 1 × 256 vector by solving its outer product, which is used as the image representation. Mapping image representations to a predicted score of 1 value using a 1 × 1 × 256 × 1 full-connectivity layer, and using l1The loss function calculates error learning. All convolutional layers use a zero-padding strategy to ensure that the resolution is not affected by the convolution. Table 1 shows a lightweight IQA convolutional neural network structure, and based on the style and convention of MEON network, the parameters of convolutional layers are referred to as "feature mapping spectrum height × feature mapping spectrum width | input channel number × output channel number | convolution step length | convolution zero padding number".
TABLE 1 lightweight IQA convolutional neural network architecture
In order to enable the lightweight IQA convolutional neural network structure to better conform to data in a real power grid, a data set of the real power grid is constructed in practice, and fine adjustment of the IQA network is achieved. 25 volunteers, including 13 males and 12 females, were selected, both of which were professional researchers in the field of non-image quality evaluation. Based on five-level evaluation criteria (custom MOS range in brackets) of ITU-R CCIR500-1 document, the volunteer's evaluation of a grid image may be: one of "excellent (90-100)", "good (80-90)", "medium (70-80)", "poor (60-70)", and "poor (60 or less)". After completing the subjective evaluation of all volunteers, the evaluation received by each image is summarized, and the grade with the highest rating number is taken as the final rating. For each class, 100 total 500 images were selected as a dataset, 80% of which were subjected to fine tuning training and the remaining 20% were tested.
In actual operation, a monitoring video sequence of a real power grid is input into a fine-tuned IQA network for evaluation, images rated as ' poor ' and ' bad ' are removed, and images rated as ' good ', good ' and ' medium ' are reserved for continuous follow-up work.
The method for detecting the target of the monitoring video sequence screened by the IQA network comprises the following steps:
constructing a target training set: the method comprises the steps of making a Pascal VOC data set, an MS COCO data set, an ImageNet data set and the like which are common at present into a Pascal VOC2007 data set format, specifically comprising an image of the data set and a label file (xml format), wherein the label file stores frame information (x) of each target*,y*,w*,h*) And a class, wherein (x)*,y*) Coordinates representing the upper left corner of the bounding box, (w)*,h*) Indicating the bezel width and height.
Training the target detection network: based on the convolutional neural network VGG16 as a base network of the detection network, a specific network structure of the VGG16 is shown in table 2 and consists of 13 convolutional layers and 3 fully-connected layers.
TABLE 2 convolutional neural network VGG16
Training a VGG16 model on a target training set, initializing parameters, and training an improved fast RCNN network by using an approximate joint training mode, wherein the specific improvements are as follows:
the input image size is unified, and the short edge is zoomed to 800, so that the detection rate of small objects such as safety instruments is improved; because the shooting distance in the surveillance video is far and near, the size difference of the target is very big, and the size of the sliding window is increased to 9 types, which are: { 32X 32, 64X 64, 96X 96,128X 128,160X 160,192X 192,224X 224,256X 256,288X 288 }.
The loss function of the module detection network consists of two parts including classification loss LclsAnd frame regression loss LregDefined as formula (one):
in the formula, i is equal to {1,2, …, NwinDenotes the reference numeral of the sliding window, NwinRepresenting the number of sliding windows used for a batch training, NregThe number of coordinate positions of the sliding window is represented, λ is a factor (in the embodiment of the present invention, λ is 10) for regulating and controlling the classification loss and the border regression loss, and p isiIndicating the probability that the window is predicted to be an object,
representing true probability (positive samples)
Negative sample
)。
The real frame coordinate information of each target in the training sample is known as (x)*,y*,w*,h*) (ii) a The information of the sliding window frame in the RPN network is (x)r,yr,wr,hr) Wherein (x)r,yr) Denotes the coordinates of the upper left corner of the sliding window, (w)r,hr) Represents the sliding window width and height; the frame information predicted by the RPN network is (x, y, w, h), wherein (x, y) represents the coordinates of the upper left corner of the predicted frame, and (w, h) represents the width and height of the predicted frame; t is tiFour-dimensional parametric vector (t) representing coordinate relation of prediction frame and sliding windowx,ty,tw,th)，
Four-dimensional parameterized vector representing coordinate relation of real frame and sliding window
The specific calculation formula is as follows:
fractional loss L in equation (one)clsBy cross entropy loss, bounding box regression loss LregBy using
The loss is specifically defined by the formula (three) to (five):
by minimizing L ({ p)i},{ti}) to train and adjust the network, the target detection network adopts a random gradient descent method to carry out back propagation, the iteration is 100000 times, the learning rate is set to be 0.001 in the first 50000 times, and the learning rate is reduced to be 0.1 in the last 10000 times.
And carrying out target detection on the monitoring video sequence by using the network obtained by training to obtain the confidence score and the predicted frame information (x, y, w, h) of the target. The confidence of each predicted frame is calculated by the softmax classifier, and when the confidence score is greater than a set threshold (in this embodiment, the set threshold is T ═ 0.75), the frame is regarded as a quasi-target, and the target frame is retained.
According to the power grid target detection method guided by blind image quality evaluation, the blind image quality evaluation network is trained, the input monitoring video sequences are screened, and video frames with the quality meeting the preset requirements are selected. The method for preprocessing any image can construct a training set on real power grid image data, and learn characteristics by using a convolutional neural network technology, so that the characteristics can be more accurately represented and the application capability in a power grid environment is improved; the blind image quality evaluation network can filter low-quality images, overcome the interference of visual angle change, illumination change, motion change and the like, and improve the detection precision; the safety behavior of the staff is analyzed, management standards and personnel safety are guaranteed, and personnel safety accidents are timely handled. The method can be used for preprocessing any source image, and has the advantages of wider application range, shortened preprocessing time and saved time cost due to the improved preprocessing method.
The embodiments provided in the present application are only a few examples of the general concept of the present application, and do not limit the scope of the present application. Any other embodiments extended according to the scheme of the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.
Claims (10)
1. A power grid target detection method guided by blind image quality evaluation is characterized by comprising the following steps:
screening an input monitoring video sequence by training a blind image quality evaluation network, and selecting a video frame with the quality meeting a preset requirement;
performing target detection on the video frames with the quality meeting the preset requirement by training a target detection network;
and analyzing the result of the target detection to obtain a conclusion of the safety state of the target human body.
2. The blind image quality evaluation guided power grid target detection method according to claim 1, wherein in the step of screening the input monitoring video sequence and selecting the video frame with the quality meeting the preset requirement by training the blind image quality evaluation network, the method comprises:
the method comprises the steps of creating a blind image quality evaluation data set by using a monitoring video acquired on the site of a power grid and combining a data enhancement technology;
training a blind image quality evaluation network according to the blind image quality evaluation data set;
separating each frame of image of an input monitoring video sequence, inputting each frame of image into a blind image quality evaluation network for evaluation, setting a quality threshold value, extracting images with the quality value larger than or equal to the quality threshold value, and arranging the images into an image set according to the original time sequence.
3. The blind image quality evaluation guided power grid target detection method according to claim 2, characterized in that in the step of training the blind image quality evaluation network according to the blind image quality evaluation dataset, the blind image quality evaluation dataset is trained by using a convolutional neural network, a VGG-F model with low computational complexity is based and a large-step convolutional kernel is selected, and all convolutional layers use zero padding.
4. The blind image quality evaluation guided power grid target detection method according to claim 2, wherein in the step of creating an image quality evaluation data set by using the monitoring video acquired on the power grid site in combination with a data enhancement technology, the method comprises:
creating a common image quality evaluation dedicated data set into an image quality evaluation data set, wherein the common image quality evaluation dedicated data set specifically comprises images of the data set and image scores, and the common image quality evaluation dedicated data set comprises a LIVE data set, a CSIQ data set and a TID2013 data set.
5. The blind image quality evaluation guided power grid target detection method according to claim 1, wherein in the step of performing target detection on the video frame with the quality meeting the preset requirement through training the target detection network, the method comprises:
constructing a target training set of a target human body, a safety helmet and a safety suit in a power grid field;
training a target detection network according to the target training set;
inputting the image sets arranged according to the original time sequence into a target detection network for target detection to obtain the confidence degrees and frame information of all targets, determining that the targets exist when the confidence degrees exceed a preset threshold value, setting the targets as quasi-targets, and reserving the frames of the targets;
and removing the redundant boxes by using a non-maximum suppression algorithm according to the quasi-target information to obtain a final detection result of the target information and the frame information.
6. The blind image quality evaluation guided power grid target detection method according to claim 1, wherein in the step of analyzing the result of target detection to obtain the conclusion of the safety state of the target human body, the method comprises:
and analyzing the safety state of the target human body according to the detection result of the final target information and the frame information to obtain the conclusion whether the posture of the target human body is safe or not and whether the wearing of the safety helmet and the safety clothes is safe or not.
7. The blind image quality evaluation guided power grid target detection method according to claim 6, wherein the step of analyzing the safety state of the target human body comprises:
judging the relative relation between the safety helmet and the target human body, wherein the safety helmet is detected in the vertical or approximately vertical direction of the target human body, the target human body and the frame of the safety helmet are intersected, and otherwise, reporting that the safety behavior of the safety helmet is abnormal;
judging whether a target human body is wearing safety clothing, collecting samples of the human body wearing safety clothing by utilizing deep layer convolution characteristics, supporting a vector machine by using a machine learning method to train a two-classification model offline, extracting the deep layer convolution characteristics of the human body wearing safety clothing in a detected frame of the target human body during online monitoring, inputting the deep layer convolution characteristics into a two-classification model classifier trained in advance to classify, judging whether the target human body is wearing the safety clothing, and obtaining a conclusion whether the target human body wearing the safety clothing is safe;
and judging whether the target human body posture is safe, and when the width-height ratio of the human body frame is greater than 1 and the ratio is maintained for 10 seconds continuously, determining that a person falling accident occurs, and obtaining a conclusion that the target human body posture is unsafe.
8. The blind image quality evaluation guided power grid target detection method according to claim 1, further comprising, after the step of analyzing the result of target detection to draw a conclusion of the safety state of the target human body:
and triggering an alarm at the working site to give an alarm when the conclusion of the safety state of the target human body is negative.
9. A power grid target detection system guided by blind image quality evaluation is characterized by comprising a blind image quality evaluation module, a target detection module and a target human body safety analysis module;
the blind image quality evaluation module is used for screening the input monitoring video sequence by training a blind image quality evaluation network and selecting a video frame with the quality meeting the preset requirement;
the target detection module is used for carrying out target detection on the video frames with the quality meeting the preset requirement through training a target detection network;
and the target human body safety analysis module is used for analyzing the result of target detection to obtain the safety state conclusion of the target human body.
10. The blind image quality evaluation guided power grid target detection system according to claim 9, wherein the blind image quality evaluation module is configured to train a blind image quality evaluation network according to a blind image quality evaluation data set, wherein the blind image quality evaluation module is further configured to train the blind image quality evaluation data set by using a convolutional neural network, select a large-step convolutional kernel based on a VGG-F model with low computational complexity, and use zero padding for all convolutional layers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182943.XA CN111091069A (en) | 2019-11-27 | 2019-11-27 | Power grid target detection method and system guided by blind image quality evaluation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911182943.XA CN111091069A (en) | 2019-11-27 | 2019-11-27 | Power grid target detection method and system guided by blind image quality evaluation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111091069A true CN111091069A (en) | 2020-05-01 |
Family
ID=70393644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911182943.XA Pending CN111091069A (en) | 2019-11-27 | 2019-11-27 | Power grid target detection method and system guided by blind image quality evaluation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111091069A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753805A (en) * | 2020-07-08 | 2020-10-09 | 深延科技(北京)有限公司 | Method and device for detecting wearing of safety helmet |
| CN112257620A (en) * | 2020-10-27 | 2021-01-22 | 广州华微明天软件技术有限公司 | Safe wearing condition identification method |
| CN116862285A (en) * | 2023-06-12 | 2023-10-10 | 深圳市名通科技股份有限公司 | Automatic operation and maintenance auditing method based on AI target detection and character recognition |
Citations (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69805132D1 (en) * | 1997-04-04 | 2002-06-06 | Tektronix Inc | Image quality assessment with spatially variable assessment threshold values |
| US20030035581A1 (en) * | 2001-08-13 | 2003-02-20 | Nokia Mobile Phones, Ltd. | Method and system for measuring perceptual distortion in images |
| CN102700569A (en) * | 2012-06-01 | 2012-10-03 | 安徽理工大学 | Mining electric locomotive passerby monitoring method based on image processing and alarm system |
| CN202906579U (en) * | 2012-11-28 | 2013-04-24 | 河海大学常州校区 | Power distribution network real-time monitoring and control system based on internet of things |
| CN103945217A (en) * | 2014-04-17 | 2014-07-23 | 四川九洲电器集团有限责任公司 | Complex wavelet domain semi-blind image quality evaluation method and system based on entropies |
| CN104021382A (en) * | 2014-06-20 | 2014-09-03 | 北京释码大华科技有限公司 | Eye image collection method and system |
| CN104318562A (en) * | 2014-10-22 | 2015-01-28 | 百度在线网络技术（北京）有限公司 | Method and device for confirming quality of internet images |
| CN106856002A (en) * | 2016-11-22 | 2017-06-16 | 上海大学 | A kind of unmanned plane shooting image quality evaluating method |
| CN107666594A (en) * | 2017-09-18 | 2018-02-06 | 广东电网有限责任公司东莞供电局 | A kind of video monitoring monitors the method operated against regulations in real time |
| CN107832802A (en) * | 2017-11-23 | 2018-03-23 | 北京智芯原动科技有限公司 | Quality of human face image evaluation method and device based on face alignment |
| CN108280399A (en) * | 2017-12-27 | 2018-07-13 | 武汉普利商用机器有限公司 | A kind of scene adaptive face identification method |
| CN108564010A (en) * | 2018-03-28 | 2018-09-21 | 浙江大华技术股份有限公司 | A kind of detection method, device, electronic equipment and storage medium that safety cap is worn |
| CN108600744A (en) * | 2018-07-17 | 2018-09-28 | 中星技术股份有限公司 | The method of picture quality control, the method and apparatus of video camera shooting image |
| CN108986075A (en) * | 2018-06-13 | 2018-12-11 | 浙江大华技术股份有限公司 | A kind of judgment method and device of preferred image |
| CN109145789A (en) * | 2018-08-09 | 2019-01-04 | 炜呈智能电力科技（杭州）有限公司 | Power supply system safety work support method and system |
| CN109146856A (en) * | 2018-08-02 | 2019-01-04 | 深圳市华付信息技术有限公司 | Picture quality assessment method, device, computer equipment and storage medium |
| CN109191971A (en) * | 2018-11-19 | 2019-01-11 | 哈尔滨学院 | A kind of preschool education interaction systems based on intelligent image identification |
| CN109379584A (en) * | 2018-11-26 | 2019-02-22 | 北京科技大学 | Camera system and image quality adjusting method under a kind of complex environment light application conditions |
| CN109671051A (en) * | 2018-11-15 | 2019-04-23 | 北京市商汤科技开发有限公司 | Picture quality detection model training method and device, electronic equipment and storage medium |
| CN109784274A (en) * | 2018-12-29 | 2019-05-21 | 杭州励飞软件技术有限公司 | Identify the method trailed and Related product |
| CN109961434A (en) * | 2019-03-30 | 2019-07-02 | 西安电子科技大学 | Non-reference picture quality appraisement method towards the decaying of level semanteme |
| CN110309719A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wears management control method and system |
| CN110309718A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wearing detection method |
| CN110414362A (en) * | 2019-07-02 | 2019-11-05 | 安徽继远软件有限公司 | Electric power image data augmentation method based on production confrontation network |
| CN110428026A (en) * | 2019-07-18 | 2019-11-08 | 国网河北省电力有限公司 | The system and method that identification power grid infrastructure project is broken rules and regulations safely |
| CN110458075A (en) * | 2019-08-05 | 2019-11-15 | 北京泰豪信息科技有限公司 | Detection method, storage medium, detection device and the detection system that safety cap is worn |
| CN111062303A (en) * | 2019-12-12 | 2020-04-24 | 深圳市捷顺科技实业股份有限公司 | Image processing method, system and computer storage medium |
-
2019
- 2019-11-27 CN CN201911182943.XA patent/CN111091069A/en active Pending
Patent Citations (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69805132D1 (en) * | 1997-04-04 | 2002-06-06 | Tektronix Inc | Image quality assessment with spatially variable assessment threshold values |
| US20030035581A1 (en) * | 2001-08-13 | 2003-02-20 | Nokia Mobile Phones, Ltd. | Method and system for measuring perceptual distortion in images |
| CN102700569A (en) * | 2012-06-01 | 2012-10-03 | 安徽理工大学 | Mining electric locomotive passerby monitoring method based on image processing and alarm system |
| CN202906579U (en) * | 2012-11-28 | 2013-04-24 | 河海大学常州校区 | Power distribution network real-time monitoring and control system based on internet of things |
| CN103945217A (en) * | 2014-04-17 | 2014-07-23 | 四川九洲电器集团有限责任公司 | Complex wavelet domain semi-blind image quality evaluation method and system based on entropies |
| CN104021382A (en) * | 2014-06-20 | 2014-09-03 | 北京释码大华科技有限公司 | Eye image collection method and system |
| CN104318562A (en) * | 2014-10-22 | 2015-01-28 | 百度在线网络技术（北京）有限公司 | Method and device for confirming quality of internet images |
| CN106856002A (en) * | 2016-11-22 | 2017-06-16 | 上海大学 | A kind of unmanned plane shooting image quality evaluating method |
| CN107666594A (en) * | 2017-09-18 | 2018-02-06 | 广东电网有限责任公司东莞供电局 | A kind of video monitoring monitors the method operated against regulations in real time |
| CN107832802A (en) * | 2017-11-23 | 2018-03-23 | 北京智芯原动科技有限公司 | Quality of human face image evaluation method and device based on face alignment |
| CN108280399A (en) * | 2017-12-27 | 2018-07-13 | 武汉普利商用机器有限公司 | A kind of scene adaptive face identification method |
| CN108564010A (en) * | 2018-03-28 | 2018-09-21 | 浙江大华技术股份有限公司 | A kind of detection method, device, electronic equipment and storage medium that safety cap is worn |
| CN108986075A (en) * | 2018-06-13 | 2018-12-11 | 浙江大华技术股份有限公司 | A kind of judgment method and device of preferred image |
| CN108600744A (en) * | 2018-07-17 | 2018-09-28 | 中星技术股份有限公司 | The method of picture quality control, the method and apparatus of video camera shooting image |
| CN109146856A (en) * | 2018-08-02 | 2019-01-04 | 深圳市华付信息技术有限公司 | Picture quality assessment method, device, computer equipment and storage medium |
| CN109145789A (en) * | 2018-08-09 | 2019-01-04 | 炜呈智能电力科技（杭州）有限公司 | Power supply system safety work support method and system |
| CN109671051A (en) * | 2018-11-15 | 2019-04-23 | 北京市商汤科技开发有限公司 | Picture quality detection model training method and device, electronic equipment and storage medium |
| CN109191971A (en) * | 2018-11-19 | 2019-01-11 | 哈尔滨学院 | A kind of preschool education interaction systems based on intelligent image identification |
| CN109379584A (en) * | 2018-11-26 | 2019-02-22 | 北京科技大学 | Camera system and image quality adjusting method under a kind of complex environment light application conditions |
| CN109784274A (en) * | 2018-12-29 | 2019-05-21 | 杭州励飞软件技术有限公司 | Identify the method trailed and Related product |
| CN109961434A (en) * | 2019-03-30 | 2019-07-02 | 西安电子科技大学 | Non-reference picture quality appraisement method towards the decaying of level semanteme |
| CN110309719A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wears management control method and system |
| CN110309718A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wearing detection method |
| CN110414362A (en) * | 2019-07-02 | 2019-11-05 | 安徽继远软件有限公司 | Electric power image data augmentation method based on production confrontation network |
| CN110428026A (en) * | 2019-07-18 | 2019-11-08 | 国网河北省电力有限公司 | The system and method that identification power grid infrastructure project is broken rules and regulations safely |
| CN110458075A (en) * | 2019-08-05 | 2019-11-15 | 北京泰豪信息科技有限公司 | Detection method, storage medium, detection device and the detection system that safety cap is worn |
| CN111062303A (en) * | 2019-12-12 | 2020-04-24 | 深圳市捷顺科技实业股份有限公司 | Image processing method, system and computer storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753805A (en) * | 2020-07-08 | 2020-10-09 | 深延科技(北京)有限公司 | Method and device for detecting wearing of safety helmet |
| CN112257620A (en) * | 2020-10-27 | 2021-01-22 | 广州华微明天软件技术有限公司 | Safe wearing condition identification method |
| CN116862285A (en) * | 2023-06-12 | 2023-10-10 | 深圳市名通科技股份有限公司 | Automatic operation and maintenance auditing method based on AI target detection and character recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112734692B (en) | Defect identification method and device for power transformation equipment | |
| CN111091069A (en) | Power grid target detection method and system guided by blind image quality evaluation | |
| WO2018028103A1 (en) | Unmanned aerial vehicle power line inspection method based on characteristics of human vision | |
| CN105184779B (en) | One kind is based on the pyramidal vehicle multiscale tracing method of swift nature | |
| CN108537154A (en) | Transmission line of electricity Bird's Nest recognition methods based on HOG features and machine learning | |
| CN106875373A (en) | Mobile phone screen MURA defect inspection methods based on convolutional neural networks pruning algorithms | |
| CN113324864B (en) | Pantograph carbon slide plate abrasion detection method based on deep learning target detection | |
| CN103488974A (en) | Facial expression recognition method and system based on simulated biological vision neural network | |
| CN109613002A (en) | A kind of glass defect detection method, apparatus and storage medium | |
| KR101183105B1 (en) | Method of establishing information of cloud data and establishing system of information of cloud data | |
| CN109506628A (en) | Object distance measuring method under a kind of truck environment based on deep learning | |
| CN109948501A (en) | The detection method of personnel and safety cap in a kind of monitor video | |
| CN111476083A (en) | Automatic identification method for wearing of safety helmet of electric power staff | |
| CN113344475B (en) | Transformer bushing defect identification method and system based on sequence modal decomposition | |
| CN116824517B (en) | Substation operation and maintenance safety control system based on visualization | |
| CN111368768A (en) | Human body key point-based employee gesture guidance detection method | |
| Kai et al. | Wearing safety helmet detection in substation | |
| CN113177439B (en) | Pedestrian crossing road guardrail detection method | |
| CN108416304B (en) | Three-classification face detection method using context information | |
| CN108520539B (en) | Image target detection method based on sparse learning variable model | |
| CN113221812A (en) | Training method of face key point detection model and face key point detection method | |
| CN112488213A (en) | Fire picture classification method based on multi-scale feature learning network | |
| CN115995097A (en) | Deep learning-based safety helmet wearing standard judging method | |
| CN115830302A (en) | Multi-scale feature extraction and fusion power distribution network equipment positioning identification method | |
| CN115578792A (en) | Indoor personnel falling early warning detection method and system based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200501 |