CN116052035A - Power plant personnel perimeter intrusion detection method based on convolutional neural network - Google Patents
Power plant personnel perimeter intrusion detection method based on convolutional neural network Download PDFInfo
- Publication number
- CN116052035A CN116052035A CN202211379670.XA CN202211379670A CN116052035A CN 116052035 A CN116052035 A CN 116052035A CN 202211379670 A CN202211379670 A CN 202211379670A CN 116052035 A CN116052035 A CN 116052035A
- Authority
- CN
- China
- Prior art keywords
- network
- neural network
- training
- key point
- power plant
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 75
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 47
- 238000000605 extraction Methods 0.000 claims abstract description 43
- 238000013528 artificial neural network Methods 0.000 claims abstract description 32
- 230000006399 behavior Effects 0.000 claims abstract description 30
- 230000009194 climbing Effects 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000010586 diagram Methods 0.000 claims abstract description 17
- 230000002493 climbing effect Effects 0.000 claims abstract description 13
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000004519 manufacturing process Methods 0.000 claims abstract description 5
- 238000013145 classification model Methods 0.000 claims abstract description 4
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 31
- 230000004913 activation Effects 0.000 claims description 16
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 7
- 238000005457 optimization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 3
- 230000009545 invasion Effects 0.000 claims 1
- 238000013135 deep learning Methods 0.000 abstract description 3
- 239000000284 extract Substances 0.000 abstract description 2
- 238000004458 analytical method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 230000002265 prevention Effects 0.000 description 4
- 239000013307 optical fiber Substances 0.000 description 3
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 239000003245 coal Substances 0.000 description 2
- 230000007123 defense Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000003546 flue gas Substances 0.000 description 1
- 230000007849 functional defect Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a power plant personnel perimeter intrusion detection method based on a convolutional neural network, which comprises two processes of algorithm model training and algorithm prediction training, wherein the algorithm model training process comprises the following steps: preparing a data set, preprocessing the data, constructing a feature extraction neural network, constructing a key point detection neural network, training a key point detection model, manufacturing a classification data set by utilizing a skeleton diagram generated by key points, constructing a climbing action classification network, and training a classification model by adopting the skeleton data set; the algorithm prediction training comprises the steps of obtaining an input video stream, fusing three network models, adding logic for judging climbing behavior and outputting a real-time prediction result; the invention is based on the human body key point detection and image classification method in deep learning, extracts image features in depth, fuses network models, sets confidence rules, and outputs more accurate recognition results.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a power plant personnel perimeter intrusion detection method based on a convolutional neural network.
Background
High risk areas in which some people are forbidden to enter are all available in the power plant, the power plant can manually add fences and warning boards to the dangerous areas, on-site constructors are reminded and warned, but unfortunately, due to the fact that safety consciousness or greedy drawing approaches are lacked, illegal behaviors of crossing the fences exist, and risks of falling injury and collision are likely to exist in the process of climbing the fences. In addition, in the coal yard area with coal rails, once people turn into the fence area, the hidden danger of serious accidents exists.
To prevent such accidents, there are many conventional perimeter intrusion detection techniques, including infrared, leaky cable, vibrating cable, pulsed electronic fence, fiber optic fence, etc. The infrared technology is mainly characterized in that a correlation mode is adopted for defense, in actual application, a plurality of infrared lights are frequently correlation-formed into a long row, namely an infrared fence, and once a person invades the periphery, the infrared signal is shielded by the human body, and the system gives an alarm. The cable is also widely applied to outdoor perimeter intrusion monitoring, and an intrusion detection scheme based on a leakage cable sensor has the characteristics of low false alarm rate and strong concealment and is not easy to be influenced by environment and topography. The pulse electronic fence sends and receives high-voltage pulse signals by the pulse generator, and forms an intelligent perimeter system with the front-end fence, and when the front-end fence is detected to be in a short circuit state, an open circuit state and the like, the alarm system is triggered. The optical fiber fence is a distributed fence sensing system, and the principle is that an optical fiber is used as a sensing medium to remotely and real-timely monitor the safety of an object in the defense setting range of the optical fiber sensor.
Although the perimeter security detection technology plays a role in security to a certain extent, the perimeter security detection technology is limited by technical conditions of the perimeter security detection technology and has a plurality of functional defects, such as: the distance which can be monitored is short, the unit distance cost is high, and the system cost is high under the condition that long-distance monitoring is required; the service life of the sensor unit is short, the sensor unit is used continuously for a long time, and the maintenance cost is high; the interference opportunities are more (electromagnetic interference, signal interference, crosstalk and the like), the sensitivity is reduced, and the false alarm rate and the missing report rate are increased; for large-scale monitoring, the traditional scheme has no positioning function, encounters invasive behavior and cannot be positioned quickly. This means that dangerous locations cannot be determined timely and accurately, and countermeasures cannot be taken timely to reduce accident losses caused by invasive actions.
Disclosure of Invention
In order to solve the problems, the invention provides a power plant personnel perimeter intrusion detection method based on a convolutional neural network, which is a human body key point detection and image classification method based on deep learning, deeply extracts image features, fuses a network model, sets confidence rules, outputs a more accurate identification result, has the advantages of low price, high efficiency, real-time detection and no dead angle leakage, saves manpower, material resources and financial resources, and has very important significance for solving perimeter prevention problems.
The technical scheme of the invention is as follows:
the invention provides a power plant personnel perimeter intrusion detection method based on a convolutional neural network, which comprises two processes of algorithm model training and algorithm predictive training;
the algorithm model training process comprises the following steps:
s11, data set preparation and data preprocessing: selecting video recordings containing climbing behaviors and normal walking of personnel in a plurality of scenes and angles from a monitoring database of a power plant, performing frame extraction processing on the video according to a certain frame rate interval, and selecting video frames of the normal behaviors and the climbing behaviors; preparing a human body key point detection data set and dividing the human body key point detection data set;
s12, constructing a feature extraction neural network: on the basis of VGG-16 classification network, setting convolution step length as 2 to replace pooling structure of each layer (pooling structure can reduce task feature extraction capability of target detection, segmentation and the like), increasing residual structure, deepening overall network depth, adding attention module and expansion convolution module with expansion coefficient of 2 in layers 2, 3 and 4 of network respectively in order to increase perception capability and feature extraction capability of the network, and increasing capability of effective feature filtration and receptive field of the network;
s13, constructing a key point detection neural network: the key point detection neural network is based on an original OpenPose key point detection algorithm, and the algorithm principle is as follows: firstly, extracting a position confidence feature map and a position affinity vector field feature map of a picture through a convolutional neural network, and then combining the position confidence feature map and the position affinity vector field feature map to output the gesture of each person through greedy reasoning and a map method, namely: for an image, firstly, all key points of all people appearing in the image are found, then the points are grouped, and the points of the same person are matched and connected;
s14, training a key point detection model, and constructing a model data set: sending the human body key point detection data set with the divided data proportion into a key point detection neural network and a skeleton extraction network, performing iterative optimization on model training by using a gradient descent method, and storing an optimal model training weight w1;
s15, utilizing a skeleton diagram generated by key points to manufacture a classification data set;
s16, constructing a climbing action classification network: the climbing action classification network adopts two layers of fully connected neural networks;
s17, training a classification model by adopting a skeleton data set: the skeleton model obtained in the step S15 is sent to a network in the step S16 for training, and classification loss is set as a cross entropy loss function (CrossEntropy Loss); finally, performing network optimization by using an Adam gradient descent method to obtain a training weight w2;
the algorithm predictive training process comprises the following steps:
s21, acquiring an input video stream;
s22, merging the feature extraction neural network, the key point detection neural network and the climbing action classification network, and adding a logic for judging the climbing action: the three network models are fused and integrated into the program. Judging the identified key points and the manually defined perimeter, and calculating how many key points fall into the perimeter range;
s23, outputting a real-time prediction result.
According to the convolutional neural network-based power plant personnel perimeter intrusion detection method, in step S11, an on-source MS COCO human body key point detection data set is adopted as the data set, and the data set is divided into training data, verification data and test data of a key point detection model according to the proportion of 8:1:1.
According to the power plant personnel perimeter intrusion detection method based on the convolutional neural network, in the step S12, the attention expansion convolutional module algorithm flow is as follows: assuming that the input of the attention-expanded convolution is any one of the intermediate feature maps F.epsilon.R H×W×C F is firstly input into the attention layer of the module, the characteristic channel is compressed through global average pooling operation to obtain a characteristic identifier of Cx1x1, and then a one-dimensional channel attention force diagram F is obtained based on a simple door mechanism and a sigmoid activation function c ∈R 1×1×C After F is obtained c Thereafter, F c The weighted feature map is obtained by performing matrix multiplication operation with the input feature map F, then the feature map is sent to a cascade expansion convolution layer, finally an output feature map after the attention expansion convolution module is obtained, and the attention map extraction process can be formulated as follows:
wherein ,σ1 Representing sigmoid activation function, sigma 0 Representing the relu activation function, w x, b x represent the weight and bias of the convolutional layer respectively,
representing an element product operation. />
According to the power plant personnel perimeter intrusion detection method based on the convolutional neural network, in the step S13, the key point detection neural network receives the feature map obtained by the feature extraction neural network in the step S12, and the key point extraction model in the step firstly sends the feature map F into an initial key point extraction network, wherein the network is formed by cascade connection of 3 multiplied by 3 and 1 multiplied by 1; this section includes two branches, one corresponding to the generated Keypoint heat map (Keypoint heat map) and part of vector field heat map (PAF heat map), followed by 2 sets of Keypoint enhancement extraction networks, which function to further generate more accurate feature maps. The original OpenPose is a 5-group enhanced extraction network, and experiments show that the accuracy is not greatly improved by a 5-layer network, so that 3 groups of modules are removed through experiments, and the reasoning speed and the precision of the model are taken as a certain choice.
Further, according to the convolutional neural network-based power plant personnel perimeter intrusion detection method of the present invention, in step S14, the training loss of the key point detection model is as follows:
wherein ,
representing location confidence map label, < >>
The field map label representing the region affinity vector, W (p), is a binary mask, W (p) =0 when the label is missing in p of the image.
According to the convolutional neural network-based power plant personnel perimeter intrusion detection method, in step S15, the weight w obtained in step S14 is loaded 1 Starting a skeleton generation model to identify video frames in the actual climbing scene of the power plant extracted in the step S11 to obtain binarized normal and climbing linesAnd classifying the two types of skeleton diagrams, marking class labels, and manufacturing a climbing classification data set according to the ratio of the training set, the verification set and the test set to 8:1:1.
According to the power plant reheat flue gas baffle operation prediction method based on the integrated hybrid model, in the step S16, the two layers of fully connected neural networks are arranged, the node of the first layer of network is set as (16384,100), the activation function is Relu, the node of the second layer of network is set as (100, 2), the classification activation function of the classification network is a softmax function, and the climbing behavior and the normal behavior are optimized in a classification mode.
8. The convolutional neural network-based power plant personnel perimeter intrusion detection method of claim 1 or 7, wherein in step S17, the cross entropy loss function mathematical formula is:
where q represents the true value of a pixel, and p represents the predicted value of a pixel.
Further, according to the convolutional neural network-based power plant personnel perimeter intrusion detection method of the present invention, in step S17, the cross entropy loss function mathematical formula is:
where q represents the true value of a pixel, and p represents the predicted value of a pixel.
According to the convolutional neural network-based power plant personnel perimeter intrusion detection method disclosed by the invention, in step S21, a camera in a power plant is used for controlling a plurality of specific scenes, the behaviors of operators in the scenes are monitored in real time, and a program is used for carrying out multithread acquisition on a plurality of cameras and carrying out frame extraction processing on acquired video data and sending the video data into an image queue to be predicted.
According to the power plant personnel perimeter intrusion detection method based on the convolutional neural network, in the step S23, an image queue to be predicted is sent into an algorithm model after fusion, the weight of the algorithm model is loaded, the climbing boundary crossing behavior is calculated according to the judgment logic set in the step S22, and the prediction result is displayed in real time.
The power plant personnel perimeter intrusion detection method based on the convolutional neural network provided by the invention has the following advantages: (1) Compared with the traditional shallow machine learning algorithm, the deep learning-based convolutional neural network algorithm can solve the problem of insufficient learning deep feature extraction, and the deep extraction is carried out on the data features, so that the risk diagnosis and prediction results are more accurate. (2) The invention uses the residual structure to deepen the network depth of the feature extraction network, so that the extracted features are more sufficient and abundant. (3) The invention uses the attention module, and makes the feature extraction network pay more attention to the key features of certain feature layers and image areas through mask weighting of the attention layers, and simultaneously suppresses invalid features of spatial areas such as image backgrounds and the like. (4) And after each attention module of the feature extraction network, the common 2D convolution operation is replaced by using the expansion convolution, so that the receptive field of the feature extraction convolution is increased, a convolution unit range can sense a feature area with a larger range, and the feature downsampling process is smoother. (5) The invention utilizes the method based on human body key point detection and image classification, and increases the robustness and accuracy of the algorithm by calculating the weight discrimination of the human body key point duty ratio and climbing behavior detection in the perimeter. (6) The perimeter intrusion prevention early warning solution of the invention is simple, convenient and quick, can complete the control by only utilizing the field monitoring without additionally adding hardware sensing equipment, has simple construction and maintenance, and is suitable for perimeter intrusion prevention detection in a large-range and long-distance scene. (7) The intelligent analysis method can collect relevant data of the intrusion perimeter in real time to carry out intelligent analysis, monitor the perimeter range, define any polygonal virtual perimeter in the sight area by a user, and give an alarm if someone intrudes. Only detecting humanoid, the false alarm rate is extremely low, and the method is not interfered by environment. (8) The invention can be used for real-time calling and monitoring, saving the alarm image and the alarm time, facilitating the power plant staff to correct the risk event, and being capable of achieving the following effect compared with the common electronic fence scheme.
Drawings
The aspects and advantages of the present application will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
In the drawings:
FIG. 1 is a flow chart of the algorithm model training of the present invention;
FIG. 2 is a flow chart of the algorithm predictive training of the present invention;
FIG. 3 is a block diagram of a feature extraction neural network of the present invention;
FIG. 4 is a schematic diagram of a residual module structure according to the present invention;
FIG. 5 is a block diagram of an attention-expanding convolution module of the present disclosure;
FIG. 6 is a block diagram of a keypoint detection neural network of the present invention;
FIG. 7 is a diagram of a climbing action classification network structure of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the principles of the present invention and not in limitation thereof.
In the present invention, the described embodiments are some, but not all, embodiments of the invention unless specifically indicated otherwise.
In the present invention, all the embodiments mentioned herein and the preferred embodiments may be combined with each other to form new technical solutions, if not specifically described.
In the present invention, all technical features mentioned herein and preferred features may be combined with each other to form new technical solutions, if not specifically stated.
The invention is further illustrated by the following examples:
in the embodiment, the climbing out-of-range behavior of a certain power plant staff is detected, the two processes of algorithm model training and algorithm predictive training are included, and the implementation process of the invention is further described through the detailed explanation of the embodiment. The implementation steps of the climbing out-of-range behavior algorithm model training of a certain power plant worker are as follows:
step S11, data set preparation and data preprocessing: and selecting a plurality of scenes, which are easy to cause climbing behaviors of personnel, and video recordings, which contain climbing behaviors of personnel and normally walk, of a certain power plant monitoring database, and intercepting effective video frames. Collecting climbing data of single people and multiple people in three time periods of early, middle and late, and performing frame extraction processing on video according to the frequency of 25 frames/second to obtain 2300 pieces of climbing behavior data and normal behavior data; preparing a human body key point detection data set and dividing the human body key point detection data set; the method comprises the steps of preparing an open source MS COCO human body key point detection data set to be divided into training data, verification data and test data of a key point detection model according to the proportion of 8:1:1;
step S12, constructing a feature extraction neural network: as shown in fig. 3, on the basis of the VGG-16 classification network, a convolution step length of 2 is set to replace a pooling structure of each layer (the pooling structure can reduce task feature extraction capability such as target detection and segmentation), a residual structure is added, the overall network depth is deepened, in order to increase the perceptibility and feature extraction capability of the network, an attention module and an expansion convolution module with an expansion coefficient of 2 are respectively added at layers 2, 3 and 4 of the network, and the capability of effective feature filtration and the receptive field of the network are increased; the structure of the attention expansion convolution module is shown in fig. 5, and a specific algorithm flow is as follows: assuming that the input of the attention-expanded convolution is any one of the intermediate feature maps F.epsilon.R H×W×C F is firstly input into the attention layer of the module, the characteristic channel is compressed through global average pooling operation to obtain a characteristic identifier of Cx1x1, and then a one-dimensional channel attention pattern F is obtained based on a simple door mechanism and a sigmoid activation function c ∈R 1×1×C After F is obtained c Thereafter, F c The weighted feature map is obtained by matrix multiplication operation with the input feature map F, then the feature map is sent into a cascade expansion convolution layer, and finally the output after the attention expansion convolution module is obtainedAnd (5) a characteristic diagram. Note that the force diagram extraction process may be formulated as follows:
wherein σ1 represents a sigmoid activation function, σ0 represents a relu activation function, w * B represents the weight and bias of the convolutional layer respectively,
representing an element product operation.
The network layer parameters of the feature extraction network module of this embodiment are shown in the following table:
| content | Convolution kernel size | Number of convolution kernels | Convolution step length | Feature map size |
| Raw data | —— | —— | —— | Different in size |
| Unified scale | —— | —— | —— | 640＊640 |
| Input layer | —— | —— | —— | 640＊640 |
| Convolutional layer C1 | 3 | 32 | 2 | 320＊320 |
| Convolutional layer C2 | 3 | 64 | 2 | 160＊160 |
| Convolutional layer C3 | 3 | 128 | 2 | 80＊80 |
| Convolutional layer C4 | 3 | 256 | 1 | 80＊80 |
| Convolutional layer C5 | 3 | 512 | 1 | 80＊80 |
Step S13, constructing a key point detection neural network: the key point detection neural network is based on an original OpenPose key point detection algorithm, and the algorithm principle is as follows: firstly, extracting a position confidence feature map and a position affinity vector field feature map of a picture through a convolutional neural network, and then combining the position confidence feature map and the position affinity vector field feature map to output the gesture of each person through greedy reasoning and a map method, namely: for an image, all key points of all people appearing in the image are found first, then the points are grouped to enable the points of the same person to be matched and connected, namely for an image, all key points of all people appearing in the image are found first, then the points are grouped to enable the points of the same person to be matched and connected.
The structure of the neural network for detecting key points is shown in fig. 6, which receives the feature map obtained by the feature extraction network in step S12, and the feature map F is first sent to an initial key point extraction network, namely stage 1 in fig. 6, where the network is formed by concatenating 3×3 convolution and 1×1 convolution. The part has two branches, which respectively correspond to the generated key point heat map (Keypoint heat map) and part vector field heat map (PAF heat map), and then a 2-group key point enhancement extraction network is used for further generating a more accurate feature map, and the structure is shown as stage t in figure 6. The original OpenPose is a 5-group enhanced extraction network, and experiments show that the accuracy is not greatly improved by a 5-layer network, so that 3 groups of modules are removed through experiments, and the reasoning speed and the precision of the model are taken as a certain choice.
The parameters of the key point detection neural network layer in this embodiment are shown in the following table:
| content | Convolution kernel size | Number of convolution kernels | Convolution step length | Feature map size |
| Input layer | 3 | 128 | 1 | 80＊80 |
| |
3,1 | 128 | 1 | 80＊80 |
| |
7,1 | 185 | 1 | 80＊80 |
| |
7,1 | 185 | 1 | 80＊80 |
| Output layer | —— | —— | —— | 640＊640 |
Step S14, training a key point detection model: the training loss for setting the key point detection model is as follows:
wherein ,
representing location confidence map label, < >>
The field label representing the region affinity vector, W (p), is a binary mask, W (p) =0 when the label is missing in p of the image.
The hardware environment is as follows: NVIDIA RTX 3090. The software environment is as follows: CUDA, cudnn, pytorchl.7, opencv, etc. Training a key point model by adopting a human body key point data set in MSCOCO, setting a batch size (batch size) as 8, setting a period (epoch) as 300, performing iterative optimization on model training by utilizing a gradient descent method, adopting SGD (generalized algorithm) by adopting the gradient descent method, setting an initial learning rate (lr) as 1e-4, setting a learning rate reduction strategy in training, and reducing lr to 1e-5 at 150 epoch. And setting and storing optimal model weight w1 by optimizing an iterative algorithm model.
Step S15, a classification data set is manufactured by utilizing a skeleton diagram generated by key points: and (3) predicting the skeleton map of 4600 pieces of climbing data processed in the step (S11) by using a key point detection model, and generating a binary normal human skeleton map with climbing behaviors. The generated skeleton diagram is subjected to data cleaning, the condition of disordered key points is removed, and the preprocessed images (about 4000 images) are divided into a training set, a verification set and a test set according to the proportion of 8:1:1;
step S16, constructing a climbing action classification network: the climbing action classification network adopts two layers of simple fully-connected neural networks; the nodes of the first layer network are set as (16384, 100), the activation function is Relu, the nodes of the second layer network are set as (100, 2), and the classification activation function of the classification network is a softmax function, which is used for performing classification optimization on climbing behaviors and normal behaviors.
The parameters of the climbing classification network of this embodiment are shown in the following table:
| content | Activation function | Number of hidden layers |
| Full connection layer F1 | ReLU function | 16384 |
| Full connection layer F2 | ReLU function | 100 |
| Classification layer F3 | Softmax function | 2 |
Step S17, training a classification model by adopting a skeleton data set: the hardware environment is as follows: NVIDIARTX 3090.
The software environment is as follows: CUDA, cudnn, pytorch 1.7.7, opencv, etc. Setting classification loss as a cross entropy loss function; a cross entropy loss function (CrossEntropy Loss), mathematically formulated as follows:
wherein q represents the true value of a certain pixel point, and p represents the predicted value of a certain pixel point;
and (3) sending the skeleton model obtained in the step S15 into the network in the step S16 for training, setting the batch size to 128, setting the epoch to 100000, adopting Adam for a gradient descent method, setting initial lr to 1e-4, setting a learning rate reduction strategy in training, and reducing lr to 1e-5 when the epoch is 50000. And setting and storing optimal model weight w2 by optimizing an iterative algorithm model.
The implementation steps of the detection and prediction of the climbing out-of-range behavior of a certain power plant worker in the embodiment are as follows:
step S21: acquiring an input video stream:
the video data acquisition is carried out on specific service scenes by utilizing the original video acquisition equipment such as a dome camera or a gun camera, a distributed control ball and the like in the power plant, and the algorithm model and the camera are in one-to-many relation, so that the data are read in a multithreading mode, and the utilization rate of the equipment is improved. In order to improve the reasoning speed of the model, frames are extracted and compressed on the video after the video of each scene is acquired, and then the video is sent to an image queue to be predicted.
Step S22: the feature extraction neural network, the key point detection neural network and the climbing action classification network are fused and the climbing out-of-range behavior judgment logic is additionally arranged: and carrying out cascade data fusion on the three models of the feature extraction network, the key point identification network and the climbing classification network, and integrating the three models into a program. And judging the identified key points and the manually defined perimeter, and calculating how many key points fall into the perimeter range.
The resolution of the picture of the scene obtained by the camera is 1920x1080, and the manually-defined perimeter range is a perimeter line formed by [ (845,5), (803,152), (780,248), (798,369), (860,410), (1085,1053), (1890,1053), (1889,5) ] coordinate points.
Step S23: outputting a real-time prediction result: and sending the image queue to be predicted into the fused algorithm model, loading the weight of the total algorithm model, predicting the obtained image queue to be predicted in real time, and displaying the recognition result.
The method is mainly applied to scenes such as important area prevention, false break-in of semi-open areas, perimeter break-in and the like. Depending on an intelligent analysis system, if people are found to cross the border, alarm image display, voice alarm and site point position voice alarm can be carried out in a monitoring room. Intelligent full-automatic analysis is achieved, personnel on duty is not needed, and manual operation is not needed. The invention is an ultra-long distance regional warning system, has simple scheme, is easy to use and control, and can meet the requirements of modern perimeter security.
The present invention is not limited to the above-mentioned embodiments, and any changes, modifications, and variations that can be easily contemplated by those skilled in the art within the scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (10)
1. The power plant personnel perimeter intrusion detection method based on the convolutional neural network is characterized by comprising two processes of algorithm model training and algorithm predictive training;
the algorithm model training process comprises the following steps:
s11, data set preparation and data preprocessing: selecting video recordings containing climbing behaviors and normal walking of personnel in a plurality of scenes and angles from a monitoring database of a power plant, performing frame extraction processing on the video, and selecting video frames of the normal behaviors and the climbing behaviors; preparing a human body key point detection data set and dividing the human body key point detection data set;
s12, constructing a feature extraction neural network: on the basis of a VGG-16 classification network, a pooling structure with a convolution step length of 2 instead of each layer is arranged, a residual structure is increased, the depth of the whole network is deepened, an attention module and an expansion convolution module with an expansion coefficient of 2 are respectively added in layers 2, 3 and 4 of the network, and the filtering capacity of effective characteristics and the receptive field of the network are increased;
s13, constructing a key point detection neural network: the key point detection neural network is based on an original OpenPose key point detection algorithm, and the algorithm principle is as follows: firstly, extracting a position confidence feature map and a position affinity vector field feature map of a picture through a convolutional neural network, and then combining the position confidence feature map and the position affinity vector field feature map to output the gesture of each person through greedy reasoning and a map method;
s14, training a key point detection model, and constructing a model data set: sending the human body key point detection data set with the divided data proportion into a key point detection neural network and a skeleton extraction network, performing iterative optimization on model training by using a gradient descent method, and storing an optimal model training weight w1;
s15, utilizing a skeleton diagram generated by key points to manufacture a classification data set;
s16, constructing a climbing action classification network: the climbing action classification network adopts two layers of fully connected neural networks;
s17, training a classification model by adopting a skeleton data set: the skeleton model obtained by processing in the step S15 is sent to a network in the step S16 for training, and classification loss is set as a cross entropy loss function; finally, performing network optimization by using a gradient descent method to obtain a training weight w2;
the algorithm predictive training process comprises the following steps:
s21, acquiring an input video stream;
s22, merging the feature extraction neural network, the key point detection neural network and the climbing action classification network, and adding a logic for judging the climbing action;
s23, outputting a real-time prediction result.
2. The method for detecting the perimeter intrusion of power plant personnel based on the convolutional neural network according to claim 1, wherein in step S11, the data set is an open source MS COCO human body key point detection data set, and the data set is divided into training data, verification data and test data of a key point detection model according to a ratio of 8:1:1.
3. The power plant personnel perimeter intrusion detection method based on convolutional neural network according to claim 2, wherein in step S12, the attention-expanding convolutional module algorithm flow is as follows: assuming that the input of the attention-expanded convolution is any one of the intermediate feature maps F.epsilon.R H×W×C F is firstly input into the attention layer of the module, the characteristic channel is compressed through global average pooling operation to obtain a characteristic identifier of Cx1x1, and then a one-dimensional channel attention force diagram F is obtained based on a simple door mechanism and a sigmoid activation function c ∈R 1×1×C After F is obtained c Thereafter, F c The weighted feature map is obtained by performing matrix multiplication operation with the input feature map F, then the feature map is sent to a cascade expansion convolution layer, finally an output feature map after the attention expansion convolution module is obtained, and the attention map extraction process can be formulated as follows:
wherein ,σ1 Representing sigmoid activation function, sigma 0 Representing the relu activation function, w * ，b * Representing the weights and offsets of the convolutional layers respectively,
representing an element product operation.
4. The method for detecting the perimeter intrusion of power plant personnel based on the convolutional neural network according to claim 3, wherein in the step S13, the keypoint detection neural network receives the feature map obtained by the feature extraction neural network in the step S12, and the keypoint extraction model in the step firstly sends the feature map F to an initial keypoint extraction network, wherein the network is formed by cascade connection of 3×3 convolution and 1×1 convolution; the method comprises two branches, wherein the generated key point heat map and partial vector field heat map are respectively corresponding, and then a 2-group key point enhancement extraction network is used for further generating a more accurate feature map.
5. The convolutional neural network-based power plant personnel perimeter intrusion detection method of claim 4, wherein in step S14, the training loss of the keypoint detection model is as follows:
wherein ,
representing location confidence map label, < >>
The field map label representing the region affinity vector, W (p), is a binary mask, W (p) =0 when the label is missing in p of the image.
6. The convolutional neural network-based power plant personnel perimeter intrusion detection method of claim 5, wherein in step S15, the weight w obtained in step S14 is loaded 1 And (3) starting a skeleton generation model to identify video frames in the actual climbing scene of the power plant extracted in the step (S11) to obtain a binary skeleton diagram of normal and climbing behaviors, classifying the two types of skeleton diagrams, marking class labels, and manufacturing a climbing classification data set according to the proportion of 8:1:1 of a training set, a verification set and a test set.
7. The method for detecting the perimeter intrusion of power plant personnel based on the convolutional neural network according to claim 6, wherein in the step S16, the two layers of fully connected neural networks are set as nodes of a first layer network (16384,100), the activation function is Relu, the nodes of a second layer network are set as nodes of a second layer network (100, 2), the classified activation function of the classified network is a softmax function, and the climbing behavior and the normal behavior are classified and optimized.
8. The convolutional neural network-based power plant personnel perimeter intrusion detection method of claim 1 or 7, wherein in step S17, the cross entropy loss function mathematical formula is:
where q represents the true value of a pixel, and p represents the predicted value of a pixel.
9. The method for detecting the perimeter intrusion of power plant personnel based on the convolutional neural network according to claim 1, wherein in step S21, the camera of the gun in the power plant is used for controlling a plurality of specific scenes, the behavior of the operators in the scenes is monitored in real time, the program carries out multithreaded acquisition on a plurality of cameras and carries out frame extraction processing on the acquired video data, and the video data are sent to an image queue to be predicted.
10. The method for detecting the perimeter invasion of the power plant personnel based on the convolutional neural network according to claim 1 or 9, wherein in step S23, the image queue to be predicted is sent into the algorithm model after fusion, the weight of the algorithm model is loaded, the climbing boundary crossing behavior is calculated according to the judgment logic set in step S22, and the prediction result is displayed in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211379670.XA CN116052035A (en) | 2022-11-04 | 2022-11-04 | Power plant personnel perimeter intrusion detection method based on convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211379670.XA CN116052035A (en) | 2022-11-04 | 2022-11-04 | Power plant personnel perimeter intrusion detection method based on convolutional neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116052035A true CN116052035A (en) | 2023-05-02 |
Family
ID=86132284
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211379670.XA Pending CN116052035A (en) | 2022-11-04 | 2022-11-04 | Power plant personnel perimeter intrusion detection method based on convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116052035A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116612603A (en) * | 2023-07-17 | 2023-08-18 | 微网优联科技(成都)有限公司 | Mobile object monitoring system and method |
-
2022
- 2022-11-04 CN CN202211379670.XA patent/CN116052035A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116612603A (en) * | 2023-07-17 | 2023-08-18 | 微网优联科技(成都)有限公司 | Mobile object monitoring system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101751744B (en) | Detection and early warning method of smoke | |
| CN112287816B (en) | Dangerous work area accident automatic detection and alarm method based on deep learning | |
| CN105809679B (en) | Mountain railway side slope rockfall detection method based on visual analysis | |
| KR102122859B1 (en) | Method for tracking multi target in traffic image-monitoring-system | |
| CN113516076B (en) | Attention mechanism improvement-based lightweight YOLO v4 safety protection detection method | |
| CN100417223C (en) | Intelligent safety protector based on omnibearing vision sensor | |
| CN110321780B (en) | Abnormal falling behavior detection method based on space-time motion characteristics | |
| CN107665326A (en) | Monitoring system, passenger transporter and its monitoring method of passenger transporter | |
| CN105678803A (en) | Video monitoring target detection method based on W4 algorithm and frame difference | |
| CN107437318B (en) | Visible light intelligent recognition algorithm | |
| CN106241533A (en) | Elevator occupant's comprehensive safety intelligent control method based on machine vision | |
| CN101635835A (en) | Intelligent video monitoring method and system thereof | |
| CN111144291A (en) | Method and device for distinguishing personnel invasion in video monitoring area based on target detection | |
| CN106815958A (en) | Warning system, alarm analysis/display device, alarm analysis/display methods | |
| CN113903081A (en) | Visual identification artificial intelligence alarm method and device for images of hydraulic power plant | |
| CN105426820A (en) | Multi-person abnormal behavior detection method based on security monitoring video data | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| CN110867046A (en) | Intelligent car washer video monitoring and early warning system based on cloud computing | |
| CN112183472A (en) | Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet | |
| CN111783700B (en) | Automatic recognition and early warning method and system for pavement foreign matters | |
| CN113191273A (en) | Oil field well site video target detection and identification method and system based on neural network | |
| CN116052035A (en) | Power plant personnel perimeter intrusion detection method based on convolutional neural network | |
| CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
| Foresti et al. | Event classification for automatic visual-based surveillance of parking lots | |
| CN105095891A (en) | Human face capturing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |