CN112434670A - Equipment and method for detecting abnormal behavior of power operation - Google Patents
Equipment and method for detecting abnormal behavior of power operation Download PDFInfo
- Publication number
- CN112434670A CN112434670A CN202011470991.1A CN202011470991A CN112434670A CN 112434670 A CN112434670 A CN 112434670A CN 202011470991 A CN202011470991 A CN 202011470991A CN 112434670 A CN112434670 A CN 112434670A
- Authority
- CN
- China
- Prior art keywords
- information
- network model
- ultra
- image
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 206010000117 Abnormal behaviour Diseases 0.000 title claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 121
- 239000010410 layer Substances 0.000 claims description 90
- 230000010365 information processing Effects 0.000 claims description 35
- 238000012549 training Methods 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 29
- 230000002159 abnormal effect Effects 0.000 claims description 27
- 238000000605 extraction Methods 0.000 claims description 19
- 230000004927 fusion Effects 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 12
- 239000002356 single layer Substances 0.000 claims description 12
- 238000012544 monitoring process Methods 0.000 claims description 8
- 230000005856 abnormality Effects 0.000 claims description 7
- 238000002372 labelling Methods 0.000 claims description 6
- 238000003860 storage Methods 0.000 claims description 3
- 238000004148 unit process Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 abstract description 5
- 238000011897 real-time detection Methods 0.000 abstract description 3
- 238000007792 addition Methods 0.000 description 9
- 240000007651 Rubus glaucus Species 0.000 description 4
- 235000011034 Rubus glaucus Nutrition 0.000 description 4
- 235000009122 Rubus idaeus Nutrition 0.000 description 4
- 208000027418 Wounds and injury Diseases 0.000 description 3
- 230000006378 damage Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 208000014674 injury Diseases 0.000 description 3
- 230000001681 protective effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention discloses equipment and a method for detecting abnormal behaviors of power operation. The invention can simultaneously identify the targets of workers, safety helmets, work clothes, gloves, grounding piles, work ropes and the like, can judge whether the workers enter a dangerous area, can intelligently identify in real time and improve the precision of various detections, and has better robustness and accuracy and more functions compared with the existing real-time detection scheme.
Description
Technical Field
The invention belongs to the technical field of abnormal behavior identification, and relates to equipment and a method for detecting abnormal behaviors in power operation.
Background
In industrial production sites, living communities, occasions with sensitive safety requirements and the like, video monitoring has important application value and practical significance for maintaining national and public safety. The detection and identification of abnormal behaviors of people are a main way for improving the intelligent degree of video monitoring, and the method integrates multiple technologies such as computer vision, pattern recognition, psychology, physiology and the like. At modern construction sites or power maintenance and other sites, workers face the danger of injury due to illegal operation, particularly casualties are easily caused due to reasons such as high-altitude falling in the construction or maintenance process, and if protective equipment can be worn correctly, the injury to personnel can be effectively reduced by standardizing operation behaviors. Therefore, provide an unusual action check out test set, detect workman's dress and action, when not correctly wearing protective apparatus, and carry out audio alert and in time remind when carrying out the violation operation, can effectively reduce the risk that workman received bodily injury.
Today, as computer vision technology develops rapidly, more and more scenes can be identified by a computer, and therefore, more and more occasions begin to use security detection technology based on video analysis. Abnormal behavior detection based on video analytics has also been investigated. The human behavior feature extraction method and system and the abnormal behavior detection method and system mentioned in the chinese patent CN102902972A have the disadvantages of various steps, complex processing, low precision, slow processing speed, and poor real-time performance, and are not suitable for being applied to the occasions with high real-time requirements; although the safety helmet detection method mentioned in chinese patent CN106295551A can realize the detection of the safety helmet under certain conditions, the comparison between the number of pixels satisfying the conditions in the statistics mentioned in the fourth step and the set threshold of the number of pixels shows that the robustness of the method is poor, and the safety helmet detection can only be realized simply.
Disclosure of Invention
According to the invention, on the basis of the existing method, the detection type diversification is realized, the accuracy and robustness of target detection are improved, whether workers wear protective equipment correctly or enter dangerous areas can be detected accurately in different scenes, and the equipment and the method for detecting the abnormal behaviors have the advantages of high accuracy, good robustness and real-time detection.
An electric power operation abnormal behavior detection device comprises an information collection unit, an information processing unit, an information display unit and a hardware control unit;
the information collecting unit is used for collecting image information and sensor information and transmitting the image information and the sensor information to the information processing unit;
the information processing unit processes the image information by using the target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, the information processing unit reminds the user; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
the information display unit is used for displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result and providing a field image information storage function;
the hardware control unit carries out traditional image processing according to the detection result information provided by the information processing unit, controls the camera to move along with the staff, and ensures that the staff is always within the monitoring range.
Further, the information collection unit comprises two USB cameras, an ultra-wideband chip, a safety helmet, a ZigBee and a power supply; one USB camera 1 is used for obtaining working images of workers, the other USB camera 2 is used for obtaining images of the grounding pile, and the ultra-wideband chip is used for collecting sensor information; the ultra-wideband chip and the power supply are embedded into the safety helmet, and the other ultra-wideband chip is arranged in a dangerous area and is communicated with the information processing unit by using ZigBee.
Further, the image information is processed by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding is carried out, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
Further, the RGB image acquired by the USB camera 2 is converted into an HSV image, a trained target detection network model is used for detecting whether the HSV image contains a grounding pile, and if not, the HSV image is abnormal.
Further, the ultra-wideband information is judged by using a threshold judgment method, namely when the distance between two ultra-wideband chips is smaller than a certain threshold, the threshold is the radius of a dangerous area, and the ultra-wideband information is judged to enter the dangerous area for voice reminding.
The invention also provides a method for detecting abnormal behaviors of power operation, which comprises the following steps:
step 1, collecting image information and sensor information;
step 2, processing the image information by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
step 3, displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result, and storing in real time;
and 4, carrying out traditional image processing on the detection result information of the target detection network model, controlling the camera to move along with the staff, and ensuring that the staff is always within the monitoring range.
Further, the specific implementation manner of processing the image information by using the target detection network model is as follows;
processing the image information by using a target detection network model in the step 2, and if the image information is abnormal, namely the detected image does not contain a set target, reminding, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
Further, the device for collecting the image information and the sensor information in the step 1 comprises two USB cameras, an ultra-wideband chip, a safety helmet, a ZigBee and a power supply; one USB camera 1 is used for obtaining working images of workers, the other USB camera 2 is used for obtaining images of the grounding pile, and the ultra-wideband chip is used for collecting sensor information; the ultra-wideband chip and the power supply are embedded into the safety helmet, and the other ultra-wideband chip is arranged in the dangerous area and is communicated with the information processing unit by using ZigBee;
and judging the ultra-wideband information by using a threshold judgment method, namely judging that the ultra-wideband information enters a dangerous area by judging that the threshold is the radius of the dangerous area when the distance between two ultra-wideband chips is smaller than a certain threshold, and carrying out voice reminding.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following advantages:
the invention can simultaneously identify the targets of workers, safety helmets, work clothes, gloves, grounding piles, work ropes and the like, can judge whether the workers enter a dangerous area, can intelligently identify in real time and improve the precision of various detections, and has better robustness and accuracy and more functions compared with the existing real-time detection scheme.
Drawings
Fig. 1 is a schematic diagram of an abnormal behavior detection apparatus according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further explained with reference to the drawings and the embodiments.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention is further illustrated below with reference to examples and schematic drawings:
fig. 1 is a schematic diagram of an embodiment of an abnormal behavior detection apparatus for power operation according to the present invention; the abnormal behavior detection device provided by the embodiment comprises an information collection unit, an information processing unit, an information display unit and a hardware control unit;
the information collecting unit is used for collecting image information and sensor information and transmitting the image information and the sensor information to the information processing unit;
the information processing unit processes the image information by using the target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, the information processing unit reminds the user; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
the information display unit is used for displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result and providing a field image information storage function;
the hardware control unit carries out traditional image processing according to the detection result information provided by the information processing unit, controls the camera to move along with the staff, and ensures that the staff is always within the monitoring range.
The information collection unit comprises 2 USB cameras, an ultra wide band chip, a ZigBee, a power supply 1 and a safety helmet. The USB camera 1 is used for obtaining working images of workers, the USB camera 2 is used for obtaining images of the grounding pile, image information collected by the USB camera is transmitted to the information processing unit through the USB port and is powered by the power supply 2 of the information processing unit, sensing information collected by the ultra-wideband is transmitted to the information processing unit through the ZigBee and is powered by the power supply 1 of the information collecting unit. The power supply 1 is embedded in the safety helmet, and the other ultra-wideband chip is arranged in a dangerous area and is communicated with the information processing unit by using ZigBee.
The information processing unit comprises a power supply 2, a CPU, a GPU, a memory bank, a magnetic disk, a mainboard and the like, the hardware forms a server, an algorithm operation platform is provided, the information transmitted by the information collection unit is analyzed by using an algorithm, a voice alarm is carried out when an abnormality is found, and the processed result information is transmitted to the information display unit and the hardware control unit.
The information display unit comprises a display screen, and the detection result of the information processing unit is displayed in the display screen in the form of an image.
The hardware control unit comprises a raspberry group, a mechanical arm and a power supply 3. Wherein the arm is placed in the workspace, is sent control arm by the raspberry and rotates, and USB camera 1 is smuggled secretly to the arm, puts USB camera 1 directly over the arm, and the target removal is caught to the camera, and the arm is sent control arm to the raspberry and rotates from top to bottom about, then takes USB camera 1 to rotate together, realizes staff's tracking function, and power 3 is that raspberry group and arm power supply.
In the information processing unit, the specific implementation manner of processing the image information by using the target detection network model is as follows;
processing image information by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output. 10000 pieces of electric power operation field data are adopted for training, the iteration times are 10000 times, the initial learning rate is 0.001, and the batch size is 256.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
Because the HSV model is sensitive to colors, work clothes, safety helmets and grounding piles which are detected by people are all objects with obvious color characteristics, so that the HSV model can be used for detection; the color characteristics of detection staff (people), working ropes and the like are not obvious, so that RGB images can be used. In specific implementation, the RGB image acquired by the USB camera 2 may also be converted into an HSV image, and a trained target detection network model is used to detect whether the HSV image contains a ground stud, and if not, it is abnormal.
In the information processing unit, the ultra-wideband information is judged by using a threshold judgment method, namely when the distance between two ultra-wideband chips is smaller than a certain threshold, the threshold is the radius of a dangerous area, and the ultra-wideband information is judged to enter the dangerous area for voice reminding.
The hardware control unit calculates the specific position of a central point of a worker in an image and the size of an image frame after receiving worker position information sent by the information processing unit.
The embodiment of the invention also provides a method for detecting abnormal behaviors of power operation, which comprises the following steps:
step 1, collecting image information and sensor information;
step 2, processing the image information by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
step 3, displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result, and storing in real time;
and 4, carrying out traditional image processing on the detection result information of the target detection network model, controlling the camera to move along with the staff, and ensuring that the staff is always within the monitoring range.
In the step 2, the image information is processed by using the target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding is carried out, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
The device for collecting the image information and the sensor information in the step 1 comprises two USB cameras, an ultra-wideband chip, a safety helmet, a ZigBee and a power supply; one USB camera 1 is used for obtaining working images of workers, the other USB camera 2 is used for obtaining images of the grounding pile, and the ultra-wideband chip is used for collecting sensor information; the ultra-wideband chip and the power supply are embedded into the safety helmet, and the other ultra-wideband chip is arranged in the dangerous area and is communicated with the information processing unit by using ZigBee;
and judging the ultra-wideband information by using a threshold judgment method, namely judging that the ultra-wideband information enters a dangerous area by judging that the threshold is the radius of the dangerous area when the distance between two ultra-wideband chips is smaller than a certain threshold, and carrying out voice reminding.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (8)
1. The equipment for detecting the abnormal behavior of the power operation is characterized by comprising an information collection unit, an information processing unit, an information display unit and a hardware control unit;
the information collecting unit is used for collecting image information and sensor information and transmitting the image information and the sensor information to the information processing unit;
the information processing unit processes the image information by using the target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, the information processing unit reminds the user; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
the information display unit is used for displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result and providing a field image information storage function;
the hardware control unit carries out traditional image processing according to the detection result information provided by the information processing unit, controls the camera to move along with the staff, and ensures that the staff is always within the monitoring range.
2. An electric power operation abnormal behavior detection apparatus according to claim 1, characterized in that: the information collection unit comprises two USB cameras, an ultra-wideband chip, a safety helmet, ZigBee and a power supply; one USB camera 1 is used for obtaining working images of workers, the other USB camera 2 is used for obtaining images of the grounding pile, and the ultra-wideband chip is used for collecting sensor information; the ultra-wideband chip and the power supply are embedded into the safety helmet, and the other ultra-wideband chip is arranged in a dangerous area and is communicated with the information processing unit by using ZigBee.
3. An electric power operation abnormal behavior detection apparatus according to claim 2, characterized in that: processing image information by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
4. An electric power operation abnormal behavior detection apparatus according to claim 3, characterized in that: and converting the RGB image acquired by the USB camera 2 into an HSV image, detecting whether the HSV image contains a grounding pile or not by using a trained target detection network model, and if not, determining that the HSV image is abnormal.
5. An abnormal behavior detection apparatus according to claim 2, wherein: and judging the ultra-wideband information by using a threshold judgment method, namely judging that the ultra-wideband information enters a dangerous area by judging that the threshold is the radius of the dangerous area when the distance between two ultra-wideband chips is smaller than a certain threshold, and carrying out voice reminding.
6. A power operation abnormal behavior detection method is characterized by comprising the following steps:
step 1, collecting image information and sensor information;
step 2, processing the image information by using a target detection network model, and if the image information is abnormal, namely the detected image does not contain a set target, reminding; processing the sensor information by using a threshold judgment method, detecting that a worker enters a dangerous area if abnormality is found, performing voice reminding, and providing specific processing result information for an information display unit and a hardware control unit;
step 3, displaying the collected field image information, the detection result of the target detection network model and the sensor information processing result, and storing in real time;
and 4, carrying out traditional image processing on the detection result information of the target detection network model, controlling the camera to move along with the staff, and ensuring that the staff is always within the monitoring range.
7. The method according to claim 6, wherein the method comprises the following steps: processing the image information by using a target detection network model in the step 2, and if the image information is abnormal, namely the detected image does not contain a set target, reminding, wherein the specific implementation mode is as follows;
firstly, constructing a target detection network model, which comprises a feature extraction module, a multi-scale fusion module and a feature detection module;
the feature extraction module is used for extracting a feature map F1 and comprises a plurality of sub-modules with the same structure, wherein each sub-module comprises 1 convolution layer and 1 maximum pooling layer;
the input of the multi-scale fusion module is a feature graph F1 extracted by the feature extraction module, which comprises a plurality of cavity convolutional layer branches, 1 connecting layer and a convolutional layer, the output feature graphs of the plurality of convolutional layer branches are input into the connecting layer, and finally multi-scale feature fusion is carried out through a traditional convolutional layer;
the characteristic detection module comprises N layers of convolution layers which are sequentially connected, wherein the back surfaces of convolution layers of other single layers except the 1 st layer and the last layer of single layers in the front N layers are respectively connected with pooling operation, then the pooled characteristic and the feature after convolution of the last layer are subjected to characteristic addition, the pooled characteristic and the full-connection operation are carried out on the characteristic after the characteristic addition, then a network is generated through an area, and a label value and a prediction detection frame are output;
then, a training set is constructed, and staff, work clothes, safety helmets, gloves, work ropes and grounding piles are marked on the images in the training set through a labeling tool;
and in the training stage, the images in the training set are sent to a target detection network model for training, and the trained target detection network model is output.
In the detection stage, images acquired by the USB camera 1 and the USB camera 2 are respectively input into a trained target detection network model, whether the images captured by the USB camera 1 comprise workers, work clothes, safety helmets, gloves and work ropes or not is output, and if any target is not included in the detection result, the detection result is abnormal; whether the output USB camera 2 includes a ground stud or not is abnormal if not.
8. The method according to claim 6, wherein the method comprises the following steps: the device for collecting the image information and the sensor information in the step 1 comprises two USB cameras, an ultra-wideband chip, a safety helmet, a ZigBee and a power supply; one USB camera 1 is used for obtaining working images of workers, the other USB camera 2 is used for obtaining images of the grounding pile, and the ultra-wideband chip is used for collecting sensor information; the ultra-wideband chip and the power supply are embedded into the safety helmet, and the other ultra-wideband chip is arranged in the dangerous area and is communicated with the information processing unit by using ZigBee;
and judging the ultra-wideband information by using a threshold judgment method, namely judging that the ultra-wideband information enters a dangerous area by judging that the threshold is the radius of the dangerous area when the distance between two ultra-wideband chips is smaller than a certain threshold, and carrying out voice reminding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011470991.1A CN112434670A (en) | 2020-12-14 | 2020-12-14 | Equipment and method for detecting abnormal behavior of power operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011470991.1A CN112434670A (en) | 2020-12-14 | 2020-12-14 | Equipment and method for detecting abnormal behavior of power operation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112434670A true CN112434670A (en) | 2021-03-02 |
Family
ID=74691593
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011470991.1A Pending CN112434670A (en) | 2020-12-14 | 2020-12-14 | Equipment and method for detecting abnormal behavior of power operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112434670A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104318732A (en) * | 2014-10-27 | 2015-01-28 | 国网冀北电力有限公司张家口供电公司 | Transformer substation field worker monitoring and management system and method based on video analysis and RFID |
| CN111126202A (en) * | 2019-12-12 | 2020-05-08 | 天津大学 | Optical remote sensing image target detection method based on void feature pyramid network |
| CN111401418A (en) * | 2020-03-05 | 2020-07-10 | 浙江理工大学桐乡研究院有限公司 | Employee dressing specification detection method based on improved Faster r-cnn |
| CN112001310A (en) * | 2020-08-24 | 2020-11-27 | 国网上海市电力公司 | Transformer substation operation field safety control system based on visual perception and space positioning |
-
2020
- 2020-12-14 CN CN202011470991.1A patent/CN112434670A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104318732A (en) * | 2014-10-27 | 2015-01-28 | 国网冀北电力有限公司张家口供电公司 | Transformer substation field worker monitoring and management system and method based on video analysis and RFID |
| CN111126202A (en) * | 2019-12-12 | 2020-05-08 | 天津大学 | Optical remote sensing image target detection method based on void feature pyramid network |
| CN111401418A (en) * | 2020-03-05 | 2020-07-10 | 浙江理工大学桐乡研究院有限公司 | Employee dressing specification detection method based on improved Faster r-cnn |
| CN112001310A (en) * | 2020-08-24 | 2020-11-27 | 国网上海市电力公司 | Transformer substation operation field safety control system based on visual perception and space positioning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10810414B2 (en) | Movement monitoring system | |
| CN109670441B (en) | Method, system, terminal and computer readable storage medium for realizing wearing recognition of safety helmet | |
| CN111898514B (en) | Multi-target visual supervision method based on target detection and action recognition | |
| CN107808139B (en) | Real-time monitoring threat analysis method and system based on deep learning | |
| CN113516076B (en) | Attention mechanism improvement-based lightweight YOLO v4 safety protection detection method | |
| CN111144263A (en) | Construction worker high-fall accident early warning method and device | |
| CN112396658A (en) | Indoor personnel positioning method and positioning system based on video | |
| CN112766050B (en) | Dressing and operation checking method, computer device and storage medium | |
| CN106778609A (en) | A kind of electric power construction field personnel uniform wears recognition methods | |
| CN103942850A (en) | Medical staff on-duty monitoring method based on video analysis and RFID (radio frequency identification) technology | |
| CN112434669B (en) | Human body behavior detection method and system based on multi-information fusion | |
| CN107948585A (en) | Video recording labeling method, device and computer-readable recording medium | |
| CN111325133B (en) | Image processing system based on artificial intelligent recognition | |
| CN111062303A (en) | Image processing method, system and computer storage medium | |
| CN112270807A (en) | Old man early warning system that tumbles | |
| CN111898580A (en) | System, method and equipment for acquiring body temperature and respiration data of people wearing masks | |
| CN112188164A (en) | AI vision-based violation real-time monitoring system and method | |
| CN111401310B (en) | Kitchen sanitation safety supervision and management method based on artificial intelligence | |
| CN113505770B (en) | Method and system for detecting clothes and hair ornament abnormity in express industry and electronic equipment | |
| CN114092875A (en) | Operation site safety supervision method and device based on machine learning | |
| CN109873990A (en) | A kind of illegal mining method for early warning in mine based on computer vision | |
| CN113314230A (en) | Intelligent epidemic prevention method, device, equipment and storage medium based on big data | |
| CN113313186A (en) | Method and system for identifying non-standard wearing work clothes | |
| CN111597919A (en) | Human body tracking method in video monitoring scene | |
| CN112434670A (en) | Equipment and method for detecting abnormal behavior of power operation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210302 |