CN114067268A - Method and device for detecting safety helmet and identifying identity of electric power operation site - Google Patents
Method and device for detecting safety helmet and identifying identity of electric power operation site Download PDFInfo
- Publication number
- CN114067268A CN114067268A CN202111364569.2A CN202111364569A CN114067268A CN 114067268 A CN114067268 A CN 114067268A CN 202111364569 A CN202111364569 A CN 202111364569A CN 114067268 A CN114067268 A CN 114067268A
- Authority
- CN
- China
- Prior art keywords
- safety helmet
- target
- network
- feature
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for detecting and identifying an identity of a safety helmet in an electric power operation field, which comprises the following steps: the method comprises the following steps of firstly, acquiring an image to be identified of an electric power operation site; inputting the image to be identified into a target detection network, and acquiring a safety helmet wearing state detection result output by the target detection network; thirdly, determining the personnel identity of a target area in the image to be recognized according to the detection result of the wearing state of the safety helmet; according to the invention, the identity of the worker can be judged and identified by identifying the type of the safety helmet of the worker on the electric power operation site, the management of a manager is assisted, the major economic loss of external personnel on the electric power operation site is prevented, and the life safety of the personnel is ensured.
Description
Technical Field
The invention relates to the technical field of target detection, in particular to a method and a device for detecting a safety helmet and identifying an identity of an electric power operation site.
Background
In order to supervise workers to wear safety helmets as required, most factories and construction sites adopt video monitoring and manual supervision methods, but the method needs a large amount of manpower, and part of corners have monitoring blind areas, so that a more reliable method is needed.
Most of traditional construction site supervision systems are realized by special engineering management systems, but the construction site supervision systems consume a large amount of manpower, are difficult to implement widely according to safety specifications due to personal negligence and other factors difficult to avoid, and have the obvious defects of low reliability, long reaction time delay, high cost, inconvenient management and the like.
Disclosure of Invention
The invention provides a method for detecting and identifying the identity of a safety helmet of an electric power operation site, which can judge and identify the identity of a worker by identifying the wearing state of the safety helmet of the worker on the electric power operation site, assist the management of a manager, prevent external personnel from causing great economic loss on the electric power operation site, and ensure the life safety of the personnel.
The invention adopts the following technical scheme.
A method for detecting and identifying an identity of a safety helmet on an electric power operation site comprises the following steps:
the method comprises the following steps of firstly, acquiring an image to be identified of an electric power operation site;
inputting the image to be identified into a target detection network, and acquiring a safety helmet wearing state detection result output by the target detection network;
and thirdly, determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
Training the target detection network by using an input sample data set, wherein the sample data set comprises helmet wearing sample images, and the generation of the sample data set comprises the following steps;
step C, collecting safety helmet wearing sample images of the power operation site, carrying out image scale normalization and image standardization on the collected safety helmet wearing sample images, and selecting specific safety helmet wearing sample images according to a preset selection standard;
step C, manually labeling areas which can be used for safety helmet detection in the specific safety helmet wearing sample image to form a sample data set;
in the step C, a method for collecting the sample data set comprises two parts, wherein one part is obtained by acquiring a video containing a safety helmet worn by a worker through a real-time monitoring system of the power operation site and extracting frames of the video at a preset frame rate to obtain a safety helmet wearing sample image; the other part is to obtain a sample image by network retrieval and downloading, wherein the sample image comprises a person wearing the safety helmet and a person not wearing the safety helmet;
in the step C I, image scale normalization and image standardization are carried out on the collected helmet wearing sample images, and specific helmet wearing sample images are selected according to a preset selection standard to form a sample data set;
the selection criteria include the following:
selecting a safety helmet wearing sample image with a safety helmet appearing under different illumination conditions;
b, selecting a safety helmet wearing sample image under the condition that different shooting angles cause different sizes of pictures occupied by the target;
selecting a safety helmet wearing sample image with medium crowd density and without mutual shielding among people;
and D, selecting a high crowd density and wearing a sample image of the safety helmet when the people are shielded.
In the step C, manual labeling is to label the image of the helmet wearing sample by using LabelImg software, and the labeling work comprises two aspects:
the working content A is that a marking frame is drawn for each area which can be used for safety helmet detection, so that a target detection network can conveniently obtain the position information of a detection target in an image;
and the working content B provides the class name of the wearing state of the safety helmet corresponding to each labeling frame, so that the target detection network can conveniently match the learned characteristics in the labeling frame with the class.
The method for training the target detection network specifically comprises the following steps:
a1, collecting a sample data set, unifying the long edge sizes of all pictures into a fixed size, and inputting the fixed size into a VGG16 convolutional neural network to extract features so as to form a feature graph;
step A2, fusing feature maps of 4 network stages after the VGG16 convolutional neural network, sequentially inputting the feature maps obtained in the step A1 into an RPN to complete feature fusion and multi-scale detection, and introducing an online difficult sample capable of enhancing the identification capability of the RPN; removing partial redundant frames by using an improved non-maximum suppression algorithm, and deleting surrounding frames of partial positive samples; screening out a suspected region of interest ROI;
step A3, inputting the suspected ROI into a read-only ROI network, introducing online difficult sample mining, and adjusting the ROI; and finally, sending the ROI into a standard ROI network, and outputting a final result, namely a target detection network model.
The VGG16 convolutional neural network is sequentially divided into 5 network stages, and the last convolutional layer of each network stage is respectively C1, C2, C3, C4 and C5;
when the C2 is fused with the feature map of C4, firstly, the C4 changes the dimensionality through a convolution kernel of 1 multiplied by 256 to enable the number of channels to be 256, and then deconvolution including quadruple bilinear interpolation upsampling operation is carried out to enable the feature maps of C4 and C2 to be the same in size; modifying the number of channels of the 2 characteristic diagram into 256 by utilizing a convolution kernel of 1 multiplied by 256, and fusing with C4, wherein the fusion operation uses an additive fusion function; in order to remove aliasing effect brought by upsampling, the fused feature map is subjected to convolution operation with convolution kernel of 3 × 3 once to generate a new feature layer P2 with 256 channels;
generating a feature layer P3 when the feature maps of C5 and C3 are fused, wherein the fusion principle adopts the feature map fusion principle of C2 and C4;
c4 and C5 respectively perform convolution operation with convolution kernel of 1 × 1 × 256 for changing the number of channels and convolution operation with convolution kernel of 3 × 3 for one time, and generate feature layers P4 and P5;
obtaining a feature pyramid formed by feature layers P5, P4, P3 and P2 through feature map fusion; the output dimension of each layer of the characteristic pyramid is fixed to be 256, the shallow layer characteristics of the characteristic pyramid are fused with deep layer characteristics to detect small targets, the strong semantic information of the high layer characteristics can detect medium and large targets, each layer of new characteristics is connected with the RPN, and interested areas are respectively predicted to realize multi-scale detection.
In the region of interest, the sizes of anchor windows corresponding to RPN/P2 and RPN/P3 with smaller receptive fields are {24 × 24, 32 × 32 and 64 × 64}, the sizes of anchor windows corresponding to RPN/P4 and RPN/P5 with larger receptive fields are {64 × 64, 128 × 128 and 256 × 256}, and the aspect ratios are {1:1, 2:1, 1:2 };
inputting the interesting regions of the 4 parts of the feature layer into an ROI pooling layer, mapping back the interesting regions to the corresponding pyramid feature layers respectively, obtaining a series of feature blocks of the interesting regions, enabling the unified size to be 7 x 7, finally longitudinally splicing the feature graphs of the 4 levels together by utilizing a cascading fusion function, sharing a classification layer and a regression layer, and expressing the classification layer and the regression layer as formulas
Wherein x isaAnd xbThe feature matrixes of different layers in the feature pyramid are obtained, and y is the feature matrix after fusion. x is the number ofa、xbAnd y ∈ RH×W×DH, W and D are the length, width and number of channels, respectively, of the feature layer. i is an element of [1, H ]]，j∈[1，W]，d∈[1，D]。
The online difficult sample mining OHEM includes: introducing OHEM for a detection task of a Faster R-CNN algorithm, listing samples with loss meeting preset requirements in a fastR-CNN stage, reforming a training sample set, sending the training sample set to a classification network and a target frame regression model, calculating the loss of a network model, and performing reverse propagation to update the network; the process comprises the following steps:
step B1, after the input image is subjected to size normalization, obtaining a feature pyramid comprising feature layers P5, P4, P3 and P2 through a VGG16 network after feature fusion, then respectively sending the feature pyramid into an RPN network, and obtaining an interested area possibly containing a target through multi-scale prediction;
step B2, inputting all the regions of interest into a read-only ROI network for forward propagation without sampling in small batches; the read-only ROI network comprises an ROI pooling layer, a full connection layer, a Softmax classification layer and a target frame regression layer; inputting the region of interest into an ROI pooling layer, mapping the region of interest to a corresponding pyramid feature layer by the ROI pooling layer, unifying the size of the pyramid feature layer to be 7 x 7, and longitudinally splicing 4 parts of feature graphs with the same size into one part by utilizing a cascade fusion function Concat layer;
step B3, inputting the characteristic diagram of the interesting region into a subsequent layer for calculation to obtain loss values of all the interesting regions, wherein the loss values represent the adaptation degree of the network structure weight, all the values are sorted, the first K samples are reserved as difficult samples, and other loss values are set to be 0;
and step B4, finding out a corresponding difficult sample according to the index value of the region of interest corresponding to the loss value, inputting the difficult sample into a standard ROI network, outputting a predicted classification result and the coordinate of the bounding box, comparing the predicted classification result with a calibrated result to calculate an error, reversely propagating and updating the whole network parameter, correcting the whole target detection network, and enabling the network detection task to be more complete.
The improved non-maxima suppression algorithm includes:
step S1, sorting all the target frames according to scores, and considering that the target frame M with the highest current score contains targets;
step S2, calculating the intersection ratio of the residual target frame and M, and when the intersection ratio of the target frame and M is less than NiConsider that the current target box and M representKeeping the original scores of two different targets;
step S3, when the intersection ratio of the target frame and M is between NiAnd NtIn between, it is considered that the current target frame and M may contain two different targets, or may represent the same target as M, so a weight penalty policy is adopted
It and IoU (M, b)i) In inverse proportion, the score is reduced, a target frame is reserved, and the next round of screening is participated;
step S4, when the intersection ratio of the target frame and M is larger than NtIf so, determining that the current target frame and the M represent a target, and discarding the current target frame;
and moving the current M into the result target frame set, forming a new set by the residual target frames, repeating the steps from S1 to S4 until the target frame set is empty, finishing screening and outputting a result.
The expression is as follows:
wherein N istIs the original threshold value, NiThe newly added threshold value is obtained.
The method of the first step is that a computer host acquires a power operation site video and performs image preprocessing, wherein the image preprocessing comprises the following steps: extracting frames of a video at a certain frame rate, and then carrying out image scale normalization and image standardization on each frame to obtain the input of a target detection network;
in the second step, the trained target detection network is an improved Faster R-CNN network model obtained by training an improved Faster R-CNN network.
And C, labeling the sample data set in the second step comprises the following category names of the wearing states of the safety helmet: personnel wearing red safety helmets, yellow safety helmet personnel, white safety helmet personnel, blue safety helmet personnel and personnel not wearing safety helmets;
the annotations in the sample dataset are saved as an XML file in the PASCAL VOC format, which can be read by Python software and used for training.
The determining the personnel identity of the target area in the image to be identified according to the detection result of the wearing state of the safety helmet comprises the following steps:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
the first feature here is a helmet wearing state.
And under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
The second feature here is the helmet category.
The method further comprises the following steps: and controlling an alarm device in a target area of a background management mechanism or an electric power operation site to alarm under the condition that the personnel do not wear the safety helmet.
A safety helmet detection and identity recognition device for an electric power operation site is used for a safety helmet detection and identity recognition method, and the device comprises:
the image to be identified acquisition module is used for acquiring an image to be identified of the electric power operation site;
the safety helmet detection module is used for inputting the image to be identified into a target detection network and acquiring a safety helmet wearing state detection result output by the target detection network;
and the identity recognition module is used for determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
The identity recognition module is used for:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
and under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
The electric power operation site safety helmet detection and identity recognition device further comprises an alarm control module, the alarm control module is connected with a background management mechanism or an alarm device in a target area of the electric power operation site, and when the identity recognition module detects that a person who does not wear the safety helmet exists in the target area of the electric power operation site, the alarm device is controlled to give an alarm.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
(1) according to the method for detecting the safety helmet and identifying the identity of the electric power operation site, whether the wearing condition of the safety helmet of a worker meets the requirements of the operation site or not can be identified through image information collected by monitoring equipment in the electric power operation site in the system operation process, and voice prompt is carried out to remind a manager of abnormal conditions;
(2) the method for detecting and identifying the safety helmet of the electric power operation site can intelligently identify the safety helmet of a worker in the electric power operation site in real time, assist management of a manager, improve the detection effect on a small target and an overlapped target, and achieve better algorithm robustness and higher speed.
Drawings
The invention is described in further detail below with reference to the following figures and detailed description:
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a schematic diagram of the architectural principles of the method of the present invention.
Detailed Description
In traditional site supervision, a supervisor can identify the identity of a person through the wearing state of a safety helmet, and the wearing state of the safety helmet can be defined according to requirements in the actual application process. In one possible implementation, the helmet wearing status indicates whether the person wears a helmet, that is, the helmet wearing status includes: wearing and not wearing. In another possible implementation, the wearing state of the safety helmet indicates whether a person wears the safety helmet and the type of the safety helmet when the person wears the safety helmet, that is, the wearing state of the safety helmet includes: unworn, wearing type a headgear, wearing type B headgear, and the like. Wherein, the type of the safety helmet can be embodied by the color but not limited.
Accordingly, in site supervision, the identity of the person can be defined as required in practical applications. For example: if the safety helmet wearing state indicates whether the person wears the safety helmet or not during supervision, the person identity can be determined as: a compliance person and a non-compliance person. If the wearing state of the safety helmet indicates whether the person wears the safety helmet and the type of the safety helmet when the person wears the safety helmet, the person identity can be determined as: managers, ordinary workers, and non-compliant personnel.
How to automatically identify the wearing state of the safety helmet by using an image identification technology plays a role in manual identification of a supervisor, is a research direction, and is also the main content of the embodiment.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other
As shown in fig. 1, in the present embodiment, there is the following process:
the method comprises the following steps of 1, firstly, collecting a sample data set of a safety helmet on the electric power operation site, and carrying out manual marking;
the process 2, improving a FasterR-CNN network, and training a safety helmet detection network model;
3, acquiring an image to be identified in the target area;
4, using a safety helmet detection network model as a target detection network;
step 5, judging the type of the safety helmet, and further determining the identity of the personnel;
the above process is basically implemented as follows:
a method for detecting and identifying an identity of a safety helmet on an electric power operation site comprises the following steps:
the method comprises the following steps of firstly, acquiring an image to be identified of an electric power operation site;
inputting the image to be identified into a target detection network, and acquiring a safety helmet wearing state detection result output by the target detection network;
and thirdly, determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
Training the target detection network by using an input sample data set, wherein the sample data set comprises helmet wearing sample images, and the generation of the sample data set comprises the following steps;
step C, collecting safety helmet wearing sample images of the power operation site, carrying out image scale normalization and image standardization on the collected safety helmet wearing sample images, and selecting specific safety helmet wearing sample images according to a preset selection standard;
step C, manually labeling areas which can be used for safety helmet detection in the specific safety helmet wearing sample image to form a sample data set;
in the step C, a method for collecting the sample data set comprises two parts, wherein one part is obtained by acquiring a video containing a safety helmet worn by a worker through a real-time monitoring system of the power operation site and extracting frames of the video at a preset frame rate to obtain a safety helmet wearing sample image; the other part is to obtain a sample image by network retrieval and downloading, wherein the sample image comprises a person wearing the safety helmet and a person not wearing the safety helmet;
in the step C I, image scale normalization and image standardization are carried out on the collected helmet wearing sample images, and specific helmet wearing sample images are selected according to a preset selection standard to form a sample data set;
the selection criteria include the following:
selecting a safety helmet wearing sample image with a safety helmet appearing under different illumination conditions;
b, selecting a safety helmet wearing sample image under the condition that different shooting angles cause different sizes of pictures occupied by the target;
selecting a safety helmet wearing sample image with medium crowd density and without mutual shielding among people;
and D, selecting a high crowd density and wearing a sample image of the safety helmet when the people are shielded.
In the step C, manual labeling is to label the image of the helmet wearing sample by using LabelImg software, and the labeling work comprises two aspects:
the working content A is that a marking frame is drawn for each area which can be used for safety helmet detection, so that a target detection network can conveniently obtain the position information of a detection target in an image;
and the working content B provides the class name of the wearing state of the safety helmet corresponding to each labeling frame, so that the target detection network can conveniently match the learned characteristics in the labeling frame with the class.
As shown in fig. 2, the method for training the target detection network specifically includes:
a1, collecting a sample data set, unifying the long edge sizes of all pictures into a fixed size, and inputting the fixed size into a VGG16 convolutional neural network to extract features so as to form a feature graph;
step A2, fusing feature maps of 4 network stages after the VGG16 convolutional neural network, sequentially inputting the feature maps obtained in the step A1 into an RPN to complete feature fusion and multi-scale detection, and introducing an online difficult sample capable of enhancing the identification capability of the RPN; removing partial redundant frames by using an improved non-maximum suppression algorithm, and deleting surrounding frames of partial positive samples; screening out a suspected region of interest ROI;
step A3, inputting the suspected ROI into a read-only ROI network, introducing online difficult sample mining, and adjusting the ROI; and finally, sending the ROI into a standard ROI network, and outputting a final result, namely a target detection network model.
The VGG16 convolutional neural network is sequentially divided into 5 network stages, and the last convolutional layer of each network stage is respectively C1, C2, C3, C4 and C5;
when the C2 is fused with the feature map of C4, firstly, the C4 changes the dimensionality through a convolution kernel of 1 multiplied by 256 to enable the number of channels to be 256, and then deconvolution including quadruple bilinear interpolation upsampling operation is carried out to enable the feature maps of C4 and C2 to be the same in size; modifying the number of channels of the 2 characteristic diagram into 256 by utilizing a convolution kernel of 1 multiplied by 256, and fusing with C4, wherein the fusion operation uses an additive fusion function; in order to remove aliasing effect brought by upsampling, the fused feature map is subjected to convolution operation with convolution kernel of 3 × 3 once to generate a new feature layer P2 with 256 channels;
generating a feature layer P3 when the feature maps of C5 and C3 are fused, wherein the fusion principle adopts the feature map fusion principle of C2 and C4;
c4 and C5 respectively perform convolution operation with convolution kernel of 1 × 1 × 256 for changing the number of channels and convolution operation with convolution kernel of 3 × 3 for one time, and generate feature layers P4 and P5;
obtaining a feature pyramid formed by feature layers P5, P4, P3 and P2 through feature map fusion; the output dimension of each layer of the characteristic pyramid is fixed to be 256, the shallow layer characteristics of the characteristic pyramid are fused with deep layer characteristics to detect small targets, the strong semantic information of the high layer characteristics can detect medium and large targets, each layer of new characteristics is connected with the RPN, and interested areas are respectively predicted to realize multi-scale detection.
In the region of interest, the sizes of anchor windows corresponding to RPN/P2 and RPN/P3 with smaller receptive fields are {24 × 24, 32 × 32, 64 × 64}, the sizes of anchor windows corresponding to RPN/P4 and RPN/P5 with larger receptive fields are {64 × 64, 128 × 128, 256 × 256}, and the aspect ratios are {1:1, 2:1, 1:2 };
inputting the interested regions of the 4 parts of the feature layer into the ROI pooling layer, mapping the interested regions back to the pyramid feature layers corresponding to the interested regions respectively to obtain a series of feature blocks of the interested regions, wherein the unified size is 7 multiplied by 7, finally, utilizing a cascade fusion function to longitudinally splice the feature graphs of the 4 levels together, sharing a classification layer and a regression layer, and expressing the classification layer and the regression layer as formulas
Wherein x isaAnd xbThe feature matrixes of different layers in the feature pyramid are obtained, and y is the feature matrix after fusion. x is the number ofa、xbAnd y ∈ RH×W×DH, W and D are the length, width and number of channels, respectively, of the feature layer. i is an element of [1, H ]]，j∈[1，W]，d∈[1，D]。
The online difficult sample mining OHEM includes: introducing OHEM for a detection task of a Faster R-CNN algorithm, listing samples with loss meeting preset requirements in a fastR-CNN stage, reforming a training sample set, sending the training sample set to a classification network and a target frame regression model, calculating the loss of a network model, and performing reverse propagation to update the network; the process comprises the following steps:
step B1, after the input image is subjected to size normalization, obtaining a feature pyramid comprising feature layers P5, P4, P3 and P2 through a VGG16 network after feature fusion, then respectively sending the feature pyramid into an RPN network, and obtaining an interested area possibly containing a target through multi-scale prediction;
step B2, inputting all the regions of interest into a read-only ROI network for forward propagation without sampling in small batches; the read-only ROI network comprises an ROI pooling layer, a full connection layer, a Softmax classification layer and a target frame regression layer; inputting the region of interest into an ROI pooling layer, mapping the region of interest to a corresponding pyramid feature layer by the ROI pooling layer, unifying the size of the pyramid feature layer to be 7 x 7, and longitudinally splicing 4 parts of feature graphs with the same size into one part by utilizing a cascade fusion function Concat layer;
step B3, inputting the characteristic diagram of the interesting region into a subsequent layer for calculation to obtain loss values of all the interesting regions, wherein the loss values represent the adaptation degree of the network structure weight, all the values are sorted, the first K samples are reserved as difficult samples, and other loss values are set to be 0;
and step B4, finding out a corresponding difficult sample according to the index value of the region of interest corresponding to the loss value, inputting the difficult sample into a standard ROI network, outputting a predicted classification result and the coordinate of the bounding box, comparing the predicted classification result with a calibrated result to calculate an error, reversely propagating and updating the whole network parameter, correcting the whole target detection network, and enabling the network detection task to be more complete.
The improved non-maxima suppression algorithm includes:
step S1, sorting all the target frames according to scores, and considering that the target frame M with the highest current score contains targets;
step S2, calculating the intersection ratio of the residual target frame and M, and when the intersection ratio of the target frame and M is less than NiConsidering that the current target frame and M represent two different targets and keeping the original score;
step S3, when the intersection ratio of the target frame and M is between NiAnd NtIn between, it is considered that the current target frame and M may contain two different targets, or may represent the same target as M, so a weight penalty policy is adopted
It and IoU (M, b)i) In inverse proportion, the score is reduced, a target frame is reserved, and the next round of screening is participated;
step S4, when the intersection ratio of the target frame and M is larger than NtIf so, determining that the current target frame and the M represent a target, and discarding the current target frame;
and moving the current M into the result target frame set, forming a new set by the residual target frames, repeating the steps from S1 to S4 until the target frame set is empty, finishing screening and outputting a result.
The expression is as follows:
wherein N istIs the original threshold value, NiThe newly added threshold value is obtained.
The method of the first step is that a computer host acquires a power operation site video and performs image preprocessing, wherein the image preprocessing comprises the following steps: extracting frames of a video at a certain frame rate, and then carrying out image scale normalization and image standardization on each frame to obtain the input of a target detection network;
in the second step, the trained target detection network is an improved Faster R-CNN network model obtained by training an improved Faster R-CNN network.
The label in the sample data set in the step C I comprises the following category names of the wearing states of the safety helmet: personnel wearing red safety helmets, yellow safety helmet personnel, white safety helmet personnel, blue safety helmet personnel and personnel not wearing safety helmets;
the annotations in the sample dataset are saved as an XML file in the PASCAL VOC format, which can be read by Python software and used for training.
The determining the personnel identity of the target area in the image to be identified according to the detection result of the wearing state of the safety helmet comprises the following steps:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
the first feature here is a helmet wearing state.
And under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
The second feature here is the helmet category.
The method further comprises the following steps: and controlling an alarm device in a target area of a background management mechanism or an electric power operation site to alarm under the condition that the personnel do not wear the safety helmet.
A safety helmet detection and identity recognition device for an electric power operation site is used for a safety helmet detection and identity recognition method, and the device comprises:
the image to be recognized acquisition module is used for acquiring an image to be recognized of the target area;
the safety helmet detection module is used for inputting the image to be identified into a target detection network and acquiring a safety helmet wearing state detection result output by the target detection network;
and the identity recognition module is used for determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
The identity recognition module is used for:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
and under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
The electric power operation site safety helmet detection and identity recognition device further comprises an alarm control module, the alarm control module is connected with a background management mechanism or an alarm device in a target area of the electric power operation site, and when the identity recognition module detects that a person who does not wear the safety helmet exists in the target area of the electric power operation site, the alarm device is controlled to give an alarm.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (15)
1. A method for detecting and identifying an identity of a safety helmet on an electric power operation site is characterized by comprising the following steps: the method comprises the following steps:
the method comprises the following steps of firstly, acquiring an image to be identified of an electric power operation site;
inputting the image to be identified into a target detection network, and acquiring a safety helmet wearing state detection result output by the target detection network;
and thirdly, determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
2. The method according to claim 1, wherein the target detection network in the second step is trained with an input sample data set, the sample data set comprises a helmet wearing sample image, and the generation of the sample data set comprises the following steps:
step C, collecting safety helmet wearing sample images of the power operation site, carrying out image scale normalization and image standardization on the collected safety helmet wearing sample images, and selecting specific safety helmet wearing sample images according to a preset selection standard;
step C, manually labeling areas which can be used for safety helmet detection in the specific safety helmet wearing sample image to form a sample data set;
the selection criteria include the following:
selecting a safety helmet wearing sample image with a safety helmet appearing under different illumination conditions;
b, selecting a safety helmet wearing sample image under the condition that different shooting angles cause different sizes of pictures occupied by the target;
selecting a safety helmet wearing sample image with medium crowd density and without mutual shielding among people;
and D, selecting a high crowd density and wearing a sample image of the safety helmet when the people are shielded.
3. The method for detecting and identifying safety helmets on electric power operation sites according to claim 2, wherein the method comprises the following steps: in the step C, manual labeling is to label the image of the helmet wearing sample by using LabelImg software, and the labeling work comprises two aspects:
the working content A is that a marking frame is drawn for each area which can be used for safety helmet detection, so that a target detection network can conveniently obtain the position information of a detection target in an image;
and the working content B provides the class name of the wearing state of the safety helmet corresponding to each labeling frame, so that the target detection network can conveniently match the learned characteristics in the labeling frame with the class.
4. The method for detecting and identifying safety helmets on electric power operation sites according to claim 3, wherein the method comprises the following steps: the method for training the target detection network specifically comprises the following steps:
a1, collecting a sample data set, unifying the long edge sizes of all pictures into a fixed size, and inputting the fixed size into a VGG16 convolutional neural network to extract features so as to form a feature graph;
step A2, fusing feature maps of 4 network stages after the VGG16 convolutional neural network, sequentially inputting the feature maps obtained in the step A1 into an RPN to complete feature fusion and multi-scale detection, and introducing an online difficult sample capable of enhancing the identification capability of the RPN; removing partial redundant frames by using an improved non-maximum suppression algorithm, and deleting surrounding frames of partial positive samples; screening out a suspected region of interest ROI;
step A3, inputting the suspected ROI into a read-only ROI network, introducing online difficult sample mining, and adjusting the ROI; and finally, sending the ROI into a standard ROI network, and outputting a final result, namely a target detection network model.
5. The method for detecting and identifying safety helmets on electric power operation sites according to claim 4, wherein the method comprises the following steps: the VGG16 convolutional neural network is sequentially divided into 5 network stages, and the last convolutional layer of each network stage is respectively C1, C2, C3, C4 and C5;
when the C2 is fused with the feature map of C4, firstly, the C4 changes the dimensionality through a convolution kernel of 1 multiplied by 256 to enable the number of channels to be 256, and then deconvolution including quadruple bilinear interpolation upsampling operation is carried out to enable the feature maps of C4 and C2 to be the same in size; modifying the number of channels of the 2 characteristic diagram into 256 by utilizing a convolution kernel of 1 multiplied by 256, and fusing with C4, wherein the fusion operation uses an additive fusion function; in order to remove aliasing effect brought by upsampling, the fused feature map is subjected to convolution operation with convolution kernel of 3 × 3 once to generate a new feature layer P2 with 256 channels;
generating a feature layer P3 when the feature maps of C5 and C3 are fused, wherein the fusion principle adopts the feature map fusion principle of C2 and C4;
c4 and C5 respectively perform convolution operation with convolution kernel of 1 × 1 × 256 for changing the number of channels and convolution operation with convolution kernel of 3 × 3 for one time, and generate feature layers P4 and P5;
obtaining a feature pyramid formed by feature layers P5, P4, P3 and P2 through feature map fusion; the output dimension of each layer of the characteristic pyramid is fixed to be 256, the shallow layer characteristics of the characteristic pyramid are fused with deep layer characteristics to detect small targets, the strong semantic information of the high layer characteristics can detect medium and large targets, each layer of new characteristics is connected with the RPN, and interested areas are respectively predicted to realize multi-scale detection.
6. The method for detecting and identifying safety helmets on electric power operation sites according to claim 5, wherein the method comprises the following steps: in the region of interest, the sizes of anchor windows corresponding to RPN/P2 and RPN/P3 with smaller receptive fields are {24 × 24, 32 × 32, 64 × 64}, the sizes of anchor windows corresponding to RPN/P4 and RPN/P5 with larger receptive fields are {64 × 64, 128 × 128, 256 × 256}, and the aspect ratios are {1:1, 2:1, 1:2 };
inputting the interested regions of the 4 parts of the feature layer into the ROI pooling layer, mapping the interested regions back to the pyramid feature layers corresponding to the interested regions respectively to obtain a series of feature blocks of the interested regions, wherein the unified size is 7 multiplied by 7, finally, utilizing a cascade fusion function to longitudinally splice the feature graphs of the 4 levels together, sharing a classification layer and a regression layer, and expressing the classification layer and the regression layer as formulas
Wherein x isaAnd xbThe feature matrixes of different layers in the feature pyramid are obtained, and y is the feature matrix after fusion. x is the number ofa、xbAnd y ∈ RH ×W×DH, W and D are the length, width and number of channels, respectively, of the feature layer. i is an element of [1, H ]]，j∈[1，W]，d∈[1，D]。
7. The method for detecting and identifying safety helmets on electric power operation sites according to claim 6, wherein the method comprises the following steps: the online difficult sample mining OHEM includes: in the FastR-CNN stage, samples with losses meeting preset requirements are mined and listed in a difficult sample pool, a training sample set is formed again, then the samples are sent to a classification network and a target frame regression model, and the loss of the network model is calculated to perform back propagation to update the network; the process comprises the following steps:
step B1, after the input image is subjected to size normalization, obtaining a feature pyramid comprising feature layers P5, P4, P3 and P2 through a VGG16 network after feature fusion, then respectively sending the feature pyramid into an RPN network, and obtaining an interested area possibly containing a target through multi-scale prediction;
step B2, inputting all the regions of interest into a read-only ROI network for forward propagation without sampling in small batches; the read-only ROI network comprises an ROI pooling layer, a full connection layer, a Softmax classification layer and a target frame regression layer; inputting the region of interest into an ROI pooling layer, mapping the region of interest to a corresponding pyramid feature layer by the ROI pooling layer, unifying the size of the pyramid feature layer to be 7 x 7, and longitudinally splicing 4 parts of feature graphs with the same size into one part by utilizing a cascade fusion function Concat layer;
step B3, inputting the characteristic diagram of the interesting region into a subsequent layer for calculation to obtain loss values of all the interesting regions, wherein the loss values represent the adaptation degree of the network structure weight, all the values are sorted, the first K samples are reserved as difficult samples, and other loss values are set to be 0;
b4, finding out a corresponding difficult sample according to the region-of-interest index value corresponding to the loss value, inputting the difficult sample into a standard ROI network, outputting a predicted classification result and the coordinates of a boundary box, comparing the predicted classification result with a calibrated result, calculating an error, and reversely propagating and updating the parameters of the whole network;
the improved non-maxima suppression algorithm includes:
step S1, sorting all the target frames according to scores, and considering that the target frame M with the highest current score contains targets;
step S2, calculating the intersection ratio of the residual target frame and M, and when the intersection ratio of the target frame and M is less than NiConsidering that the current target frame and M represent two different targets and keeping the original score;
step S3, when the intersection ratio of the target frame and M is between NiAnd NtIn between, then the current target box and M are considered to possibly contain two different purposesThe target and M may represent the same target, so a weight penalty strategy is adopted
It and IoU (M, b)i) In inverse proportion, the score is reduced, a target frame is reserved, and the next round of screening is participated;
step S4, when the intersection ratio of the target frame and M is larger than NtIf so, determining that the current target frame and the M represent a target, and discarding the current target frame;
moving the current M into the result target frame set, forming a new set by the residual target frames, repeating the steps from S1 to S4 until the target frame set is empty, finishing screening and outputting a result;
the expression is as follows:
wherein N istIs the original threshold value, NiThe newly added threshold value is obtained.
8. The method for detecting and identifying safety helmets on electric power operation sites according to claim 1, wherein the method comprises the following steps: the method of the first step is that a computer host acquires a power operation site video and performs image preprocessing, wherein the image preprocessing comprises the following steps: extracting frames of a video at a certain frame rate, and then carrying out image scale normalization and image standardization on each frame to obtain the input of a target detection network;
in the second step, the trained target detection network is an improved Faster R-CNN network model obtained by training an improved Faster R-CNN network.
9. The method for detecting and identifying safety helmets on electric power operation sites according to claim 3, wherein the method comprises the following steps: and C, labeling the sample data set in the second step comprises the following category names of the wearing states of the safety helmet: red, yellow, white, blue, and unworn.
10. The method for detecting and identifying safety helmets on electric power operation sites according to claim 3, wherein the method comprises the following steps: the annotations in the sample data set are saved as an XML file in the paschaloc format, which can be read by Python software and used for training.
11. The method for detecting and identifying safety helmets on electric power operation sites according to claim 1, wherein the method comprises the following steps: the determining the personnel identity of the target area in the image to be identified according to the detection result of the wearing state of the safety helmet comprises the following steps:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
and under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
12. The method of claim 11, further comprising: and controlling an alarm device in a target area of a background management mechanism or an electric power operation site to alarm under the condition that the personnel do not wear the safety helmet.
13. A safety helmet detection and identity recognition device for an electric power operation site is used for a safety helmet detection and identity recognition method for the electric power operation site, and is characterized in that: the device comprises:
the image to be identified acquisition module is used for acquiring an image to be identified of the electric power operation site;
the safety helmet detection module is used for inputting the image to be identified into a target detection network and acquiring a safety helmet wearing state detection result output by the target detection network;
and the identity recognition module is used for determining the personnel identity of the target area in the image to be recognized according to the detection result of the wearing state of the safety helmet.
14. The device of claim 13, wherein the identification module is configured to:
acquiring a first characteristic from the detection result of the wearing state of the safety helmet, and determining whether a person in the target area wears the safety helmet or not according to the first characteristic;
and under the condition that the person wears the safety helmet, acquiring a second characteristic from the detection result of the wearing state of the safety helmet, and determining the identity of the person according to the second characteristic.
15. The electrical work site safety helmet detection and identification device of claim 13, further comprising an alarm control module, wherein the alarm control module is connected to a background management mechanism or an alarm device in a target area of the electrical work site, and when the identification module detects that a person without a safety helmet is present in the target area of the electrical work site, the alarm device is controlled to alarm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111364569.2A CN114067268A (en) | 2021-11-17 | 2021-11-17 | Method and device for detecting safety helmet and identifying identity of electric power operation site |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111364569.2A CN114067268A (en) | 2021-11-17 | 2021-11-17 | Method and device for detecting safety helmet and identifying identity of electric power operation site |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114067268A true CN114067268A (en) | 2022-02-18 |
Family
ID=80277570
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111364569.2A Pending CN114067268A (en) | 2021-11-17 | 2021-11-17 | Method and device for detecting safety helmet and identifying identity of electric power operation site |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114067268A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115171006A (en) * | 2022-06-15 | 2022-10-11 | 武汉纺织大学 | Detection method for automatically identifying personnel entering electric power dangerous area based on deep learning |
| CN116824517A (en) * | 2023-08-31 | 2023-09-29 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
-
2021
- 2021-11-17 CN CN202111364569.2A patent/CN114067268A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115171006A (en) * | 2022-06-15 | 2022-10-11 | 武汉纺织大学 | Detection method for automatically identifying personnel entering electric power dangerous area based on deep learning |
| CN116824517A (en) * | 2023-08-31 | 2023-09-29 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
| CN116824517B (en) * | 2023-08-31 | 2023-11-17 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| WO2021022970A1 (en) | Multi-layer random forest-based part recognition method and system | |
| CN110852316B (en) | Image tampering detection and positioning method adopting convolution network with dense structure | |
| CN110264440B (en) | Large-scale train displacement fault detection method and system based on deep learning | |
| CN114067268A (en) | Method and device for detecting safety helmet and identifying identity of electric power operation site | |
| Chen et al. | An improved Yolov3 based on dual path network for cherry tomatoes detection | |
| CN109543744B (en) | Multi-category deep learning image identification method based on Loongson group and application thereof | |
| CN113034506B (en) | Remote sensing image semantic segmentation method and device, computer equipment and storage medium | |
| CN112270681B (en) | Method and system for detecting and counting yellow plate pests deeply | |
| CN114170144A (en) | Power transmission line pin defect detection method, equipment and medium | |
| CN111079645A (en) | Insulator self-explosion identification method based on AlexNet network | |
| CN117197896A (en) | Medical waste temporary storage station abnormal behavior real-time detection method based on deep learning | |
| CN115797637A (en) | Semi-supervised segmentation model based on uncertainty between models and in models | |
| CN115564031A (en) | Detection network for glass defect detection | |
| CN113378962B (en) | Garment attribute identification method and system based on graph attention network | |
| CN114359578A (en) | Application method and system of pest and disease damage identification intelligent terminal | |
| CN113221667A (en) | Face and mask attribute classification method and system based on deep learning | |
| CN109816022A (en) | A kind of image-recognizing method based on three decisions and CNN | |
| CN117351409A (en) | Intelligent concrete dam face operation risk identification method | |
| CN106960188B (en) | Weather image classification method and device | |
| CN117372944A (en) | Construction site bare soil monitoring method and terminal | |
| CN117372853A (en) | Underwater target detection algorithm based on image enhancement and attention mechanism | |
| CN117197746A (en) | Safety monitoring system and method based on deep learning | |
| CN115240163A (en) | Traffic sign detection method and system based on one-stage detection network | |
| CN115719428A (en) | Face image clustering method, device, equipment and medium based on classification model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |