CN111222478A - Construction site safety protection detection method and system - Google Patents
Construction site safety protection detection method and system Download PDFInfo
- Publication number
- CN111222478A CN111222478A CN202010028838.7A CN202010028838A CN111222478A CN 111222478 A CN111222478 A CN 111222478A CN 202010028838 A CN202010028838 A CN 202010028838A CN 111222478 A CN111222478 A CN 111222478A
- Authority
- CN
- China
- Prior art keywords
- safety protection
- detection
- real
- monitoring
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V10/507—Summing image-intensity values; Histogram projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a construction site safety protection detection method and a construction site safety protection detection system.A target detection network fused with a CLAHE algorithm is constructed, and the target detection network is trained and optimized by utilizing a training set; the method comprises the steps of obtaining a model weight file according to a target detection network after training optimization, reading the model weight file into an ImageAI library fused with a non-maximum suppression algorithm to obtain a safety protection appliance detection model, obtaining a to-be-detected monitoring video, detecting whether workers wear safety protection appliances in each real-time monitoring image in the to-be-detected monitoring video or not by adopting the safety protection appliance detection model, and if the detection result of M continuous real-time monitoring images indicates that the workers do not wear safety equipment, sending alarm information to a field administrator terminal. The method and the system can reliably and effectively detect the wearing condition of the safety appliance, reduce manpower and material resources for construction site supervision, improve the timeliness of supervision and reduce the probability of safety accidents.
Description
Technical Field
The application belongs to the technical field of video image processing, and particularly relates to a construction site safety protection detection method and system.
Background
According to the statistical report published by the housing of the people's republic of China and the ministry of urban and rural construction, the number of accidents in the construction industry tends to be gradually reduced. Causes of construction safety production accidents include falls, slips, collisions, electric shocks, etc., wherein, in most worker death events caused by falls, the cause of death is that the head is not protected by regulations so that the head is severely impacted. Literature studies have analyzed the relationship between building site deaths and safety equipment usage and have shown that 47.3% of fatal victims have not used or used the safety gear correctly (e.g., safety helmets, safety belts, etc.). To ensure that workers comply with relevant regulations and correctly wear safety helmets and safety harnesses at a job site, it is important to develop an efficient method of automatically monitoring the safety of a job site. Currently, some construction sites establish a video monitoring system for workers within the construction scope, but it is extremely difficult to identify all helmet violations at any time for a large number of monitoring screens. Therefore, there is an urgent need to develop a smart vision based system to perform this monitoring task.
In recent years, image recognition technology has been greatly advanced and has been applied to the fields of face recognition, vehicle detection, pedestrian detection, and the like, and in addition, automatic vision detection has also been widely applied to the industrial field. It is known from previous investigations that the current methods for identifying safety helmets and safety belts are mainly based on traditional image processing algorithms or algorithms based on a combination of machine learning and image processing. The problems of low detection precision or detection rate, high missed detection probability and the like generally exist.
Disclosure of Invention
The application aims to provide a building site safety protection detection method and system, which can reliably and effectively detect the condition that a safety protection tool is not worn, reduce a large amount of manpower and material resources paid for building site safety supervision at present, and improve the timeliness of building site safety supervision.
In order to achieve the purpose, the technical scheme adopted by the application is as follows:
a construction site safety protection detection method is used for detecting whether safety protection tools are worn by workers on a construction site in real time and sending alarm information to a site administrator terminal when the safety protection tools are not worn, and comprises the following steps:
step S1, acquiring a training image of whether the calibrated worker wears the safety protection appliance, and dividing the training image into a training set and a test set;
s2, constructing a RetinaNet target detection network fused with a CLAHE algorithm, and training and optimizing the RetinaNet target detection network by utilizing the training set;
step S3, obtaining a model weight file according to the RetinaNet target detection network after training optimization, reading the model weight file into an ImageAI library fused with a non-maximum suppression algorithm to obtain a safety protection appliance detection model, verifying whether the safety protection appliance detection model reaches a convergence condition or not by using the test set, and outputting the safety protection appliance detection model if the safety protection appliance detection model reaches the convergence condition; otherwise, re-entering step S1;
and S4, acquiring real-time monitoring videos to be detected, acquiring real-time monitoring images according to the monitoring videos to be detected, detecting whether workers wear safety protection appliances in each real-time monitoring image by adopting the safety protection appliance detection model, and if the detection result of M continuous real-time monitoring images is that no safety protection appliances are worn, sending alarm information to a field administrator terminal, wherein M is a set threshold parameter.
Further, the method for detecting the safety protection of the construction site further comprises the following steps:
when a monitoring video to be detected is obtained, monitoring position information matched with the monitoring video to be detected is obtained at the same time, and when real-time monitoring images are obtained according to the monitoring video to be detected, corresponding monitoring position information is generated for each real-time monitoring image; then, the sending of the alarm information to the field administrator terminal includes: and sending the real-time monitoring image and the corresponding monitoring position information of the worker not wearing the safety protection appliance to a field administrator terminal.
Further, the RetinaNet target detection network fused with the CLAHE algorithm, when processing an image, includes:
s2.1, establishing a CLAHE algorithm balanced image, wherein the CLAHE algorithm is as follows:
step S2.1.1, dividing the original image into non-overlapping grids, and respectively calculating histograms of all the grids;
step S2.1.2, clipping the histogram to avoid being affected by noise, if the number of pixels of any intensity value is larger than a predetermined threshold, fixing it to the threshold, and uniformly assigning the clipped number of pixels to the histogram;
step S2.1.3, performing histogram equalization on the histogram obtained in step S2.1.2, combining adjacent grids and eliminating boundary artifacts using bilinear interpolation, and for the pixel in the center of the grid, using interpolation of four adjacent pixels;
s2.2, establishing a depth residual error network RestNet50 to extract the characteristics of the balanced image processed by the CLAHE algorithm, and accelerating and compressing the depth residual error network RestNet50 in the characteristic extraction process;
s2.3, establishing an FPN network to recombine the extracted features;
and S2.4, identifying the recombined features through classifying and positioning the two sub-networks, and outputting whether the label wearing the safety protection appliance exists and the corresponding confidence coefficient.
Further, the test set is used for verifying whether the safety protection appliance detection model reaches a convergence condition, and if the safety protection appliance detection model reaches the convergence condition, the safety protection appliance detection model is output; otherwise re-entering step S1, including:
adopting an evaluation index mAP as a convergence condition, and when the evaluation index mAP of the safety protection appliance detection model is greater than a threshold value K, achieving the convergence condition; otherwise, the convergence condition is not satisfied.
The application also provides a construction site safety protection detection system, which is used for the safety protection appliance detection model obtained by the method to detect whether workers on a construction site wear safety protection appliances in real time and send alarm information to a site administrator terminal when the workers do not wear the safety protection appliances, the construction site safety protection detection system comprises a detection server, the site administrator terminal connected with the detection server and monitoring equipment, wherein the detection server acquires real-time monitoring video to be detected acquired by the monitoring equipment, obtains real-time monitoring images according to the monitoring video to be detected, adopts the safety protection appliance detection model to detect whether the workers wear the safety protection appliances in each real-time monitoring image, and sends the alarm information to the site administrator terminal if the detection result of continuous M real-time monitoring images indicates that the workers do not wear the safety protection appliances, and M is a set threshold parameter.
Further, when the detection server acquires the monitoring video to be detected, the detection server simultaneously acquires monitoring position information matched with the monitoring video to be detected, and when real-time monitoring images are obtained according to the monitoring video to be detected, corresponding monitoring position information is generated for each real-time monitoring image; then, the sending of the alarm information to the field administrator terminal includes: and sending the real-time monitoring image and the corresponding monitoring position information of the worker not wearing the safety protection appliance to a field administrator terminal.
Further, building site safety protection detecting system still includes the alarm unit with supervisory equipment installation nearby, detection server, in the detection result of continuous M real time monitoring image for not wearing safety protection apparatus, then to alarm unit sends alarm information.
The application provides a building site safety protection detection method and system, application degree of depth learning frame, use RetinaNet target detection algorithm, add CLAHE reinforcing image contrast, creatively use ImageAI storehouse to test and verify, a safety protection apparatus detection model of taking into account precision and speed has finally been proposed, safety protection apparatus detection model's practicality is strong, it is accurate to detect, the probability of missing to examine is low, and when detecting that the staff does not wear safety protection apparatus, produce corresponding alarm information and remind the staff in time to wear safety protection apparatus, thereby reduce the probability that building site incident takes place.
Drawings
FIG. 1 is a flowchart of a worksite safety protection detection method of the present application;
fig. 2 is a schematic structural diagram of a deep residual error network ResNet 50;
FIG. 3 is a view showing the structure of the FPN;
FIG. 4 is a schematic view of the worksite safety protection detection system of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
As shown in fig. 1, in an embodiment, a worksite safety protection detection method is provided, which is used for detecting whether a worker wears safety protection equipment in real time and sending alarm information to a worksite administrator terminal when a worker who does not wear safety equipment appears, and includes:
and step S1, acquiring a training image of whether the calibrated worker wears the safety protection appliance, and dividing the training image into a training set and a test set.
In order to obtain a more comprehensive and effective data set, facilitate subsequent model training or testing and improve the accuracy of model identification, when the data set is manufactured, a plurality of monitoring videos are firstly obtained and summarized to obtain training videos of various cameras, various shooting angles, various scenes and various workers not wearing safety equipment.
And then obtaining a plurality of training images according to the training video, and calibrating whether the worker wears the safety protection appliance or not for each training image. When the training images are calibrated whether workers wear safety protection tools or not, the training videos are converted into tens of thousands of frame images through libav software, and then the training videos are calibrated through a visual image labeling tool labelImg, so that an XML file conforming to the PASCAL VOC format is generated.
In this embodiment, to simplify manual calibration and facilitate subsequent identification, the training images of the workers wearing the security devices are correspondingly calibrated, and the training images of the workers not wearing the security devices are not calibrated. And 70% of the calibrated training images are used as a training set, and the remaining 30% of the calibrated training images are used as a test set.
And S2, constructing a RetinaNet target detection network fused with a CLAHE algorithm, and training and optimizing the RetinaNet target detection network by utilizing the training set.
In the construction of the deep network, based on the target detection task for wearing safety protection appliances, software Anaconda, a language Python3.6 and a tensoflow deep learning framework are used, and a target detection network RetinaNet is combined. The complex scene of the construction site is synthesized, so that a Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm is integrated into a training model to improve the image Contrast.
And (3) adopting a loss function FocalLoss to accelerate the model when training the target detection network RetinaNet. Ten thousand pieces of data are trained by using a keras deep neural network model, and a model weight file is output.
When the RetinaNet target detection network fused with the CLAHE algorithm is adopted to process the image, the specific implementation steps are as follows:
s2.1, establishing a CLAHE algorithm balanced image, wherein the CLAHE algorithm is as follows:
step S2.1.1, the original image is divided into non-overlapping local regions (meshes). The minimum size of the grid should be 32 x 32. And computes histograms of all grids separately.
Step S2.1.2, cut histogram to avoid being affected by noise. If the number of pixels for any intensity value is greater than a predetermined threshold, it should be fixed to that threshold. In this case, in order to equalize the total number of pixels, the number of clipped pixels is uniformly assigned to the histogram.
Step S2.1.3, histogram equalization is performed on the histogram obtained in step S2.1.2. Adjacent grids are combined and boundary artifacts are eliminated using bilinear interpolation. For the pixel in the center of the grid, an interpolation of four neighboring pixels is used. Although the original method was developed for grayscale images, it has been used for color images with different schemes.
The CLAHE algorithm is used to equalize the images in the process of improving the image contrast. The CLAHE considers the influence of surrounding areas while the local histogram is balanced, and the processed image has the characteristic of adapting to the difference of gray level distribution of different parts of the image after the local histogram is balanced and has the effect of relatively coordinating the gray level distribution of the whole image after the global histogram is balanced. The contrast of the image processed by the CLAHE algorithm is obviously improved.
The combined prior RetinaNet model is composed of three modules: the idea of feature extraction network, Feature Pyramid Network (FPN) and sub-network, the further construction operation of this embodiment is:
s2.2, establishing a depth residual error network RestNet50 to extract the characteristics of the balanced image h (S) processed by the CLAHE algorithm, and accelerating and compressing the depth residual error network RestNet50 in the characteristic extraction process.
The defect of linear CNN is overcome by using a deep residual error network ResNet50 in a feature extraction network, quick connection is added in a convolution feedforward network, and the self mapping result of the network is directly added into the output of an overlying layer. As shown in fig. 2, in the structure of the deep residual network ResNet50, a data set is first input into a convolutional layer of 7 × 7 × 64, i.e., {7 × 7, conv,64}, then passes through a maximum pooling layer of 3 × 3 to reduce the size of a feature map generated by the convolutional layer, and then passes through 16 building blocks (building blocks) of digital sums (3+4+6+3) at dotted arrows, each building block is 3 layers, i.e., 16 × 3 ═ 48 layers, and the convolutional layer at the beginning of 7 × 7 × 64 and the fully-connected layer at the end together constitute a 50-layer network in ResNet.
And S2.3, establishing an FPN network to recombine the extracted features.
The feature pyramid network FPN combines the shallow feature and the deep feature to obtain a strong semantic information feature, thereby improving the detection performance. FPN includes two paths: a bottom-up path and a top-down path. The bottom-up path is to perform deep convolution on an input image through a backbone network to extract shallow features, and feature layers obtained by extracting the shallow features are marked as C1, C2, C3, C4 and C5 respectively. The top-down path is to perform up-sampling on the deep feature map with strong semantics to obtain deep feature layers C5 ', C4 ', C3 ', C2 ' and C1 ' respectively; and then transversely connecting the obtained features with the previous layer of features to obtain fused feature layers, and performing convolution on each fused feature layer by adopting a convolution kernel of 3 multiplied by 3 to obtain feature layers P2, P3, P4 and P5 for further classification and positioning. The FPN structure is shown in FIG. 3.
The retinaNet model uses the idea of Region candidate network (RPN) in fast R-CNN for reference, and a mapping point of the center of a current sliding window on an original image is called a candidate frame (Anchor), and the Anchor is used as the center to respectively generate candidate regions at five different levels of the FPN.
The RetinaNet model is added when the Anchor is generated
Three different sizes and three different length-width ratios {1:2,1:1, 2: 1}, 9 anchors can be generated, the area sizes of the anchors are {512 } on P1, P2, P3, P4 and P5 respectively2,2562,1282,642,322}。
And S2.4, identifying the recombined features through classifying and positioning the two sub-networks, and outputting a label indicating whether a worker wears safety protection equipment and corresponding confidence.
Classifying and locating two subnetworks includes a classification subnetwork and a border prediction subnetwork, wherein:
the classification sub-network predicts for each Anchor the probability of the occurrence of an object that is not wearing a safety gear. A certain layer in a FPN five-layer network result structure is connected with a Full Convolution Network (FCN), after convolution, a ReLU is used as an excitation function, and finally a Sigmoid function is used for predicting classification conditions.
The focus loss function (Focal loss) is mainly used for solving the problems of unbalance of positive and negative samples and unbalance of difficult and easy samples in one-stage target detection. The focus loss function is improved on the basis of a commonly used cross-entropy loss function. The cross entropy loss function is expressed as follows:
where y represents a class sample and y ∈ { + -1 }, p ∈ [0, 1}, and]the representative model predicts the probability that the output is the class sample y-1. For the sake of brevity and clarity, we define pt：
We can therefore rewrite (1) to CE (p, y) ═ CE (p)t)＝-log(pt)。
To solve the problem of positive and negative sample imbalance, the focus loss function introduces a weight ratio parameter α e [0,1], which takes α when y is 1 and 1- α when y is-1.
So the cross entropy function at this time can be expressed as CE (p)t)＝-αtlog(pt)。
To solve the problem of imbalance of difficult and easy samples, the focus loss function introduces a training specific gravity value gamma for reducing the easy samples, each of which is multiplied by (1-p)t)γTherefore, combining the two parameters α and γ can determine the expression of the focus loss function as:
FL(pt)＝-αt(1-pt)γlog(pt) (4)
combining the steps S2.1-S2.4 to obtain the fusion CLAHE algorithm and the RetinaNet target detection network. After the construction is completed, when the RetinaNet target detection network is optimized by using the training set, the embodiment trains a detection model by using transfer learning, namely, a model pre-trained on ImageNet1K large-scale data by using a deep residual error network ResNet50 is used for initializing RetinaNet. Specifically, during initialization:
fine-tuning the network by using a training sample of the construction site scene, wherein the number of classes in an input _ data layer and the number of outputs in a cls _ score layer in the modified model are 4; modifying the number of outputs in the bbox _ pred layer to 64, i.e. the product of the total number of categories 4 and the coordinates of the centre point (x, y) and the width and height (w, h); the Anchor intersection ratio is modified to be 0.7, namely more than 0.7 is marked as a positive sample, and less than 0.3 is marked as a negative sample. The hyper-parameter lr of the model is initialized to 0.001, the batch _ size is initialized to 25, the epoch is initialized to 25, and the dropout is initialized to 0.8.
And after the initialization is finished, optimizing model parameters through a back propagation algorithm, and then obtaining the optimized RetinaNet target detection network.
Step S3, obtaining a model weight file according to the RetinaNet target detection network after training optimization, reading the model weight file into an ImageAI library fused with a Non-maximum suppression (NMS) algorithm to obtain a safety protection appliance detection model, verifying whether the safety protection appliance detection model reaches a convergence condition by using the test set, and outputting the safety protection appliance detection model if the safety protection appliance detection model reaches the convergence condition; otherwise, the process re-enters step S1.
Since the optimized RetinaNet target detection network can give a large number of predicted results, but most of the predicted values have very low confidence (confidence score), only those predicted results with confidence higher than a certain threshold are considered.
In the testing stage, in order to test the intuitiveness and the effectiveness of the RetinaNet target detection network, an NMS algorithm is used in an ImageAI library of python to inhibit redundant boxes, boxes with the overlapping degree higher than threshold are combined, redundant candidate frames are eliminated, and the optimal object detection position is found.
The essence of NMS is to search for local maxima and suppress non-maxima elements. In object detection, after the characteristics of the sliding window are extracted and classified and identified by a classifier, each window obtains a classification (category) and a score (confidence). NMS algorithms are commonly used to address the situation where many windows contain or mostly intersect other windows due to sliding windows.
In order to facilitate setting of a reasonable convergence condition and obtain a safety protection appliance detection model with better performance, in one embodiment, an evaluation index mAP (mean accuracy value) is used as the convergence condition, and when the evaluation index mAP of the safety protection appliance detection model is greater than a threshold value K, the convergence condition is reached; otherwise, the convergence condition is not satisfied, the data set is continuously modified (the training images are expanded or the bad training images are eliminated), the parameters of the safety protection appliance detection model are modified or the safety protection appliance detection model is improved. The threshold K is set to 80% in this embodiment.
Specifically, in order to determine the correctness of each candidate frame, iou (intersection over union) is used as a metric for evaluating the correctness of the bounding box.
And the performance evaluation index mAP is calculated as follows:
defining the correct detection value TP (true Positives), the false detection value FP (false Positives) and the number of objects missed to be detected FN (false Negatives) of each category in the image, and calculating the Recall and Precision as follows:
further given the Precision value Precision of class C in the imageCThe true number of classes C in the image N (TP)CThe number of all targets of the class C in the image N (total)CThe ratio of:
the mAP (mean Average precision) in target detection is the mean of the Average precision values of all classes, calculated as follows:
in the formula, APCRepresents the average Precision value, sigma Precision, of class CCRepresenting the sum of the average precision values for class C in all test images, N (TotalImages)CIndicates the number of classes C, Sigma AP, contained in all test imagesCRepresents the sum of the average precision values of all classes, and N (classes) represents the number of classes.
And S4, acquiring real-time monitoring videos to be detected, acquiring real-time monitoring images according to the monitoring videos to be detected, detecting whether workers wear safety protection tools in each real-time monitoring image by adopting the safety protection tool detection model, and if the detection result of the continuous M real-time monitoring images indicates that the workers do not wear safety equipment, sending alarm information to a field administrator terminal.
In an embodiment, in order to ensure the requirements of remote control and concurrency of the system, an Ngnix server is introduced, so that when detection is performed: firstly, acquiring a real-time monitoring video to be detected, uploading the real-time monitoring video to an Ngnix server end to ensure the requirements of remote control and concurrence of a system, and acquiring the monitoring video to be detected of the Ngnix server end by utilizing FFmpeg, splitting the monitoring video into frames and acquiring a real-time monitoring image; and reading the real-time monitoring image into an ImageAI library with a construction site safety protection detection model for real-time detection.
And the detection result is output after one detection, and comprises a target frame, a label for judging whether the worker wears the safety protection tool or not and confidence. The confidence coefficient is greater than 0.5, which indicates that the worker without the safety equipment is present, and the closer the value is to 1, which indicates that the accuracy of the detected worker without the safety equipment is higher, and the corresponding target frame, the label "Nhat", "Nbelt" and the value of the confidence coefficient are displayed in the image when the worker without the safety equipment is present; the confidence coefficient is less than 0.5, which indicates that no worker wears the safety equipment, and the detection marks a rectangular frame on the target under the detection condition that the confidence coefficient is less than 0.5.
In one embodiment, in order to timely remind a site administrator of a worker who does not wear a safety protection tool, the method for detecting the safety protection of the construction site further includes: when a monitoring video to be detected is obtained, monitoring position information matched with the monitoring video to be detected is obtained at the same time, and when real-time monitoring images are obtained according to the monitoring video to be detected, corresponding monitoring position information is generated for each real-time monitoring image; then send alarm information to the site manager terminal, including: and sending the real-time monitoring image of the worker without wearing the safety equipment and the corresponding monitoring position information to a field administrator terminal.
In this embodiment, through sending alarm information to field administrator terminal to field administrator in time discovers the staff who does not wear the safety protection apparatus and the position of the staff who does not wear the safety protection apparatus, can in time go to and carry out corresponding processing.
Of course, in order to further improve the timeliness of finding out the worker who does not wear the safety protection equipment, an audio or video alarm message may be generated near the worker who wears the safety protection equipment, which requires a certain number of alarm units to be installed in advance in a designated area according to actual conditions.
According to the method, the idea of RetinaNet is mainly utilized, a depth residual error network ResNet50 is used for completing primary extraction of image features in feature extraction on an image after preprocessing enhancement, and then a Feature Pyramid Network (FPN) is used for completing fine extraction of the image features; and finally, entering a sub-network to classify and position the target. The algorithm ensures accurate detection of the target and also saves training time.
The video stream that this application was shot through the camera control is real-time to be inputed and carry out real-time monitoring detection in building site safety protection detection training model that builds, and the processing speed of every video frame image is about 0.1s in the aspect of image processing, and the practicality is strong, detect accurate, the probability of missing the detection is low. When detecting the staff who do not wear the safety protection appliance, the cloud server sends the detection result to the site manager, so that the site manager can timely remind the staff who do not wear the safety protection appliance to wear the safety protection appliance.
As shown in fig. 4, in one embodiment, a worksite safety protection detection system is further provided, which is used for the safety protection appliance detection model obtained by the worksite safety protection detection method, detecting whether safety protection appliances are worn by workers on a worksite in real time, and sending alarm information to a site administrator terminal when safety protection appliances are not worn, the worksite safety protection detection system includes a detection server, a site administrator terminal connected to the detection server, and a monitoring device, the detection server acquires real-time monitoring videos to be detected collected by the monitoring device, obtains real-time monitoring images according to the monitoring videos to be detected, detects whether workers wear safety protection appliances in each real-time monitoring image by using the safety protection appliance detection model, and if the detection result of M continuous real-time monitoring images is that no safety protection appliances are worn, and sending alarm information to a field administrator terminal, wherein M is a set threshold parameter.
It should be noted that, in fig. 4, the cloud server is the detection server, and details are not described below.
Specifically, in an embodiment, when acquiring to-be-detected monitoring videos of a plurality of monitoring devices, the detection server simultaneously acquires monitoring position information matched with the to-be-detected monitoring videos, and when obtaining real-time monitoring images according to the to-be-detected monitoring videos, generates corresponding monitoring position information for each real-time monitoring image;
and when the detection server sends alarm information to the field administrator terminal, sending a real-time monitoring image of a worker not wearing the safety equipment and corresponding monitoring position information to the field administrator terminal.
Specifically, in an embodiment, the worksite safety protection detection system further includes an alarm unit installed nearby the monitoring device, and the detection server further includes a monitoring position information triggering unit adjacent to the monitoring position information according to the monitoring position information corresponding to the real-time monitoring image of the worker who has no wearing the safety device when detecting that the detection result of the M continuous real-time monitoring images is that the worker who has no wearing the safety device exists, so as to remind the worker who has no wearing the safety protection appliance to wear the safety protection appliance.
Further limitations regarding the worksite safety protection inspection system may be found in the above-mentioned limitations regarding the worksite safety protection inspection method, which will not be described in detail herein.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (7)
1. The utility model provides a building site safety protection detection method for whether the staff of real-time detection building site dresses the safety protection apparatus to when appearing not wearing the safety protection apparatus send alarm information to on-the-spot administrator terminal, its characterized in that, building site safety protection detection method includes:
step S1, acquiring a training image of whether the calibrated worker wears the safety protection appliance, and dividing the training image into a training set and a test set;
s2, constructing a RetinaNet target detection network fused with a CLAHE algorithm, and training and optimizing the RetinaNet target detection network by utilizing the training set;
step S3, obtaining a model weight file according to the RetinaNet target detection network after training optimization, reading the model weight file into an ImageAI library fused with a non-maximum suppression algorithm to obtain a safety protection appliance detection model, verifying whether the safety protection appliance detection model reaches a convergence condition or not by using the test set, and outputting the safety protection appliance detection model if the safety protection appliance detection model reaches the convergence condition; otherwise, re-entering step S1;
and S4, acquiring real-time monitoring videos to be detected, acquiring real-time monitoring images according to the monitoring videos to be detected, detecting whether workers wear safety protection appliances in each real-time monitoring image by adopting the safety protection appliance detection model, and if the detection result of M continuous real-time monitoring images is that no safety protection appliances are worn, sending alarm information to a field administrator terminal, wherein M is a set threshold parameter.
2. The worksite safety protection inspection method of claim 1, further comprising:
when a monitoring video to be detected is obtained, monitoring position information matched with the monitoring video to be detected is obtained at the same time, and when real-time monitoring images are obtained according to the monitoring video to be detected, corresponding monitoring position information is generated for each real-time monitoring image; then, the sending of the alarm information to the field administrator terminal includes: and sending the real-time monitoring image and the corresponding monitoring position information of the worker not wearing the safety protection appliance to a field administrator terminal.
3. The worksite safety protection detection method as claimed in claim 1, wherein the RetinaNet target detection network fused with the CLAHE algorithm comprises, when processing the image:
s2.1, establishing a CLAHE algorithm balanced image, wherein the CLAHE algorithm is as follows:
step S2.1.1, dividing the original image into non-overlapping grids, and respectively calculating histograms of all the grids;
step S2.1.2, clipping the histogram to avoid being affected by noise, if the number of pixels of any intensity value is larger than a predetermined threshold, fixing it to the threshold, and uniformly assigning the clipped number of pixels to the histogram;
step S2.1.3, performing histogram equalization on the histogram obtained in step S2.1.2, combining adjacent grids and eliminating boundary artifacts using bilinear interpolation, and for the pixel in the center of the grid, using interpolation of four adjacent pixels;
s2.2, establishing a depth residual error network RestNet50 to extract the characteristics of the balanced image processed by the CLAHE algorithm, and accelerating and compressing the depth residual error network RestNet50 in the characteristic extraction process;
s2.3, establishing an FPN network to recombine the extracted features;
and S2.4, identifying the recombined features through classifying and positioning the two sub-networks, and outputting whether the label wearing the safety protection appliance exists and the corresponding confidence coefficient.
4. The worksite safety protection detection method according to claim 1, wherein the test set is used for verifying whether the safety protection appliance detection model reaches a convergence condition, and if the convergence condition is reached, the safety protection appliance detection model is output; otherwise re-entering step S1, including:
adopting an evaluation index mAP as a convergence condition, and when the evaluation index mAP of the safety protection appliance detection model is greater than a threshold value K, achieving the convergence condition; otherwise, the convergence condition is not satisfied.
5. A construction site safety protection detection system is characterized in that the construction site safety protection detection system is used for detecting whether workers on a construction site wear safety protection appliances in real time according to a safety protection appliance detection model obtained by the method of any one of claims 1 to 4 and sending alarm information to a site administrator terminal when the workers do not wear the safety protection appliances, the construction site safety protection detection system comprises a detection server, the site administrator terminal and monitoring equipment, wherein the site administrator terminal is connected with the detection server, the detection server acquires real-time monitoring videos to be detected and collected by the monitoring equipment, real-time monitoring images are obtained according to the monitoring videos to be detected, the safety protection appliance detection model is used for detecting whether the workers wear the safety protection appliances in each real-time monitoring image, and if the detection result of M continuous real-time monitoring images is that the safety protection appliances are not worn, and sending alarm information to a field administrator terminal, wherein M is a set threshold parameter.
6. The construction site safety protection detection system according to claim 5, wherein the detection server simultaneously acquires monitoring position information matched with the monitoring video to be detected when acquiring the monitoring video to be detected, and generates corresponding monitoring position information for each real-time monitoring image when obtaining the real-time monitoring image according to the monitoring video to be detected; then, the sending of the alarm information to the field administrator terminal includes: and sending the real-time monitoring image and the corresponding monitoring position information of the worker not wearing the safety protection appliance to a field administrator terminal.
7. The worksite safety protection detection system according to claim 6, further comprising an alarm unit installed nearby the monitoring device, wherein the detection server sends alarm information to the alarm unit when the detection result of the continuous M real-time monitoring images indicates that no safety protection tool is worn.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010028838.7A CN111222478A (en) | 2020-01-11 | 2020-01-11 | Construction site safety protection detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010028838.7A CN111222478A (en) | 2020-01-11 | 2020-01-11 | Construction site safety protection detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111222478A true CN111222478A (en) | 2020-06-02 |
Family
ID=70810408
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010028838.7A Withdrawn CN111222478A (en) | 2020-01-11 | 2020-01-11 | Construction site safety protection detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111222478A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112183472A (en) * | 2020-10-28 | 2021-01-05 | 西安交通大学 | Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet |
| CN112241694A (en) * | 2020-09-25 | 2021-01-19 | 上海荷福人工智能科技（集团）有限公司 | Method for identifying unworn safety belt based on CenterNet |
| CN112686823A (en) * | 2020-12-31 | 2021-04-20 | 广西慧云信息技术有限公司 | Automatic image enhancement method based on illumination transformation network |
| CN113780322A (en) * | 2021-02-09 | 2021-12-10 | 北京京东振世信息技术有限公司 | Safety detection method and device |
| CN113902792A (en) * | 2021-11-05 | 2022-01-07 | 长光卫星技术有限公司 | Building height detection method and system based on improved RetinaNet network and electronic equipment |
| CN113971829A (en) * | 2021-10-28 | 2022-01-25 | 广东律诚工程咨询有限公司 | Intelligent detection method, device, equipment and storage medium for wearing condition of safety helmet |
| CN114202796A (en) * | 2020-08-27 | 2022-03-18 | 中国电信股份有限公司 | Model adaptive target detection method and device |
| CN114694073A (en) * | 2022-04-06 | 2022-07-01 | 广东律诚工程咨询有限公司 | Intelligent detection method and device for wearing condition of safety belt, storage medium and equipment |
| CN117011890A (en) * | 2023-07-10 | 2023-11-07 | 三峡科技有限责任公司 | Construction personnel protection article detection method based on improved YOLOv7 model |
| CN117152846A (en) * | 2023-10-30 | 2023-12-01 | 云南师范大学 | Student behavior recognition method, device and system and computer readable storage medium |
| CN113902792B (en) * | 2021-11-05 | 2024-06-11 | 长光卫星技术股份有限公司 | Building height detection method, system and electronic equipment based on improved RETINANET network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014108109A2 (en) * | 2012-12-22 | 2014-07-17 | Wesp Gmbh | It-shirt |
| CN110046557A (en) * | 2019-03-27 | 2019-07-23 | 北京好运达智创科技有限公司 | Safety cap, Safe belt detection method based on deep neural network differentiation |
| CN110516529A (en) * | 2019-07-09 | 2019-11-29 | 杭州电子科技大学 | It is a kind of that detection method and system are fed based on deep learning image procossing |
-
2020
- 2020-01-11 CN CN202010028838.7A patent/CN111222478A/en not_active Withdrawn
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014108109A2 (en) * | 2012-12-22 | 2014-07-17 | Wesp Gmbh | It-shirt |
| CN110046557A (en) * | 2019-03-27 | 2019-07-23 | 北京好运达智创科技有限公司 | Safety cap, Safe belt detection method based on deep neural network differentiation |
| CN110516529A (en) * | 2019-07-09 | 2019-11-29 | 杭州电子科技大学 | It is a kind of that detection method and system are fed based on deep learning image procossing |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114202796B (en) * | 2020-08-27 | 2023-08-04 | 中国电信股份有限公司 | Model self-adaptive target detection method and device |
| CN114202796A (en) * | 2020-08-27 | 2022-03-18 | 中国电信股份有限公司 | Model adaptive target detection method and device |
| CN112241694A (en) * | 2020-09-25 | 2021-01-19 | 上海荷福人工智能科技（集团）有限公司 | Method for identifying unworn safety belt based on CenterNet |
| CN112183472A (en) * | 2020-10-28 | 2021-01-05 | 西安交通大学 | Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet |
| CN112686823A (en) * | 2020-12-31 | 2021-04-20 | 广西慧云信息技术有限公司 | Automatic image enhancement method based on illumination transformation network |
| CN113780322A (en) * | 2021-02-09 | 2021-12-10 | 北京京东振世信息技术有限公司 | Safety detection method and device |
| CN113780322B (en) * | 2021-02-09 | 2023-11-03 | 北京京东振世信息技术有限公司 | Safety detection method and device |
| CN113971829A (en) * | 2021-10-28 | 2022-01-25 | 广东律诚工程咨询有限公司 | Intelligent detection method, device, equipment and storage medium for wearing condition of safety helmet |
| CN113902792A (en) * | 2021-11-05 | 2022-01-07 | 长光卫星技术有限公司 | Building height detection method and system based on improved RetinaNet network and electronic equipment |
| CN113902792B (en) * | 2021-11-05 | 2024-06-11 | 长光卫星技术股份有限公司 | Building height detection method, system and electronic equipment based on improved RETINANET network |
| CN114694073B (en) * | 2022-04-06 | 2023-06-06 | 广东律诚工程咨询有限公司 | Intelligent detection method, device, storage medium and equipment for wearing condition of safety belt |
| CN114694073A (en) * | 2022-04-06 | 2022-07-01 | 广东律诚工程咨询有限公司 | Intelligent detection method and device for wearing condition of safety belt, storage medium and equipment |
| CN117011890A (en) * | 2023-07-10 | 2023-11-07 | 三峡科技有限责任公司 | Construction personnel protection article detection method based on improved YOLOv7 model |
| CN117152846A (en) * | 2023-10-30 | 2023-12-01 | 云南师范大学 | Student behavior recognition method, device and system and computer readable storage medium |
| CN117152846B (en) * | 2023-10-30 | 2024-01-26 | 云南师范大学 | Student behavior recognition method, device and system and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111222478A (en) | Construction site safety protection detection method and system | |
| CN111353413B (en) | Low-missing-report-rate defect identification method for power transmission equipment | |
| CN111784685B (en) | Power transmission line defect image identification method based on cloud edge cooperative detection | |
| CN111339882B (en) | Power transmission line hidden danger detection method based on example segmentation | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN110084166B (en) | Intelligent substation smoke and fire identification and monitoring method based on deep learning | |
| CN112052797A (en) | MaskRCNN-based video fire identification method and system | |
| CN101751744B (en) | Detection and early warning method of smoke | |
| CN112507865B (en) | Smoke identification method and device | |
| CN113642474A (en) | Hazardous area personnel monitoring method based on YOLOV5 | |
| CN111414807A (en) | Tidal water identification and crisis early warning method based on YO L O technology | |
| CN112183472A (en) | Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet | |
| CN114648714A (en) | YOLO-based workshop normative behavior monitoring method | |
| CN106339657A (en) | Straw incineration monitoring method and device based on monitoring video | |
| CN115512387A (en) | Construction site safety helmet wearing detection method based on improved YOLOV5 model | |
| CN116052082A (en) | Power distribution station room anomaly detection method and device based on deep learning algorithm | |
| CN117576632B (en) | Multi-mode AI large model-based power grid monitoring fire early warning system and method | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| CN117726991A (en) | High-altitude hanging basket safety belt detection method and terminal | |
| CN117523437A (en) | Real-time risk identification method for substation near-electricity operation site | |
| CN117197746A (en) | Safety monitoring system and method based on deep learning | |
| CN115830302B (en) | Multi-scale feature extraction fusion power distribution network equipment positioning identification method | |
| CN112001336A (en) | Pedestrian boundary crossing alarm method, device, equipment and system | |
| CN117423157A (en) | Mine abnormal video action understanding method combining migration learning and regional invasion | |
| CN113065379A (en) | Image detection method and device fusing image quality and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200602 |