CN112329532A - Automatic tracking safety helmet monitoring method based on YOLOv4 - Google Patents
Automatic tracking safety helmet monitoring method based on YOLOv4 Download PDFInfo
- Publication number
- CN112329532A CN112329532A CN202011066394.2A CN202011066394A CN112329532A CN 112329532 A CN112329532 A CN 112329532A CN 202011066394 A CN202011066394 A CN 202011066394A CN 112329532 A CN112329532 A CN 112329532A
- Authority
- CN
- China
- Prior art keywords
- face
- picture
- yolov4
- file
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012549 training Methods 0.000 claims abstract description 58
- 238000010276 construction Methods 0.000 claims abstract description 17
- 238000002360 preparation method Methods 0.000 claims abstract description 8
- 238000012360 testing method Methods 0.000 claims description 31
- 238000005070 sampling Methods 0.000 claims description 12
- 238000009432 framing Methods 0.000 claims description 10
- 238000007639 printing Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 5
- 238000013459 approach Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000012797 qualification Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 15
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention discloses an automatic tracking safety helmet monitoring method based on YOLOv 4.A YOLOv4 network training module performs early preparation work, trains aiming at a construction site scene, and acquires a weight file and a configuration file; the face coding library establishing module performs early preparation work to establish a face coding library for constructors; the automatic tracking identification module identifies the picture shot by the ball machine by using the weight and the configuration file obtained by the YOLOv4 network training module, detects whether a constructor wears a safety helmet or not, controls the ball machine to track according to the position information if the constructor not wears the safety helmet is detected, shoots a face image and identifies the face image, displays the corresponding name if the corresponding person is found in the face code library, and otherwise displays unknown; the alarm module gives an alarm to remind the personnel who do not wear the safety helmet and the safety officer.
Description
Technical Field
The invention belongs to the technical field of deep learning, and relates to an automatic tracking safety helmet monitoring method based on YOLOv 4.
Background
The safety helmet has a protective effect on the head, and when constructors are impacted or extruded by high falling objects and hard objects, the impact force can be reduced, and the injury to the head of a human body can be eliminated or lightened. Plays a key role in the life safety of constructors.
However, in the real situation, there are still many construction personnel who do not wear the safety helmet on the construction site, and the safety accidents caused by the safety helmet are more continuous. The reason why the worker does not wear the safety helmet includes the following points:
1. often forget at job site or rest area, lead to constructor to wear when business turn over job site.
2. The weather is hot in summer, and the safety helmet is uncomfortable to wear, so when the safety supervisor is not at present, the safety helmet can be stolen and taken down.
3. Some constructors consider that it is safe enough on the open ground or consider that the construction site is close to completion, and consider that there is not much safety problem due to the lucky psychology, so there is no need to wear safety helmet.
Therefore, the method can monitor the condition that the constructor wears the safety helmet in real time, reflect the condition to the safety supervisor in time, and greatly reduce the possibility of safety accidents caused by the fact that the constructor does not wear the safety helmet.
In recent years, with the further development of Chinese economy and technology, especially the development of big data, artificial intelligence and Internet of things technology, a management mode of deep integration of the Internet and the traditional building industry, namely an intelligent construction site, begins to appear, and gradually replaces the traditional safety supervisor mode in the past. Wherein, the safety helmet detects just is the important ring in wisdom building site. This patent is exactly a monitoring system of automatic tracking safety helmet based on YOLOv 4.
Currently, target detection algorithms commonly used in the industry include SSD issued in 2015, RetinaNet, Mask R-CNN and Cascade R-CNN issued in 2017, and YOLOv3 issued in 2018. This application uses YOLOv4, released in month 4 of 2020.
As can be seen from fig. 1, compared to YOLOv3, the accuracy is greatly improved and the AP is improved by about 10% while the running speed is kept unchanged.
Therefore, YOLOv4 has guaranteed better speed and precision simultaneously, more accords with the demand that wisdom building site safety helmet detected and need guarantee real-time and accuracy simultaneously. In addition, at present, many GPUs are required to be trained in parallel due to the fact that the scale of a model is too large, and the YOLOv4 simplifies and optimizes some newly proposed algorithms, so that the YOLOv4 can be trained on one GPU, such as 1080Ti, the training cost is reduced, and the method is more convenient to apply to a production environment.
The Face _ recognition Face recognition is based on a deep learning model in a C + + open source library Dlib which is advanced in the industry, and can be operated on a CPU through a Dlib HOG or a GPU through a Dlib CNN. Is a powerful, simple and efficient model.
At present, monitoring CCD cameras are mainly divided into a gun camera and a ball camera. The monitoring position of the bolt is fixed, and only a certain monitoring position is right aligned, so that the monitoring range is limited. The dome camera integrates a camera system, a zoom lens and an electronic pan-tilt, and is better than a gun type camera in stability and controllability. The effects of integral explosion prevention and water prevention can be achieved, and under the outdoor condition, the combination of the gunlock and the holder cannot be compared favorably. In addition, the biggest difference between the ball machine and the gun camera is a cloud deck system, which can control the horizontal and vertical rotation of the ball machine and can also control the zooming focus and the aperture of a lens. The monitoring range of the ball machine is much larger than that of a fixed gun, and 360-degree rotation can be generally achieved, so that a large area can be monitored. .
Disclosure of Invention
In order to solve the problems, the safety helmet is worn by constructors and other personnel entering a construction site according to safety regulations, and a safety supervisor is required to supervise. However, because of the complex situation of the construction site, the manpower cannot effectively supervise the whole construction site, and the safety supervisor cannot always supervise the construction site. And constructors have a lucky psychology, often do not wear safety helmets, and need an intelligent safety helmet monitoring system to supplement the shortage of manpower. The existing safety helmet detection system has the contradiction that only the safety helmet can be detected and the face can not be identified in the large-range monitoring process, and the face can be identified in the small-range detection process but the large-range monitoring process can not be carried out.
In order to achieve the above object, the technical solution of the present invention is a method for monitoring an auto-tracing crash helmet based on YOLOv4, wherein the system for monitoring an auto-tracing crash helmet based on YOLOv4 comprises a YOLOv4 network training module, a face code library establishing module, an auto-tracing recognition module connected to both the YOLOv4 network training module and the face code library establishing module, and an alarm module connected to the auto-tracing recognition module, and the method using the above system comprises the following steps:
s10, collecting data;
s20, marking data and organizing a data set;
s30, setting a YOLOv4 framework training configuration file for training: setting a test configuration file, carrying out testing and performance statistics, modifying the prior frame size of YOLOv4 according to the test configuration file, retraining and testing the previous frame size, and acquiring an optimal weight file and a configuration file;
s40, photographing and coding the face of the constructor, and establishing a face database;
s50, identifying the picture shot by the dome camera in real time by using a YOLOv4 frame to obtain position information and type information, framing the position in the monitoring picture and printing the type information;
s60, the ball machine tracks according to the position information;
s70, adopting face _ recognition frame to recognize human face;
and S80, giving an alarm.
Preferably, the YOLOv4 network training module performs preliminary preparation work, trains aiming at the construction site scene, and acquires a weight file and a configuration file; the face coding library establishing module performs early preparation work to establish a face coding library for constructors; the automatic tracking identification module identifies the picture shot by the ball machine by using the weight and the configuration file obtained by the YOLOv4 network training module, detects whether a constructor wears a safety helmet or not, controls the ball machine to track according to the position information if the constructor not wears the safety helmet is detected, shoots a face image and identifies the face image, displays the corresponding name if the corresponding person is found in the face code library, and otherwise displays unknown; the alarm module gives an alarm to remind the personnel who do not wear the safety helmet and the safety officer.
Preferably, said collecting data comprises the steps of:
s11, shooting by a ball machine on a construction site to obtain a video sample;
and S12, determining sampling time according to the time length of the video sample to obtain more samples, sampling the obtained video sample, obtaining the image sample, naming the image sample, and storing the image sample in a folder to form an image data set.
Preferably, the sampling is every 10 seconds.
Preferably, the annotation data, organizing the data set, comprises the steps of:
s21, labeling the dataset with labellimg: set as 3 categories, respectively person, with a helmet aqm _ zc and without a helmet aqm _ yc, choose to generate a lyolo formatted txt markup file, the format is:
class_id x y w h；
class _ id is id number of the class;
wherein, x is the x coordinate (transverse direction) of the central point of the target/the total width of the picture; y is the y coordinate of the center of the target (longitudinal)/total height of the picture; w is the total width of the broadband/picture of the target frame; h is the height of the target frame/the total height of the picture;
s22, putting all pictures to be trained and tested into a JPEGImages folder, and putting a corresponding yolo format label file into a labels folder;
s23, generating a training set and a testing set by using the python script file: generating a train.txt and a test.txt, respectively giving lists of a training picture file and a test picture file, and containing a path and a file name of each picture.
Preferably, the setting YOLOv4 framework training configuration file is used for training, and comprises the following steps:
s31, setting configuration files according to a YOLOv4 framework, training a sample training set, and generating the optimal weight of the training, wherein the configuration files are aqm.names, aqm.data and YOLOv 4-aqm.cfg; the names file is used for setting classified types, the data file is used for setting the number of types, the paths of the test set and the training set, the paths of the names file and the paths of the final storage weight file, the cfg file is used for setting parameters related to training, and according to the performance of the training host, batch, subdivisions, max _ batchs, steps, scales, angle, qualification, exposur and hue are configured;
s32, copying a copy of yolov4-aqm.cfg, renaming to yolov4-aqm-test.cfg, modifying batch to 1, modifying subdivisions to 1, and starting training after configuration is completed;
s33, obtaining the prior frame size of the data set by using k-means clustering, modifying the prior frame size of the cfg configuration file according to performance statistics, then performing training and testing again, and finally obtaining a better weight file and a better cfg configuration file.
Preferably, the photographing coding of the face of the constructor and the establishment of the face database comprise the following steps:
s41, photographing and sampling the faces of all constructors, and renaming the pictures as constructor names;
and S42, loading the head portrait of the recorded constructor by using the face _ recognition framework, converting the head portrait into a face code, and storing the face code as a face code library.
Preferably, the identifying a picture shot by a dome camera in real time by using a YOLOv4 frame to obtain the position information and the category information, framing a specific position in the monitoring picture and printing the category information includes the following steps:
s51, calling a camera by using opencv to obtain a real-time monitoring video picture;
s52, extracting each frame of the video, performing color space conversion, and converting the video from the BGR format of opencv images to the RBG format required by YOLOv 4;
s53, scaling the picture to fit the image size required by YOLOv 4;
s54, inputting the processed single-frame image into YOLOv4, and converting and identifying to obtain identified position information and type information;
s55, if no person wearing the safety helmet exists, framing out the human face and the safety helmet, and marking out the category information; if the person without the safety helmet wears the safety helmet, the head position is framed, the category information is marked, and the coordinate information is output.
Preferably, the ball machine tracks according to the position information, and comprises the following steps:
s61, if the person without wearing the safety helmet is identified, judging whether the person is close to the edge in the picture of the dome camera according to the head coordinate of the person;
s62, if the edge is approached, controlling the ball machine pan-tilt to move according to the coordinates, and enabling the target to approach the center of the picture; otherwise, skipping the step;
s63, recognizing the face, and recognizing the face by judging whether the face _ recognition framework can be positioned at the face position; if the human face is not identified, amplifying the picture, returning to opencv to call the camera to acquire a frame of picture, and starting a new cycle; and if the human face is successfully identified, carrying out the next step.
Preferably, the face recognition with the face _ recognition framework includes the following steps:
s71, inputting the picture obtained in the step 6 into a face _ recognition framework;
s72, the face _ recognition framework locates the face in the screenshot;
s73, coding the human face according to the position of the human face in the image;
s74, comparing the obtained code with a face database;
and S75, if finding that the corresponding constructor exists, printing the name on the monitoring picture, otherwise, displaying unknown.
The invention has at least the following specific beneficial effects:
1. the safety helmet monitoring system based on YOLOv4 detects the condition that a constructor wears a safety helmet in continuous frames in real time, and performs face recognition and gives an alarm if the constructor does not wear the safety helmet. The detection ensures good real-time performance and accuracy, and can detect a plurality of people in the picture at the same time. Meanwhile, the safety helmet can be directly detected when the face shot by the dome camera at a high position is too small. And then amplifying the picture to perform face recognition. Instead of advanced face positioning, the safety helmet is identified according to the position of the face.
2. The safety helmet monitoring system based on YOLOv4 has the advantages of high detection speed and small detection error, and can eliminate common interference, such as misjudgment caused by material stacking; no special requirement is made on the environment in the detection process; the cost is low, and other equipment is not needed for detection personnel except a camera, a display, a host and related alarm equipment in daily use; the traceability is realized, the face and the name of a person who does not wear the safety helmet can be recorded, and the follow-up playback and check are facilitated; the method has wide application scenes, is not limited to detection of construction sites, is also suitable for detection of wearing of the safety helmet by constructors in occasions such as workshops and tunnels, and is also suitable for detection of wearing conditions of the safety helmet of personnel riding a battery car or a motorcycle on a road.
Drawings
FIG. 1 is a comparison of YOLOv4 with other prior art algorithms;
FIG. 2 is a flowchart illustrating the steps of a YOLOv 4-based method for automatically tracking headwear monitoring in accordance with an embodiment of the present invention;
FIG. 3 is a system block diagram of a YOLOv 4-based method for automatically tracking helmet surveillance in accordance with an embodiment of the present invention;
FIG. 4 is a flowchart of the step S30 of the YOLOv 4-based automatic tracking helmet monitoring method in accordance with an embodiment of the present invention;
FIG. 5 is a flowchart of the step S40 of the YOLOv 4-based automatic tracking helmet monitoring method in accordance with an embodiment of the present invention;
FIG. 6 is a flowchart of steps S50-70 of a YOLOv 4-based automatic tracking crash helmet monitoring method in accordance with an embodiment of the method of the present invention;
fig. 7 is a flowchart of S80 of a method for automatically tracking crash helmet monitoring based on YOLOv4 in accordance with an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
On the contrary, the invention is intended to cover alternatives, modifications, equivalents and alternatives which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following detailed description of the present invention, certain specific details are set forth in order to provide a better understanding of the present invention. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details.
Before introducing the overall scheme, the following phrases and symbols are defined:
1. face coding: firstly, the position of the face is positioned, a face image is input according to the positioned position, and a 128-dimensional list is output after coding processing, wherein the 128-dimensional list comprises various information of the face.
2. Labeling data: processing the collected images by using labelimg software, manually framing the positions of the objects to be recognized, and calibrating the type and category information
3. A priori block: the priori block is used for helping the user to determine the width and the height of a common target, and the determined width and the determined height can be used for helping the user to perform prediction when the user performs prediction.
Face _ recognition framework: the face recognition method is a powerful, simple and easy-to-use face recognition open source project, is the most concise face recognition library in the world, and can extract, recognize and operate faces by using Python and a command line tool. The face recognition of the face _ recognition framework is based on a deep learning model in a C + + open source library dlib which is advanced in the industry, and a tested face data set is used for testing, so that the accuracy rate is up to 99.38%. Has the functions of: 1. and positioning the position of the human face. 2. And identifying key points of the human face. 3. The face detection method can be matched with other python libraries, such as opencv, to realize real-time face detection.
Referring to fig. 2, a technical solution of the present invention, which is an embodiment of the present invention, is a flow chart of steps of an auto-tracing helmet monitoring method based on YOLOv4, and a block diagram of an auto-tracing helmet monitoring system based on YOLOv4 is shown in fig. 3, which includes a YOLOv4 network training module, 10, a face code library establishing module 20, an auto-tracing recognition module 30 connected to both the YOLOv4 network training module 10 and the face code library establishing module 20, and an alarm module 40 connected to the auto-tracing recognition module 30, and the method using the above system includes the following steps:
s10, collecting data;
s20, marking data and organizing a data set;
s30, setting a YOLOv4 framework training configuration file for training: setting a test configuration file, carrying out testing and performance statistics, modifying the prior frame size of YOLOv4 according to the test configuration file, retraining and testing the previous frame size, and acquiring an optimal weight file and a configuration file;
s40, photographing and coding the face of the constructor, and establishing a face database;
s50, identifying the picture shot by the dome camera in real time by using a YOLOv4 frame to obtain position information and type information, framing the position in the monitoring picture and printing the type information;
s60, the ball machine tracks according to the position information;
s70, adopting face _ recognition frame to recognize human face;
and S80, giving an alarm.
The YOLOv4 network training module performs early-stage preparation work, trains aiming at a construction site scene, and acquires a weight file and a configuration file; the face coding library establishing module performs early preparation work to establish a face coding library for constructors; the automatic tracking identification module identifies the picture shot by the ball machine by using the weight and the configuration file obtained by the YOLOv4 network training module, detects whether a constructor wears a safety helmet or not, controls the ball machine to track according to the position information if the constructor not wears the safety helmet is detected, shoots a face image and identifies the face image, displays the corresponding name if the corresponding person is found in the face code library, and otherwise displays unknown; the alarm module gives an alarm to remind the personnel who do not wear the safety helmet and the safety officer.
S10, collecting data includes the steps of:
s11, shooting by a ball machine on a construction site to obtain a video sample;
and S12, determining sampling time according to the time length of the video sample to obtain more samples, sampling the obtained video sample, obtaining the image sample, naming the image sample, and storing the image sample in a folder to form an image data set.
The sampling is every 10 seconds.
S20, labeling data and organizing a data set, wherein the method comprises the following steps:
s21, labeling the dataset with labellimg: set as 3 categories, respectively person, with a helmet aqm _ zc and without a helmet aqm _ yc, choose to generate a lyolo formatted txt markup file, the format is:
class_id x y w h；
class _ id is id number of the class;
wherein, x is the x coordinate (transverse direction) of the central point of the target/the total width of the picture; y is the y coordinate of the center of the target (longitudinal)/total height of the picture; w is the total width of the broadband/picture of the target frame; h is the height of the target frame/the total height of the picture;
s22, putting all pictures to be trained and tested into a JPEGImages folder, and putting a corresponding yolo format label file into a labels folder;
s23, generating a training set and a testing set by using the python script file: generating a train.txt and a test.txt, respectively giving lists of a training picture file and a test picture file, and containing a path and a file name of each picture.
S30, setting a YOLOv4 framework training configuration file for training, comprising the following steps:
s31, setting configuration files according to a YOLOv4 framework, training a sample training set, and generating the optimal weight of the training, wherein the configuration files are aqm.names, aqm.data and YOLOv 4-aqm.cfg; the names file is used for setting classified types, the data file is used for setting the number of types, the paths of the test set and the training set, the paths of the names file and the paths of the final storage weight file, the cfg file is used for setting parameters related to training, and according to the performance of the training host, batch, subdivisions, max _ batchs, steps, scales, angle, qualification, exposur and hue are configured;
s32, copying a copy of yolov4-aqm.cfg, renaming to yolov4-aqm-test.cfg, modifying batch to 1, modifying subdivisions to 1, and starting training after configuration is completed;
s33, obtaining the prior frame size of the data set by using k-means clustering, modifying the prior frame size of the cfg configuration file according to performance statistics, then performing training and testing again, and finally obtaining a better weight file and a better cfg configuration file.
S40, the photographing coding of the face of the constructor and the establishment of the face database comprise the following steps:
s41, photographing and sampling the faces of all constructors, and renaming the pictures as constructor names;
and S42, loading the head portrait of the recorded constructor by using the face _ recognition framework, converting the head portrait into a face code, and storing the face code as a face code library.
The above is the preliminary preparation work, and the following is the system work flow.
S50, the method for identifying the picture shot by the dome camera in real time by using the YOLOv4 frame to obtain the position information and the category information, framing the specific position in the monitoring picture and printing the category information comprises the following steps:
s51, calling a camera by using opencv to obtain a real-time monitoring video picture;
s52, extracting each frame of the video, performing color space conversion, and converting the video from the BGR format of opencv images to the RBG format required by YOLOv 4;
s53, scaling the picture to fit the image size required by YOLOv 4;
s54, inputting the processed single-frame image into YOLOv4, and converting and identifying to obtain identified position information and type information;
s55, if no person wearing the safety helmet exists, framing out the human face and the safety helmet, and marking out the category information; if the person without the safety helmet wears the safety helmet, the head position is framed, the category information is marked, and the coordinate information is output.
S60, the ball machine tracks according to the position information, and the method comprises the following steps:
s61, if the person without wearing the safety helmet is identified, judging whether the person is close to the edge in the picture of the dome camera according to the head coordinate of the person;
s62, if the edge is approached, controlling the ball machine pan-tilt to move according to the coordinates, and enabling the target to approach the center of the picture; otherwise, skipping the step;
s63, recognizing the face, and recognizing the face by judging whether the face _ recognition framework can be positioned at the face position; if the human face is not identified, amplifying the picture, returning to opencv to call the camera to acquire a frame of picture, and starting a new cycle; and if the human face is successfully identified, carrying out the next step.
S70, the face recognition with the face _ recognition framework includes the following steps:
s71, inputting the picture obtained in S60 into the face _ recognition framework;
s72, the face _ recognition framework locates the face in the screenshot;
s73, coding the human face according to the position of the human face in the image;
s74, comparing the obtained code with a face database;
and S75, if finding that the corresponding constructor exists, printing the name on the monitoring picture, otherwise, displaying unknown.
S80, the alarm sending comprises the following steps:
and S81, if the person without the safety helmet is present, storing the face picture of the person in the background of the system.
S82, recording the name of the person, and the time and place when the non-wearing safety helmet is detected.
And S83, giving an alarm at the intelligent construction site system to remind a safety supervisor.
And S84, sending a short message to the mobile phone of the safety supervisor. If the personnel information exists in the database, the short message is also sent to the mobile phone of the personnel for reminding.
Only the face-recognition framework is applied by the face-code library establishing module and the automatic tracking identification module, and the face-code library establishing module 20: and (3) coding the photos of the constructors by using a face _ recognition framework, converting image information into coding information, and establishing a face coding library.
The automatic tracking recognition module 30: and performing real-time face recognition by using a face _ recognition framework, namely encoding the face of the person without wearing the safety helmet in real time and comparing the face with the information in the face encoding library. If the same is found, the corresponding person is found. If not, the person is not recorded.
The invention adopts the high-definition network spherical camera, and the ball machine can be arranged at the high position of a construction site, such as a tower crane and the like, so as to carry out large-scale monitoring. When the safety helmet is detected, the whole monitoring picture can be directly subjected to large-scale multi-target identification. When the situation that the safety helmet is not worn is found, the ball machine is controlled according to the detected coordinates to track the target, the picture is enlarged, and accurate face recognition is carried out on the target.
The invention combines the safety helmet monitoring module and the face monitoring module for use, and directly uses YOLOv4 to detect the safety helmet in a normal state. When the safety helmet is not worn, the face recognition is carried out. Instead of detecting the face first and then detecting whether the safety helmet is worn. Therefore, the detection speed and the detection precision are better and the cost is reduced by using lower hardware configuration.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. The automatic tracking safety helmet monitoring method based on the YOLOv4 is characterized in that an automatic tracking safety helmet monitoring system based on the YOLOv4 comprises a YOLOv4 network training module, a face code library establishing module, an automatic tracking recognition module and an alarm module, wherein the automatic tracking recognition module is connected with the YOLOv4 network training module and the face code library establishing module, the alarm module is connected with the automatic tracking recognition module, and the method adopting the system comprises the following steps:
s10, collecting data;
s20, marking data and organizing a data set;
s30, setting a YOLOv4 framework training configuration file for training: setting a test configuration file, carrying out testing and performance statistics, modifying the prior frame size of YOLOv4 according to the test configuration file, retraining and testing the previous frame size, and acquiring an optimal weight file and a configuration file;
s40, photographing and coding the face of the constructor, and establishing a face database;
s50, identifying the picture shot by the dome camera in real time by using a YOLOv4 frame to obtain position information and type information, framing the position in the monitoring picture and printing the type information;
s60, the ball machine tracks according to the position information;
s70, adopting face _ recognition frame to recognize human face;
and S80, giving an alarm.
2. The method of claim 1, wherein the YOLOv4 network training module performs a preliminary work of training for a worksite scene to obtain a weight file and a configuration file; the face coding library establishing module performs early preparation work to establish a face coding library for constructors; the automatic tracking identification module identifies the picture shot by the ball machine by using the weight and the configuration file obtained by the YOLOv4 network training module, detects whether a constructor wears a safety helmet or not, controls the ball machine to track according to the position information if the constructor not wears the safety helmet is detected, shoots a face image and identifies the face image, displays the corresponding name if the corresponding person is found in the face code library, and otherwise displays unknown; the alarm module gives an alarm to remind the personnel who do not wear the safety helmet and the safety officer.
3. The method of claim 1, wherein said collecting data comprises the steps of:
s11, shooting by a ball machine on a construction site to obtain a video sample;
and S12, determining sampling time according to the time length of the video sample to obtain more samples, sampling the obtained video sample, obtaining the image sample, naming the image sample, and storing the image sample in a folder to form an image data set.
4. The method of claim 3, wherein the sampling is every 10 seconds.
5. The method of claim 1, wherein the annotation data, organizing a data set, comprises the steps of:
s21, labeling the dataset with labellimg: set as 3 categories, respectively person, with a helmet aqm _ zc and without a helmet aqm _ yc, choose to generate a lyolo formatted txt markup file, the format is:
class_id x y w h；
class _ id is id number of the class;
wherein, x is the horizontal coordinate of the central point x of the target/the total width of the picture; y is the y longitudinal coordinate of the center of the target/total height of the picture; w is the total width of the broadband/picture of the target frame; h is the height of the target frame/the total height of the picture;
s22, putting all pictures to be trained and tested into a JPEGImages folder, and putting a corresponding yolo format label file into a labels folder;
s23, generating a training set and a testing set by using the python script file: generating a train.txt and a test.txt, respectively giving lists of a training picture file and a test picture file, and containing a path and a file name of each picture.
6. The method of claim 1, wherein the setup YOLOv4 framework training profile, training, comprises the steps of:
s31, setting configuration files according to a YOLOv4 framework, training a sample training set, and generating the optimal weight of the training, wherein the configuration files are aqm.names, aqm.data and YOLOv 4-aqm.cfg; the names file is used for setting classified types, the data file is used for setting the number of types, the paths of the test set and the training set, the paths of the names file and the paths of the final storage weight file, the cfg file is used for setting parameters related to training, and according to the performance of the training host, batch, subdivisions, max _ batchs, steps, scales, angle, qualification, exposur and hue are configured;
s32, copying a copy of yolov4-aqm.cfg, renaming to yolov4-aqm-test.cfg, modifying batch to 1, modifying subdivisions to 1, and starting training after configuration is completed;
s33, obtaining the prior frame size of the data set by using k-means clustering, modifying the prior frame size of the cfg configuration file according to performance statistics, then performing training and testing again, and finally obtaining a better weight file and a better cfg configuration file.
7. The method of claim 1, wherein said encoding the face of the constructor by taking a picture, creating a face database, comprises the steps of:
s41, photographing and sampling the faces of all constructors, and renaming the pictures as constructor names;
and S42, loading the head portrait of the recorded constructor by using the face _ recognition framework, converting the head portrait into a face code, and storing the face code as a face code library.
8. The method of claim 1, wherein the identifying the shot by the ball machine in real time by using the YOLOv4 frame to obtain the position information and the category information, and framing the specific position in the monitoring picture and printing the category information comprises the following steps:
s51, calling a camera by using opencv to obtain a real-time monitoring video picture;
s52, extracting each frame of the video, performing color space conversion, and converting the video from the BGR format of opencv images to the RBG format required by YOLOv 4;
s53, scaling the picture to fit the image size required by YOLOv 4;
s54, inputting the processed single-frame image into YOLOv4, and converting and identifying to obtain identified position information and type information;
s55, if no person wearing the safety helmet exists, framing out the human face and the safety helmet, and marking out the category information; if the person without the safety helmet wears the safety helmet, the head position is framed, the category information is marked, and the coordinate information is output.
9. The method of claim 1, wherein the ball machine is tracked based on location information, comprising the steps of:
s61, if the person without wearing the safety helmet is identified, judging whether the person is close to the edge in the picture of the dome camera according to the head coordinate of the person;
s62, if the edge is approached, controlling the ball machine pan-tilt to move according to the coordinates, and enabling the target to approach the center of the picture; otherwise, skipping the step;
s63, recognizing the face, and recognizing the face by judging whether the face _ recognition framework can be positioned at the face position; if the human face is not identified, amplifying the picture, returning to opencv to call the camera to acquire a frame of picture, and starting a new cycle; and if the human face is successfully identified, carrying out the next step.
10. The method according to claim 1, wherein the face recognition using the face _ recognition framework comprises the following steps:
s71, inputting the picture obtained in S60 into the face _ recognition framework;
s72, the face _ recognition framework locates the face in the screenshot;
s73, coding the human face according to the position of the human face in the image;
s74, comparing the obtained code with a face database;
and S75, if finding that the corresponding constructor exists, printing the name on the monitoring picture, otherwise, displaying unknown.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066394.2A CN112329532A (en) | 2020-09-30 | 2020-09-30 | Automatic tracking safety helmet monitoring method based on YOLOv4 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011066394.2A CN112329532A (en) | 2020-09-30 | 2020-09-30 | Automatic tracking safety helmet monitoring method based on YOLOv4 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112329532A true CN112329532A (en) | 2021-02-05 |
Family
ID=74314444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011066394.2A Pending CN112329532A (en) | 2020-09-30 | 2020-09-30 | Automatic tracking safety helmet monitoring method based on YOLOv4 |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112329532A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113158995A (en) * | 2021-05-21 | 2021-07-23 | 西安建筑科技大学 | Multi-target tracking detection method, system, equipment and storage medium |
| CN113283369A (en) * | 2021-06-08 | 2021-08-20 | 苏州市伏泰信息科技股份有限公司 | Port wharf operating personnel safety protection measure monitoring system and method |
| CN113283296A (en) * | 2021-04-20 | 2021-08-20 | 晋城鸿智纳米光机电研究院有限公司 | Helmet wearing detection method, electronic device and storage medium |
| CN113361319A (en) * | 2021-04-02 | 2021-09-07 | 中国电子科技集团公司第五十四研究所 | Indoor scene detection method based on target detection and personnel attribute identification |
| CN113537183A (en) * | 2021-09-17 | 2021-10-22 | 江苏巨亨智能科技有限公司 | Electric vehicle helmet identification method based on artificial intelligence |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188724A (en) * | 2019-06-05 | 2019-08-30 | 中冶赛迪重庆信息技术有限公司 | The method and system of safety cap positioning and color identification based on deep learning |
| CN110309718A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wearing detection method |
| CN110399905A (en) * | 2019-07-03 | 2019-11-01 | 常州大学 | The detection and description method of safety cap wear condition in scene of constructing |
| CN110807429A (en) * | 2019-10-23 | 2020-02-18 | 西安科技大学 | Construction safety detection method and system based on tiny-YOLOv3 |
| CN111191586A (en) * | 2019-12-30 | 2020-05-22 | 安徽小眯当家信息技术有限公司 | Method and system for inspecting wearing condition of safety helmet of personnel in construction site |
| CN111241959A (en) * | 2020-01-06 | 2020-06-05 | 重庆大学 | Method for detecting person without wearing safety helmet through construction site video stream |
| CN111523432A (en) * | 2020-04-16 | 2020-08-11 | 重庆大学 | Intelligent construction site safety helmet detection system and method thereof |
-
2020
- 2020-09-30 CN CN202011066394.2A patent/CN112329532A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309718A (en) * | 2019-05-27 | 2019-10-08 | 安徽继远软件有限公司 | A kind of electric network operation personnel safety cap wearing detection method |
| CN110188724A (en) * | 2019-06-05 | 2019-08-30 | 中冶赛迪重庆信息技术有限公司 | The method and system of safety cap positioning and color identification based on deep learning |
| CN110399905A (en) * | 2019-07-03 | 2019-11-01 | 常州大学 | The detection and description method of safety cap wear condition in scene of constructing |
| CN110807429A (en) * | 2019-10-23 | 2020-02-18 | 西安科技大学 | Construction safety detection method and system based on tiny-YOLOv3 |
| CN111191586A (en) * | 2019-12-30 | 2020-05-22 | 安徽小眯当家信息技术有限公司 | Method and system for inspecting wearing condition of safety helmet of personnel in construction site |
| CN111241959A (en) * | 2020-01-06 | 2020-06-05 | 重庆大学 | Method for detecting person without wearing safety helmet through construction site video stream |
| CN111523432A (en) * | 2020-04-16 | 2020-08-11 | 重庆大学 | Intelligent construction site safety helmet detection system and method thereof |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361319A (en) * | 2021-04-02 | 2021-09-07 | 中国电子科技集团公司第五十四研究所 | Indoor scene detection method based on target detection and personnel attribute identification |
| CN113283296A (en) * | 2021-04-20 | 2021-08-20 | 晋城鸿智纳米光机电研究院有限公司 | Helmet wearing detection method, electronic device and storage medium |
| CN113158995A (en) * | 2021-05-21 | 2021-07-23 | 西安建筑科技大学 | Multi-target tracking detection method, system, equipment and storage medium |
| CN113283369A (en) * | 2021-06-08 | 2021-08-20 | 苏州市伏泰信息科技股份有限公司 | Port wharf operating personnel safety protection measure monitoring system and method |
| CN113537183A (en) * | 2021-09-17 | 2021-10-22 | 江苏巨亨智能科技有限公司 | Electric vehicle helmet identification method based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112329532A (en) | Automatic tracking safety helmet monitoring method based on YOLOv4 | |
| CN108062349B (en) | Video monitoring method and system based on video structured data and deep learning | |
| CN108009473B (en) | Video structuralization processing method, system and storage device based on target behavior attribute | |
| CN110543867B (en) | Crowd density estimation system and method under condition of multiple cameras | |
| CN111191586B (en) | Method and system for inspecting wearing condition of safety helmet of personnel in construction site | |
| CN113516076B (en) | Attention mechanism improvement-based lightweight YOLO v4 safety protection detection method | |
| CN102799935B (en) | Human flow counting method based on video analysis technology | |
| CN111726586A (en) | Production system operation standard monitoring and reminding system | |
| CN112396658B (en) | Indoor personnel positioning method and system based on video | |
| CN111241959A (en) | Method for detecting person without wearing safety helmet through construction site video stream | |
| CN113903081A (en) | Visual identification artificial intelligence alarm method and device for images of hydraulic power plant | |
| CN112560745B (en) | Method for discriminating personnel on electric power operation site and related device | |
| CN107133611A (en) | A kind of classroom student nod rate identification with statistical method and device | |
| CN107241572A (en) | Student's real training video frequency tracking evaluation system | |
| CN113743256B (en) | Intelligent early warning method and device for site safety | |
| CN112184773A (en) | Helmet wearing detection method and system based on deep learning | |
| CN112434828A (en) | Intelligent identification method for safety protection in 5T operation and maintenance | |
| CN112434827A (en) | Safety protection identification unit in 5T fortune dimension | |
| CN113111771A (en) | Method for identifying unsafe behaviors of power plant workers | |
| CN111738115A (en) | Mobile camera automatic identification method and system based on artificial intelligence | |
| CN113807240A (en) | Intelligent transformer substation personnel dressing monitoring method based on uncooperative face recognition | |
| CN114067396A (en) | Vision learning-based digital management system and method for live-in project field test | |
| CN113593162A (en) | Stranger passage monitoring method and device based on video AI | |
| CN209803819U (en) | Face recognition recorder | |
| CN115953815A (en) | Monitoring method and device for infrastructure site |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210205 |