CN113052107B - Method for detecting wearing condition of safety helmet, computer equipment and storage medium - Google Patents
Method for detecting wearing condition of safety helmet, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113052107B CN113052107B CN202110357038.4A CN202110357038A CN113052107B CN 113052107 B CN113052107 B CN 113052107B CN 202110357038 A CN202110357038 A CN 202110357038A CN 113052107 B CN113052107 B CN 113052107B
- Authority
- CN
- China
- Prior art keywords
- image
- head
- frame
- target
- targets
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000007637 random forest analysis Methods 0.000 claims abstract description 31
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 54
- 238000004422 calculation algorithm Methods 0.000 description 23
- 238000010276 construction Methods 0.000 description 22
- 238000003066 decision tree Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 208000012260 Accidental injury Diseases 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 239000002243 precursor Substances 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a method for detecting the wearing condition of a safety helmet, a computer device and a storage medium, wherein the method comprises the following steps: acquiring a time sequence image of a preset area; detecting head targets in each frame of image to obtain head targets contained in each frame of image, and category, category confidence and image area of the head targets, wherein the category comprises: head targets wearing and not wearing helmets; performing target tracking, and associating the same head target in the multi-frame images; extracting preset image features of an image area where each head target is located for each head target detected in each frame of image; and for each head target contained in each frame of image, determining the category of each head target corresponding to the current image by using a random forest classifier according to the category and the category confidence degree detected by each head target in the current image and the preamble image of the current image and the preset image features extracted from the current image. Thus, the detection accuracy is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method for detecting a wearing condition of a helmet, a computer device, and a storage medium.
Background
The safety helmet is an important protective article for effectively protecting the heads of operators, preventing and relieving various injury accidents and ensuring the life safety of the operators on a construction site. A large number of facts prove that the accident occurrence frequency of a construction site can be effectively reduced and the accident injury risk can be reduced by correctly wearing the safety helmet.
In the related art, no detection is performed on the construction site. In addition, the monitoring distance of the construction site is far, the coverage range is large, the personnel are in an active state, the personnel are large in size difference due to the fact that the personnel are distributed at different distances, in addition, the construction site is also affected by weather and the like, and when the detection scheme in the related technology is applied to the construction site, a large amount of missed detection or detection errors exist.
In summary, there is no effective solution for accurately detecting the wearing condition of the helmet on the construction site.
Disclosure of Invention
In order to solve the technical problems described above or at least partially solve the technical problems described above, the present application provides a method for detecting the wearing condition of a helmet, a computer device and a storage medium.
In a first aspect, the present application provides a method of detecting a wear condition of a helmet, comprising: acquiring a time sequence image of a preset area; detecting a head target in each frame of image to obtain a head target contained in each frame of image, and a category of the head target, a category confidence and an image area where the head target is located, wherein the category comprises: a head target wearing a helmet and a head target not wearing a helmet; performing multi-target tracking on all detected head targets, and associating the same head target in the multi-frame image; extracting preset image features of an image area where each head target is located for each head target detected in each frame of image; and for each head target contained in each frame of image, determining the category of each head target corresponding to the current image by using a random forest classifier according to the category and the category confidence degree detected by each head target in the current image and the preamble image of the current image and the preset image features extracted from the current image.
In certain embodiments, the above method further comprises: for each head target detected by each frame of image, determining the linear distance between the image area of each head target in the current image and the image area of each head target in the previous image of the current image; for each head target contained in each frame of image, determining the category of each head target corresponding to the current image by using a random forest classifier according to the category and the category confidence degree detected by each head target in the current image and the preamble image of the current image, the preset image features extracted from the current image and the straight line distance.
In certain embodiments, the above method further comprises: for a newly detected head object in each frame of image, a random forest classifier is used in the subsequent image to determine that the newly detected head object corresponds to the category of the subsequent image.
In some embodiments, the preset image features include: image local texture features and/or color histogram features.
In some embodiments, the image local texture features described above are described using a local binary pattern (Local Binary Pattern, abbreviated as LBP) operator.
In some embodiments, acquiring a time-series image of a preset region includes: acquiring a video acquired by a camera, wherein the field of view of the camera covers a preset area; and extracting multi-frame images from the video according to preset conditions to obtain a time sequence image of a preset area.
In certain embodiments, the above method further comprises: detecting a human body target in each frame of image; and alarming the wearing condition of the safety helmet according to the number of head targets and the number of human targets in each frame of image.
In certain embodiments, the above method further comprises: detecting a human body in each frame of image to obtain a human body target contained in each frame of image; for each frame of image, if the number of head targets determined as not wearing the safety helmet is 0 and the number of head targets determined as wearing the safety helmet is not equal to the number of detected human body targets, marking the image as an abnormal frame; if the number of the head targets which are determined to not wear the safety helmet is larger than 0, marking the image as an alarm frame; and alarming according to the condition that the continuous multi-frame images are marked as abnormal frames and/or alarming frames.
In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor; the computer program when executed by a processor performs any of the steps of the method of detecting a helmet wearing condition described above.
In a third aspect, the present application provides a computer readable storage medium having stored thereon a program for detecting a helmet wearing condition, which when executed by a processor, implements the steps of any one of the methods for detecting a helmet wearing condition described above.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the head targets wearing the safety helmet and not wearing the safety helmet are subjected to target detection and target tracking to obtain the category, the category confidence coefficient and the image area where the head targets are located in the current image and the front image of the current image, the preset image characteristics of the area where the head targets are located are extracted, and the head targets corresponding to the category of the current image are determined according to the category and the category confidence coefficient of the head targets in the current image and the front image of the head targets and the preset image characteristics of the head targets in the current image by using the random forest classifier, so that the head targets wearing the safety helmet and not wearing the safety helmet can be accurately identified, and especially, the head targets wearing the safety helmet and not wearing the safety helmet can be accurately identified under the conditions of long monitoring distance and large coverage range.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic diagram of an embodiment of a system for detecting a wearing condition of a helmet according to an embodiment of the present application;
FIG. 2 is a flowchart of an embodiment of a method for detecting a wearing condition of a helmet according to an embodiment of the present application;
FIG. 3 is a flowchart of an example of a method for detecting a wearing condition of a helmet according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an embodiment of a random forest classifier provided by an embodiment of the present application;
FIG. 5 is an example of LBP characteristics at different luminance according to an embodiment of the present application;
FIG. 6 is an example of color histogram features of different color helmets according to an embodiment of the present application; and
fig. 7 is a schematic hardware diagram of an implementation manner of a computer device according to an embodiment of the present application.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present application, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
An embodiment of the present application provides a system for detecting a wearing condition of a helmet, as shown in fig. 1, a system 100 includes: one or more cameras 101, computer devices 102, and clients 103. It should be appreciated that embodiments of the present application are not limited in this regard, as, for example, a camera and computer device may be integrated.
In the embodiment of the present application, the camera 101 is disposed at a construction site, and its field of view covers at least a partial area of the construction site, so as to collect video images of the covered area. In some examples, camera 101 is fixed to cover an area, and video images of the area are acquired according to preset conditions. In some examples, the job site is divided into a plurality of sub-areas, each sub-area being provided with a camera 101 to acquire video images of the sub-area. In some examples, camera 101 is configured to be adjustable, which can cover different areas at different times to monitor multiple areas.
In the embodiment of the application, the computer device 102 is in communication connection with the camera 101, and the computer device 102 is configured to control the camera 101, receive video images collected by the camera 101, and detect the wearing condition of the helmet according to the video images collected by the camera 101. In some examples, the computer device 102 is configured to alert based on the detection results and send an alert message to the client 103 to inform the job site of the helmet wear.
In the embodiment of the application, because the environment of a construction site is often bad, the algorithm is greatly influenced by factors such as reflection, similar objects, shooting distance and the like, a large number of missed detection and false detection can occur, and particularly, the detection results among different frames are unstable.
In addition, in order to cover a wide range, the camera 101 is generally erected far from the working surface, and the helmet is small, so that the false alarm rate is high. In some cases, the camera 101 needs to cruise over a large range, and the detected targets vary widely in size, resulting in a high false alarm rate.
Furthermore, on-site constructors often need to bend or squat when working, and shielding by a bracket or a wall often occurs, so that detection is difficult, or head position detection is inaccurate after detection. In addition, a group of people gather together to operate, and after detecting the human body, the head position picture cannot be cut, so that the safety helmet detection cannot be realized.
To at least partially solve the above-mentioned problems, an embodiment of the present application provides a method for detecting a wearing condition of a helmet. The method comprehensively utilizes the safety helmet wearing detection method of the multi-target tracking and random forest classifier, is oriented to an open construction site, solves the problems that detection is missed, the detection result is unstable, the full coverage detection of the construction site cannot be realized and the like when whether the safety helmet is correctly worn by a person on the construction site is detected, realizes the detection of the safety helmet wearing of the person on the construction site, and can realize online real-time detection.
According to the actual construction environment, the camera 101 is reasonably installed, and the preset point and the cruising period of the camera 101 can be set, so that an on-site image is obtained. And monitoring the construction area to be monitored, and judging whether the situation that the safety helmet is not worn exists in the area. In some examples, the situation that the algorithm has missed detection and false detection is considered, and alarm output is carried out by comprehensively judging detection results of a human body target, a head target wearing a safety helmet and a head target not wearing the safety helmet in a multi-frame video, so that alarm accuracy is improved.
A method of detecting a helmet wearing condition according to an embodiment of the present application is described below with reference to the system 100 shown in fig. 1.
Fig. 2 is a flowchart of an embodiment of a method for detecting a wearing condition of a helmet according to the present application, as shown in fig. 2, the method includes steps S202 to S210. It should be understood that, although steps have numbers in the embodiment of the present application, this is not a limitation on the execution sequence of steps, and the sequence of steps or synchronous execution may be adjusted as needed.
Step S202, acquiring a time sequence image of a preset area.
In step S202, the time-series image may be all frame images of the video image or may be images extracted from the video image. In practical applications, images are extracted from video images by setting frame extraction conditions, such as the number of frames extracted per second, etc.
Step S204, detecting the head target in each frame of image to obtain the head target contained in each frame of image, the category of the head target, the category confidence and the image area where the head target is located.
In step S204, as an exemplary illustration, object detection is performed by the YOLO algorithm using an object detection model based on deep learning, but the embodiment of the present application is not limited thereto, and other object detection methods are also possible.
In step S204, the categories of the head targets include: a head target wearing a helmet and a head target not wearing a helmet.
In step S204, a head target is detected, and the type of the detected head target is determined. The category confidence is the probability that the head target is identified as the head target wearing the helmet or the head target wearing the helmet.
In step S204, as an exemplary illustration, the image area where the head target is located is a target frame on the image, and the image area is represented by the vertex pixel coordinates of the target frame, but the embodiment of the present application is not limited thereto.
Step S206, performing multi-target tracking on all the detected head targets, and associating the same head target in the multi-frame image.
In step S206, a plurality of head targets can be detected on each frame of image, and each head target is tracked by a multi-target tracking method, so that the same head target in the multi-frame image is associated. As an exemplary illustration, in some examples, a SORT algorithm is used for multi-objective tracking.
In step S206, the detected identical head targets are assigned identical Identifications (IDs), and the identical head targets are identical targets.
Step S208, for each head target detected in each frame of image, extracting preset image features of an image area where each head target is located.
In step S208, one or more preset image features are extracted, the plurality of image features facilitating the final classification of the head object.
Step S210, for each head object contained in each frame of image, determining that each head object corresponds to the category of the current image by using a random forest classifier according to the category and the category confidence degree detected by each head object in the current image and the previous image of the current image and the preset image features extracted from the current image.
The category of the head object detected by each frame of image is determined using a random forest classifier, through step S210. The classification of the head targets is comprehensively judged by using a random forest classifier through the classification and the classification confidence degree of each head target detected in the current image and the front image of the current image and the preset image features extracted from the current image, so that the accuracy of head target identification is improved.
According to the method provided by the embodiment of the application, the head targets wearing the safety helmet and not wearing the safety helmet are subjected to target detection and target tracking to obtain the category, the category confidence coefficient and the image area where the head targets are located in the current image and the front image of the current image, the preset image characteristics of the area where the head targets are located are extracted, and the head targets corresponding to the category of the current image are determined according to the category and the category confidence coefficient of the head targets in the current image and the front image of the head targets and the preset image characteristics of the head targets in the current image by using the random forest classifier, so that the head targets wearing the safety helmet and not wearing the safety helmet can be accurately identified, and especially, the head targets wearing the safety helmet and not wearing the safety helmet can be accurately identified under the conditions of long monitoring distance and large coverage range.
In some examples, the predetermined image feature is a color histogram feature. The color histogram is a global feature describing the surface properties of the scene corresponding to the image or image area, and the accuracy of judging whether the head target wears the helmet is improved by the color histogram feature because the helmet target has fixed several types of colors.
In some examples, the predetermined image feature is an image local texture feature. When the brightness change is obvious, the local texture feature change of the image is small, so that the feature information of the safety helmet in the outdoor scene can be better reflected. In a preferred example, the image specific texture feature is characterized by using an LBP operator or an improved operator thereof, wherein the LBP feature is an operator for describing the local feature of the image, and has significant advantages of gray invariance, rotation invariance and the like. Herein, "LBP operator" is a class of operators, including improvements therein, and is not limited to one of them.
In some examples, the preset image features include color histogram features and image local texture features, so as to improve accuracy of judging whether the head target wears the helmet through the color histogram features, and improve adaptability to scene darkness through the image local texture features.
It should be appreciated that in embodiments of the present application, the preset image features are not limited to color histogram features and image local texture features, and other image features are also contemplated, and embodiments of the present application are not limited thereto, and one or more other image features may be employed by those skilled in the art. According to the embodiment of the application, on the basis of the detection result of the current image, the random forest classifier is used for further determining the category of the head target corresponding to the current image by combining the detection result of the precursor image and the image characteristics of the image area where the head target is located in the image, so that the identification accuracy is improved.
In some examples, to increase robustness and reduce target tracking errors, a straight line distance between an image region in which each head target is located in the current image and an image region in a preceding image of the current image is also determined for each head target detected for each frame of image. In step S210, for each head object included in each frame of image, a random forest classifier is used to determine, according to the category and the category confidence level detected by each head object in the current image and the previous image of the current image, the preset image feature extracted from the current image, and the above straight line distance, the category of each head object corresponding to the current image.
In the above step S210, for the newly detected head object in each frame of image, a random forest classifier is used to determine the category of the newly detected head object in the subsequent image.
In some examples, in step S202, a video acquired by a camera is acquired, where a field of view of the camera covers a preset area; and extracting multi-frame images from the video according to preset conditions to obtain a time sequence image of a preset area.
In some examples, the above method further comprises: detecting a human body target in each frame of image; and alarming the wearing condition of the safety helmet according to the number of head targets and the number of human targets in each frame of image. And the quantity of the head targets and the quantity of the human targets are compared, so that the reliability of alarming is improved.
In some examples, the above method further comprises: and tracking the track of the head target without the safety helmet to obtain a moving track diagram of the personnel without the safety helmet. In some examples, upon alerting, a map of the movement trajectories of persons not wearing the headgear is sent to the client to inform the relevant person.
The embodiment of the application also provides an alarm strategy, and the method further comprises the following steps: detecting a human body in each frame of image to obtain a human body target contained in each frame of image. For each frame of image, if the number of head targets determined as not wearing the safety helmet is 0 and the number of head targets determined as wearing the safety helmet is not equal to the number of detected human body targets, marking the image as an abnormal frame; if the number of the head targets which are determined to not wear the safety helmet is larger than 0, marking the image as an alarm frame; and alarming according to the condition that the continuous multi-frame images are marked as abnormal frames and/or alarming frames. For example, within the sliding window, if consecutive m-frame images are marked as alert frames, an alert is issued.
An example of an embodiment of the present application will be described below taking the YOLO algorithm for target detection and the SORT algorithm for target tracking as an example.
The method comprises the steps of (1) marking samples by using the head of a person wearing the safety helmet, the head of a person not wearing the safety helmet and the whole body of the person, and training a detection model; and (3) selecting category and category confidence degree for the output of the model reasoning process, training a random forest classifier by combining the detection information of the previous frame of the image and other image features of the positioning area, and further making a decision on the final result of the model.
The flow chart of the present example is shown in fig. 3, and the present example is described below with reference to fig. 3.
Camera configuration and scheduling
According to the actual construction environment, the camera is reasonably installed and deployed, and the preset point and the cruising period of the camera are set, so that the on-site image is obtained. And (3) carrying out real-time image acquisition on the key areas by adopting a fixed camera.
The camera preset points are set, so that the whole coverage of the construction site can be realized through multiple cameras; defining a camera cruising period and a preset point setting rule; and automatically scheduling a preset point of the camera, calling the camera to a designated preset point according to defined logic timing, combining with an intelligent detection algorithm, performing video frame extraction, and determining the frame number extracted per second according to the algorithm requirement to perform detection processing.
Model training and target detection
The target detection is performed by using a target detection algorithm based on deep learning, wherein the YOLO algorithm has a relatively high detection speed and high accuracy, and the YOLOv3 algorithm is adopted in the example.
Sample marking and model training are carried out on the heads of construction site personnel and personnel wearing safety helmets and personnel not wearing safety helmets. And extracting proper video frames from the monitoring video of the construction site as sample data, taking 80% of all pictures as training sets and the rest 20% as test sets. And detecting personnel on a construction site in real time by using the trained target detection model, inputting each frame of video image in the detection process, and outputting the detected target position (the position of a human body and the position of the head), the category (whether a safety helmet is worn or not) and the category confidence degree of the current frame.
Multi-target tracking
The target in the video image is determined by adopting a multi-target tracking algorithm, the movement of the target is continuously tracked, and the movement track of the detected target can be drawn in real time in the detection window.
The effect of the multi-target Tracking algorithm is now closely related to the result of target Detection, since the mainstream multi-target Tracking algorithm is a TBD (Tracking-by-Detection) strategy. Here, the SORT (Simple Online And Realtime Tracking) algorithm is taken as an example for illustration, but DeepSORT or other Multiple Object Tracking, MOT algorithms may also be used.
The SORT algorithm consists of a kalman filter and a hungarian algorithm. On the premise of utilizing the SORT algorithm to track the target, a detector is utilized to detect the target, and if the target detection is inaccurate, the tracking effect is poor. The method comprises the following specific steps:
1) Detecting a first frame of the video by utilizing YOLO, establishing a tracker by utilizing detected target information (Box, target frame) and initializing, distributing an ID for each target, processing the Box information detected by the first frame by utilizing a Kalman filter, and generating state prediction and covariance prediction of a second frame.
2) And detecting a second frame of the video by utilizing YOLO, solving an IOU (interaction-Over-Union) from the obtained new target information and the target information predicted by a Kalman filter of the previous frame, obtaining the largest unique matching (data association part) of the IOU in the two frames by using a Hungary bipartite graph matching algorithm, and removing the matching pair with the matching value smaller than a threshold value, so that the same target in the front video frame and the rear video frame is matched.
3) And updating the Kalman tracker by using the matched target detection Box in the second frame, calculating Kalman gain, state updating and covariance updating, and outputting a state updating value as a tracking Box of the second frame. And (3) re-initializing the tracker for the target which is not matched in the second frame, and allocating a new ID.
4) Repeating the second step and the third step until the video is finished.
Target detection and processing
For the same frame of image, detection is respectively carried out based on a human body detection model, a safety helmet wearing model and a safety helmet not wearing model, and a YOLOv3 or R-CNN similar algorithm can be adopted to obtain corresponding target detection results, namely target positions (image areas), types and type confidence degrees. Establishing a target set:
S P (x) = { all human targets in the current frame }, assuming that the current frame is F (x) }, the previous frame is denoted F (x-1), and the next frame is denoted F (x+1); each target is expressed as T (x, y), wherein y is ID or 0 of target tracking, and the human body target is not used as a tracking target and is only used as a basis for alarm decision output.
S H (x) = { head targets of all helmets worn in current frame }, each target is denoted as H1 (x, z), where z is the ID of target tracking.
S N (x) = { head targets of all unworn helmets in current frame }, each target is denoted H2 (x, z), where z is the ID of target tracking.
For set S H (x) And S is equal to N (x) And (3) voting decisions are made using a random forest classifier to determine the final class.
Random forest classifier design
A random forest is a classifier composed of a plurality of decision trees, and its classification decision is determined by the majority of the classification results of the sub decision trees constituting the forest, and is an integrated learning method based on the decision trees, as shown in fig. 4. Compared with other classification algorithms, the random forest can better tolerate noise and has better generalization performance. The construction of random forests is as follows:
1) Setting the number T of decision trees to be constructed of a random forest;
2) Bootstrap resampling is carried out on the sample data to generate a plurality of sample subsets; that is, randomly taking out one sample from N samples at a time, thus taking N times, and finally obtaining N samples, wherein repeated samples are possibly taken;
3) Random extraction is used to construct the features of the decision tree: randomly selecting m features from all candidate features each time, and taking the m features as candidate features for decision under the current node;
4) Constructing a decision tree by taking each resampled sample set as a training sample by utilizing the selected representative features;
5) After a plurality of decision trees with the number given in advance are obtained, voting is respectively carried out on the output result of each tree, and the decision with the largest number of votes is used as the final decision output of the random forest.
For the same target, the designed random forest classifier selects the category and the category confidence coefficient of the target in the current frame, the category and the category confidence coefficient of the target in the previous frame (if the target is a new ID, the random forest classifier is not passed, the voting decision of the next frame is waited), the straight line distance of the same target in the current frame and the previous frame, the Circular LBP characteristic and the color histogram characteristic of the image area where the target is located as classification basis to train the random forest classifier.
The Circular LBP is improved from a common LBP feature, and the LBP feature is an operator for describing local features of the image, and has the remarkable advantages of gray invariance, rotation invariance and the like. As shown in fig. 5, when the brightness change is relatively obvious, the LBP feature of the image is very small, so that the feature information of the helmet in the outdoor scene can be better reflected. The color histogram is a global feature describing the surface properties of the scene to which the image or image area corresponds, and is a color histogram of different color helmets as shown in fig. 6, since the helmet object has several fixed classes of colors.
In this example, the random forest classifier training steps are as follows:
1) For an image to be detected, firstly, a trained YOLOv3 model is used for calculating and reasoning the image to be detected, and the position, the category and the confidence information of the target are obtained.
2) And for the target detected by YOLOv3 in the current frame, the LBP characteristic and the color histogram characteristic of the target are acquired, and the target in the previous frame is associated by utilizing an SORT algorithm.
3) Aiming at the target obtained by YOLOv3 positioning in the current frame, selecting a category, category confidence, category and category confidence of the same target in the previous frame, straight line distance of the same target in the previous frame and LBP characteristics and color histogram characteristics of a target area as characteristic attributes to train a random forest classifier.
And classifying the detection result of each frame by using a trained random forest classifier, and determining a final detection result according to the final classification result of the decision tree.
Processing of decision output
Alarming the detected personnel without the safety helmet in the monitoring video, tracking the detected personnel without the safety helmet to form a moving track diagram, and designing an alarming logic in order to further reduce the influence of false detection and omission of an algorithm and frequent alarming on a system:
assuming that the current frame is F (x), counting the number of human body targets detected in the frame F (x) as P (x), the number of head targets wearing the safety helmet as H (x), and the number of head targets not wearing the safety helmet as N (x);
if N (x) =0 and H (x) =p (x), then it is indicated that there is no person not wearing the helmet, no alarm is required;
if N (x) =0 and H (x) > P (x), there is a large probability of missed detection, and the head may be blocked, and the marked frame F (x) is an abnormal frame;
if N (x) =0 and H (x) < P (x), it is indicated that there may be a serious human body occlusion in the current frame, and the marked frame F (x) is an abnormal frame;
if N (x) >0, the personnel without the safety helmet exist in a large probability, a target T (x, y) without the safety helmet in a set S (x) = { current frame F (x) is established, and a marked frame F (x) is an alarm frame;
for the above information, a sliding window W (m) is established for counting the alarm results of consecutive m frames (where m can be set according to the situation), and if the targets without the safety helmet appear continuously in the m frames, the targets in the set are alerted, including the generation of the video before the alert is triggered and the moving track of the alert target. For the continuously occurring abnormal frames, the alarm can be set according to the user demand, so that the operator on duty can pay attention to the possible abnormality at any time.
In this example, an alarm is given for a worker who is not wearing a helmet; continuous detection and detection can be performed according to user settings, and multiple groups of alarm information are generated; and the track tracking of personnel not wearing the safety helmet is realized, and alarm information and video are formed.
The embodiment also provides computer equipment. The computer device 20 of the present embodiment includes at least, but is not limited to: a memory 21, a processor 22, which may be communicatively coupled to each other via a system bus, as shown in fig. 7. It should be noted that fig. 7 only shows a computer device 20 having components 21-22, but it should be understood that not all of the illustrated components are required to be implemented, and that more or fewer components may be implemented instead.
In the present embodiment, the memory 21 (i.e., readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 21 may be an internal storage unit of the computer device 20, such as a hard disk or memory of the computer device 20. In other embodiments, the memory 21 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device 20. Of course, the memory 21 may also include both internal storage units of the computer device 20 and external storage devices. In this embodiment, the memory 21 is typically used to store an operating system and various types of application software installed on the computer device 20, such as program codes of a method for detecting the wearing condition of the helmet. Further, the memory 21 may be used to temporarily store various types of data that have been output or are to be output.
The processor 22 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 22 is generally used to control the overall operation of the computer device 20. In this embodiment, the processor 22 is configured to execute the program code stored in the memory 21 or process data, such as the program code of the method for detecting the wearing condition of the helmet, so as to implement the method for detecting the wearing condition of the helmet.
The present embodiment also provides a computer-readable storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., on which a computer program is stored, which when executed by a processor, performs the corresponding functions. The computer readable storage medium of the present embodiment is used for storing a program for detecting a wearing condition of a helmet, and the steps of the method for detecting a wearing condition of a helmet are implemented when the program is executed by a processor.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.
Claims (9)
1. A method of detecting a helmet wear condition, comprising:
acquiring a time sequence image of a preset area;
detecting head targets in each frame of image to obtain head targets contained in each frame of image, and categories, category confidence degrees and image areas where the head targets are located, wherein the categories comprise: a head target wearing a helmet and a head target not wearing a helmet;
performing multi-target tracking on all detected head targets, and associating the same head target in multi-frame images;
extracting preset image features of an image area where each head target is located for each head target detected in each frame of image;
for each head target contained in each frame of image, determining the category of each head target corresponding to the current image by using a random forest classifier according to the category and the category confidence degree detected by each head target in the current image and the previous image of the current image and the preset image features extracted from the current image;
further comprises:
for each head target detected by each frame of image, determining a linear distance between an image area where each head target is located in the current image and an image area where each head target is located in a preceding image of the current image;
for each head target contained in each frame of image, determining the category of each head target corresponding to the current image by using a random forest classifier according to the category and the category confidence degree detected by each head target in the current image and the previous image of the current image, the preset image features extracted from the current image and the straight line distance.
2. The method as recited in claim 1, further comprising: for the newly detected head object in each frame of image, determining in a subsequent image that the newly detected head object corresponds to a category of the subsequent image using the random forest classifier.
3. The method of claim 2, wherein the preset image features comprise: image local texture features and/or color histogram features.
4. A method according to claim 3, characterized in that the image local texture features are described using an LBP operator.
5. The method of claim 1, wherein acquiring a time series image of the predetermined area comprises:
acquiring a video acquired by a camera, wherein the visual field of the camera covers a preset area;
and extracting multi-frame images from the video according to preset conditions to obtain a time sequence image of the preset area.
6. The method according to any one of claims 1 to 5, further comprising: detecting a human body target in each frame of image; and alarming the wearing condition of the safety helmet according to the number of head targets and the number of human targets in each frame of image.
7. The method according to any one of claims 1 to 5, further comprising:
detecting a human body in each frame of image to obtain a human body target contained in each frame of image;
for each frame of image, if the number of head targets determined to be unworn with the safety helmet is 0 and the number of head targets determined to be unworn with the safety helmet is not equal to the number of detected human body targets, marking the image as an abnormal frame; if the number of the head targets which are determined to not wear the safety helmet is larger than 0, marking the image as an alarm frame;
and alarming according to the condition that the continuous multi-frame images are marked as abnormal frames and/or alarming frames.
8. A computer device, the computer device comprising:
a memory, a processor, and a computer program stored on the memory and executable on the processor;
the computer program, when executed by the processor, carries out the steps of a method of detecting a helmet wearing condition as claimed in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that it has stored thereon a program for detecting a wearing condition of a helmet, which, when executed by a processor, implements the steps of the method for detecting a wearing condition of a helmet according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110357038.4A CN113052107B (en) | 2021-04-01 | 2021-04-01 | Method for detecting wearing condition of safety helmet, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110357038.4A CN113052107B (en) | 2021-04-01 | 2021-04-01 | Method for detecting wearing condition of safety helmet, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113052107A CN113052107A (en) | 2021-06-29 |
| CN113052107B true CN113052107B (en) | 2023-10-24 |
Family
ID=76517534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110357038.4A Active CN113052107B (en) | 2021-04-01 | 2021-04-01 | Method for detecting wearing condition of safety helmet, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113052107B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658219A (en) * | 2021-07-22 | 2021-11-16 | 浙江大华技术股份有限公司 | High-altitude parabolic detection method, device and system, electronic device and storage medium |
| CN113553963A (en) * | 2021-07-27 | 2021-10-26 | 广联达科技股份有限公司 | Detection method and device of safety helmet, electronic equipment and readable storage medium |
| CN113743214B (en) * | 2021-08-02 | 2023-12-12 | 国网安徽省电力有限公司超高压分公司 | Intelligent cradle head camera |
| CN113554682B (en) * | 2021-08-03 | 2023-03-17 | 同济大学 | Target tracking-based safety helmet detection method |
| CN116958707B (en) * | 2023-08-18 | 2024-04-23 | 武汉市万睿数字运营有限公司 | Image classification method, device and related medium based on spherical machine monitoring equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108288033A (en) * | 2018-01-05 | 2018-07-17 | 电子科技大学 | A kind of safety cap detection method merging multiple features based on random fern |
| CN110602449A (en) * | 2019-09-01 | 2019-12-20 | 天津大学 | Intelligent construction safety monitoring system method in large scene based on vision |
| CN110852283A (en) * | 2019-11-14 | 2020-02-28 | 南京工程学院 | Helmet wearing detection and tracking method based on improved YOLOv3 |
| CN111191581A (en) * | 2019-12-27 | 2020-05-22 | 深圳供电局有限公司 | Safety helmet detection method and device based on electric power construction and computer equipment |
| CN111723749A (en) * | 2020-06-23 | 2020-09-29 | 广东电网有限责任公司 | Method, system and equipment for identifying wearing of safety helmet |
| CN111815577A (en) * | 2020-06-23 | 2020-10-23 | 深圳供电局有限公司 | Method, device, equipment and storage medium for processing safety helmet wearing detection model |
| CN111913799A (en) * | 2020-07-14 | 2020-11-10 | 北京华夏启信科技有限公司 | Video stream online analysis task scheduling method and computer equipment |
| CN111914636A (en) * | 2019-11-25 | 2020-11-10 | 南京桂瑞得信息科技有限公司 | Method and device for detecting whether pedestrian wears safety helmet |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10565439B2 (en) * | 2017-10-10 | 2020-02-18 | Caterpillar Inc. | Method and system for tracking workers at worksites |
-
2021
- 2021-04-01 CN CN202110357038.4A patent/CN113052107B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108288033A (en) * | 2018-01-05 | 2018-07-17 | 电子科技大学 | A kind of safety cap detection method merging multiple features based on random fern |
| CN110602449A (en) * | 2019-09-01 | 2019-12-20 | 天津大学 | Intelligent construction safety monitoring system method in large scene based on vision |
| CN110852283A (en) * | 2019-11-14 | 2020-02-28 | 南京工程学院 | Helmet wearing detection and tracking method based on improved YOLOv3 |
| CN111914636A (en) * | 2019-11-25 | 2020-11-10 | 南京桂瑞得信息科技有限公司 | Method and device for detecting whether pedestrian wears safety helmet |
| CN111191581A (en) * | 2019-12-27 | 2020-05-22 | 深圳供电局有限公司 | Safety helmet detection method and device based on electric power construction and computer equipment |
| CN111723749A (en) * | 2020-06-23 | 2020-09-29 | 广东电网有限责任公司 | Method, system and equipment for identifying wearing of safety helmet |
| CN111815577A (en) * | 2020-06-23 | 2020-10-23 | 深圳供电局有限公司 | Method, device, equipment and storage medium for processing safety helmet wearing detection model |
| CN111913799A (en) * | 2020-07-14 | 2020-11-10 | 北京华夏启信科技有限公司 | Video stream online analysis task scheduling method and computer equipment |
Non-Patent Citations (2)
| Title |
|---|
| A Review on Deep Learning Based Helmet Detection;Arya K M 等;《ICSEE》;全文 * |
| 基于深度学习的安全帽佩戴检测与跟踪;秦嘉 等;计算机与现代化(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113052107A (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113052107B (en) | Method for detecting wearing condition of safety helmet, computer equipment and storage medium | |
| CN111079663B (en) | High-altitude parabolic monitoring method and device, electronic equipment and storage medium | |
| CN108053427B (en) | Improved multi-target tracking method, system and device based on KCF and Kalman | |
| CN108062349B (en) | Video monitoring method and system based on video structured data and deep learning | |
| CN108009473B (en) | Video structuralization processing method, system and storage device based on target behavior attribute | |
| CN108052859B (en) | Abnormal behavior detection method, system and device based on clustering optical flow characteristics | |
| US7479980B2 (en) | Monitoring system | |
| US8744125B2 (en) | Clustering-based object classification | |
| JP5301973B2 (en) | Crime prevention device and program | |
| US9111148B2 (en) | Unsupervised learning of feature anomalies for a video surveillance system | |
| US6774905B2 (en) | Image data processing | |
| CN109377694B (en) | Monitoring method and system for community vehicles | |
| JP2004534315A (en) | Method and system for monitoring moving objects | |
| CN110738178A (en) | Garden construction safety detection method and device, computer equipment and storage medium | |
| CN109255360B (en) | Target classification method, device and system | |
| CN111191507A (en) | Safety early warning analysis method and system for smart community | |
| CN109963113B (en) | Method and device for monitoring interested target | |
| CN116704411A (en) | Security control method, system and storage medium based on Internet of things | |
| CN116778673A (en) | Water area safety monitoring method, system, terminal and storage medium | |
| CN110969642B (en) | Video filtering method and device, electronic equipment and storage medium | |
| CN112052878A (en) | Radar shielding identification method and device and storage medium | |
| CN114973573A (en) | Target intrusion determination method and device, storage medium and electronic device | |
| SE519700C2 (en) | Image Data Processing | |
| CN112347989A (en) | Reflective garment identification method and device, computer equipment and readable storage medium | |
| EP1261951B1 (en) | Surveillance method, system and module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |