CN115410111A - A helmet detection method based on structural reparameterization for edge devices - Google Patents

Abstract

The invention relates to a safety helmet detection method applied to edge devices based on structural reparameterization, comprising the following steps: 1) constructing a dense reparameterization module; 2) constructing a standard YOLOv3-tiny model and training data suitable for safety helmet detection 3) Refactor and train the standard YOLOv3-tiny model; 4) Equivalently convert the trained reconstructed model into an inference model, and perform helmet detection. Compared with the prior art, the present invention has the advantages of high real-time performance, high accuracy and strong generalization ability, can avoid gradient dispersion and gradient explosion, reduce feature redundancy, and improve network learning ability.

Description

A Hard Hat Detection Method Based on Structure Reparameterization for Edge Devices

technical field

The invention relates to the technical fields of deep neural network structure reparameterization technology and target detection technology, in particular to a safety helmet detection method applied to edge devices based on structure reparameterization.

Background technique

Engineering, construction and other industries are typical labor-intensive industries. Their working environment is complex and safety accidents occur frequently. The death of brain trauma caused by objects falling from high altitude is a common typical accident in the construction industry. As an effective safety protection equipment, safety helmets can It can block the impact energy of falling objects from high places and reduce head shock injuries, and is widely used in construction sites. In construction site safety management, real-time and accurate supervision of helmet wearing behavior is an extremely important link. Site safety management requires detection equipment to be small in size and highly mobile, so embedded edge computing devices are often used, which leads to the lack of real-time and accuracy of the helmet detection algorithm.

Many works at home and abroad have studied the automatic identification technology of helmets. Dalal et al. first proposed to realize the automatic detection of helmets by extracting gradient histogram features; Feng Jie et al. combined with Adaboost classifier to detect the position of helmets, Judging whether to wear a helmet or not based on the positional relationship between the person and the helmet; Hu Tian et al. put forward a neural network helmet identification model based on the analysis of wavelet transform and deep learning in the application of helmet identification; Liu Xiaohui et al. The detection method locates the face, and then uses the support vector machine (SVM) to realize the identification of the helmet; Liu Yunbo et al. judge whether to wear a helmet by detecting the distribution of pixel chromaticity values in the upper 1/3 part of the moving target. Although the above methods can achieve more accurate identification of helmets in specific scenarios, there are still problems such as high environmental requirements, poor real-time performance, weak generalization ability, and complicated user operation processes.

In recent years, with the in-depth research and popularization of deep learning technology, the deep network target detection method represented by the YOLOv3 (You Only LookOnce V3) model not only has good real-time performance, but also has high accuracy. However, complex and diverse convolutional structures lead to fragmented network structures, increased network complexity, high feature redundancy, low memory access efficiency, and poor flexibility, which seriously hinders the YOLOv3-based helmet detection algorithm in weak computing power. , Deploy applications on edge smart devices with low memory.

Contents of the invention

The purpose of the present invention is to provide a safety helmet detection method applied to edge devices based on structure reparameterization in order to overcome the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved through the following technical solutions:

A safety helmet detection method applied to an edge device based on structural reparameterization, the method comprising the following steps:

1) Construct a dense reparameterization module;

2) Construct a standard YOLOv3-tiny model and training data set suitable for helmet detection;

3) Refactor and train the standard YOLOv3-tiny model;

4) Equivalently convert the trained reconstruction model into an inference model, and perform helmet detection.

Described step 1) specifically comprises the following steps:

11) Define the basic unit;

12) Construct transformation structure;

13) Build a learning structure;

14) Construct a dense reparameterization module DR-Block: The dense reparameterization module DR-Block is composed of a transformation structure cascaded with a learning structure. The parameter of the transformation structure is F _trans (x;C _{rep_in} ), and the learning structure The parameter of is F _learn (x _learn ;2×C _{rep_in} ,C _{rep_out} ,K _rep ,S _rep ), and x _learn =F _trans (x;C _{rep_in} ), then the construction of dense reparameterization module DR-Block is recorded as F _rep (x; C _{rep_in} , C _{rep_out} , K _rep , S _rep ).

In the step 11), the basic unit is composed of a convolutional layer cascaded with a batch normalization layer, which is denoted as F _basic (x; C _{basic_in} , C _{basic_out} , K _basic , S _basic ), where x is Input, C _{basic_in} is the number of input channels, C _{basic_out} is the number of output channels, K _basic is the convolution kernel size, and S _basic is the step size. Specifically, the number of input channels of the convolutional layer is C _{basic_in} , the number of output channels is C _{basic_out} , the size of the convolution kernel is K _basic , and the number of channels of the batch normalization layer is C _{basic_out} .

Described step 12) is specifically:

First, 4 basic units are cascaded to form a deep cascaded network structure to achieve over-parameterization of the network, then a jump connection is added between any two basic units to achieve model integration of different levels of features, and finally the output of each basic unit is Spliced together as one, denoted as F _trans (x;C _{trans_in} ).

In the step 13), the learning structure is composed of one basic unit, denoted as F _learn (x; C _{learn_in} , C _{learn_out} , K _learn , S _learn ).

Described step 2) specifically comprises the following steps:

21) Collect and mark the safety helmet detection data set in the construction site and network pictures, and perform standard data preprocessing;

22) Build a standard YOLOv3-tiny model, and set the number of detection categories to 2, specifically human body without helmet and human body with helmet.

Described step 3) specifically comprises the following steps:

31) For all non-1×1 convolutional layers and their cascaded batch normalization layers in the standard YOLOv3-tiny model, replace them with the dense reparameterization module DR-Block, and the replaced reconstruction model is denoted as DR-Net; For the standard YOLOv3-tiny model, the number of input channels is C _in , the number of output channels is C _out , the convolution kernel is K (K≠1), and the step size is S. Replace it with F _rep (x;C _in , C _out ,K,S);

32) Using the helmet detection data set, the reconstructed model DR-Net is trained through the YOLOv3-tiny standard training parameters and training strategy.

Described step 4) specifically comprises the following steps:

偏置为

41) Convert the basic unit into a single convolutional layer, the weight of the convolution in the basic unit is denoted as w _conv , the mean value of the batch normalization layer is denoted as μ, the standard deviation is σ, the scaling factor is γ, and the offset coefficient is β , then the weight of the reconstructed single convolutional layer F′ _basic is

biased to

42) Convert the convolution using skip connection to a single convolutional layer, the weight of the converted single convolutional layer F′ _{skip_connect} is w′=concat([w _prev ,w]), and the bias is b′=concat( [b _prev ,b]), where concat represents the cascade operation, w _prev is the weight of the equivalent convolution, b _prev is the bias of the equivalent convolution, w is the weight of the converted convolution, and b is the converted convolution bias;

43) converting two cascaded convolutions into a single convolutional layer;

44) Transform the transformation structure into a single 1×1 convolutional layer, follow step 41) convert all convolutions in the transformation structure into a single convolutional layer, and then follow step 42) to sequentially reconstruct all skip connections into a single convolution layer, and finally a single convolutional layer F′ _trans equivalent to the entire transformation structure is obtained;

45) Convert DR-Block to a single convolutional layer;

46) Convert all DR-Blocks in DR-Net into a single convolutional layer according to step 45), so as to obtain the inference stage structure required for deployment;

47) Extract the scene image frame, input the model obtained in step 46) to carry out helmet detection, and output the detection result.

Described step 43) is specifically:

First, transpose the first and second dimensions of the weight w ₁ of the first convolution, and record the result as w ₁ ′, then the weight of the reconstructed single convolutional layer F′ _cascade is w′=Conv2d(w ₂ ,w ₁ ′), the offset is b′=b ₂ +(b ₁ ×w ₂ ), where Conv2d represents a two-dimensional convolution operation, b ₁ is the offset of the first convolution, and w ₂ is the second The weight of the first convolution, b ₂ is the bias of the second convolution.

Described step 45) is specifically:

First, convert the transformation structure into a single convolutional layer _F'trans according to step 44), and then convert the cascaded structure of the single convolutional layer _F'trans and the learning structure into a single convolutional layer _F'rep , F' according to step 43 _rep is a single convolutional layer structure after DR-Block reparameterization.

Compared with the prior art, the present invention has the following advantages:

1. In view of the lack of real-time performance of the existing hard hat detection algorithm based on deep learning, the present invention uses a yolov3-tiny network for training and reasoning to achieve high real-time performance.

2. Due to the pursuit of real-time performance, the YOLOv3-tiny network has shallow network depth and low accuracy rate. The present invention adopts the network re-parameterization method to decouple the training phase and the reasoning phase of the network, and use complex structures to model in the training phase. Learning, improve the accuracy of the network, convert the complex structure equivalently into the original yolov3-tiny simple structure in the reasoning stage, and restore the real-time performance of the network.

3. Due to the limited training data and insufficient generalization ability of the model, the present invention realizes over-parameterization of the network and increases the capacity of the model by adding a deep cascade structure into the re-parameterization structure of the network, thereby generating an implicit regularization effect on the network. The generalization ability of the original model is improved.

4. Due to the large number of layers in the deep network, there are dispersion and explosion problems in the gradient return, the training is slow and difficult to converge, the existing method does not substantially change the network depth, and does not perform additional processing on the return gradient, and the present invention passes The multi-layer cascaded batch normalization layer adjusts the multi-level distribution of the return gradient, which better ensures the effectiveness of the gradient return and avoids gradient dispersion and gradient explosion to a certain extent.

5. Due to the large number of deep network parameters, there is a lot of redundancy, and the learning ability of the network is weakened. The existing methods mainly enhance the expressive ability of a single convolutional layer by designing multi-branch structures of different scales and complexity, but introduce a large number of Feature redundancy affects the ability to improve performance. This method realizes model integration of different levels of features through dense connections, greatly reduces feature redundancy, improves the learning ability of the network, and further improves the accuracy of the original model. Rate.

Description of drawings

Fig. 1 is the flow chart of method design of the present invention.

Figure 2 is a diagram of DR-Block structure and reparameterization transformation.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

Example

As shown in Figure 1, the present invention provides a safety helmet detection method applied to edge devices based on structural reparameterization, including the following steps:

First, build a dense reparameterization module as follows

S1. Define the "basic unit": the parameters of the "basic unit" are: the input is x, the number of input channels C _{basic_in} , the number of output channels C _{basic_out} , the size of the convolution kernel K _basic , and the step size is S _basic , the "basic unit""Includes 1 convolutional layer cascaded with 1 batch normalization layer, where the number of input channels of the convolutional layer is C _{basic_in} , the number of output channels is C _{basic_out} , and the size of the convolution kernel is K _basic ; the role of the batch normalization layer The number of channels is C _{basic_out} , and the "basic unit" is F _basic (x;C _{basic_in} ,C _{basic_out} ,K _basic ,S _basic );

然后在任意两个“基本单元”之间添加跳跃连接，用于实现不同层级特征的模型集成，从而降低特征冗余度；最后将每个“基本单元”的输出拼接为一体，记“变换结构”为F_trans(x;C_{trans_in})；S2. On the basis of S1, build a "transformation structure": the parameters of the "transformation structure" are x, the number of input channels C _{trans_in} , and the number of output channels 2×C _{trans_in} . The "transformation structure" includes: first cascade 4 A "basic unit" to form a deep cascaded network structure, which is used to achieve network overparameterization, introduce implicit regularization effects, and increase model capacity; among them, the introduction of multi-layer batch normalization layers makes reparameterization The module effectively adjusts the gradient of the network return to avoid gradient disappearance and gradient explosion. Among them, the parameter of the i-th "basic unit" is

Then add a skip connection between any two "basic units" to achieve model integration of different levels of features, thereby reducing feature redundancy; finally, the output of each "basic unit" is spliced into one, and the "transformation structure" is recorded. ” is F _trans (x; C _{trans_in} );

S3. On the basis of S1, define a "learning structure" for extracting important features and expanding the receptive field: the parameters of the "learning structure" are x, the number of input channels C _{learn_in} , the number of output channels C _{learn_out} , convolution The kernel size is K _learn and the step size is S _learn . The structure includes: 1 "basic unit" whose parameters are F _learn (x;C _{learn_in} ,C _{learn_out} ,K _learn ,S _learn );

S4. On the basis of S2 and S3, build a "densely reparametrized block (densely reparametrized block)", which is recorded as DR-Block. The parameters of this DR-Block are: input is x, input channel number C _{rep_in} , output channel number C _{rep_out} , convolution kernel size K _rep , step size S _rep . The DR-Block includes a "transformation structure" concatenated with a "learning structure", wherein the parameter of the "transformation structure" is F _trans (x;C _{rep_in} ), and the parameter of the "learning structure" is F _learn (x _learn ;2×C _{rep_in} , C _{rep_out} , K _rep , S _rep ), wherein, x _learn = F _trans (x; C _{rep_in} ), the DR-Block is recorded as F _rep (x; C _{rep_in} , C _{rep_out} , K _rep , S _rep ).

Second, build a YOLOv3-tiny model and training data set suitable for helmet detection

S5. Collect and mark the safety helmet detection data set in the construction site and network pictures, and perform standard data preprocessing;

S6. Build a standard YOLOv3-tiny model, and set the number of detection categories to 2 (two categories: human body without helmet and human body with helmet);

Third, follow the steps below to reconstruct and train the original YOLOv3-tiny model

S7. Given the network model built in S6, replace all non-1×1 convolutional layers and their cascaded batch normalization layers in the original model with DR-Block, and the replaced model is denoted as DR-Net, For the convolution whose original parameter is the number of input channels is C _in , the number of output channels is C _out , the convolution kernel is K (K≠1), and the step size is S, replace it with F _rep (x;C _in ,C _out , K, S);

S8. Use the data set made in S5, and use the YOLOv3-tiny standard training parameters and training strategy to train the reconstructed model in S7;

Finally, follow the steps below to equivalently convert the trained reconstruction model into an inference model and perform detection

偏置为

S9. Convert the "basic unit" into a single convolutional layer. The weight of the convolution in the "basic unit" is denoted as w _conv ; the mean value of the batch normalization layer is denoted as μ, the standard deviation is σ, the scaling factor is γ, and the partial The shift coefficient is β; then the weight of the reconstructed single convolutional layer F′ _basic is

biased to

S10, based on S9, convert the convolution using skip connections into a single convolutional layer. The input of the structure can be expressed as the original input x obtained by equivalent convolution F _prev , and the weight of the equivalent convolution is denoted as w _prev , and the bias is denoted as b _prev ; the weight of the convolution in the structure is denoted as w, The bias is denoted as b. Then the weight of the converted single convolutional layer F′ _{skip_connect} is w′=concat([w _prev ,w]), and the bias is b′=concat([b _prev ,b]), where concat represents a cascade operation ;

S11. Based on S10, convert the two cascaded convolutions into a single convolutional layer. Denote the weight of the first convolution as w ₁ and the bias as b ₁ ; denote the weight of the second convolution as w ₂ and the bias as b ₂ . First, the first and second dimensions of w ₁ are transposed, and the result is recorded as w ₁ ′, then the weight of the reconstructed single convolutional layer F′ _cascade is w′=Conv2d(w ₂ ,w ₁ ′), The bias is b'=b ₂ +(b ₁ ×w ₂ ), where Conv2d represents a two-dimensional convolution operation.

S12: Based on S11, convert the "transform structure" into a single 1×1 convolutional layer. First, convert all convolutions in the "transformation structure" into a single convolutional layer according to S9; then reconstruct all skip connections into a single convolutional layer in sequence according to S10; finally obtain a single convolution equivalent to the entire "transformation structure" Multiply F′ _trans .

S13. Based on S12, convert the DR-Block into a single convolutional layer. First, convert the "transformation structure" into a single convolutional layer _F'trans according to S12; then convert the cascade structure of _F'trans and "learning structure" into a single convolutional layer _F'rep according to S11. F' _rep is a single convolutional layer structure after DR-Block reparameterization;

S14. Finally, convert all DR-Blocks in DR-Net into a single convolutional layer according to S13, so as to obtain the inference stage structure required for deployment;

S15. Extracting the on-site image frame, inputting the model obtained in S14 for helmet detection, and outputting the detection result.

Table 1 shows the experimental results of the present invention on the hard hat detection data set. The results show that the present invention can detect edge devices in a plug-and-play manner without changing the network structure of the original method and without adding additional reasoning overhead. The accuracy of the helmet detection task on the above can be improved.

Table 1 The improvement effect of the present invention on the accuracy of the helmet detection algorithm

index AP@0.5-0.95 AP@0.5 original model 78.1 92.1 DR-Block 81.2 (↑3.1) 96.4 (↑4.3)

Since the DR-Block is plug-and-play, the present invention can be applied to various tasks and promote efficient deployment and real-time computing of deep networks on edge devices.

To sum up, the present invention aims at improving the performance of the hard hat detection algorithm on the edge device, and designs a dense linear composite structure, which is used to replace the convolution of the hard hat detection algorithm in the training phase, and reconstructed into a simple convolution of the original model in the inference phase. In order to achieve performance improvement while keeping the inference speed unchanged.

The present invention first constructs a dense network re-parameterization structure (DR-Block). This structure first realizes network over-parameterization through a deep cascading structure, increases the model capacity, thereby exerts an implicit regularization effect on the network and improves the efficiency of the network. The generalization ability of the model; secondly, the cascaded batch normalization layer is used to adjust the multi-level distribution of the return gradient, which better ensures the effectiveness of the gradient return and avoids gradient dispersion and gradient Explosion; Finally, through dense connection, the model integration of different levels of features is realized, which greatly reduces feature redundancy and improves the learning ability of the original model. Secondly, the present invention constructs a yolov3-tiny network with strong real-time performance, and adapts it to the helmet detection task. Subsequently, the present invention constructs the training and reasoning structure of the yolov3-tiny network, and uses DR-Block to replace the original network convolution and perform training in the training stage to achieve network performance improvement; finally, the DR-Block is equivalently converted to the original yolov3 -Simple convolution of the tiny network, so as to achieve performance improvement without adding additional reasoning overhead, and deploy and apply it for the helmet detection task.

Claims (10)

1. A safety helmet detection method based on structural parameterization applied to edge equipment is characterized by comprising the following steps:

1) Constructing a dense heavy parameterization module;

2) Constructing a standard YOLOv3-tiny model and a training data set which are suitable for safety helmet detection;

3) Reconstructing and training a standard YOLOv3-tiny model;

4) Equivalently converting the trained reconstruction model into a reasoning model, and detecting the safety helmet.

2. The method for detecting a safety helmet based on structural parameterization applied to edge equipment according to claim 1, wherein the step 1) specifically comprises the following steps:

11 Define basic units;

12 Constructing a transformation structure;

13 Build a learning structure;

14 Constructing a dense heavy parameterization module DR-Block: the dense heavy parameterization module DR-Block consists of 1The transformation structure is formed by cascading 1 learning structure, and the parameter of the transformation structure is F _trans (x；C _{rep_in} ) The parameter of the learning structure is F _learn (x _learn ；2×C _{rep_in} ，C _{rep_out} ，K _rep ，S _rep ) And x is _learn ＝F _trans (x；C _{rep_in} ) If the building density reparameterization module DR-Block is marked as F _rep (x；C _{rep_in} ，C _{rep_out} ，K _rep ，S _rep )。

3. The method for inspecting safety helmet based on structural parameterization applied to edge device of claim 2, wherein in the step 11), the basic unit is formed by cascading 1 convolution layer and 1 batch normalization layer, which is denoted as F _basic (x；C _{basic_in} ，C _{basic_out} ，K _basic ，S _basic ) Wherein x is the input, C _{basic_in} Is input channel number, C _{basic_out} Is the number of output channels, K _basic As the size of the convolution kernel, S _basic Is the step length; specifically, the number of input channels of the convolutional layer is C _{basic_in} The number of output channels is C _{basic_out} Convolution kernel size of K _basic The number of batch normalization layer action channels is C _{basic_out} 。

4. The method for detecting a safety helmet based on structural parameterization applied to edge equipment according to claim 3, wherein the step 12) is specifically as follows:

firstly, cascading 4 basic units to form a deep cascading network structure to realize over-parameterization of the network, then adding jump connection between any two basic units to realize model integration of different hierarchical features, and finally, splicing the output of each basic unit into a whole, and marking as F _trans (x；C _{trans_in} )。

5. The safety helmet detection method based on structural parameterization applied to the edge device according to claim 4,in the step 13), the learning structure is composed of 1 basic unit, and is marked as F _learn (x；C _{learn_in} ，C _{learn_out} ，K _learn ，S _learn )。

6. The method for detecting safety helmet based on structural parameterization applied to edge device according to claim 1, wherein the step 2) specifically comprises the following steps:

21 Collecting and labeling a safety helmet detection data set in a construction site and a network picture, and performing standard data preprocessing;

22 Building a standard YOLOv3-tiny model, and setting the number of detection categories as 2, specifically a body without a safety helmet and a body with a safety helmet.

7. The method for detecting safety helmet based on structural parameterization applied to edge equipment according to claim 6, wherein the step 3) comprises the following steps:

31 Replacing all non-1 multiplied by 1 convolution layers and cascaded batch normalization layers in the standard YOLOv3-tiny model by a dense parameterization module DR-Block, and marking the reconstructed model after replacement as DR-Net; for the standard YOLOv3-tiny model, the number of input channels is C _in The number of output channels is C _out The convolution kernel is K (K ≠ 1) and the convolution with the step length S is replaced by F _rep (x；C _in ；C _out ，K，S)；

32 Adopting a helmet detection data set, and training the reconstructed model DR-Net through a Yolov3-tiny standard training parameter and a training strategy.

8. The method for detecting safety helmet based on structural parameterization applied to edge device according to claim 2, wherein the step 4) comprises the following steps:

41 Convert the elementary units into a single convolutional layer, the weight of the convolution in the elementary units is denoted as w _conv The mean of the batch normalization layer is denoted as μ, the standard deviation is σ, the scaling factor is γ, and the deviation isA shift coefficient of beta, the reconstructed single convolutional layer F' _basic Has a weight of

Is biased to

42 Conversion of convolution using skip connection into a single convolution layer, converted single convolution layer F' _{skip_connect} Weight of (b) is w' = concat ([ w ] _prev ，w]) Bias is b '= concat ([ b' = b) _prev ，b]) Wherein concat represents a cascade operation, w _preu Weights for equivalent convolution, b _prev The offset of the equivalent convolution, w is the weight of the converted convolution, and b is the offset of the converted convolution;

43 Two concatenated convolutions are converted into a single convolutional layer;

44 Convert the transform structure to a single 1 × 1 convolutional layer, convert all convolutions in the transform structure to a single convolutional layer as per step 41), and then reconstruct all skip connections into a single convolutional layer in sequence as per step 42), finally obtaining a single convolutional layer F 'equivalent to the entire transform structure' _trans ；

45 Convert DR-Block to a single convolutional layer;

46 All DR-blocks in the DR-Net are converted into a single convolution layer according to the step 45) so as to obtain an inference phase structure required by deployment;

47 Extracting the field image frames, inputting the field image frames into the model obtained in the step 46) for safety helmet detection, and outputting a detection result.

9. The method for detecting a safety helmet based on structural parameterization applied to edge equipment according to claim 8, wherein the step 43) is specifically as follows:

first the weight w of the first convolution ₁ Transposing the first and second dimensions of (1), the result being denoted w ₁ 'then reconstructed single convolution layer F' _cascade Is w' = in weightConv2d(w ₂ ，w ₁ ') offset b' = b ₂ +(b ₁ ×w ₂ ) Where Conv2d denotes a two-dimensional convolution operation, b ₁ Offset for the first convolution, w ₂ As a weight of the second convolution, b ₂ Is the offset of the second convolution.

10. The method for detecting safety helmet based on structural parameterization applied to edge device according to claim 8, wherein the step 45) is specifically as follows:

first converting the transformed structure to a single convolutional layer F 'as per step 44)' _trans Then, a single winding of laminate F' _trans The cascade structure with the learning structure is converted to a single convolutional layer F 'according to step 43' _rep ，F′ _rep The single convolution layer structure after the DR-Block reparameterization is obtained.

Images