CN110427821A - A kind of method for detecting human face and system based on lightweight convolutional neural networks - Google Patents

Abstract

The invention belongs to the technical field of computer vision detection, and in particular relates to a face detection method and system based on a lightweight convolutional neural network. The key problem that needs to be solved in this solution is to optimize the model into a lightweight model, reduce the amount of calculation, and improve the calculation speed. Secondly, while realizing the lightweight of the network, it is necessary to ensure the accuracy of the face detection model. Therefore, it is necessary to balance the lightweight and Accuracy, it is necessary to be able to study how to improve the accuracy of the algorithm based on the lightweight face detection network model is the key problem to be solved by this solution. This case has certain advantages in terms of detection accuracy, model size, and detection speed. Compared with the face detection algorithm based on VGG16, the network guarantees a certain accuracy and has advantages in detection speed and model size.

Description

A face detection method and system based on lightweight convolutional neural network

technical field

The invention belongs to the technical field of computer vision detection, and in particular relates to a face detection method and system based on a lightweight convolutional neural network.

Background technique

Detection problem is a traditional problem of computer vision, and face detection is one of the most important ones. With the increasing maturity of computer imaging technology and the development of face detection algorithms, face detection has begun to play an increasingly important role in today's society, such as airport security check, company check-in, public place monitoring, electronic equipment password-free entry, etc. etc., face detection has its place.

Existing face detection methods can be roughly divided into two categories. One is based on template matching. By comparing the template with the picture to be detected, it is determined whether the picture is a human face, and a template is established for the invariant features of the face, such as contour features, color features, texture features, etc. The similarity of the face template determines whether the area contains a face. The algorithm based on template matching relies on the pre-established face template in the static scene. In the dynamic scene with large scale changes, the effect is relatively poor. The other is based on feature statistics, combining artificially constructed features with classifier algorithms in machine learning such as artificial neural networks and Adaboost, and performing statistical learning in a large number of face samples to determine whether a certain area in the image to be tested can The purpose of face detection is achieved through the correct classification of the classifier. The detection algorithm based on feature statistics is subject to the artificially designed feature operator, and cannot fully obtain the higher-level representation of the image.

Convolutional neural network was quickly used in face detection after its success in image classification, and its accuracy greatly surpassed the previous AdaBoost framework. Currently, there are some high-precision and efficient algorithms.

The face detection network technology based on deep learning has made a breakthrough, but there are still many deficiencies in the balance between real-time performance and accuracy. As the depth of the convolutional layer increases, it can extract better features, and the detection accuracy will increase, but the larger the calculation, the slower the detection speed, such as the SSD framework and various improved SSD models derived from it. Although the detection accuracy of the SSD model based on VGG-Net has been improved, it is difficult to meet the needs of real-time applications due to the large amount of calculation and the slow detection speed of the model. To achieve real-time face detection, it is necessary to prune and optimize the neural network, which will sacrifice a certain detection accuracy. Therefore, how to maintain the detection accuracy and compress the network at the same time to make the detection speed faster is a current issue. The main technical problems faced by face detection.

Contents of the invention

In order to solve the technical defects existing in the prior art, the present invention proposes a face detection method and system based on a lightweight convolutional neural network.

The present invention is realized through the following technical solutions:

A face detection method based on lightweight convolutional neural network, which comprises steps:

S1, data processing, establishing a face database, performing image preprocessing, and generating training samples;

S2, based on the input samples, feature extraction is performed using a lightweight convolutional neural network; the lightweight convolutional neural network includes convolutional layers with convolution kernel sizes of 3x3 and 1x1 respectively;

S3, based on the feature fusion module, the different feature layers of the lightweight convolutional neural network are fused;

S4, anchor frame selection;

S5, output the multi-scale feature map;

S6, face candidate area mapping matching;

S7, face classification regression;

S8, establishing a non-maximum suppression constraint;

S9, outputting the detection result.

Further, in the step S2, the backbone network of the lightweight convolutional neural network CNN includes: 1 A block and 5 B blocks connected in sequence, wherein the number of channels of convolution in the A block is set to 16. The number of channels of each B block is set to 16, 32, 64, 128, and 128 in sequence.

Further, the A block implements downsampling through a convolutional layer with a convolution kernel size of 3×3 and a step size of 2, and then adds a convolution layer with a convolution kernel size of 1×1 and a step size of 1 to achieve For the purpose of input feature dimensionality reduction, each convolutional layer is followed by Batch Normalization and nonlinear function ReLU for improving the nonlinear characteristics of the network.

Further, the B block implements downsampling through a maximum pooling layer with a convolution kernel size of 2×2, and its step size is 2, and then sequentially connects the convolution kernel sizes of 3×3, 1×1, and 3× 3 convolutional layers, each convolutional layer has a step size of 1, and is sequentially connected to the non-linear feature Batch Normalization and non-linear function ReLU used to improve the network.

Further, in the step S3, the feature fusion module includes a convolution layer with a convolution kernel size of 3×3 and 1×1 respectively, and a step size of 1, which uses deconvolution for upsampling, And the low-level feature map and the transformed high-level feature map are fused together by element-wise summation.

The present invention also provides a face detection system based on a lightweight convolutional neural network, based on an SSD detection framework, and adopts a designed lightweight convolutional neural network as a backbone network during feature extraction, which includes:

The data processing module is used to establish a face database, perform image preprocessing, and generate training samples;

A lightweight convolutional neural network module, based on the input samples, utilizes a lightweight convolutional neural network for feature extraction; the lightweight convolutional neural network includes convolution kernels whose sizes are 3x3 and 1x1 respectively Floor;

The feature fusion module is used to fuse different feature layers of the backbone network and output multi-scale feature maps;

Anchor box selection module, used to set the aspect ratio of the anchor box.

Preferably, the lightweight convolutional neural network includes: 1 A block and 5 B blocks connected in sequence, wherein the number of channels of convolution in the A block is set to 16, and the number of channels of each B block is set to 16, 32, 64, 128, 128.

Preferably, the A block implements downsampling through a convolutional layer with a convolution kernel size of 3×3 and a step size of 2, and then adds a convolution layer with a convolution kernel size of 1×1 and a step size of 1 to achieve For the purpose of input feature dimensionality reduction, each convolutional layer is followed by Batch Normalization and nonlinear function ReLU for improving the nonlinear characteristics of the network.

Preferably, the B block implements downsampling through a maximum pooling layer with a convolution kernel size of 2×2, and its step size is 2, and then sequentially connects convolution kernel sizes of 3×3, 1×1, and 3× 3 convolutional layers, each convolutional layer has a step size of 1, and is sequentially connected to the non-linear feature Batch Normalization and non-linear function ReLU used to improve the network.

Preferably, the feature fusion module includes a convolution layer with convolution kernel sizes of 3×3 and 1×1 respectively, and the step size is 1, which uses deconvolution for upsampling, and adopts the method of element summation The low-level feature maps and the transformed high-level feature maps are fused together.

Compared with the prior art, the present invention has at least the following beneficial effects or advantages: the program starts from the design of a lightweight face detection network, and analyzes the face detection on the basis of the original deep learning neural network model theoretical research. The key constraints of the neural network model in terms of size and operation efficiency, through the analysis of the key constraints, optimize the network; in the SSD framework, the designed lightweight network is used as the backbone network for face detection, reducing the number of models storage space, reducing the computer memory occupation during model establishment and initialization; the fusion of multiple convolutional layer features is realized on the convolutional network, and the designed feature fusion module is used in a lightweight face detection network, so that The algorithm has a certain improvement in accuracy; compared with the traditional SSD detection algorithm, considering the particularity of the shape and proportion of the face, this solution sets the aspect ratio of the detection frame to 1 to reduce complex network calculations and improve deep learning. The computational efficiency of neural networks.

Description of drawings

The present invention will be described in further detail below in conjunction with accompanying drawing;

Fig. 1 (a) is A block structural diagram of the present invention;

Fig. 1 (b) is the B block structural diagram of the present invention;

Fig. 1 (c) is the backbone network structural diagram of face detection of the present invention; Fig. 2 is the feature fusion module diagram of the present invention;

Fig. 3 is a flow chart of face detection in the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The mainstream convolutional neural network model has deep layers, large parameters, and a huge amount of calculation, which cannot realize real-time detection. According to experiments, the factors that affect the time-consuming in convolution calculation are mainly divided into two aspects: the dimension of the input feature map and the volume The number of cores. The larger the width and height of the input feature map, the more times the convolution kernel needs to slide and calculate in the horizontal and vertical directions; the larger the number of channels of the input feature map, the convolution kernel needs linear combination calculations for each local connection calculation. The more parameters; in the forward operation, the input features need to be combined and calculated with each neuron in the convolutional neural network. The more convolution kernels are included, the more iterative calculations with the convolution kernels will be during the calculation of this layer. big.

From the above analysis, it can be seen that the convolutional neural network model can improve the performance of the model by increasing the depth of the model and the number of convolution kernels in each layer, but it brings the problem of increased calculation; if only using a smaller resolution image As an input, too little information will make the network underfit. To solve this problem, this scheme is based on the SSD detection framework, and adopts the designed lightweight neural network in feature extraction, which can reduce the network scale (parameter amount, calculation amount) while ensuring a certain recognition accuracy. Design the network to replace the 3x3 convolution kernel with a 1x1 convolution kernel, which reduces the parameter amount by 9 times, but this may affect the effectiveness of the extracted features of the convolution network. In order to ensure the recognition efficiency, only Partial replacement; at the same time, the network reduces the channel size of the input data, which can reduce the amount of parameters. The structure of the backbone part of the network is shown in Figure 1(c). A block (as shown in Figure 1(a)) implements downsampling through a convolutional layer with a convolution kernel size of 3×3 and a step size of 2, followed by Batch Normalization and nonlinear function ReLU are used to improve the nonlinear characteristics of the network, and then add a 1×1 convolution layer to achieve the purpose of dimensionality reduction of input features, thereby optimizing the calculation time of the model, and the number of convolutional channels in A block Set to 16. B block (as shown in Figure 1(b)) achieves downsampling through the 2×2 maximum pooling layer, with a step size of 2, and then sequentially connects the convolution kernels with sizes of 3×3, 1×1, and 3×3 The convolutional layer, each convolutional layer has a step size of 1, followed by Batch Normalization and nonlinear function ReLU to improve the nonlinear expression ability of the model. In the entire backbone network, a total of 5 B blocks are used, and the number of channels is set to 16, 32, 64, 128, and 128 in sequence.

The different detection branches in the SSD structure are independent of each other, so it is easy to have the problem that the same object is detected by the detection frames of different sizes at the same time. In this case, the use of feature fusion can increase the connection between different layers and reduce the appearance of repeated boxes. On the other hand, the low-level feature semantic information is relatively small, and more attention is paid to detailed information, and the target position is accurate; the high-level feature semantic information is relatively rich, but the target The position is relatively rough, and the feature fusion module utilizes the high-resolution of low-level features and high-level semantic information of high-level features at the same time, and achieves the prediction effect by fusing the features of these different layers, and the prediction is performed separately on each fused feature layer , which can improve the detection accuracy of small-sized faces.

The feature fusion module is shown in Figure 2. The number of channels of Conv3×3 and Conv1×1 is 256, and the step size is 1. Deconvolution is used for upsampling, and element-wise sum is used. The method fuses the low-level feature maps and the transformed high-level feature maps together. Through the designed feature fusion module, the different feature layers of the backbone network are fused, and the corresponding feature maps of different scales can be output, and finally output together with the auxiliary convolutional layer added after the backbone network as the candidate frame of each type of detection frame. To achieve the purpose of improving the detection accuracy, while ensuring a smaller model and higher operating efficiency.

In this case, different feature maps of a single network can use different sizes of detection frames to achieve multi-scale detection effects. At the same time, parameters can be shared to reduce the amount of calculation and improve the detection speed. In theory, the receptive field of the detection frame is uniform. In fact, the middle input has a heavier impact on the output, similar to a Gaussian distribution. According to the above theory and the detection frame design method of SSD, this technology sets the detection frame conforming to multi-scale face detection. And after analyzing the face data, most of the faces are square. Due to the characteristics of the shape of the face, in this case, the aspect ratio of the detection frame is set to 1, which can reduce the number of network calculations and obtain, thereby improving the detection speed.

The face detection process is shown in Figure 3. This case is based on the SSD detection framework, and the designed lightweight neural network is used as the backbone network in feature extraction, which greatly reduces the parameters of the neural network. Secondly, we will design the lightweight neural network as the backbone network. The fusion of multiple convolutional features in the magnitude convolution network improves the accuracy of the algorithm, maintains the multi-scale prediction characteristics of SSD, and improves the accuracy of detection, thereby ensuring the accuracy of network face detection. Compared with the face detection algorithm based on VGG16, this network has certain advantages in comprehensive factors such as algorithm accuracy, detection speed, and model size. In the setting of the detection frame, the shape of the general object is uncertain, so SSD sets the anchor point frame with different ratios, but due to the particularity of the face, the ratio of the general face is fixed. In order to reduce unnecessary calculations, this paper sets the aspect ratio of the detection frame to 1, which can reduce the number of network calculations and thus improve the detection speed.

The present invention also provides a computer-readable storage medium on which a computer program is stored, which is characterized in that, when the program is executed by a processor, the steps of the face detection method based on a lightweight convolutional neural network are realized.

The present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements a lightweight convolutional neural network-based The steps of the face detection method.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. protected range. Without departing from the spirit and scope of the present invention, any modifications, equivalent replacements, improvements, etc., also fall within the protection scope of the present invention.

Claims (10)

1. a kind of method for detecting human face based on lightweight convolutional neural networks, which is characterized in that comprising steps of

Face database is established in S1, data processing, carries out image preprocessing, generates training sample；

S2, the sample based on input carry out feature extraction using lightweight convolutional neural networks；The lightweight convolution mind It include the convolutional layer that convolution kernel size is respectively 3x3 and 1x1 through network；

S3 is merged the different characteristic layer of lightweight convolutional neural networks based on Fusion Features module；

S4, anchor point frame are chosen；

S5 exports Analysis On Multi-scale Features figure；

S6, the mapping matching of face candidate area；

S7, face classification return；

S8 establishes non-maxima suppression constraint；

S9, output test result.

2. the method for detecting human face according to claim 1 based on lightweight convolutional neural networks, which is characterized in that in institute It states in step S2, the core network of the lightweight convolutional neural networks CNN includes: sequentially connected 1 A block and 5 B Block, wherein the port number of convolution is set as 16 in A block, the port number of each B block sets gradually as 16,32, 64、128、128。

3. the method for detecting human face according to claim 2 based on lightweight convolutional neural networks, which is characterized in that described A block by convolution kernel size be 3 × 3, step-length be 2 convolutional layer realize it is down-sampled, thereafter add convolution kernel size be 1 × 1, the convolutional layer that step-length is 1 achievees the purpose that input feature vector dimensionality reduction, is sequentially connected behind each convolutional layer for improving network Nonlinear characteristic Batch Normalization and nonlinear function ReLU.

4. the method for detecting human face according to claim 3 based on lightweight convolutional neural networks, which is characterized in that described B block is down-sampled by the maximum pond layer realization that convolution kernel size is 2 × 2, and step-length 2 has then been sequentially connected volume The convolutional layer that product core size is 3 × 3,1 × 1,3 × 3, each convolutional layer step-length are 1, and are sequentially connected for improving net below The nonlinear characteristic Batch Normalization and nonlinear function ReLU of network.

5. the method for detecting human face according to claim 1 based on lightweight convolutional neural networks, which is characterized in that in institute It states in step S3, the Fusion Features module includes the convolutional layer that convolution kernel size is respectively 3 × 3 and 1 × 1, and step-length is 1, it is up-sampled using deconvolution, and by low-level feature figure and transformed high-level characteristic figure by the way of element summation It is fused together.

6. a kind of face detection system based on lightweight convolutional neural networks, based on SSD detection framework, in feature extraction Light weight convolution grade neural network is as core network designed by Shi Caiyong characterized by comprising

Data processing module carries out image preprocessing, generates training sample for establishing face database；

Light weight convolution grade neural network module, the sample based on input carry out feature using lightweight convolutional neural networks It extracts；The lightweight convolutional neural networks include the convolutional layer that convolution kernel size is respectively 3x3 and 1x1；

Fusion Features module merges for the different characteristic layer to core network, and exports Analysis On Multi-scale Features figure；

Anchor point frame chooses module, for the ratio of width to height of anchor point frame to be arranged.

7. the face detection system according to claim 6 based on lightweight convolutional neural networks, which is characterized in that described Lightweight convolutional neural networks include: sequentially connected 1 A block and 5 B block, wherein convolution in A block Port number is set as 16, and it is 16,32,64,128,128 that the port number of each B block, which is set gradually,.

8. the face detection system according to claim 7 based on lightweight convolutional neural networks, which is characterized in that described A block by convolution kernel size be 3 × 3, step-length be 2 convolutional layer realize it is down-sampled, thereafter add convolution kernel size be 1 × 1, the convolutional layer that step-length is 1 achievees the purpose that input feature vector dimensionality reduction, is sequentially connected behind each convolutional layer for improving network Nonlinear characteristic Batch Normalization and nonlinear function ReLU.

9. the face detection system according to claim 8 based on lightweight convolutional neural networks, which is characterized in that described B block is down-sampled by the maximum pond layer realization that convolution kernel size is 2 × 2, and step-length 2 has then been sequentially connected volume The convolutional layer that product core size is 3 × 3,1 × 1,3 × 3, each convolutional layer step-length are 1, and are sequentially connected for improving net below The nonlinear characteristic Batch Normalization and nonlinear function ReLU of network.

10. the face detection system according to claim 6 based on lightweight convolutional neural networks, which is characterized in that institute Stating Fusion Features module includes the convolutional layer that convolution kernel size is respectively 3 × 3 and 1 × 1, and step-length is 1, uses deconvolution It is up-sampled, and is fused together low-level feature figure and transformed high-level characteristic figure by the way of element summation.