CN108090417A - A kind of method for detecting human face based on convolutional neural networks - Google Patents

Abstract

The present invention relates to a kind of face detection method based on convolutional neural network, comprises the following steps: 1) establishes face detection model, and this model adopts RFCN network structure, and described RFCN network structure comprises the feature extraction layer based on feature fusion; 2 ) to obtain a sample set; 3) to train the face detection model established in step 1); 4) to perform face detection on the picture to be tested with the trained face detection model. Compared with the prior art, the present invention has the advantages of higher accuracy rate and recall rate, better adaptability to complex scenes, and the like.

Description

A Face Detection Method Based on Convolutional Neural Network

technical field

The invention relates to the technical field of face recognition, in particular to a face detection method based on a convolutional neural network.

Background technique

Face detection is a research topic involving computer vision, pattern recognition, artificial intelligence and other fields. Because of its wide application value in commercial, medical and military fields, it has always been a hot research topic. However, in real-world scenarios, faces in complex images are often severely occluded, which poses a huge challenge to face detection, so it is still a research to propose a face detection method that can adapt to severe occlusions. difficulty.

Literature "Object Detection via Region-based Fully Convolutional Networks" (Dai, J., Li, Y., He, K., Sun, J.: R-FCN:.In: 30th Conference on Neural Information Processing Systems, pp.379 -387.Barcelona) discloses a target detection method based on regional fully convolutional neural network. The basic network of this method uses ResNet101, which is an RFCN network structure. The sub-network is divided into Region ProposalNetwork (RPN) and classification network. The overall network structure is shown in the figure 1. The process of extracting feature maps by ResNet consists of 4 stages, which are recorded as res1, res2, res3, and res4. After res4, it is connected with the RPN subnetwork and the classification subnetwork through the convolution operation. The RPN sub-network and the classification sub-network share the feature maps extracted by ResNet, so that the feature extraction only needs to be performed once, which greatly improves the calculation efficiency.

The RPN network is used to extract region proposals, which are possible face regions. The rpn_bbox_pred layer returns the offset of each region relative to the anchor. The anchor is a rectangular box of different scales and aspect ratios generated based on the original input image. The offset for each anchor obtained by adding rpn_bbox_pred to each anchor value is the position of the region that the RPN layer needs to output. rpn_cls_prob outputs the probability that each region is a foreground object and background. The proposal layer integrates the results of the rpn_bbox_pred layer and the rpn_cls_prob layer, sorts them according to the foreground probability, and then uses the non maximum suppression (NMS) algorithm to obtain several regions. (Extract 2000 during training and 300 during testing). The classification network continues to extract features based on the fifth stage res5 of ResNet to obtain score maps with a depth of C*k*k. k is a hyperparameter, with a value of 3; C represents the number of categories of the final classification (including background classes), with a value of 2 (face|background). RFCN uses the Position-sensitive ROIPooling layer to perform position-based average pooling on the score maps for each region obtained by the RPN network. It extracts features for each position of the region separately, and obtains the final result by voting for all positions. Through the RPN sub-network and the classification sub-network, the region where the face is located and the probability that each region is a face can be obtained.

In the training process of this target detection method, the public data set WIDER FACE is selected as the sample set, the RFCN model pre-trained on ImageNet is obtained first, and then the training starts on the prepared sample set. Finally, the trained model is used for face detection.

Although the above-mentioned existing methods can obtain certain accuracy, there are still the following disadvantages: 1. Sensitive to face occlusion, and it is difficult to detect when there are many occlusions. The mAP on WIDER FACE is only 0.77; 2. For smaller people Poor face or profile detection.

Contents of the invention

The object of the present invention is to provide a face detection method based on a convolutional neural network in order to overcome the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved through the following technical solutions:

A face detection method based on convolutional neural network, comprising the following steps:

1) set up face detection model, this model adopts RFCN network structure, and described RFCN network structure comprises the feature extraction layer based on feature fusion;

2) Obtain a sample set;

3) training the face detection model set up in step 1);

4) Perform face detection with the trained face detection model on the picture to be tested.

Further, in the feature extraction layer, the output layer of res3 and the output layer of res4 are superimposed and fused.

Further, in the step 2), the number of samples in the sample set is greater than 30,000.

Further, the training of the step 3) adopts the caffe framework, including:

301) pre-training the face detection model on ImageNet;

302) Using the sample set to train the pre-trained face detection model again.

Further, in the step 4), multi-scale detection is performed on the picture to be tested.

Further, the multi-scale detection is specifically:

401) Perform scaling processing of multiple sizes on the picture to be tested;

402) Use the trained face detection model to perform face detection on the pictures obtained under each size, and obtain multiple face detection results;

403) Merging and screening the multiple face detection results to obtain a final detection result.

Further, in the step 403), the NMS algorithm is used to merge and filter the multiple face detection results.

Compared with the prior art, the present invention has the following beneficial effects:

1) The present invention establishes an improved face detection model. In the feature extraction process, the output of res3 and the output of res4 are fused, and the fused features can be applied to the two subnets of RPN and classification network at the same time Among them, the accuracy and recall rate of face detection are greatly improved.

2) The present invention adopts a multi-scale detection method when detecting a picture to be tested, and can obtain more information about occluded faces and small-resolution faces, further improving the accuracy and accuracy of face detection. Rate.

3) The number of samples in the sample set of the present invention is greater than 30,000, which ensures the accuracy of the detection model.

4) The present invention has a good adaptation effect in complex scenes, especially for scenes with severe face occlusion and small faces. After a large number of tests, the accuracy rate and recall rate both reach more than 90%.

Description of drawings

Fig. 1 is the overall network structure schematic diagram of existing method;

FIG. 2 is a schematic diagram of an existing RPN subnetwork structure;

Fig. 3 is a schematic diagram of the network structure of the feature fusion of the present invention;

Fig. 4 is a schematic diagram of the overall network structure of the present invention;

Fig. 5 is a schematic diagram of the detection process of the present invention;

Fig. 6 is a schematic diagram of the detection effect of the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is carried out on the premise of the technical solution of the present invention, and detailed implementation and specific operation process are given, but the protection scope of the present invention is not limited to the following embodiments.

The present invention provides a kind of face detection method based on convolution neural network, comprises the following steps: 1) establishes face detection model, and this model adopts RFCN network structure, and described RFCN network structure comprises the feature extraction layer based on feature fusion; 2 ) to obtain a sample set; 3) to train the face detection model established in step 1); 4) to perform face detection on the picture to be tested with the trained face detection model. Through the above method, face detection with higher accuracy and recall rate can be performed on complex scenes.

The key points of the above detection method are:

a. Model structure improvement

The improvement of the model structure mainly lies in the feature fusion of the middle layer of the network. In the structure of ResNet101, from the first four stages to res4, a total of 4 pooling operations have been performed. The deeper the convolutional pooling network, the larger the receptive field of each feature map, and the more advanced the learned semantic features. However, for severely occluded faces or small faces, the features they have are limited, and the extraction of high-level semantic features makes their limited local features easier to lose. That is to say, for objects with limited exposed features, a large receptive field is not as effective for detection as a small receptive field. Therefore, in order to correctly detect small faces and severely occluded faces, the present invention superimposes and fuses the output layer of res3 and the output layer of res4, so that the features learned by the network at the res4 layer have both high-level semantic features and low-level local features. The reason for choosing to fuse in res4 is that the features obtained by fusion can be applied to the two subnets of RPN and classification network at the same time. The network structure of feature fusion is shown in Figure 3. res4b22_relu is the output of res4, and res4b22_dcov is the upsampling of the output of res4, so that the feature map of res4 and the feature map of res3 maintain the same size, and res3_scale expands the number of channels of res3 , keep the depth of the same size as the res4feature map. Since the deconvolution operation can only be doubled to an even or odd number, and whether the operation before pooling is even or odd is uncertain, it is necessary to use the crop operation to crop the feature map after deconvolution to the same size as res3. The improved overall network structure is shown in Figure 4.

b. Training stage

Step 1: Make a sample

The source of the sample set is the public dataset WIDER FACE, which contains a total of 32,203 pictures and 393,703 face samples. Produced according to the format of the PASCAL VOC dataset, PASCAL VOC provides a set of standardized and excellent datasets for image recognition and classification, so face samples are produced according to this standard. The sample storage specifications are as follows: JPEGImages stores sample pictures containing faces, and Annotations stores detailed information about the corresponding sample pictures and the bounding box coordinates of the face objects in the pictures. Coordinate composition, Annotation is stored in xml file format.

Step 2: Train the model

For the training of the model, the caffe framework is used. First obtain the improved RFCN model pre-trained on ImageNet, and then start training on the prepared sample set. The training hyperparameters are shown in Table 1.

Table 1: Hyperparameter settings for training

iterations 500000 batch size 1 base learning rate 0.001 k 3 momentum 0.9 scale 1,2,4 weight_decay 0.0005 ratio 0.5,1,2

The model file face_model.caffemodel is obtained after training, and the face can be detected by using this model file.

c. Detect faces

For face detection in complex scenes, we hope to obtain more information about occluded faces and small-resolution faces. Therefore, during the detection, multi-scale processing is performed on the picture, and the pictures of each scale are detected once, and then the detection results are merged and screened using the NMS algorithm to obtain the final detection result. The process is shown in Figure 5 .

The mAP on WIRDER FACE reaches 0.897, and the detection effect is shown in Figure 6.

The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make many modifications and changes according to the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection defined by the claims.

Claims (7)

1. a face detection method based on convolutional neural network, is characterized in that, comprises the following steps: 1) set up face detection model, this model adopts RFCN network structure, and described RFCN network structure comprises the feature extraction layer based on feature fusion; 2) Obtain a sample set; 3) training the face detection model set up in step 1); 4) Perform face detection with the trained face detection model on the picture to be tested. 2. the face detection method based on convolutional neural network according to claim 1, is characterized in that, in the feature extraction layer, the output layer of res3 and the output layer of res4 are superimposed and fused. 3. the hand-raised detection method based on deep learning according to claim 1, is characterized in that, in described step 2), the sample quantity of sample set is greater than 30,000. 4. the face detection method based on convolutional neural network according to claim 1, is characterized in that, the training of described step 3) adopts caffe framework, comprises: 301) pre-training the face detection model on ImageNet; 302) Using the sample set to train the pre-trained face detection model again. 5. the face detection method based on convolutional neural network according to claim 1, is characterized in that, in described step 4), described picture to be tested is carried out multi-scale detection. 6. the face detection method based on convolutional neural network according to claim 5, is characterized in that, described multi-scale detection is specifically: 401) Perform scaling processing of multiple sizes on the picture to be tested; 402) Use the trained face detection model to perform face detection on the pictures obtained under each size, and obtain multiple face detection results; 403) Merging and screening the multiple face detection results to obtain a final detection result. 7. the face detection method based on convolutional neural network according to claim 6, is characterized in that, in described step 403), adopts NMS algorithm to carry out merging screening to described multiple face detection results.