CN106485230B - Training of face detection model based on neural network, face detection method and system - Google Patents

Abstract

The present invention provides a neural network-based face detection model training, face detection method and system, the training method: according to the bias information of the predicted face frame relative to the default face frame, and the real face frame relative to the default The bias information of the face frame is calculated to predict the loss function of the face frame bias network layer; according to the confidence of the default face frame, the loss function of the predicted face frame confidence network layer is calculated; the error of the two loss functions is calculated And the error is fed back to the neural network to adjust the weights in the neural network; repeated iterative training until it converges to obtain the face detection model, so that the predicted face frame contains the face more accurately. Detection method: Input the face image to be tested into the trained face detection model to output bias information and confidence; calculate the corresponding predicted face frame according to the bias information; select a confidence greater than the preset confidence threshold Or the predicted face frame corresponding to the highest confidence level is used as the face detection result.

Description

Training of face detection model based on neural network, face detection method and system

technical field

The invention relates to the technical field of image processing, in particular to a neural network-based face detection model training, face detection method and system.

Background technique

The main task of face detection is to judge whether there is a face on a given face image, and if there is a face, give the position and size of the face. The commonly used process of face detection mainly includes the following three steps: (1) select a rectangular area from the image as an observation window; (2) extract features from the observation window to describe the content it contains; (3) carry out Classification discrimination, to determine whether the window contains a face. By continuously repeating the above three steps, until all observation windows on the face image are traversed. If any observation window is judged to contain a human face, the position and size of the window can be the position and size of the detected human face; conversely, if all windows do not contain a human face, the given face image is considered There are no faces.

The currently commonly used face detectors are based on the Viola-Jones face detector designed by Paul Viola and Michael Jones in 2001; first, use the Haar feature to construct an integral map to achieve fast calculation of features; second, use effective feature classification processor method, for example, the AdaBoost algorithm; finally, the window is discriminated from coarse to fine through cascading. Due to the non-rigid attribute structure of the face and the complex and changeable external environment, the face detection technology has problems such as high false detection and low face detection rate. In response to this problem, a lot of follow-up work has improved the Viola-Jones face detector, for example, selecting features with stronger descriptive capabilities (LBP, HOG), improving the classifier algorithm and cascade structure, etc., making face detection performance has been improved.

In recent years, with the development of deep learning, some face detection methods based on deep neural networks have gradually emerged, such as CascadeCNN, Faceness, Faster RCNN, etc. Compared with traditional face detection methods, the features extracted by deep neural networks It has stronger robustness and descriptive ability, so it has higher detection rate and lower false detection.

Although the current face detector has made great progress, there are still some problems in the following aspects:

(1) At present, most face detectors use a sliding window method to select the observation window, so after traversing a face image, it is necessary to calculate and distinguish a large number of observation windows, so the calculation amount is large; and for face images For faces of different sizes, it is necessary to construct an image pyramid, or use observation windows of different scales, and the face detection speed is slow.

(2) Most face detection algorithms have many steps, and each step is relatively independent. Any problem in any step will affect the final face detection result.

(3) The face detection method based on deep learning, although the effect is good, needs to scale the input face image to a fixed size, causing the face in the image to be stretched, distorted, deformed, etc., affecting the final face detection result .

Contents of the invention

In view of the above-mentioned shortcomings of the prior art, the object of the present invention is to provide a neural network-based face detection model training, face detection method and system, which can realize multi-scale detection of faces.

In order to achieve the above purpose and other related purposes, the present invention provides a training method for a face detection model based on a neural network. The network layer of the face frame confidence; wherein, the network layer of the human face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and the network layer of the human face detection Each cell in is bound to six default face frames, which are set according to the network layer scale of the corresponding face detection; each layer of face detection network layer is connected to a layer of predicted face frame A biased network layer and a network layer predicting the confidence of the face frame; the method also includes:

When the model training instruction is received, the face images in the training set are input into the neural network for training;

Calculate the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the bias information of the real face frame relative to the corresponding default face frame; and Calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame;

According to the bias information of the predicted face frame relative to the corresponding default face frame, and the bias information of the real face frame relative to the corresponding default face frame, calculate the network layer for predicting the bias of the face frame A loss function; and according to the confidence of the default human face frame, calculate the loss function of the network layer for predicting the confidence of the human face frame;

Calculate the error of the loss function of the network layer of the bias of the predicted face frame and the loss function of the network layer of the predicted face frame confidence, and feed back the error to the neural network through backpropagation, Updating the network weight parameters of the neural network according to the error and adjusting the predicted face frame according to the updated network weight parameters;

Repeat iterative training until the error between the adjusted predicted face frame and the real face frame is within the preset error range, and output the face detection model.

Preferably, the bias information of the predicted face frame relative to the corresponding default face frame is calculated by the network layer of the predicted face frame bias, including:

Calculate the offset information of the predicted face frame relative to the corresponding default face frame according to the following formula:

t _x ＝(x- x _a )/w _a , t _y ＝(y- y _a )/h _a

t _w =log(w/w _a ),t _h =log(h/h _a )

Among them, (x, y, w, h) are the center point coordinates, width and height of the predicted face frame; (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame High; (t _x , t _y , t _w , t _h ) is the offset information of the predicted face frame relative to the corresponding default face frame;

The calculation of the offset information of the real face frame relative to the corresponding default face frame includes:

According to each default face frame and the corresponding real face frame, the offset information of the real face frame relative to the corresponding default face frame is calculated according to the following formula:

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame; (x ^* , y ^* , w ^* , h ^* ) are the center point of the real face frame coordinates, width and height; is the offset information of the real face frame relative to the corresponding default face frame.

Preferably, the predicted face frame is calculated according to the bias information of the predicted face frame relative to the corresponding default face frame, and the bias information of the real face frame relative to the corresponding default face frame The loss function of the biased network layer, including:

Select the corresponding default face frame when the relative area of the default face frame relative to the corresponding real face frame is greater than the preset first relative area threshold as the sampling default face frame; wherein, the relative area is the default The area of the intersection area of the face frame and the real face frame is divided by the area of the union area of the default face frame and the real face frame;

According to the bias information of the predicted face frame relative to the corresponding sampled default face frame, and the bias information of the real face frame relative to the corresponding sampled default face frame, the predicted face frame offset is calculated according to the following formula The loss function Loss ₁ of the network layer:

Among them, N _reg is the number of sampled default face frames, L _reg corresponds to the regression loss function of the spatial bias of the kth (k is a positive integer between 1 and N _reg ) default face frames, T=t _x , t _y , t _w , t _h are the bias information of the predicted face frame relative to the corresponding sampled default face frame, z is an element belonging to the set {x, y, w, h}, is the offset information of the real face frame relative to the corresponding sampled default face frame, smooth _L1 represents the smooth L1 loss function, which is a variant of the L1 norm loss function, and l represents the input variable of the smooth _L1 function.

According to the confidence that the default human face frame contains a human face, the loss function of the network layer that calculates the predicted human face frame confidence includes:

When the relative area of the default face frame to the corresponding real face frame is greater than the preset first relative area threshold, the corresponding default face frame is used as a positive sample, and the default face frame is compared to the corresponding The default face frame corresponding to when the relative area of the real face frame is less than or equal to the preset first relative area threshold is used as a negative sample;

Select all positive samples and some negative samples according to the preset ratio of positive and negative samples;

Calculate the loss function Loss ₂ of the network layer for predicting the confidence of the face frame according to the following formula:

L _cls (p,p ^* )＝-[p ^* logp+(1-p ^* )log(1-p)

Among them, N _cls is the total number of positive and negative samples selected, Corresponding to the classification loss function of the i-th category (i is a positive integer between 1 and N _cls ), p is the confidence that the selected positive or negative sample contains the face, and p ^* is the selected positive or negative sample contains The true probability of the face, p ^* is 1 for positive samples, and p ^* is 0 for negative samples.

Preferably, the calculation of the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame includes:

Adopt stochastic gradient descent method to calculate the gradient of the loss function of the network layer of described prediction face frame offset and the loss function of the network layer of described prediction face frame confidence degree;

The obtained gradient value is used as an error between the loss function of the network layer predicting the bias of the face frame and the loss function of the network layer predicting the confidence of the face frame.

Another object of the present invention is to provide a face detection method based on neural network, said neural network comprising: a network layer for face detection, a network layer for predicting the bias of the face frame, and a network for predicting the confidence of the face frame layer; wherein, the network layer of the human face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and each cell in the network layer of the human face detection is bound Determine six default face frames, the default face frame is set according to the network layer scale of the corresponding face detection; the network layer of each layer of face detection is connected to a network layer that predicts the bias of the face frame and a network layer Layer predicts the network layer of face frame confidence; Described method comprises:

When the face detection instruction is received, the face image to be detected is input into the trained face detection model for face detection;

For the face image to be detected, the network layer of the predicted face frame bias in the trained face detection model outputs the bias information of the predicted face frame relative to the corresponding default face frame, and the trained The network layer predicting the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face;

Calculate the corresponding predicted face frame according to the offset information of each default face frame and the predicted face frame relative to each default face frame;

In the predicted human face frame, select the predicted human face frame corresponding to the confidence greater than the preset confidence threshold as the final human face detection result, or select the corresponding predicted human face frame in the predicted human face frame The predicted face frame is used as the final face detection result.

Preferably, the calculation of the corresponding predicted face frame according to each default face frame and the offset information of the corresponding predicted face frame relative to each default face frame includes:

According to the offset information of each default face frame and the corresponding predicted face frame relative to each default face frame, the corresponding predicted face frame is calculated according to the following formula:

x＝t _x *w _a +x _a ,y＝t _y *h _a +y _a

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of each default face frame; (x, y, w, h) are the predictions corresponding to each default face frame The center point coordinates, width and height of the face frame; (t _x , _ty , t _w , t _h ) is the offset information of the corresponding predicted face frame relative to each default face frame.

Preferably, for the face image to be detected, the bias information of the predicted face frame relative to the corresponding default face frame is output through the network layer of the predicted face frame bias in the trained face detection model, and passed After the network layer of predicting the confidence of the face frame in the trained face detection model outputs the confidence that each default face frame contains the face, the method also includes:

Filtering out default face frames whose confidence is less than or equal to a preset confidence threshold;

Calculate the corresponding predicted face frame according to the bias information of each remaining default face frame and the corresponding predicted face frame relative to each of the remaining default face frames;

The predicted face frames corresponding to the remaining default face frames are used as the final face detection result.

Preferably, after calculating the corresponding predicted face frame according to the offset information of each default face frame and the corresponding predicted face frame relative to each default face frame, the method further includes:

Calculate the relative area of each two predicted face frames;

If the relative area of every two predicted human face frames is greater than the preset second relative area threshold, then the two predicted human face frames are used as sampling predicted human face frames; wherein, the relative areas are two predicted human faces The area of the intersection area of the frame is divided by the area of the union area of the two predicted face frames;

In the sampled predicted face frame, select the sampled predicted face frame corresponding to the confidence greater than the preset confidence threshold as the final face detection result, or select the highest confidence in the sampled predicted face frame The sampling prediction face frame corresponding to degree is used as the final face detection result.

Another object of the present invention is to provide a training system for a face detection model based on a neural network. degree of network layer; wherein, the network layer of the human face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and each of the network layers of the human face detection The cell is bound to six default face frames, which are set according to the network layer scale of the corresponding face detection; each network layer of face detection is connected to a network that predicts the bias of the face frame Layer and a network layer for predicting the confidence of the face frame; the system includes: an input module, a first calculation module, a second calculation module, a third calculation module, a feedback and update module, and an iterative output module; wherein,

The input module is configured to input the face images in the training set into the neural network for training when receiving a model training instruction;

The first calculation module is used to calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame offset, and calculate the offset information of the real face frame relative to the corresponding default The bias information of the face frame; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame;

The second computing module is configured to calculate according to the bias information of the predicted face frame relative to the corresponding default face frame, and the bias information of the real face frame relative to the corresponding default face frame Predict the loss function of the network layer of human face frame offset; And according to the confidence degree that described default human face frame comprises people's face, calculate the loss function of the network layer of predicting human face frame confidence degree;

The third calculation module is used to calculate the error between the loss function of the network layer of the predicted face frame bias and the loss function of the network layer of the predicted face frame confidence;

The feedback and update module is configured to feed back the error into the neural network through backpropagation, update the network weight parameters of the neural network according to the error, and adjust the prediction according to the updated network weight parameters face frame;

The iterative output module is used to repeat iterative training until the error between the adjusted predicted face frame and the real face frame is within a preset error range, and output the face detection model.

Another object of the present invention is to provide a human face detection system based on a neural network, the neural network comprising: a network layer for human face detection, a network layer for predicting the bias of the human face frame, and a network for predicting the confidence of the human face frame layer; wherein, the network layer of the human face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and each cell in the network layer of the human face detection is bound Determine six default face frames, the default face frame is set according to the network layer scale of the corresponding face detection; the network layer of each layer of face detection is connected to a network layer that predicts the bias of the face frame and a network layer The network layer of layer prediction face frame confidence degree; Described system comprises: input module, output module, calculation module, selection module; Wherein,

The input module is configured to input the face image to be detected into the trained face detection model for face detection when receiving the face detection instruction;

The output module is used to output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame offset in the trained face detection model for the face image to be detected, And output the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame in the trained face detection model;

The calculation module is used to calculate the corresponding predicted face frame according to the bias information of each default face frame and predicted face frame relative to each default face frame;

The selection module is configured to select a predicted face frame corresponding to a confidence greater than a preset confidence threshold in the predicted face frame as the final face detection result, or, in the predicted face frame The predicted face frame corresponding to the highest confidence level is selected as the final face detection result.

As mentioned above, the training of the face detection model based on the neural network, the face detection method and system of the present invention have the following beneficial effects compared with the prior art:

(1) In the embodiment of the present invention, it is not necessary to select the observation window in a sliding window manner, nor to construct an image pyramid or use a multi-scale observation window and to calculate and discriminate a large number of observation windows, but according to the neural network Different network layers correspond to different sizes of receptive fields in the original face image. The network layer used for face detection is selected. The higher the number of network layers corresponds to the larger the receptive field of the original face image, the lower the number of network layers corresponds to The smaller the receptive field of the original face image, the face detection is performed directly through the face detection network layer. Compared with the existing technology, the calculation amount is small, and the low-level network layer can be selected to detect small-sized faces. The layer is used to detect large-sized human faces, realize multi-scale detection and bias regression of human faces, and make the predicted human face frame better contain human faces. Compared with the prior art, the human face detection of the embodiment of the present invention is more accurate and Face detection is faster.

(2) In the embodiment of the present invention, the model training directly adopts the end-to-end mode, and the face image in the input training set directly outputs the position and size of the corresponding predicted face frame, which is simpler and faster than the existing multi-step method; and, Due to the end-to-end training mode, the network weight parameters of the neural network are directly fed back according to the error between the loss function of the network layer for predicting the bias of the face frame and the loss function of the network layer for the confidence of the predicted face frame Adjust to make the predicted face frame and the real face frame closer, so that the predicted face frame contains the face more accurately; therefore, it has a higher detection rate than the method based on multiple independent sub-steps in the prior art .

(3) In the embodiment of the present invention, it is not necessary to zoom the input face image, but directly input the face image in the training set into the neural network to train the face detection model, and directly input the face image to be detected into Face detection is performed in the trained face detection model, which can avoid the influence of face image stretching, distortion, deformation and other factors on the face detection results.

Description of drawings

Fig. 1 is shown as the implementation flowchart of the training method of the face detection model based on neural network in the embodiment of the present invention;

Fig. 2 is shown as the implementation flowchart of the face detection method based on neural network in the embodiment of the present invention;

Fig. 3 is shown as the composition structure diagram of the training system of the face detection model based on neural network in the embodiment of the present invention;

FIG. 4 is a schematic diagram showing the composition and structure of the neural network-based face detection system in the embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention.

Please refer to attached picture. It should be noted that the diagrams provided in the embodiments of the present invention are only schematically illustrating the basic idea of the present invention, and only the components related to the present invention are shown in the diagrams rather than the number and shape of components in actual implementation. and size drawing, the type, quantity and proportion of each component can be changed arbitrarily during actual implementation, and the component layout type may also be more complex.

In the embodiment of the present invention, the neural network includes: a network layer for face detection, a network layer for predicting the bias of the face frame, and a network layer for predicting the confidence of the face frame; wherein, the network layer for face detection is Selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, each cell in the network layer of the face detection is bound to six default face frames, and the default face The frame is set according to the network layer scale of the corresponding face detection; each network layer of face detection is connected with a network layer for predicting the bias of the face frame and a network layer for predicting the confidence of the face frame.

Among them, since VGG-16 is currently a popular fully convolutional neural network structure, it has good expression ability of abstract features, and has achieved good results in object classification and target recognition, such as ImageNet competition, face recognition and other fields. , so VGG-16 is selected as the basic network structure of the fully convolutional neural network. The network parameter configuration information of the VGG-16 fully convolutional neural network structure is shown in Table 1:

Table 1

Wherein, since the receptive fields of different sizes are conducive to detecting face images of different scales, the network layer can be selected as the network layer of face detection according to the receptive fields of different network layers in the neural network corresponding to the face images in the training set; Here, the three network layers conv3_3, conv4_3, and conv5_3 in the neural network can be selected for multi-scale face detection according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, conv3_3, The three network layers of conv4_3 and conv5_3 correspond to the receptive fields of the face images in the training set as shown in Table 2:

Table 2

Among them, an additional network layer for predicting the bias of the face frame and a network layer for predicting the confidence of the face frame can be added behind the network layer of each layer of face detection; that is: each layer The network layer of face detection is connected with a network layer for predicting the bias of the face frame and a network layer for predicting the confidence of the face frame, and a network layer for predicting the bias of the face frame and a network layer for The layers of the network that predict the confidence of the face box are parallelized.

Among them, six default face frames are bound to each cell (feature mapcell) of the three network layers conv3_3, conv4_3, and conv5_3, which can be set according to the scale of each layer of the three network layers conv3_3, conv4_3, and conv5_3 The size of the six default face frames bound to each cell in each layer, the scale of each layer and the recommended size of the default face frame are shown in Table 3:

table 3

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The embodiment of the present invention proposes a training method of a neural network-based face detection model, as shown in Figure 1, the method includes:

Step S100: When receiving a model training instruction, input the face images in the training set into the neural network for training.

In this step, the face detection model trained on the ImageNet image library (1000 categories, 1.2 million training images) can be used as the initialization face detection model; then use the CelebFaces image library ( 202,599 images, each containing a face) and the AFLW image library (21,080 images, containing a total of 24,386 faces). Augmentation, e.g., scaling, adding blurring noise, contrast changes, etc., to enrich training samples.

Step S101: Calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the offset of the real face frame relative to the corresponding default face frame information; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame.

In this step, each default face frame corresponds to a predicted face frame and a real face frame, that is, there is a one-to-one correspondence between the default face frame, the predicted face frame, and the real face frame.

In this step, the bias information (t _x , t _y , t _w , t _h ) of the output predicted face frame relative to the corresponding default face frame is calculated according to the following formula:

t _x ＝(xx _a )/w _a , t _y ＝(yy _a )/h _a

t _w =log(w/w _a ),t _h =log(h/h _a )

Among them, (x, y, w, h) are the center point coordinates, width and height of the predicted face frame; (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame High; (t _x , t _y , t _w , t _h ) is the offset information of the predicted face frame relative to the corresponding default face frame;

The calculation of the offset information of the real face frame relative to the corresponding default face frame includes:

Calculate the offset information of the real face frame relative to the corresponding default face frame according to the following formula:

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame; (x ^* , y ^* , w ^* , h ^* ) are the center point of the real face frame coordinates, width and height; is the offset information of the real face frame relative to the corresponding default face frame.

Step S102: According to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, calculate the bias of the predicted face frame The loss function of the network layer; and according to the confidence that the default face frame contains the face, calculate the loss function of the network layer that predicts the confidence of the face frame.

In this step, only the bias information corresponding to the default face frame whose relative area IOU of the default face frame relative to the real face frame is greater than the preset first relative area threshold can be selected for regression calculation, and the face frame regression is predicted The loss function of is calculated as follows:

First select the default face frame corresponding to the default face frame when the relative area IOU of the corresponding real face frame is greater than the preset first relative area threshold as the default face frame for sampling; wherein the relative area is the area of the intersection area of the default face frame and the real face frame divided by the area of the union area of the default face frame and the real face frame;

According to the bias information of the predicted face frame relative to the corresponding sampled default face frame, and the bias information of the real face frame relative to the corresponding sampled default face frame, the predicted face frame offset is calculated according to the following formula The loss function of the network layer:

Among them, N _reg is the number of sampled default face frames, L _reg corresponds to the regression loss function of the spatial bias of the kth (k is a positive integer between 1 and N _reg ) default face frames, T=t _x , t _y , t _w , t _h are the bias information of the predicted face frame relative to the corresponding sampled default face frame, z is an element belonging to the set {x, y, w, h}, is the offset information of the real face frame relative to the corresponding sampled default face frame, smooth _L1 represents the smooth L1 loss function, which is a variant of the L1 norm loss function, and l represents the input variable of the smooth _L1 function;

Then, use the default face frame corresponding to the default face frame when the relative area of the corresponding real face frame is greater than the preset first relative area threshold as a positive sample, and compare the default face frame with respect to The corresponding default face frame when the relative area of the corresponding real face frame is less than or equal to the preset first relative area threshold is used as a negative sample;

Select all positive samples and some negative samples according to the preset ratio of positive and negative samples;

Calculate the loss function of the network layer that predicts the confidence of the face frame according to the following formula:

L _cls (p,p ^* )＝-[p ^* logp+(1-p ^* )log(1-p)

Among them, N _cls is the total number of positive and negative samples selected, Corresponding to the classification loss function of the i-th category (i is a positive integer between 1 and N _cls ), p is the confidence that the selected positive or negative sample contains the face, and p ^* is the selected positive or negative sample contains The true probability of the face, p ^* is 1 for positive samples, and p ^* is 0 for negative samples.

In the embodiment of the present invention, the preset first relative area threshold can be set according to actual requirements, and the preset first relative area threshold is not specifically limited here. Preferably, the preset first relative area threshold is 0.5. For the calculation of face classification confidence, since the relative area IOU between most of the default face frames and the corresponding real face frames in face detection is less than 0.5, the number of negative samples will be much higher than that of positive samples. Therefore, in order to balance the positive and negative samples to avoid the misinterpretation or missed detection of the face model caused by the imbalance of positive and negative samples, a positive and negative sample ratio of 1:3 can be used to calculate the confidence of face classification, in which the confidence of negative samples is selected The higher part is used for the training of the face detection model.

Finally, the above two loss functions can be fused to obtain the following total loss function:

Loss＝Loss ₁ +λLoss ₂

Among them, λ is used to balance the two loss functions and is set to 1 by default.

It should be noted that: λ can be set according to actual needs.

Step S103: Calculate the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame, and feed back the error to the neural network through backpropagation In the network, the network weight parameters of the neural network are updated according to the error and the predicted face frame is adjusted according to the updated network weight parameters.

In this step, the stochastic gradient descent method can be used to calculate the gradient of the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame; the obtained gradient value is used as the The error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame.

In this step, the formula for updating the network weight parameters is as follows:

Among them, W _i ^l is the weight of the parameters of the l layer after i iterations, is the updated weight of the i+1 iteration, Initialize with a Gaussian distribution with a mean of 0 and a variance of 1, α is the learning rate, m is the momentum, and λ is the weight decay coefficient.

In this step, the stochastic gradient descent method is used for gradient feedback update, and the parameters used for learning network weight parameters need to be set in advance. The parameters used include: initial learning rate, momentum, and weight decay; where the initial learning rate is set to 0.001, and The momentum is set to 0.9 and the initial weight decay is set to 0.0005.

Step S104: Determine whether the error between the adjusted predicted face frame and the real face frame is within the preset error range; if the adjusted error between the predicted face frame and the real face frame is within the preset error range If the error between the adjusted predicted human face frame and the real human face frame is not within the preset error range, then go to step S101 according to the adjusted predicted human face frame. The face frame continues to execute.

In this step, the preset error range can be set according to actual needs, based on the principle that the adjusted predicted face frame iteratively converges to the real face frame, that is, the weight parameters of the neural network are updated so that the predicted face frame The frame converges to the real face frame, usually after 360000 iterations.

The embodiment of the present invention proposes a face detection method based on a neural network, as shown in Figure 2, the method includes:

Step S200: When a face detection instruction is received, input the face image to be detected into the trained face detection model for face detection.

In this step, the trained face detection model is the face detection model obtained when the error between the adjusted predicted face frame and the real face frame is within the preset error range by repeated iterative training in steps S100-S104 .

Step S201: Output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias in the trained face detection model for the face image to be detected, and pass the training The network layer that predicts the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face.

In this step, the confidence of the default face frame containing the face is the probability that the area contained in the default face frame belongs to the background or the face.

Step S202: According to each default face frame and the offset information of the predicted face frame relative to each default face frame, calculate the corresponding predicted face frame.

Specifically, according to the offset information of each default face frame and the corresponding predicted face frame relative to each default face frame, the corresponding predicted face frame is calculated according to the following formula:

x＝t _x *w _a +x _a ,y＝t _y *h _a +y _a

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of each default face frame; (x, y, w, h) are the predictions corresponding to each default face frame The center point coordinates, width and height of the face frame; (t _x , _ty , t _w , t _h ) is the offset information of the corresponding predicted face frame relative to each default face frame.

Step S203: Select the predicted face frame corresponding to the confidence greater than the preset confidence threshold in the predicted face frame as the final face detection result, or select the predicted face frame corresponding to the highest confidence as the The final face detection result.

In this step, since there is a one-to-one correspondence between the default face frame and the predicted face frame, the predicted face frame corresponding to a confidence degree greater than the preset confidence threshold or the highest confidence degree means that the confidence degree is greater than the predetermined The default face frame of the set confidence threshold or the predicted face frame corresponding to the default face frame with the highest confidence.

In this step, the confidence threshold may be preset according to actual needs, and the confidence threshold is not specifically limited here.

In a preferred embodiment of the present invention, for the face image to be detected, the network layer of the predicted face frame offset in the trained face detection model outputs the predicted face frame with respect to the corresponding default face The bias information of the frame, and output the confidence of each default face frame containing the face through the network layer of the predicted face frame confidence in the trained face detection model, in order to reduce the calculation amount of the predicted face frame ,Also includes:

First filter out the default face frames whose confidence is less than or equal to a preset confidence threshold;

Then according to the bias information of each remaining default human face frame and corresponding predicted human face frame with respect to each remaining said default human face frame, calculate the corresponding predicted human face frame;

Finally, the predicted face frames corresponding to the remaining default face frames are used as the final face detection result.

In a preferred embodiment of the present invention, before calculating the corresponding predicted face frame, first filter out the default face frame whose confidence is less than or equal to the preset confidence threshold, and then according to each of the remaining default The face frame and the corresponding predicted face frame are calculated with respect to the offset information of each of the remaining default face frames, and the corresponding predicted face frame can be calculated, which can reduce the calculation amount of the predicted human face frame, thereby improving the calculation speed and reduce computation time.

In another preferred embodiment of the present invention, since a face may correspond to multiple overlapping predicted face frames, there is a large amount of redundancy, and the non-maximum suppression method can be used to remove redundant and repeated face frames, specifically , remove the predicted face frame whose relative area IOU of every two predicted face frames is greater than the preset second relative area threshold, and only keep the predicted face frame whose relative area IOU of every two predicted face frames is higher, so , after calculating the corresponding predicted face frame according to the offset information of each default face frame and the corresponding predicted face frame relative to each default face frame, it also includes:

Calculate the relative area of each two predicted face frames;

If the relative area of every two predicted human face frames is greater than the preset second relative area threshold, then the two predicted human face frames are used as sampling predicted human face frames; wherein, the relative areas are two predicted human faces The area of the intersection area of the frame is divided by the area of the union area of the two predicted face frames;

In the sampled predicted face frame, select the sampled predicted face frame corresponding to the confidence greater than the preset confidence threshold as the final face detection result, or select the highest confidence in the sampled predicted face frame The corresponding sampling prediction face frame is used as the final face detection result.

In another preferred embodiment of the present invention, the relative areas of every two predicted face frames are calculated for the predicted face frames, and if the relative areas of each other are larger than the preset second relative area threshold, the two predicted face frames are considered The face frame contains the same face, therefore, if the relative area of each two predicted face frames is greater than the preset second relative area threshold, the two predicted face frames are used as the sample predicted face frame, and the In the sampled predicted face frame, the sampled predicted face frame corresponding to the confidence greater than the preset confidence threshold is selected as the final face detection result, or the predicted face frame corresponding to the highest confidence is selected as the final face detection result. face detection results, so that the final face detection results better contain faces.

Wherein, the second relative area threshold can be preset according to actual needs, and the second relative area threshold is not specifically limited here, preferably, the second relative area threshold is preset as 0.3.

In order to realize the above-mentioned method, the embodiment of the present invention provides a training system of a face detection model based on a neural network. Since the principle and method of solving the problem of this system are similar, therefore, the implementation process and implementation principle of the system can refer to the above-mentioned method The implementation process and implementation principles are described, and the repetitions will not be repeated.

The embodiment of the present invention proposes a training system for a face detection model based on a neural network. The neural network includes: a network layer for face detection, a network layer for predicting the bias of the face frame, and a network layer for predicting the confidence of the face frame. Network layer; wherein, the network layer of the human face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and each cell in the network layer of the human face detection Binding six default face frames, the default face frame is set according to the network layer scale of the corresponding face detection; the network layer of each layer of face detection is connected to a network layer that predicts the bias of the face frame and One layer predicts the network layer of face frame confidence; As shown in Figure 3, this system comprises: input module 300, the first calculation module 301, the second calculation module 302, the third calculation module 303, feedback and update module 304, iterative output module 305; wherein,

The input module 300 is configured to input the face images in the training set into the neural network for training when receiving a model training instruction;

The first calculation module 301 is used to calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame offset, and calculate the offset information of the real face frame relative to the corresponding The bias information of the default face frame; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame;

The second calculation module 302 is configured to, according to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, Computing the loss function of the network layer for predicting the offset of the face frame; and calculating the loss function of the network layer for predicting the confidence of the face frame according to the confidence of the default face frame;

The third calculation module 303 is used to calculate the error between the loss function of the network layer of the predicted face frame bias and the loss function of the network layer of the predicted face frame confidence;

The feedback and update module 304 is configured to feed back the error into the neural network through backpropagation, update the network weight parameters of the neural network according to the error, and adjust the network weight parameters according to the updated network weight parameters. Predict the face frame;

The iterative output module 305 is configured to repeat iterative training until the error between the adjusted predicted face frame and the real face frame is within a preset error range, and output the face detection model.

In specific implementation, the first calculation module 301 is specifically used for:

Calculate the offset information of the predicted face frame relative to the corresponding default face frame according to the following formula:

t _x ＝(xx _a )/w _a , t _y ＝(yy _a )/h _a

t _w =log(w/w _a ),t _h =log(h/h _a )

Among them, (x, y, w, h) are the center point coordinates, width and height of the predicted face frame; (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame High; (t _x , t _y , t _w , t _h ) is the offset information of the predicted face frame relative to the corresponding default face frame;

The offset information of the real face frame relative to the corresponding default face frame is calculated according to the following formula:

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of the default face frame; (x ^* , y ^* , w ^* , h ^* ) are the center point of the real face frame coordinates, width and height; is the offset information of the real face frame relative to the corresponding default face frame.

In a specific implementation, the second calculation module 302 is specifically used for:

Select the corresponding default face frame when the relative area of the default face frame relative to the corresponding real face frame is greater than the preset first relative area threshold as the sampling default face frame; wherein, the relative area is the default The area of the intersection area of the face frame and the real face frame is divided by the area of the union area of the default face frame and the real face frame;

According to the bias information of the predicted face frame relative to the corresponding sampled default face frame, and the bias information of the real face frame relative to the corresponding sampled default face frame, the predicted face frame offset is calculated according to the following formula The loss function of the network layer:

Among them, N _reg is the number of sampled default face frames, L _reg corresponds to the regression loss function of the spatial bias of the kth (k is a positive integer between 1 and N _reg ) default face frames, T=t _x , t _y , t _w , t _h are the bias information of the predicted face frame relative to the corresponding sampled default face frame, z is an element belonging to the set {x, y, w, h}, is the offset information of the real face frame relative to the corresponding sampled default face frame, smooth _L1 represents the smooth L1 loss function, which is a variant of the L1 norm loss function, and l represents the input variable of the smooth _L1 function;

According to the confidence that the default human face frame contains a human face, the loss function of the network layer that calculates the predicted human face frame confidence includes:

When the relative area of the default face frame to the corresponding real face frame is greater than the preset first relative area threshold, the corresponding default face frame is used as a positive sample, and the default face frame is compared to the corresponding The default face frame corresponding to when the relative area of the real face frame is less than or equal to the preset first relative area threshold is used as a negative sample;

Select all positive samples and some negative samples according to the preset ratio of positive and negative samples;

Calculate the loss function of the network layer that predicts the confidence of the face frame according to the following formula:

L _cls (p,p ^* )＝-[p ^* logp+(1-p ^* )log(1-p)

Among them, N _cls is the total number of positive and negative samples selected, Corresponding to the classification loss function of the i-th category (i is a positive integer between 1 and N _cls ), p is the confidence that the selected positive or negative sample contains the face, and p ^* is the selected positive or negative sample contains The true probability of the face, p ^* is 1 for positive samples, and p ^* is 0 for negative samples.

In specific implementation, the third calculation module 303 is specifically used for:

Adopt stochastic gradient descent method to calculate the gradient of the loss function of the network layer of described prediction face frame offset and the loss function of the network layer of described prediction face frame confidence degree;

The obtained gradient value is used as an error between the loss function of the network layer predicting the bias of the face frame and the loss function of the network layer predicting the confidence of the face frame.

The above division manner of the functional modules is only a preferred implementation manner given by the embodiment of the present invention, and the division manner of the functional modules does not constitute a limitation of the present invention. For the convenience of description, each part of the system described above is divided into various modules or units by function and described separately. The system can be a distributed system or a centralized system. If it is a distributed system, the above-mentioned functional modules can be realized by hardware devices respectively, and each hardware device interacts through a communication network; if it is a centralized system, the above-mentioned functional modules can be integrated in one hardware device.

In practical applications, when the input module 300, the first calculation module 301, the second calculation module 302, the third calculation module 303, the feedback and update module 304, and the iterative output module 305 are integrated in one hardware device, the The input module 300, the first calculation module 301, the second calculation module 302, the third calculation module 303, the feedback and update module 304, and the iterative output module 305 can be located in the central processing unit (CPU), microprocessor ( MPU), Digital Signal Processor (DSP) or Field Programmable Gate Array (FPGA) implementation.

In order to realize the above method, the embodiment of the present invention also provides a face detection system based on neural network. Since the principle and method of solving the problem of this system are similar to the method, therefore, the implementation process and implementation principle of the system can refer to the implementation of the aforementioned method The process and implementation principles are described, and the repetitions will not be repeated.

An embodiment of the present invention proposes a neural network-based face detection system, the neural network comprising: a network layer for face detection, a network layer for predicting the bias of the face frame, and a network layer for predicting the confidence of the face frame; Wherein, the network layer of the face detection is selected according to the receptive fields of the face images in the training set corresponding to different network layers in the neural network, and each cell in the network layer of the face detection is bound to six A default face frame, the default face frame is set according to the network layer scale of the corresponding face detection; the network layer of each layer of face detection is connected to a network layer that predicts the bias of the face frame and a layer of prediction The network layer of face frame confidence degree; As shown in Figure 4, this system comprises: input module 400, output module 401, calculation module 402, selection module 403; Wherein,

The input module 400 is configured to input the face image to be detected into the trained face detection model for face detection when receiving the face detection instruction;

The output module 401 is used to output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame offset in the trained face detection model for the face image to be detected And each default face frame contains the confidence of the face;

The calculation module 402 is used to calculate the corresponding predicted face frame according to each default face frame and the bias information of the predicted face frame relative to each default face frame;

The selecting module 403 is configured to select a predicted human face frame corresponding to a confidence degree greater than a preset confidence threshold in the predicted human face frame as the final human face detection result, or, in the predicted human face frame, Select the predicted face frame corresponding to the highest confidence level in the frame as the final face detection result.

In specific implementation, the calculation module 402 is specifically used for:

According to the offset information of each default face frame and the corresponding predicted face frame relative to each default face frame, the corresponding predicted face frame is calculated according to the following formula:

x＝t _x *w _a +x _a ,y＝t _y *h _a +y _a

Among them, (x _a , y _a , w _a , h _a ) are the center point coordinates, width and height of each default face frame; (x, y, w, h) are the predictions corresponding to each default face frame The center point coordinates, width and height of the face frame; (t _x , _ty , t _w , t _h ) is the offset information of the corresponding predicted face frame relative to each default face frame.

Further, the system also includes:

A filtering module 404, configured to filter out default face frames whose confidence is less than or equal to a preset confidence threshold;

The calculation module 402 is further configured to calculate a corresponding predicted face frame according to each of the remaining default face frames and the offset information of the corresponding predicted face frame relative to each of the remaining default face frames;

The selection module 403 is further configured to use the predicted face frames corresponding to the remaining default face frames as the final face detection result.

Further, the system also includes:

The judging module 405 is used to calculate the relative area of each two predicted human face frames, and when the relative area of each two predicted human face frames is greater than the preset second relative area threshold, the two predicted human face frames are used as Sampling prediction face frame; Wherein, described relative area is the area of the intersecting area of two prediction face frames divided by the area of the union area of two prediction face frames;

The selection module 403 is further configured to select, in the sampled predicted face frame, a sampled predicted face frame corresponding to a confidence greater than a preset confidence threshold as the final face detection result, or, in the In the sampled predicted face frame, the sampled predicted face frame corresponding to the highest confidence is selected as the final face detection result.

The above division manner of the functional modules is only a preferred implementation manner given by the embodiment of the present invention, and the division manner of the functional modules does not constitute a limitation of the present invention. For the convenience of description, each part of the system described above is divided into various modules or units by function and described separately. The system can be a distributed system or a centralized system. If it is a distributed system, the above-mentioned functional modules can be realized by hardware devices respectively, and each hardware device interacts through a communication network; if it is a centralized system, the above-mentioned functional modules can be integrated in one hardware device.

In practical applications, when the input module 400, output module 401, calculation module 402, selection module 403, filter module 404, and judgment module 405 are integrated in one hardware device, the input module 400, output module 401, calculation module Module 402, selection module 403, filter module 404, judgment module 405 can be positioned at central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA) in this hardware equipment )accomplish.

In order to illustrate the embodiment of the present invention more clearly, the training process and detection process of the neural network-based face detection model will be described in detail below with specific embodiments.

Embodiment one

Step S1: When receiving a model training instruction, input the face images in the training set into the neural network for training.

Step S2: Calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the offset of the real face frame relative to the corresponding default face frame information; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame.

Step S3: According to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, calculate the predicted face frame offset The loss function of the network layer; and according to the confidence that the default face frame contains the face, calculate the loss function of the network layer that predicts the confidence of the face frame.

Step S4: Calculate the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame, and feed back the error to the neural network through backpropagation In the network, the network weight parameters of the neural network are updated according to the error and the predicted face frame is adjusted according to the updated network weight parameters.

Step S5: Determine whether the error between the adjusted predicted face frame and the real face frame is within the preset error range; if the adjusted error between the predicted face frame and the real face frame is within the preset error range , then output the face detection model; if the error between the adjusted predicted face frame and the real face frame is not within the preset error range, then go to step S1 and continue to execute according to the adjusted predicted face frame.

Step S6: When receiving the face detection instruction, input the face image to be detected into the trained face detection model for face detection.

Step S7: Output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias in the trained face detection model for the face image to be detected, and pass the training The network layer that predicts the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face.

Step S8: According to each default face frame and the bias information of the predicted face frame relative to each default face frame, calculate the corresponding predicted face frame.

Step S9: Select the predicted face frame corresponding to the confidence greater than the preset confidence threshold in the predicted face frame as the final face detection result, or select the highest confidence in the predicted face frame The predicted face frame corresponding to degree is used as the final face detection result.

Embodiment two

Step S1: When receiving a model training instruction, input the face images in the training set into the neural network for training.

Step S2: Calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the offset of the real face frame relative to the corresponding default face frame information; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame.

Step S3: According to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, calculate the predicted face frame offset The loss function of the network layer; and according to the confidence that the default face frame contains the face, calculate the loss function of the network layer that predicts the confidence of the face frame.

Step S4: Calculate the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame, and feed back the error to the neural network through backpropagation In the network, the network weight parameters of the neural network are updated according to the error and the predicted face frame is adjusted according to the updated network weight parameters.

Step S5: Determine whether the error between the adjusted predicted face frame and the real face frame is within the preset error range; if the adjusted error between the predicted face frame and the real face frame is within the preset error range , then output the face detection model; if the error between the adjusted predicted face frame and the real face frame is not within the preset error range, then go to step S1 and continue to execute according to the adjusted predicted face frame.

Step S6: When receiving the face detection instruction, input the face image to be detected into the trained face detection model for face detection.

Step S7: Output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias in the trained face detection model for the face image to be detected, and pass the training The network layer that predicts the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face.

Step S8: Filter out default face frames whose confidence is less than or equal to a preset confidence threshold.

Step S9: According to each of the remaining default face frames and the offset information of the corresponding predicted face frames relative to each of the remaining default face frames, calculate the corresponding predicted face frame.

Step S10: Use the predicted face frames corresponding to the remaining default face frames as the final face detection result.

Embodiment three

Step S1: When receiving a model training instruction, input the face images in the training set into the neural network for training.

Step S2: Calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the offset of the real face frame relative to the corresponding default face frame information; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame.

Step S3: According to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, calculate the predicted face frame offset The loss function of the network layer; and according to the confidence that the default face frame contains the face, calculate the loss function of the network layer that predicts the confidence of the face frame.

Step S4: Calculate the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame, and feed back the error to the neural network through backpropagation In the network, the network weight parameters of the neural network are updated according to the error and the predicted face frame is adjusted according to the updated network weight parameters.

Step S5: Determine whether the error between the adjusted predicted face frame and the real face frame is within the preset error range; if the adjusted error between the predicted face frame and the real face frame is within the preset error range , then output the face detection model; if the error between the adjusted predicted face frame and the real face frame is not within the preset error range, then go to step S1 and continue to execute according to the adjusted predicted face frame.

Step S6: When receiving the face detection instruction, input the face image to be detected into the trained face detection model for face detection.

Step S7: Output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias in the trained face detection model for the face image to be detected, and pass the training The network layer that predicts the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face.

Step S8: According to each default face frame and the bias information of the predicted face frame relative to each default face frame, calculate the corresponding predicted face frame.

Step S9: Calculate the relative area of each two predicted human face frames; if the relative area of each two predicted human face frames is greater than the preset second relative area threshold, then use the two predicted human face frames as the sampling predicted human face frame.

Step S10: Select the sampled predicted face frame corresponding to the confidence greater than the preset confidence threshold in the sampled predicted face frame as the final face detection result, or, in the sampled predicted face frame Select the predicted face frame corresponding to the highest confidence level as the final face detection result.

Embodiment Four

Step S1: When receiving a model training instruction, input the face images in the training set into the neural network for training.

Step S2: Calculate the offset information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias, and calculate the offset of the real face frame relative to the corresponding default face frame information; and calculate the confidence that each default face frame contains a face through the network layer that predicts the confidence of the face frame.

Step S3: According to the offset information of the predicted face frame relative to the corresponding default face frame, and the offset information of the real face frame relative to the corresponding default face frame, calculate the predicted face frame offset The loss function of the network layer; and according to the confidence that the default face frame contains the face, calculate the loss function of the network layer that predicts the confidence of the face frame.

Step S4: Calculate the error between the loss function of the network layer that predicts the bias of the face frame and the loss function of the network layer that predicts the confidence of the face frame, and feed back the error to the neural network through backpropagation In the network, the network weight parameters of the neural network are updated according to the error and the predicted face frame is adjusted according to the updated network weight parameters.

Step S5: Determine whether the error between the adjusted predicted face frame and the real face frame is within the preset error range; if the adjusted error between the predicted face frame and the real face frame is within the preset error range , then output the face detection model; if the error between the adjusted predicted face frame and the real face frame is not within the preset error range, then go to step S1 and continue to execute according to the adjusted predicted face frame.

Step S6: When receiving the face detection instruction, input the face image to be detected into the trained face detection model for face detection.

Step S7: Output the bias information of the predicted face frame relative to the corresponding default face frame through the network layer of the predicted face frame bias in the trained face detection model for the face image to be detected, and pass the training The network layer that predicts the confidence of the face frame in the face detection model outputs the confidence that each default face frame contains a face.

Step S8: Filter out default face frames whose confidence is less than or equal to a preset confidence threshold.

Step S9: According to each of the remaining default face frames and the offset information of the corresponding predicted face frames relative to each of the remaining default face frames, calculate the corresponding predicted face frame.

Step S10: Calculate the relative area of each two predicted human face frames; if the relative area of each two predicted human face frames is greater than the preset second relative area threshold, then use the two predicted human face frames as the sampling predicted human face frame. face frame;

Step S11: Take the sampled predicted face frame as the final face detection result, or select the sampled predicted face frame corresponding to the highest confidence level among the sampled predicted face frames as the final face detection result.

In summary, the training method and system of the face detection model based on the neural network, the face detection method and the system of the present invention have the following beneficial effects compared with the prior art:

(1) In the embodiment of the present invention, it is not necessary to select the observation window in a sliding window manner, nor to construct an image pyramid or use a multi-scale observation window and to calculate and discriminate a large number of observation windows, but according to the neural network Different network layers correspond to different sizes of receptive fields in the original face image. The network layer used for face detection is selected. The higher the number of network layers corresponds to the larger the receptive field of the original face image, the lower the number of network layers corresponds to The smaller the receptive field of the original face image, the face detection is performed directly through the face detection network layer. Compared with the existing technology, the calculation amount is small, and the low-level network layer can be selected to detect small-sized faces. The layer is used to detect large-sized human faces, realize multi-scale detection and bias regression of human faces, and make the predicted human face frame better contain human faces. Compared with the prior art, the human face detection of the embodiment of the present invention is more accurate and Face detection is faster.

(2) In the embodiment of the present invention, the model training directly adopts the end-to-end mode, and the face image in the input training set directly outputs the position and size of the corresponding predicted face frame, which is simpler and faster than the existing multi-step method; and, Due to the end-to-end training mode, the network weight parameters of the neural network are directly fed back according to the error between the loss function of the network layer for predicting the bias of the face frame and the loss function of the network layer for the confidence of the predicted face frame Adjust to make the predicted face frame and the real face frame closer, so that the predicted face frame contains the face more accurately; therefore, it has a higher detection rate than the method based on multiple independent sub-steps in the prior art .

(3) In the embodiment of the present invention, it is not necessary to zoom the input face image, but directly input the face image in the training set into the neural network to train the face detection model, and directly input the face image to be detected into Face detection is performed in the trained face detection model, which can avoid the influence of face image stretching, distortion, deformation and other factors on the face detection results.

The above-mentioned embodiments only illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed in the present invention should still be covered by the claims of the present invention.

Claims (10)

1. A training method of a face detection model based on a neural network is characterized in that the neural network comprises the following steps: a network layer for face detection, a network layer for predicting face frame bias and a network layer for predicting face frame confidence; the network layer of the face detection is selected according to the receptive fields of face images in training sets corresponding to different network layers in the neural network, each cell element in the network layer of the face detection is bound with six default face frames, and the default face frames are set according to the scale of the network layer of the corresponding face detection; each layer of network layer for face detection is connected with a layer of network layer for predicting face frame bias and a layer of network layer for predicting face frame confidence; the method comprises the following steps:

when a model training instruction is received, inputting the face images in the training set into the neural network for training;

calculating the offset information of the predicted face frame relative to the corresponding default face frame and calculating the offset information of the real face frame relative to the corresponding default face frame through the network layer of the offset of the predicted face frame; calculating the confidence coefficient of each default face frame including the face through a network layer for predicting the confidence coefficient of the face frame;

calculating a loss function of a network layer of the offset of the predicted face frame according to the offset information of the predicted face frame relative to the corresponding default face frame and the offset information of the real face frame relative to the corresponding default face frame; calculating a loss function of a network layer for predicting the confidence of the face frame according to the confidence of the face contained in the default face frame;

calculating the error of the loss function of the network layer of the predicted face frame bias and the loss function of the network layer of the predicted face frame confidence, feeding the error back to the neural network through back propagation, updating the network weight parameter of the neural network according to the error and adjusting the predicted face frame according to the updated network weight parameter;

and repeating iterative training until the error between the adjusted predicted face frame and the actual face frame is within a preset error range, and outputting a face detection model.

2. The training method according to claim 1, wherein the calculating, by the network layer of the predicted face frame bias, bias information of the predicted face frame with respect to a corresponding default face frame includes:

calculating the bias information of the predicted face frame relative to the corresponding default face frame according to the following formula:

t_x＝(x-x_a)/w_a,t_y＝(y-y_a)/h_a

t_w＝log(w/w_a),t_h＝log(h/h_a)

wherein, (x, y, w, h) is the coordinate of the central point of the predicted face frame, the width and the height; (x)_a,y_a,w_a,h_a) Is a default face frameThe center point coordinates, width and height of; (t)_x,t_y,t_w,t_h) Bias information for the predicted face frame relative to a corresponding default face frame;

the calculating of the bias information of the real face frame relative to the corresponding default face frame includes:

calculating the bias information of the real face frame relative to the corresponding default face frame according to the following formula:

wherein (x)_a,y_a,w_a,h_a) The coordinates, width and height of the center point of the default face frame are used; (x)^*,y^*,w^*,h^*) Coordinates, width and height of a central point of a real face frame;and the offset information of the real face frame relative to the corresponding default face frame is obtained.

3. The training method according to claim 2, wherein the calculating a loss function of a network layer of the predicted face frame bias according to the bias information of the predicted face frame relative to the corresponding default face frame and the bias information of the real face frame relative to the corresponding default face frame comprises:

selecting the default face frame corresponding to the situation that the relative area of the default face frame relative to the corresponding real face frame is larger than a preset first relative area threshold value as a sampling default face frame; the relative area is the area of the intersection area of the default face frame and the real face frame divided by the area of the union area of the default face frame and the real face frame;

based on the predicted face frameCalculating the offset information relative to the corresponding sampling default face frame and the offset information of the real face frame relative to the corresponding sampling default face frame according to the following formula to predict the Loss function Loss of the network layer of the face frame offset₁：

Wherein N is_regDefault number of face boxes for sampling, L_regRegression loss function of spatial bias corresponding to k-th default face box, k being 1 and N_regPositive integer between, T ═ T_x，t_y，t_w，t_hIs the bias information of the predicted face frame with respect to the corresponding sampled default face frame, z is an element belonging to the set { x, y, w, h }, T ═ T }^* _x，t^* _y，t^* _w，t^* _hThe's is the offset information of the real face frame relative to the corresponding sampling default face frame, smooth_L1Represents the smoothing L1 loss function, which is a variation of the L1 norm loss function, L represents smooth_L1An input variable of the function;

the calculating a loss function of a network layer for predicting the confidence of the face frame according to the confidence that the default face frame contains the face comprises:

taking the default face frame corresponding to the situation that the relative area of the default face frame relative to the corresponding real face frame is larger than a preset first relative area threshold value as a positive sample, and taking the default face frame corresponding to the situation that the relative area of the default face frame relative to the corresponding real face frame is smaller than or equal to the preset first relative area threshold value as a negative sample;

selecting all positive samples and part of negative samples according to a preset positive-negative sample proportion;

calculating Loss function Loss of the network layer for predicting the confidence coefficient of the face frame according to the following formula₂：

L_cls(p,p^*)＝-[p^*logp+(1-p^*)log(1-p)]

Wherein N is_clsFor the total number of positive and negative samples selected,classification loss function corresponding to ith class, i being 1 and N_clsP is the confidence that the selected positive sample or negative sample contains the face, p^*For the true probability that the selected positive or negative sample contains a face, p of the positive sample^*Is 1, p of negative example^*Is 0.

4. A training method as claimed in any one of claims 1 to 3, wherein said calculating an error between the loss function of the network layer for the predicted face frame bias and the loss function of the network layer for the predicted face frame confidence comprises:

calculating the gradient of the loss function of the network layer of the predicted face frame bias and the gradient of the loss function of the network layer of the predicted face frame confidence by adopting a random gradient descent method;

and taking the obtained gradient value as the error between the loss function of the network layer of the predicted face frame bias and the loss function of the network layer of the confidence coefficient of the predicted face frame.

5. A face detection method based on a neural network is characterized in that the neural network comprises the following steps: a network layer for face detection, a network layer for predicting face frame bias and a network layer for predicting face frame confidence; the network layer of the face detection is selected according to the receptive fields of face images in training sets corresponding to different network layers in the neural network, each cell element in the network layer of the face detection is bound with six default face frames, and the default face frames are set according to the scale of the network layer of the corresponding face detection; each layer of network layer for face detection is connected with a layer of network layer for predicting face frame bias and a layer of network layer for predicting face frame confidence; the method comprises the following steps:

when a face detection instruction is received, inputting a face image to be detected into the face detection model according to any one of claims 1 to 4 for face detection;

for the face image to be detected, outputting the offset information of the predicted face frame relative to the corresponding default face frame through the network layer for predicting the offset of the face frame in the face detection model, and outputting the confidence coefficient of each default face frame including the face through the network layer for predicting the confidence coefficient of the face frame in the face detection model;

calculating corresponding predicted face frames according to each default face frame and the bias information of the corresponding predicted face frame relative to each default face frame;

and selecting the predicted face frame corresponding to the confidence coefficient greater than a preset confidence coefficient threshold value from the predicted face frames as a final face detection result, or selecting the predicted face frame corresponding to the highest confidence coefficient from the predicted face frames as the final face detection result.

6. The detection method according to claim 5, wherein the calculating the corresponding predicted face frame according to each default face frame and the offset information of the corresponding predicted face frame relative to each default face frame comprises:

according to each default face frame and the bias information of the corresponding predicted face frame relative to each default face frame, calculating the corresponding predicted face frame according to the following formula:

x＝t_x*w_a+x_a,y＝t_y*h_a+y_a

wherein (x)_a,y_a,w_a,h_a) Coordinates, width and height of a center point of each default face frame; (x, y, w, h) coordinates, width and height of a center point of the predicted face frame corresponding to each default face frame; (t)_x,t_y,t_w,t_h) Bias information for the corresponding predicted face frame relative to each default face frame.

7. The detection method according to claim 5 or 6, wherein after outputting, for the face image to be detected, the offset information of the predicted face frame relative to the corresponding default face frame through the network layer for predicting the offset of the face frame in the trained face detection model, and outputting the confidence that each default face frame contains a face through the network layer for predicting the confidence of the face frame in the trained face detection model, the method further comprises:

filtering out a default face frame with the confidence coefficient smaller than or equal to a preset confidence coefficient threshold value;

calculating corresponding predicted face frames according to the rest default face frames and the bias information of the corresponding predicted face frames relative to the rest default face frames;

and taking the predicted face frames corresponding to the rest default face frames as final face detection results.

8. The method of claim 7, wherein after computing the corresponding predicted face frame according to each default face frame and the offset information of the corresponding predicted face frame relative to each default face frame, the method further comprises:

calculating the relative area of every two predicted face frames;

if the relative area of every two predicted face frames is larger than a preset second relative area threshold value, taking the two predicted face frames as sampling predicted face frames; wherein the relative area is the area of the intersection region of the two predicted face frames divided by the area of the union region of the two predicted face frames;

and selecting the sampling prediction face frame corresponding to the confidence coefficient which is greater than a preset confidence coefficient threshold value from the sampling prediction face frames as a final face detection result, or selecting the sampling prediction face frame corresponding to the highest confidence coefficient from the sampling prediction face frames as the final face detection result.

9. A training system for a face detection model based on a neural network, the neural network comprising: a network layer for face detection, a network layer for predicting face frame bias and a network layer for predicting face frame confidence; the network layer of the face detection is selected according to the receptive fields of face images in training sets corresponding to different network layers in the neural network, each cell element in the network layer of the face detection is bound with six default face frames, and the default face frames are set according to the scale of the network layer of the corresponding face detection; each layer of network layer for face detection is connected with a layer of network layer for predicting face frame bias and a layer of network layer for predicting face frame confidence; the system comprises: the device comprises an input module, a first calculation module, a second calculation module, a third calculation module, a feedback and update module and an iteration output module; wherein,

the input module is used for inputting the face images in the training set into the neural network for training when receiving a model training instruction;

the first calculation module is used for calculating the offset information of the predicted face frame relative to the corresponding default face frame and calculating the offset information of the real face frame relative to the corresponding default face frame through the network layer of the offset of the predicted face frame; calculating the confidence coefficient of each default face frame including the face through a network layer for predicting the confidence coefficient of the face frame;

the second calculation module is used for calculating a loss function of a network layer of the offset of the predicted face frame according to the offset information of the predicted face frame relative to the corresponding default face frame and the offset information of the real face frame relative to the corresponding default face frame; calculating a loss function of a network layer for predicting the confidence of the face frame according to the confidence of the face contained in the default face frame;

the third calculation module is used for calculating the error between the loss function of the network layer of the predicted face frame bias and the loss function of the network layer of the predicted face frame confidence;

the feedback and updating module is used for feeding the error back to the neural network through back propagation, updating the network weight parameters of the neural network according to the error and adjusting the predicted face frame according to the updated network weight parameters;

and the iteration output module is used for repeating iteration training until the error between the adjusted predicted face frame and the adjusted real face frame is within a preset error range, and outputting a face detection model.

10. A face detection system based on a neural network, the neural network comprising: a network layer for face detection, a network layer for predicting face frame bias and a network layer for predicting face frame confidence; the network layer of the face detection is selected according to the receptive fields of face images in training sets corresponding to different network layers in the neural network, each cell element in the network layer of the face detection is bound with six default face frames, and the default face frames are set according to the scale of the network layer of the corresponding face detection; each layer of network layer for face detection is connected with a layer of network layer for predicting face frame bias and a layer of network layer for predicting face frame confidence; the system comprises: the device comprises an input module, an output module, a calculation module and a selection module; wherein,

the input module is used for inputting a face image to be detected into a trained face detection model for face detection when receiving a face detection instruction;

the output module is used for outputting the offset information of the predicted face frame relative to the corresponding default face frame by the network layer for predicting the offset of the face frame by the face detection model according to any one of claims 1 to 4 aiming at the face image to be detected, and outputting the confidence coefficient of each default face frame including the face by the network layer for predicting the confidence coefficient of the face frame by the face detection model;

the calculation module is used for calculating corresponding predicted face frames according to each default face frame and the bias information of the predicted face frames relative to each default face frame;

and the selection module is used for selecting the predicted face frame corresponding to the confidence coefficient which is greater than a preset confidence coefficient threshold value from the predicted face frames as a final face detection result, or selecting the predicted face frame corresponding to the highest confidence coefficient from the predicted face frames as the final face detection result.