CN103778414A - Real-time face recognition method based on deep neural network - Google Patents

Abstract

The invention provides a real-time face recognition method based on a deep neural network and adjacent element analysis. According to the invention, at first, a multi-layer neural network is practiced through a large-scale face database having the advantage of good diversity, wherein layers besides the last layer are non-linear layers, and the tail layer is a linear layer; then, the obtained network is practiced continuously on the basis of a hybrid face database through the supervised adjacent element analysis method to make the network has more deep understanding of face images so as to achieve the purposes of shorting the face image distance between same individuals and increasing the face image distance between different individuals; and finally, in the actual face recognition stage, the invention provides a concept of 'search radius', so the recognition time can be shortened on the premise of ensuring the recognition rate to realize real-time face recognition. According to the invention, the advantage of high recognition rate can be realized, and at the same time, according to the invention, the advantage of fast recognition speed can be realized, so that the methof of the invention is suitable for being applied in real-time face recognition tasks.

Description

Real-time face recognition methods based on degree of depth neural network

Technical field

The invention belongs to area of pattern recognition, relate to a kind of face identification method, particularly a kind of face identification method that can carry out in real time recognition of face task.

Background technology

As the one of biological identification technology, recognition of face is by feat of characteristics such as untouchable, good user's experience, and the discrimination growing steadily, and has huge market potential and scientific research value.Recognition of face belongs to the one of image recognition, the heavy difficult point of image recognition is to give the ability of machine perception implicit information that image contains, and as a kind of feature extracting method that extracts data the deep information, degree of depth neural network has certain inspiration to the face recognition technology based on image.

At present, degree of depth neural network has had multinomial breakthrough at area of pattern recognition: Microsoft adopts degree of depth nerual network technique to carry out speech recognition, has reached the highest now phonetic recognization rate; Baidu adopt the degree of depth convolution networking that Yann professor LeCun of New York University proposes Baidu know figure and wait application, and set up the Baidu degree of depth and learnt research institute; Latest news is pointed out, can accomplish the degree of thinking independently on the basis of degree of depth learning system large scale database in " study " is stored in server that Google builds.

But, rarely have researcher by degree of depth Application of Neural Network in recognition of face field, applicant thinks that reason is 2 points: it is that each individuality is set up a model that the framework of degree of depth neural network+softmax that one side is traditional needs multiple images, poor for the uncertain task applicability of this class number of recognition of face; On the other hand, the needed dimension of picture of recognition of face is larger, and generally, more than 30 × 30 pictures just can reach promising result, has so just strengthened the difficulty of training degree of depth network model.

Summary of the invention

For probing into the recognition effect of degree of depth neural network for recognition of face task, the present invention proposes a kind of real-time face recognition methods based on degree of depth neural network.First utilize the preferably extensive face database of diversity to train a kind of multilayer neural network, wherein every layer except last one deck is non-linear layer, and tail layer is linear layer; Then utilize the method for the contiguous meta analysis that has supervision on the basis of mixing face database, to continue the network that training obtains, network is deepened for the understanding of facial image, reach the object that shortens facial image distance between same individuality, increases facial image distance between Different Individual, last in the actual recognition of face stage, the present invention proposes the concept of one " search radius ", under the prerequisite of guaranteeing discrimination, shorten identification required time, realize real-time recognition of face.

In order to realize above object, the inventive method mainly comprises the following steps:

Step (1). obtain network training data, specifically: choose the good face database of diversity as extensive face database.In addition, from multiple face databases, choose the synthetic face database that mixes of a part of image sets, wherein the stronger image of part illumination variation is carried out to unitary of illumination operation, reduce the impact of illumination.People face part in image during employing Viola-Jones human-face detector detects and shears extensive face database and mixes face database.By extensive face database with mix face database in the pixel value of every image be stretched as row by row or column, being combined into line number is that image pixel is counted, columns is the matrix of image number, matrix, divided by 255, makes data be distributed in 0-1 scope.

Step (2). use on the basis of face parts of images in the extensive face database that degree of depth neural network obtains in step (1) with without monitor mode training degree of depth neural network, specific algorithm is as follows.

The present invention is take Autoencoder degree of depth neural network as framework, by recovering facial image in extensive face database, to reach the object of training degree of depth neural network, first use the every layer network of reverse conduction Algorithm for Training, last same with the general performance of reverse conduction algorithm adjustment network.The concrete step of algorithm is poly-as follows:

1) the weight penalty factor of each layer network of initialization, weight scaling, weighted value, bias,

The parameters such as integrated data size.Weight penalty factor is to control the network weight mistake that does not allow training obtain

Greatly, cause over-fitting, suppose that initialized weight proportion is W _s, have:

W _s＝sqrt(6)/sqrt(v+h+1)(1)

In formula (1), v represents visible layer nodes, and h represents hidden layer node number.Can obtain:

W＝2 ^*w _s ^*(rand(h,v)-0.5)(2)

W′＝2 ^*w _s ^*(rand(v,h)-0.5)(3)

In formula (2) (3), W, W ' are respectively the initializes weights of visible layer, hidden layer, and rand (m, n) is for being created on the function of m × n matrix of uniform random number composition between (0,1).

2) determine the loss function of network.The target of Autoencoder is to update network, strengthens the recovery capability of network for raw data, and the loss function of network is:

$\begin{array}{c}J(W,b)=\left[\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}J(W,b;x^{\left(i\right)},y^{\left(i\right)})\right]+\frac{\mathrm{\λ}}{2}\underset{l=1}{\overset{n_t-1}{\mathrm{\Σ}}}\underset{i=1}{\overset{s_t}{\mathrm{\Σ}}}\underset{j=1}{\overset{s_{l+1}}{\mathrm{\Σ}}}{\left(W_{\mathrm{ji}}^{\left(l\right)}\right)}^2\\ =\left[\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\left(\frac{1}{2}\right||r_{W,b}\left(x^{\left(i\right)}\right)-y^{\left(i\right)}{\left|\right|}^2)\right]+\frac{\mathrm{\λ}}{2}\underset{l=1}{\overset{n_l-1}{\mathrm{\Σ}}}\underset{i=1}{\overset{s_l}{\mathrm{\Σ}}}\underset{j=1}{\overset{s_{l+1}}{\mathrm{\Σ}}}{\left(W_{\mathrm{ji}}^{\left(i\right)}\right)}^2\end{array}\left(4\right)$

In formula (4), m is data dimension, i.e. visible node layer number, and r is the data of rebuilding through network, and W, b are respectively weight and biasing, and y be actual input, and λ is that weight is punished parameter, object is to avoid weight excessive and cause over-fitting, n _tfor the number of plies, s _t, s _t+1for input layer and output layer nodes.

3) ask for every layer network loss function for weight and bigoted partial derivative.For convenience of calculation, the present invention uses a kind of intermediate variable a, δ ₁, δ ₂, for linear layer, have:

a＝W ^*y+b(5)

δ ₂＝-(y-r _W,b(x))(6)

δ ₁＝((W′) ^T*δ ₂)(7)

For non-linear layer, suppose that activation function is sigmoid, has:

a＝1./(1+exp(-(W ^*y+b)))(8)

δ ₂＝-(y-r _W,b(x)). ^*r _W,b(x). ^*(1-r _W,b(x))(9)

δ ₁＝((W′) ^T*δ ₂). ^*a. ^*(1-a)(10)

Objective function for the partial derivative of visible layer, hidden layer weight and biasing is:

$\frac{\∂J}{\∂W}=(\frac{1}{m}*{\mathrm{\δ}}_1*y)+\mathrm{\λsum}\left(W\right)---\left(11\right)$

$\frac{\∂J}{\∂b}=\frac{1}{m}*\mathrm{sum}({\mathrm{\δ}}_1,2)---\left(12\right)$

$\frac{\∂J}{{\∂W}^{\′}}=(\frac{1}{m}*{\mathrm{\δ}}_2*{\mathrm{\α}}^T)+\mathrm{\λsum}\left(W^{\′}\right)---\left(13\right)$

$\frac{\∂J}{\∂b^{\′}}=\frac{1}{m}*\mathrm{sum}({\mathrm{\δ}}_2,2)---\left(14\right)$

In formula (11) (12) (13) (14), sum (x), sum (x, 2) be respectively ask for x all elements and, ask for the function of all row sums of x.Try to achieve after partial derivative, utilize gradient descent method to upgrade weight and biasing.

4) after the weight training of every layer, regard all-network as an entirety, utilize gradient descent method to upgrade each layer of weight, further improve the recovery capability of network for data.

Step (3). to mix face database as input data, the degree of depth neural network that step (2) is obtained is carried out Training, improves the cognitive ability of network for face.

Here the present invention adopts the method for contiguous meta analysis (Neighbourhood Components Analysis) to shorten the distance of facial image between same individual, increases the distance of facial image between Different Individual, and then improves accuracy of face identification.

Contiguous meta analysis is a kind of learning distance metric method, and it defines the distance of single data field remainder data in new transformed space with a square of Euclidean distance, and function definition is as follows:

$P_{\mathrm{ij}}=\left\{\begin{array}{cc}\frac{{\exp }^{-\left|\right|{\mathrm{Ax}}_i-{\mathrm{Ax}}_j{\left|\right|}^2}}{{\mathrm{\Σ}}_k{\exp }^{-\left|\right|{\mathrm{Ax}}_i-{\mathrm{Ax}}_j{\left|\right|}^2}}& j\≠i\\ 0& j=i\end{array}\right.\left(15\right)$

In formula (15), x _i, x _j, x _kbe respectively i, j, the corresponding mapping (enum) data of the individual facial image in k position, A is mapping space, is expressed as certain layer that neural network is concrete in the present invention.Definition loss function is f, is determined by following formula:

$\underset{i}{\mathrm{\Σ}}\underset{j\∈C_i}{\mathrm{\Σ}}{P}_{\mathrm{ij}}=\underset{i}{\mathrm{\Σ}}{P}_i---\left(16\right)$

In formula (16), C _ifor the corresponding face set of individual i.Same step (2), the present invention is by asking for the partial derivative of loss function f for every layer network

, utilize gradient descent method iteration optimization network model, improve the understandability of network for face difference.

Step (4). through above step, can obtain well recovering and deep layer network that can deep understanding facial image.For concrete identifying, the present invention proposes the concept of one " search radius (SearchRadius) ", concrete steps are:

For the multidimensional data after face mapping, use following formula to ask for a binary representation data y _ab:

$y_{\mathrm{\&alpha;b}}=\left\{\begin{array}{cc}1& x_{\mathrm{\&alpha;b}\&GreaterEqual;m_b}\\ 0& x_{\mathrm{\&alpha;b}}<m_b\end{array}\right.---\left(17\right)$

In formula (17), x _abbe the b dimension component that a opens facial image, m _bfor the intermediate value of face images b dimension component.This formula has guaranteed being uniformly distributed for every one dimension component 0 and 1 of token image.Here binary representation data y, _abalso can be regarded as the Hash feature of energy token image.

In actual identifying, the view data to be identified representing according to degree of depth network is asked for this binary form registration, use all images in binary form registration and face database to ask for Hamming distance, according to the size of set search radius R, use the front R of Hamming distance minimum to open image as secondary contrast images, once obtain the Euclidean distance with image to be identified, Euclidean distance and the threshold value of gained minimum compare, and are less than or equal to threshold value and are defined as this facial image (threshold value by etc. wrong rate ERR determine).

The present invention, compared with existing many face identification methods, has following features:

Degree of depth neural network can be extracted the further feature of image, be suitable for this task higher to image understanding Capability Requirement of recognition of face, network training process of the present invention comprises without monitor procedure and has monitor procedure simultaneously, the main training network of monitor procedure of recovery capability without to(for) data, there is monitor procedure to be used for the understandability of training network for face, the extensive face database using without monitor procedure has guaranteed the applicability of network for the face beyond training data, there is in monitor procedure each individuality contain and comprise multi-pose, multiple expression, multiple images of many illumination, for e-learning Different Individual facial image why not the same and same individual facial image equally have effect why, therefore recognition of face rate of the present invention has kept higher level.

For concrete identifying, guaranteeing under the prerequisite of discrimination, the proposition of " search radius " concept, get around image to be identified and storehouse and neutralized the directly contrast of the larger image do of this image Hamming distance, so just avoided with image library in all image autumn rain Euclidean distances, greatly reduce identification required time.

Accompanying drawing explanation

Fig. 1 is the invention process process flow diagram;

Fig. 2 is part facial image in the extensive face database of the invention process;

Fig. 3 is that the invention process is mixed part facial image in face database;

Fig. 4 is that the invention process is trained the ground floor network weight obtaining;

Fig. 5 is the invention process FAR and FRR curve;

Fig. 6 is that the invention process recognition time is with " search radius " change curve;

Fig. 7 is that the invention process recognition correct rate is with " search radius " change curve;

Embodiment

Below in conjunction with accompanying drawing, embodiments of the invention are elaborated: the present embodiment is implemented under take technical solution of the present invention as prerequisite, provided detailed embodiment and concrete operating process.

As shown in Figure 1, the present embodiment comprises the steps:

Step 1. obtain network training data, specifically: use diversity good LFW face database as without network training storehouse in monitor procedure, referring to Fig. 2, use parts of images in CMU-PIE, Georgia Tech, CaltechFaces and VidTiMIT face database to be combined into mixing face database (containing 2311 images, each individuality contains multiple images under multiple states), referring to Fig. 3, as the training data having in monitor procedure, wherein the stronger image of part illumination variation is carried out to unitary of illumination operation, reduce the impact of illumination.People face part in image during employing Viola-Jones human-face detector detects and shears LFW and mixes face database.By extensive face database with mix face database in the pixel value of every image be stretched as row by row or column, being combined into line number is that image pixel is counted, columns is the matrix of image number, matrix, divided by 255, makes data be distributed in 0-1 scope.

Step 2. use on the basis of the LFW face parts of images that degree of depth neural network obtains in step 1 to train a kind of three layer depth neural networks without monitor mode, two-layer is wherein non-linear layer, activation function is sigmoid, tail layer is linear layer, net structure is 4096-256-128-64, and specific algorithm is as follows.

(1) parameter of the every layer network of initialization.

(2) ask the loss function of determining network.The network recovery data that the loss function here refers to and input data least mean-square error, add the penalty term to network weight, prevents network over-fitting.

(3) ask the loss function of determining network for the weight of network and the partial derivative of biasing, then utilize Gradient Descent method progressively to upgrade weight, here, gradient descent method adopts L-BFGS method.

Step 3. adopt contiguous first method on the basis of 3/4 mixing face database (approximately containing 2310 images), the network obtaining further to be trained, same, also adopt L-BFGS method progressively to upgrade weight here.

Step 4. remaining 1/4 to mixing in face database, be about 770 images, as test pattern.In test process, the Hamming distance of binary form registration between first movement images, in the size of basis " search radius " R, choose the image of the front R of Hamming distance as image to be selected, carry out secondary contrast, secondary contrast adopts Euclidean distance, and the minimum image of distance is as target image.In this actual identifying, this enforcement is used different " search radius ", probes into the validity of " search radius " concept of the present invention's proposition.

The network weight being obtained by this enforcement of Fig. 4 can find out, facial contour is more clear, and known the present invention trains the degree of depth neural network obtaining better for the recovery capability of facial image.

As seen from Figure 5, the false rejection rate FRR of this enforcement and the intersection point ERR of false acceptance rate FAR are lower, illustrate that the discrimination of algorithm is higher, and robustness is stronger, and adaptability is better, is easily generalized in general recognition of face task.

As seen from Figure 6 along with the continuous increase of " search radius ", identification required time also presents linear growth, known in the time that " search radius " reaches 100 left and right by table 4.11, recognition correct rate reaches steady state (SS), is stabilized in 94% left and right, in the time that " search radius " reaches 100, check corresponding Fig. 7, obtaining 770 images, to be all completed recognition time be 1.5833s, and every face picture of average test needs 2.056ms, meets the required condition of real-time face recognition system.

Claims (1)

1. the real-time face recognition methods based on degree of depth neural network, is characterized in that the method comprises the steps:

Step (1). obtain network training data, specifically: choose the good face database of diversity as extensive face database; In addition, from multiple face databases, choose the synthetic face database that mixes of a part of image sets, wherein the stronger image of part illumination variation is carried out to unitary of illumination operation, reduce the impact of illumination; People face part in image during employing Viola-Jones human-face detector detects and shears extensive face database and mixes face database; By extensive face database with mix face database in the pixel value of every image be stretched as row by row or column, being combined into line number is that image pixel is counted, columns is the matrix of image number, matrix, divided by 255, makes data be distributed in 0-1 scope;

Step (2). use on the basis of face parts of images in the extensive face database that degree of depth neural network obtains in step (1) with without monitor mode training degree of depth neural network, specific algorithm is as follows;

1) the weight penalty factor of each layer network of initialization, weight scaling, weighted value, bias, integrated data size; Weight penalty factor is to control not allow to train the network weight obtaining excessive, causes over-fitting, supposes that initialized weight proportion is W _s, have:

W _sin=sqrt (6)/sqrt (v+h+1) (1) formula (1), v represents visible layer nodes, and h represents hidden layer node number; Can obtain:

W＝2 ^*w _s ^*(rand(h,v)-0.5)(2)

W′＝2 ^*w _s ^*(rand(v,h)-0.5)(3)

In formula (2) (3), W, W ' are respectively the initializes weights of visible layer, hidden layer, and rand (m, n) is for being created on the function of m × n matrix of uniform random number composition between (0,1);

2) determine the loss function of network; The target of Autoencoder is to update network, strengthens the recovery capability of network for raw data, and the loss function of network is:

$\begin{array}{c}J(W,b)=\left[\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}J(W,b;x^{\left(i\right)},y^{\left(i\right)})\right]+\frac{\mathrm{\&lambda;}}{2}\underset{l=1}{\overset{n_t-1}{\mathrm{\&Sigma;}}}\underset{i=1}{\overset{s_t}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{s_{l+1}}{\mathrm{\&Sigma;}}}{\left(W_{\mathrm{ji}}^{\left(l\right)}\right)}^2\\ =\left[\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}\left(\frac{1}{2}\right||r_{W,b}\left(x^{\left(i\right)}\right)-y^{\left(i\right)}{\left|\right|}^2)\right]+\frac{\mathrm{\&lambda;}}{2}\underset{l=1}{\overset{n_l-1}{\mathrm{\&Sigma;}}}\underset{i=1}{\overset{s_l}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{s_{l+1}}{\mathrm{\&Sigma;}}}{\left(W_{\mathrm{ji}}^{\left(i\right)}\right)}^2\end{array}\left(4\right)$

In formula (4), m is data dimension, i.e. visible node layer number, and r is the data of rebuilding through network, and W, b are respectively weight and biasing, and y be actual input, and λ is that weight is punished parameter, object is to avoid weight excessive and cause over-fitting, n _tfor the number of plies, s _t, s _t+1for input layer and output layer nodes;

3) ask for every layer network loss function for weight and bigoted partial derivative; For convenience of calculation, use a kind of intermediate variable a, δ ₁, δ ₂, for linear layer, have:

a＝W ^*y+b(5)

δ ₂＝-(y-r _W,b(x))(6)

δ ₁＝((W′) ^T*δ ₂)(7)

For non-linear layer, suppose that activation function is sigmoid, has:

a＝1./(1+exp(-(W ^*y+b)))(8)

δ ₂＝-(y-r _W,b(x)). ^*r _W,b(x). ^*(1-r _W,b(x))(9)

δ ₁＝((W′) ^T*δ ₂). ^*a. ^*(1-a)(10)

Objective function for the partial derivative of visible layer, hidden layer weight and biasing is:

$\frac{\&PartialD;J}{\&PartialD;W}=(\frac{1}{m}*{\mathrm{\&delta;}}_1*y)+\mathrm{\&lambda;sum}\left(W\right)---\left(11\right)$

$\frac{\&PartialD;J}{\&PartialD;b}=\frac{1}{m}*\mathrm{sum}({\mathrm{\&delta;}}_1,2)---\left(12\right)$

$\frac{\&PartialD;J}{{\&PartialD;W}^{\&prime;}}=(\frac{1}{m}*{\mathrm{\&delta;}}_2*{\mathrm{\&alpha;}}^T)+\mathrm{\&lambda;sum}\left(W^{\&prime;}\right)---\left(13\right)$

$\frac{\&PartialD;J}{\&PartialD;b^{\&prime;}}=\frac{1}{m}*\mathrm{sum}({\mathrm{\&delta;}}_2,2)---\left(14\right)$

In formula (11) (12) (13) (14), sum (x), sum (x, 2) be respectively ask for x all elements and, ask for the function of all row sums of x; Try to achieve after partial derivative, utilize gradient descent method to upgrade weight and biasing;

4) after the weight training of every layer, regard all-network as an entirety, utilize gradient descent method to upgrade each layer of weight, further improve the recovery capability of network for data;

Step (3). to mix face database as input data, the degree of depth neural network that step (2) is obtained is carried out Training, improves the cognitive ability of network for face;

Here adopt the method for contiguous meta analysis to shorten the distance of facial image between same individual, increase the distance of facial image between Different Individual, and then improve accuracy of face identification;

Contiguous meta analysis is a kind of learning distance metric method, and it defines the distance of single data field remainder data in new transformed space with a square of Euclidean distance, and function definition is as follows:

$P_{\mathrm{ij}}=\left\{\begin{array}{cc}\frac{{\exp }^{-\left|\right|{\mathrm{Ax}}_i-{\mathrm{Ax}}_j{\left|\right|}^2}}{{\mathrm{\&Sigma;}}_k{\exp }^{-\left|\right|{\mathrm{Ax}}_i-{\mathrm{Ax}}_j{\left|\right|}^2}}& j\&NotEqual;i\\ 0& j=i\end{array}\right.\left(15\right)$

In formula (15), x _i, x _j, x _kbe respectively i, j, the corresponding mapping (enum) data of the individual facial image in k position, A is mapping space, is expressed as certain layer that neural network is concrete; Definition loss function is f, is determined by following formula:

$\underset{i}{\mathrm{\&Sigma;}}\underset{j\&Element;C_i}{\mathrm{\&Sigma;}}{P}_{\mathrm{ij}}=\underset{i}{\mathrm{\&Sigma;}}{P}_i---\left(16\right)$

In formula (16), C _ifor the corresponding face set of individual i; By asking for the partial derivative of loss function f for every layer network

, utilize gradient descent method iteration optimization network model, improve the understandability of network for face difference;

Step (4). for the multidimensional data after face mapping, use following formula to ask for a binary representation data y _ab:

$y_{\mathrm{\&alpha;b}}=\left\{\begin{array}{cc}1& x_{\mathrm{\&alpha;b}\&GreaterEqual;m_b}\\ 0& x_{\mathrm{\&alpha;b}}<m_b\end{array}\right.---\left(17\right)$

In formula (17), x _abbe the b dimension component that a opens facial image, m _bfor the intermediate value of face images b dimension component; This formula has guaranteed being uniformly distributed for every one dimension component 0 and 1 of token image;

In identifying, the view data to be identified representing according to degree of depth network is asked for this binary form registration, use all images in binary form registration and face database to ask for Hamming distance, according to the size of set search radius R, use the front R of Hamming distance minimum to open image as secondary contrast images, once obtain and the Euclidean distance of image to be identified, Euclidean distance and the threshold value of gained minimum compare, and are less than or equal to threshold value and are defined as this facial image.

Images