WO2016008071A1

WO2016008071A1 - Face verification method and system

Info

Publication number: WO2016008071A1
Application number: PCT/CN2014/082149
Authority: WO
Inventors: 王亮; 谭铁牛; 王威; 黄岩
Original assignee: 中国科学院自动化研究所
Priority date: 2014-07-14
Filing date: 2014-07-14
Publication date: 2016-01-21
Also published as: CN104363981B; CN104363981A

Abstract

The present invention relates to a face verification method and system, the method comprising: preprocessing high-dimensional facial feature data through principal component analysis and linear discriminant analysis, the preprocessing comprising setting a data dimension after dimension reduction through principal component analysis; establishing a discriminant high-order Boltzmann machine and setting the number of nodes for a hidden layer; reducing model parameters of the discriminant high-order Boltzmann machine by employing a strategy of tensor diagonalization; inputting pairs of facial data into the discriminant high-order Boltzmann machine, using a random gradient descent algorithm to maximize the conditional probability of relationship categories, so as to iteratively optimize a weight of the Boltzmann machine and obtain an ultimate discriminant high-order Boltzmann machine; inputting pairs of facial data to be verified into the discriminant high-order Boltzmann machine model and obtaining corresponding verification result data. The present invention introduces data relationship category information into an unsupervised Boltzmann machine model, enhancing discrimination ability and being more suitable for face verification having precision requirements.

Description

Face test method and system

The present invention relates to a face verification method and system, and more particularly to a face verification method and system based on a discriminant high-order Boltzmann machine. Background technique

Face verification technology has been a hot research topic in the field of pattern recognition due to its wide application, for example, as an identity authentication application in customs and banking.

Given a pair of face images, the goal of face verification is to determine whether the two face images represent the same person. The previous work was mainly to study face verification in a constrained environment, that is, the face images in the experiment were collected under controlled environmental factors, including fixed illumination, gestures and expressions. However, in practical applications, these factors are uncontrollable. For example, a face image in video surveillance may contain any illumination. Therefore, many researchers have recently turned to the face verification problem in unconstrained environments. Face images under unconstrained environmental factors have very large intra-class variances and can be challenging to handle.

The existing face verification problem consists of two steps: data feature representation and data relationship metric. For data feature representation, people usually use some artificially set feature descriptors to represent face data, such as LBP, S I FT and Gabor. After the paired face data representation is obtained, the cosine distance is usually used to measure the similarity of each pair of images, and then it can be judged whether the pair of images represents the same person. Summary of the invention

The object of the present invention is to provide a face verification method and system, aiming at the defects of the prior art. To achieve face verification that is more suitable for accuracy requirements.

To achieve the above object, the present invention provides a face verification method, the method comprising: Step 1: using principal component analysis and linear discriminant analysis to separately preprocess high-dimensional facial feature data, including setting principal component analysis to reduce Dimensional data dimension;

Step 2: Establish a discriminative high-order Boltzmann machine, and set the number of nodes of the hidden layer;

Step 3. Using a tensor diagonalization strategy to reduce the model parameters of the discriminative high-order Boltzmann machine;

Step 4: Input the paired face data into the discriminant high-order Boltzmann machine, and use the stochastic gradient descent algorithm to maximize the conditional probability of the relationship class, thereby iteratively optimizing the weight of the Boltzmann machine. Thereby obtaining the final discriminant high-order Boltzmann machine;

Step 5: Input the paired face data to be verified to the discriminative high-order Boltzmann machine model, and obtain corresponding verification result data.

The present invention also provides a face verification system, the system comprising: a network establishment module, a network weight optimization module, and a data verification module, wherein:

The network establishing module is configured to establish a discriminative high-order Boltzmann machine, and set a number of nodes of a network hidden layer;

The network weight optimization module uses a stochastic gradient descent algorithm to minimize the objective function of the Boltzmann machine to obtain an optimized Boltzmann machine weight, thereby obtaining a final discriminant high-order Boltzmann machine model. ;

The data verification module is configured to input paired face data to be verified to the discriminative high-order Boltzmann machine, obtain two node values of the category layer, and compare the relative sizes of the two values to obtain a person Face verification result data.

The face verification method and system of the present invention, by introducing data relationship category information into the unsupervised Boltzmann machine model, enhances the discriminative power of the model, and is more suitable for face verification with accuracy requirements. DRAWINGS 1 is a flow chart of a method for verifying a face of the present invention;

2 is a schematic view showing the structure of a discriminative high-order Boltzmann machine used in the present invention; FIG. 3 is a schematic diagram of a tensor diagonalization method;

4 is a schematic diagram of a face verification system of the present invention;

Figure 5 shows the experimental effect changes for different input data dimensions and different number of hidden nodes. detailed description

The technical solution of the present invention will be further described in detail below through the accompanying drawings and embodiments. The inventive face verification method and system mainly propose a more effective data relationship measurement method, which can improve the accuracy of face verification to a large extent.

FIG. 1 is a flowchart of a method for verifying a face of the present invention. As shown in the figure, the present invention specifically includes the following steps:

Step 101: Preprocessing the high-dimensional facial feature data by using principal component analysis and linear discriminant analysis, wherein the data dimension of the principal component analysis after dimension reduction is set;

Among them, Principal Component Analysis (PGA) is a commonly used data dimension reduction method. This method needs to set the dimensional dimension after dimensionality reduction, usually from several hundred to several thousand. Linear Discriminant Analysis (LDA) is a commonly used supervised subspace learning method that can make the discriminability of data enhanced.

Step 102: Establish a discriminative high-order Boltzmann machine, and set a number of nodes of the hidden layer. The discriminant high-order Boltzmann machine is a multi-layer network structure, including two data input layers and one hidden layer. Contains layers and a category output layer. The two input layers of the discriminative high-order Boltzmann machine are paired face training data, such as a feature representation of a face image. In an embodiment of the invention, all face data is required to maintain the same size, for example, a vector of the same length; the class output layer is used to represent the verification result of the paired face training data, since the result of the face verification has only two types, That is, matching or not matching, where the output layer only contains two nodes, corresponding to the two categories; the discriminant high-order Boltzmann machine has a network weight, which is used to obtain the next layer node value according to the current layer node value. . The number of nodes of the input layer and the output layer of the discriminative high-order Boltzmann machine is fixed, but the number of nodes of the hidden layer needs to be manually adjusted to optimize the effect of the model.

Fig. 2 is a block diagram showing the structure of a discriminating high-order Boltzmann machine used in an embodiment of the present invention. As shown in Figure 2, this is a four-layer network with circular points in each layer representing network nodes. The leftmost bottom face layers representing two data input layer, the input layer I are dimensional column vector x, each dimension yeR ^Ixl, the vector is represented by a node, which values are normalized to 0 The mean and the variance is 1. The highest layer is the category output layer, and the face verification category is represented as a 2-dimensional vector zeM ^2xl , and the vector value is [1, 0] or [0, 1]. The two cases correspond to or do not match. The hidden layer in the middle is a K-dimensional vector heffi ^Kxl , and each dimension of the vector takes a value of 0 or 1. The values of h and z can all be calculated from the vector values of their previous layers, where h is interconnected with the X and y layers: yt

d _t +∑U _kt h _k

e ^k

=∑ '' ^{+∑ k} ,

Where _σ (χ) = 1 / (1 + , x = ( _Xl L , y = ( _yj , h = ) _keK , z = ( ) _te(i2 ). c = (c _k ) _keK and d = (d _t ) are the offset parameters of the hidden layer and the class layer respectively. W = (, ) is the connection weight between the two input layers and the hidden layer, and 11 = (11 _1{1 ) ₁ ^^ ₁₂₎ is The weight of the connection between the hidden layer and the category layer.

Step 103: Using a tensor diagonalization strategy to reduce the model parameters of the discriminative high-order Boltzmann machine; the discriminant high-order Boltzmann machine contains a large number of model parameters that need to be learned, when the input data dimension For a few hundred hours, the three-dimensional weight tensor W may contain millions of weight parameters. Thus, in the case where the amount of training data is not very large, it is easy to cause overfitting. Therefore, the tensor diagonalization method is used here to reduce the parameters, as shown in Figure 3. On the h-axis, each h _k corresponds to a weight matrix slice:

w _k = (w ) where each weight \3⁄4 in the matrix corresponds to the connection relationship of nodes _Xi and _yj . Implied assumption Any node in x is related to the node in y. Such assumptions are not appropriate for face data. Specifically, we usually use local descriptors to extract features of facial key attachments, such as extracting LBP features from faces, noses, and mouths, and then concatenating these features to obtain the final features. So each node in X and y only represents a local area of the face, as shown by R1 - R5 in Figure 3. When performing face verification, people usually care about whether the same position in the two face images, such as whether the eyes are matched, does not care whether one's eyes match the nose of another person. Therefore, only when the neighboring _Xi nodes are the same as the face regions represented by the _yj nodes, they have a strong correlation, as shown in the block diagonalization matrix in Figure 3, where black represents correlation, white For no relevance. So for any node _Xi , we define the node associated with its - neighborhood as { _yj , j G [is, i + 4, and then reduce each weight matrix slice w. _k to only retain 2s + l Weights on the diagonal: w _k = (w _ijk ) Thus the magnitude of the parameter in the entire weight tensor W is reduced from 0(n ³ ) to 0(n ² ). Step 104: Input the paired face data into the discriminant high-order Boltzmann machine, and use a stochastic gradient descent algorithm to maximize the conditional probability of the relationship class, thereby iteratively optimizing the weight of the Boltzmann machine. Thus, the final discriminant high-order Boltzmann machine is obtained; taking the network in Fig. 2 as an example, the two input layer data, the middle hidden layer and the class output layer represent a discriminative high-order Boltz Man machine, whose energy function E(x, y, h, z) is defined as:

E (x, y,h,z) (x ) ²

- ∑( y factory bj ) ² -∑ ^C A -∑d _t z _t

j k t

Where a = and b = (b)^ represent the bias term parameters of the two input layers, respectively. The probability distribution P (x, y, z) of the input layer data x, y and its relational category z can be obtained on the basis of the energy function:

In order to learn the model parameters, it is common practice to minimize the objective function by using a stochastic gradient descent algorithm, that is, to minimize the negative logarithm of the maximum logarithm function L _gra :

L _gen =- ^lo g∑p( ^x ",y", ^z ") where the index of the training data is represented. The objective function !^ The gradient of the parameter W can be calculated as follows:

Where <. _>q represents the expectation about the variable q. However, calculating the second term in the above equation is very difficult because the calculation process needs to traverse all the values of the variables x, y, h, and z. use! ^ The model learning process as the objective function is shown in Figure 6. In addition, in order to simplify the calculation process, we also tried a discriminant objective function, that is, the conditional probability of the relationship category under the negative logarithm:

L _dls =-log∑P(z«lx«, y«) where the conditional distribution P(zlx, y) for z can be expressed as:

Unlike the objective function L _g , the target function in this case! ^ The gradient of the parameter W can be accurately calculated:

ΜΜ

∑σ(Μ _Λ ) 2 (M _t , _k )P(z _t , lx,y

M _tk =c _k +3⁄4 _k x _i y _j +u _to

Ij

After getting the gradient, we adjust the parameter pairs in an iterative way: dW

Where f represents a constant learning rate. Step 1105: Input the paired face data to be verified, such as X and y, to the trained discriminative high-order Boltzmann machine model to obtain an output layer node value z, by comparing the relative values of the two values. The size can be used to judge the result of face verification. If > then does not match, otherwise it matches.

4 is a schematic diagram of a face verification system according to the present invention. As shown in the figure, the present invention specifically includes: a network establishment module 1, a network weight optimization module 2, and a data verification module 3, wherein:

Specifically, the network establishing module 1 is configured to establish a discriminative high-order Boltzmann machine and set the number of nodes of the network hidden layer; the discriminative high-order Boltzmann machine is a multi-layer network structure, including two input data. Layer, an implicit layer and a category output layer. The input layer is training paired face data, the output layer represents the verification result of training paired face data; the discriminant discriminant high-order Boltzmann machine has network weight, which is used to obtain the next according to the current layer node value. Layer node value.

The network weight optimization module 2 uses the stochastic gradient descent algorithm to minimize the objective function of the Boltzmann machine to obtain the optimized Boltzmann machine weight, and obtain the final discriminant high-order Boltzmann machine model.

The data verification module 3 is used to input the paired face data to be verified to the discriminant high-order Boltzmann machine, obtain two node values of the class layer, and compare the relative sizes of the two values to obtain the face verification result.

In order to explain in detail the specific embodiment of the present invention, two image data sets (LFW and WDRef) are illustrated here as an example. The LFW data set contains 1 3233 face images from 5,749 individuals. One of the 1,680 people has one image, while the rest have more than two images. The WDRef data set includes 99,773 face images on the network, where WDRef does not overlap with the face data in the LFW. All face images are collected directly from the Internet, with very large changes in lighting, posture, and more.

We performed face verification experiments in two settings:

1) Under the original data, there is supervised learning. The data of LFW is divided into 10 copies, 9 of which are trained, and the remaining 1 is tested, and this is repeated 10 times.

2) Supervised learning under external data. Training with WDRef data, in the LFW database Test on it.

Specific steps are as follows:

Step S1, for all data, the principal high-dimensional face LBP feature is reduced to 2000 dimension by principal component analysis, and then processed by linear discriminant analysis.

Step S2, using a four-layer discriminative high-order Boltzmann machine model, the two input layers, the hidden layer and the output layer respectively contain 2000, 2000, 1000 and 2 nodes.

In step S3, the tensor diagonalization strategy is used to eliminate the network weights outside the diagonal line, and the weight quantity is reduced to ² ).

Step S4, using the gradient descent algorithm to optimize the objective function ^ l s of the network, wherein the gradient can be accurately calculated rather than approximated. Optimization is done in an iterative manner, where setting the maximum number of iterations to 30 guarantees convergence.

In step S6, the paired face data of the test is input into the trained model, and the verified result is output on the output layer. The result of the output is a two-dimensional vector. Comparing the relative sizes of the two values determines the match or does not match.

We compared some of the most accurate methods for face verification today. By comparing the R0G curves, we found that our method achieved the best experimental results under both experimental settings. In addition, we tested the experimental results of the present invention using different input data dimensions and different numbers of hidden nodes. As shown in Figure 5, the horizontal axis represents the number of hidden nodes (number of hi dden un i ts), and the vertical axis represents the accuracy of face recognition (accuracy). Each curve in the graph represents a data dimension in different implied The face verification accuracy changes at the node. The input data dimension has changed from 200 to 1000 dimensions, and the number of hidden nodes has changed from 200 to 1200. As can be seen from the figure, the number of hidden nodes is fixed, and as the dimension of the input data is increased, the accuracy is increased. For any dimension of input data, when the number of hidden nodes is greater than 400, the accuracy remains stable and no longer rises.

A person skilled in the art should further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, in order to clearly illustrate hardware and software. Interchangeability, according to the above description The composition and steps of the various examples are generally described. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. Software modules can be placed in random access memory

(RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, GD-ROM, or any other form of storage known in the art In the middle.

The above described embodiments of the present invention are further described in detail, and the embodiments of the present invention are intended to be illustrative only. The scope of the protection, any modifications, equivalents, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

Claim

A method for verifying a face, the method comprising:

Step 1. Pre-processing the high-dimensional facial feature data by principal component analysis and linear discriminant analysis, including setting the data dimension of the principal component analysis after dimension reduction;

2. The method according to claim 1, wherein the model of the discriminative high-order Boltzmann machine in step 2 comprises two data input layers, one hidden layer and one category output. Layer four network structure.

3. The method according to claim 2, wherein the two data input layers are paired training face data, the class output layer represents a face verification result; and the discriminant high order glass The Erzmann machine has a network weight to obtain the next layer of node values based on the current layer node value.

4. The method according to claim 3, wherein the tensor diagonalization strategy in step 3 is to process the three-dimensional weight cube between the two data input layers and the hidden layer into multiple A two-dimensional weight matrix.

5. The method according to claim 1, wherein the objective function of the discriminant high-order Boltzmann machine is a conditional probability of a relationship class of a negative logarithm.

6. A face verification system, the system comprising: a network establishment module, a network weight optimization module, and a data verification module, wherein:

The network establishing module is configured to establish a discriminative high-order Boltzmann machine, and set a network implied The number of nodes in the layer;