CN102262729B - Fused face recognition method based on integrated learning - Google Patents

Abstract

The invention discloses a fused face recognition method based on integrated learning, which comprises the following steps: 1.) inputting a face image to be identified; 2.) identification based on ART2 (Adaptive Resonance Theory 2) face recognition method: if the ART2 network system has a returned recognition result, receiving the recognition result, which means the identification is successful, else, entering the next step; 3.) identification based on feature face recognition method: assuming that the threshold of the method is S and the identification score of the face image by the method is s, if s>S, receiving the recognition result, which means that the identification is successful, else, entering the subsequent fused identification step; and 4.) fused identification: sequencing by using the degrees of similarity which are respectively given out by the two single recognition methods, and comparing the recognition results, wherein the first ones in the sequence are identical, receiving the recognition result, which means that the identification is successful, else, the identification fails. By adopting the concept of integrated learning, the ART2 face recognition method and feature face recognition method are fused and complemented, thereby overcoming the limitations in the single face recognition method and enhancing the integral recognition performance.

Description

Face identification method is merged in mixing based on integrated study

Technical field

The present invention relates to the Automatic face recognition technical field, specifically, relate to a kind of mixing based on integrated study and merge face identification method.

Background technology

Face recognition technology is based on people's face feature, to facial image or the video flowing of input, judges at first whether it exists people's face, if there is people's face, then provides further the information such as position, size of each face.And according to these information, further extract the identity characteristic that contains in everyone face, and itself and known people's face are compared, thereby identify the identity of everyone face.

The advantage of recognition of face is its naturality and the characteristics of not discovered by tested individuality.So-called naturality refers to that the biological characteristic that utilizes when this recognition method is carried out individual identification with the mankind is identical.For example recognition of face, human also by observe and relatively people's face distinguish and confirm identity, the identification that has in addition naturality also has speech recognition, bodily form identification etc., and fingerprint recognition, iris recognition etc. do not have naturality, because the mankind or other biological are individual by this type of biological characteristic difference.The characteristics of not discovered are also very important for a kind of recognition methods, and this can make this recognition methods not offensive, and are not easy to be cheated because be not easy to arouse people's attention.Recognition of face has the characteristics of this respect, it utilizes visible light to obtain human face image information fully, and be different from fingerprint recognition or iris recognition, need to utilize electronic pressure transmitter to gather fingerprint, perhaps utilize the infrared collection iris image, these special acquisition modes are easy to be discovered by the people, thereby more likely by impersonation.

Although recognition of face has the incomparable advantage of a lot of other identifications, also there are many difficulties in itself.Recognition of face is considered to one of the most difficult research topic of living things feature recognition field even artificial intelligence field.The difficulty of recognition of face mainly is that people's face brings as the characteristics of biological characteristic.People's face characteristics visually are: the difference between the Different Individual is little, and the structure of the somebody of institute face is all similar, even the construction profile of human face is all very similar.Such characteristics are favourable for utilizing people's face to position, and are disadvantageous for utilizing people's face to distinguish the human individual still; The profile of people's face is very unstable, the people can produce by the variation of face a lot of expressions, and in different viewing angles, the visual pattern of people's face also differs greatly, in addition, recognition of face also is subjected to the impact of the many factors such as a lot of overcovers (such as mouth mask, sunglasses, hair, beard etc.), age of illumination condition (for example day and night, indoor and outdoors etc.), people's face.

There is certain limitation in single face identification method, such as in the situation that error rates such as EER() fixing, reduce the FAR(false acceptance rate) will cause the FRR(false rejection rate) suddenly raise; If reduce the FRR(false rejection rate) then will cause the FAR(false acceptance rate) suddenly raise.Under the prior art level, adopt single face identification method can't make both reach simultaneously minimum.

Summary of the invention

Technical matters to be solved by this invention is: the thought that adopts integrated study, provide a kind of mixing based on integrated study to merge face identification method, to merge complementation based on face identification method and the eigenface recognition methods of ART2, overcome the existing limitation of single face identification method, improve whole recognition performance.

For solving the problems of the technologies described above, technical scheme of the present invention is: face identification method is merged in the mixing based on integrated study, may further comprise the steps:

1.) the facial image of input identity to be identified;

2.) based on the evaluation of ART2 face identification method, if the ART2 network system has recognition result to return, then receive recognition result, identify successfully; Otherwise, enter next step and identify;

3.) based on the evaluation of eigenface recognition methods, the threshold value of setting the eigenface recognition methods is S, with the eigenface recognition methods facial image is identified, must be divided into s, if s〉S, then receive recognition result, identify successfully; Otherwise, enter follow-up fusion steps and identify;

4.) merge authentication step, utilize the ordering of the similarity that ART2 face identification method and eigenface recognition methods provide respectively, the recognition result of these two kinds of recognition methodss is compared, if first in the ordering is identical, then receive recognition result, identify successfully; Otherwise, identify unsuccessfully.

As preferred technical scheme, described step 2.) in, comprise attention subsystem and directed subsystem based on the ART2 network system in the ART2 face identification method; Described attention subsystem is used for the vector x of the expression facial image of input is processed, and finishes the comparison of competition selection and similarity, and described facial image pattern is local binary patterns, i.e. LBP; Described directed subsystem be used for to check whether similarity has reached the identification decision standard and made corresponding actions, requires then resonates if reach similarity, otherwise reset; Described attention subsystem is divided into F1 relatively layer and F2 identification layer, described F1 comparison layer is used for the vector x of the expression face characteristic of input is carried out denoising, the normalization pre-service, entering the F2 identification layer through up filtering channel afterwards is at war with, select and activate its storage prototype node the most similar to input pattern at the F2 identification layer, selected node is sent into the Vigilance test that directed subsystem carries out similarity through the pattern prototype that the downstream feedback passage stores in passage, directed subsystem is carried out relatively and two functions of resetting: preset a warning parameter, relatively whether the middle layer model of input and the similarity degree that comprises between the feedback model of template prototypical information are higher than this warning parameter, if, then enter resonance state, export final recognition result, identify successfully; Otherwise, send the current node that is activated of replacement ripple shielding; Described ART2 network system also comprises a storage organization, and this storage organization not only is used for exporting final recognition result, also stores chartered mode flag according to similarity ordering from high to low, for fusion steps is offered help.In this step, at area of pattern recognition, things to be identified, such as people's face, fingerprint, character etc. all are called pattern, and the middle level refers in the ART2 network, one deck that input pattern arrives later on through denoising, normalization, the output of this one deck can be regarded middle layer model as.Mark is exactly the classification under this pattern, is exactly everyone unique ID in recognition of face.

As preferred technical scheme, described step 3.) in, as follows based on the eigenface recognition methods:

Extract N facial image as training sample, vector x of each composition of sample _i, x _iPixel grey scale by facial image consists of, i.e. x _iDimension be D=w * h, wherein, w is the length of image, h is the width of image, consists of a sample vector collection { x by N vector ₁, x ₂..., x _N;

The first step is asked the average vector of this sample vector collection according to formula (3);

$\stackrel{\‾}{x}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{x}_i---\left(3\right)$

Second step, asking the deviation matrix M of this sample vector collection, M is D * N dimension;

M={y ₁, y ₂..., y _N, wherein

The 3rd goes on foot, and asks the covariance matrix C of this sample vector collection according to formula (4);

$C={\mathrm{DD}}^T=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{y}_i{y}_i^T---\left(4\right)$

The 4th goes on foot, and asks the eigenvalue λ of this covariance matrix C _iWith corresponding proper vector e _i, this stack features vector e _iBe quadrature, any facial image can be by this stack features vector e _iExpression is with proper vector e _iBy its eigenvalue λ _iArrange from big to small λ ₁〉=λ ₂〉=... 〉=λ _d〉=... 〉=λ _D, e ₁〉=e ₂〉=... 〉=e _d〉=... 〉=e _D

In the 5th step, select from big to small d eigenvalue λ according to formula (5) _iCharacteristic of correspondence vector e _iForm transformation matrix W=[e ₁, e ₂..., e _d], d＜＜D, in the subspace of structure, x _iBe expressed as z _i=W ^Ty _i

$\frac{\underset{j=1}{\overset{d}{\mathrm{\Σ}}}{\mathrm{\λ}}_j}{\underset{i=1}{\overset{D}{\mathrm{\Σ}}}{\mathrm{\λ}}_i}\≥0.85---\left(5\right)$

In the 6th step, the facial image t for to be identified extracts its eigenwert, and this list of feature values is shown

Use formula (6) complementation chordal distance is measured t and each x _iSimilarity, then use nearest neighbor classifier that facial image is identified, and provide similarity ordering from high to low

$d(t,x_i)=\frac{s\·z_i}{\left|\right|s\left|\right|\left|\right|z_i\left|\right|}---\left(6\right).$

Wherein, cosine distance is a kind of in the distance metric, and modal Euclidean distance is the air line distance between 2 at ordinary times, and the cosine distance is two vectors that 2 are regarded as in certain space, asks the cosine of the angle of these two vectors, as a kind of distance metric.

Owing to having adopted technique scheme, the invention has the beneficial effects as follows: the present invention adopts the thought of integrated study, to merge complementation based on face identification method and the eigenface recognition methods of ART2, quick and precisely learning ability that the ART2 model is had and the rotation of local binary patterns feature based on the face identification method of ART2, the translation invariant characteristic combines, and consider the complementarity of local feature and global characteristics, merge again the Eigenface that uses based on global characteristics, thereby can overcome the existing limitation of single face identification method, can utilize better so different features and sorter to improve whole recognition performance.

Description of drawings

The present invention is further described below in conjunction with drawings and Examples.

Fig. 1 is that the ART2 network system in the embodiment of the invention forms schematic diagram;

Fig. 2 is the LBP operator schematic diagram in the local binary patterns in the embodiment of the invention;

Fig. 3 is that the face identification method process flow diagram is merged in the mixing based on integrated study in the embodiment of the invention.

Embodiment

As shown in Figure 3, merge face identification method based on the mixing of integrated study, may further comprise the steps:

1.) the facial image of input identity to be identified;

2.) based on the evaluation of ART2 face identification method, if the ART2 network system has recognition result to return, then receive recognition result, identify successfully; Otherwise, enter next step and identify;

3.) based on the evaluation of eigenface recognition methods, the threshold value of setting the eigenface recognition methods is S, with the eigenface recognition methods facial image is identified, must be divided into s, if s〉S, then receive recognition result, identify successfully; Otherwise, enter follow-up fusion steps and identify;

4.) merge authentication step, utilize the ordering of the similarity that ART2 face identification method and eigenface recognition methods provide respectively, the recognition result of these two kinds of recognition methodss is compared, if first in the ordering is identical, then receive recognition result, identify successfully; Otherwise, identify unsuccessfully.Similarity, i.e. similarity degree between two facial images, similarity is larger, and is better matchingly, and matching score is just higher.

Wherein, the face identification method based on ART2 reaches as follows to the adjustment of the method:

Adaptive resonance theory (Adaptive Resonance theory, ART) is a kind of study mechanism without tutor's state of conflict neural network.The appearance of this theory is so that attain full development in the application of area of pattern recognition without tutor's clustering learning and neural network in real time.ART1, ART2, ART3 are based on three kinds of basic models of ART theory, and their common feature is: have the unsupervised learning function, have the ability that clock signal is carried out real-time learning, processes in real time; Can the object of having learnt be responded fast, automatically identification.These characteristics are so that the ART neural network is very suitable for the application of area of pattern recognition, so it is once widely concern occurring being subject to.In these three kinds of models especially with ART2 neural network because of applied widely, superior performance, network complexity is moderate and receive much attention.

As shown in Figure 1, comprise attention subsystem and directed subsystem based on the ART2 network system in the ART2 face identification method; Described attention subsystem is used for the vector x of the expression facial image of input is processed, and finishes the comparison of competition selection and similarity, and described facial image pattern is local binary patterns, i.e. LBP; Described directed subsystem be used for to check whether similarity has reached the identification decision standard and made corresponding actions, requires then resonates if reach similarity, otherwise reset; Described attention subsystem is divided into F1 relatively layer and F2 identification layer, described F1 comparison layer is used for the vector x of the expression face characteristic of input is carried out denoising, the pre-service such as normalization, ART2 network input pattern after steady enters the F2 identification layer through up filtering channel and is at war with, select and activate its storage prototype node the most similar to input pattern at the F2 identification layer, selected node is sent into the Vigilance test that directed subsystem carries out similarity through the pattern prototype that the downstream feedback passage stores in passage, directed subsystem is carried out relatively and two functions of resetting: preset a warning parameter ρ, relatively whether the middle layer model u of input and the similarity degree r that comprises between the feedback model p of template prototypical information are higher than this warning parameter ρ (0＜ρ≤1, c=0.1), if, then enter resonance state, export final recognition result, identify successfully; Otherwise, send the current node that is activated of replacement ripple shielding.Wherein, u and p are the output modes of two layers in the ART2 network, and r is similarity, and c is a parameter of ART2 network, and value is 0.1, and warning parameter ρ also is a parameter, has provided hereinafter value 0.9, and these several parameters all are the parameters in the ART2 network.

What the ART2 network system of prior art was exported is final recognition result, the present invention makes an adjustment the ART2 network system, increased a storage organization, this storage organization not only can be exported final recognition result, also store chartered mode flag according to similarity ordering from high to low, for follow-up fusion steps is offered help.

As the input vector of system, i.e. the character representation of people's face, the face characteristic that the present invention selects is local binary patterns (Local Binary Pattern, LBP) feature.LBP is the texture analysis method that typical structure combines with statistics.

As shown in Figure 2, the LBP operator is commonly defined as 3 * 3 window, with the gray-scale value g of window center point _cAs threshold value, the grey scale pixel value of other positions is g in the window _p

$s\left(x\right)=\left\{\begin{array}{cc}1& x\&GreaterEqual;0\\ 0& x<0\end{array}\right.---\left(1\right)$

$\mathrm{LBP}=\underset{p=0}{\overset{p-1}{\mathrm{\&Sigma;}}}s(g_p-g_c)2^p---\left(2\right)$

P is the neighborhood territory pixel number of samples, for example when neighborhood is 3 * 3 window, and p=8, the LBP operator is as shown in Figure 2.For a neighborhood, can obtain the LBP value of center pixel according to formula (1) and formula (2), from computation rule, can find out this value between 0～255, therefore can add up the LBP value of all pixels, formation LBP histogram.If a local binary patterns only comprises the at the most conversion of 2 positions, then this local binary patterns is even pattern (Uniform Pattern), such as 00100000 and 11111111, and 58 kinds altogether of this patterns, add non-homogeneous local binary patterns as a kind of pattern, amount to 59 kinds.This algorithm is that the radius shown in employing Fig. 2 is 2 pixels, and the even local binary patterns histogram of 8 sampled points (p=8) is as the feature (59 dimension) of facial image.

Wherein, as follows based on the qualification process of eigenface recognition methods:

The main thought of the method is to use pivot analysis (Principal Component Analysis), it is the method that with a kind of feature of lesser amt sample is described to reach reduction feature space dimension, the basis of method is the Karhunen-Loeve expansion, is called for short the K-L expansion.Briefly, its principle is exactly with a high dimension vector, by a special eigenvectors matrix, projects in the vector space of a low-dimensional, is characterized by a low dimensional vector, and only loses some less important information.That is to say, the vector sum eigenvectors matrix by low-dimensional characterizes can reconstruct corresponding original high dimension vector substantially.The method has obtained maximum Data Dimensionality Reduction under the prerequisite of minimal loss information.

Be provided with N facial image as training sample, vector x of each composition of sample _i, x _iPixel grey scale by facial image consists of, i.e. x _iDimension be D=w * h, wherein, w is the length of image, h is the width of image, consists of a sample vector collection { x by N vector ₁, x ₂..., x _N;

The first step is asked the average vector of this sample vector collection according to formula (3);

$\stackrel{\&OverBar;}{x}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}_i---\left(3\right)$

Second step, asking the deviation matrix M of this sample vector collection, M is D * N dimension;

M={y ₁, y ₂..., y _N, wherein

The 3rd goes on foot, and asks the covariance matrix C of this sample vector collection according to formula (4);

$C={\mathrm{DD}}^T=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{y}_i{y}_i^T---\left(4\right)$

The 4th goes on foot, and asks the eigenvalue λ of this covariance matrix C _iWith corresponding proper vector e _i, this stack features vector is quadrature, any facial image can be by this stack features vector representation, with proper vector e _iBy its eigenvalue λ _iArrange from big to small λ ₁〉=λ ₂〉=... 〉=λ _d〉=... 〉=λ _D, e ₁〉=e ₂〉=... 〉=e _d〉=... 〉=e _D

In the 5th step, select from big to small d eigenvalue λ according to formula (5) _iCharacteristic of correspondence vector e _iForm transformation matrix W=[e ₁, e ₂..., e _d], d＜＜D, in the subspace of structure, x _iBe expressed as z _i=W ^Ty _i

$\frac{\underset{j=1}{\overset{d}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_j}{\underset{i=1}{\overset{D}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_i}\&GreaterEqual;0.85---\left(5\right)$

In the 6th step, the facial image t for to be identified extracts its eigenwert, and this list of feature values is shown

Use formula (6) complementation chordal distance is measured t and each x _iSimilarity, then use nearest neighbor classifier that facial image is identified, and provide similarity ordering from high to low

$d(t,x_i)=\frac{s\&CenterDot;z_i}{\left|\right|s\left|\right|\left|\right|z_i\left|\right|}---\left(6\right).$

Wherein, fusion authentication method of the present invention is as follows:

In the description to people's face, local message has the illumination of people's face, expression shape change, blocks and the advantage such as a small amount of displacement is insensitive.Global information is described the integrity attributes such as architectural feature such as face contour and face organ, and local message is then described such as detail attribute such as face organ's characteristics or other singularity characteristics.Therefore, global information and local message have good complementarity, and both combinations can be reached better recognition effect.The present invention considers to utilize this point well, and the employed feature of ART2 recognition methods is a kind of feature based on local grain, and the feature that is based on the overall situation that the eigenface recognition methods is used.The Integrated Strategy that the present invention uses is as follows:

As shown in Figure 3, for each subalgorithm a threshold value that confidence level is very high is set, makes it mistake can not occur and know, the threshold value of ART2 recognition methods (warning parameter ρ) T is 0.9, and the threshold value of eigenface recognition methods (cosine distance) S is 0.99.For the facial image of identity to be identified, if the ART2 network system has recognition result to return, then identify successfully; Otherwise the use characteristic face recognition method is identified it, must be divided into s, if s〉S, then identify successfully; Otherwise, the ordering that utilizes ART2 recognition methods and eigenface recognition methods to provide respectively according to similarity, the identity result who identifies for these two kinds of methods compares, if first is identical in the ordering, thinks that then identification is successfully, otherwise identifies unsuccessfully.Among the figure: Rank1 is the ordering to classification number corresponding to similarity that produces by based on the ART2 recognition methods; Rank2 is the ordering to classification number corresponding to similarity that produces by based on the eigenface recognition methods.

The above is giving an example of best mode for carrying out the invention, and the part of wherein not addressing in detail is those of ordinary skills' common practise.Protection scope of the present invention is as the criterion with the content of claim, and any equivalent transformation that carries out based on technology enlightenment of the present invention is also within protection scope of the present invention.

Claims (2)

1. merge face identification method based on the mixing of integrated study, it is characterized in that, may further comprise the steps:

1.) the facial image of input identity to be identified;

2.) based on the evaluation of ART2 face identification method, if the ART2 network system has recognition result to return, then receive recognition result, identify successfully; Otherwise, enter next step and identify;

In this step, comprise attention subsystem and directed subsystem based on the ART2 network system in the ART2 face identification method; Described attention subsystem is used for the vector x of the expression facial image of input is processed, and finishes the comparison of competition selection and similarity, and described facial image pattern is local binary patterns, i.e. LBP; Described directed subsystem be used for to check whether similarity has reached the identification decision standard and made corresponding actions, requires then resonates if reach similarity, otherwise reset; Described attention subsystem is divided into F1 relatively layer and F2 identification layer, described F1 comparison layer is used for the vector x of the expression face characteristic of input is carried out denoising, the normalization pre-service, entering the F2 identification layer through up filtering channel afterwards is at war with, select and activate its storage prototype node the most similar to input pattern at the F2 identification layer, selected node is sent into the Vigilance test that directed subsystem carries out similarity through the pattern prototype that the downstream feedback passage stores in passage, directed subsystem is carried out relatively and two functions of resetting: preset a warning parameter, relatively whether the middle layer model of input and the similarity degree that comprises between the feedback model of template prototypical information are higher than this warning parameter, if, then enter resonance state, export final recognition result, identify successfully; Otherwise, send the current node that is activated of replacement ripple shielding;

Described ART2 network system also comprises a storage organization, and this storage organization not only is used for exporting final recognition result, also stores chartered mode flag according to similarity ordering from high to low, for fusion steps is offered help;

3.) based on the evaluation of eigenface recognition methods, the threshold value of setting the eigenface recognition methods is S, with the eigenface recognition methods facial image is identified, must be divided into s, if s〉S, then receive recognition result, identify successfully; Otherwise, enter follow-up fusion steps and identify;

4.) merge authentication step, utilize the ordering of the similarity that ART2 face identification method and eigenface recognition methods provide respectively, the recognition result of these two kinds of recognition methodss is compared, if first in the ordering is identical, then receive recognition result, identify successfully; Otherwise, identify unsuccessfully.

2. face identification method is merged in the mixing based on integrated study as claimed in claim 1, it is characterized in that described step 3.) in, as follows based on the eigenface recognition methods:

Extract N facial image as training sample, vector x of each composition of sample _i, x _iPixel grey scale by facial image consists of, i.e. x _iDimension be D=w * h, wherein, w is the length of image, h is the width of image, consists of a sample vector collection { x by N vector ₁, x ₂..., x _N;

The first step is asked the average vector of this sample vector collection according to formula (3);

$\stackrel{\&OverBar;}{x}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{x}_i---\left(3\right)$

Second step, asking the deviation matrix M of this sample vector collection, M is D * N dimension;

M={y ₁, y ₂..., y _N, wherein

The 3rd goes on foot, and asks the covariance matrix C of this sample vector collection according to formula (4);

$C={\mathrm{DD}}^T=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}{y}_i{y}_i^T---\left(4\right)$

The 4th goes on foot, and asks the eigenvalue λ of this covariance matrix C _iWith corresponding proper vector e _i, this stack features vector e _iBe quadrature, any facial image can be by this stack features vector e _iExpression is with proper vector e _iBy its eigenvalue λ _iArrange from big to small λ ₁〉=λ ₂〉=... 〉=λ _d〉=... 〉=λ _D, e ₁〉=e ₂〉=... 〉=e _d〉=... 〉=e _D

In the 5th step, select from big to small d eigenvalue λ according to formula (5) _iCharacteristic of correspondence vector e _iForm transformation matrix W=[e ₁, e ₂..., e _d], d＜＜D, in the subspace of structure, x _iBe expressed as z _i=W ^Ty _i

$\frac{\underset{j=1}{\overset{d}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_j}{\underset{i=1}{\overset{D}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_i}\&GreaterEqual;0.85---\left(5\right)$

The 6th step represented with t for facial image to be identified, extracted the eigenwert of t, and this list of feature values is shown

Use formula (6) complementation chordal distance is measured t and each x _iSimilarity, then use nearest neighbor classifier that facial image is identified, and provide similarity ordering from high to low

$d(t,x_i)=\frac{s\&CenterDot;z_i}{\left|\right|s\left|\right|\left|\right|z_i\left|\right|}---\left(6\right).$

Images