CN106203356A - A kind of face identification method based on convolutional network feature extraction - Google Patents

Abstract

The present invention relates to a kind of technology lacking face image restoration and identification, be specifically related to a kind of face identification method based on convolutional network feature extraction, belong to living things feature recognition field.First with blocking core model algorithm, original missing image is carried out matrix recovery process, the recovery matrix that the information that obtains is recovered substantially, then utilize low-rank matrix decomposition algorithm that recovery matrix is carried out the extraction of low-rank information, and it is converted into matrix convolution core by vector form, extract, followed by convolutional neural networks, the characteristics of image recovered and encode, obtain the final feature of every image, finally by SVM, feature samples is trained and Classification and Identification.The present invention is avoided that in traditional method because image lacks the low discrimination problem brought, and the missing image of disparate databases is attained by preferable result.

Description

Face recognition method based on convolutional network feature extraction

Technical Field

The invention relates to a technology for restoring and identifying a missing face image, in particular to a face identification method based on convolutional network feature extraction, and belongs to the field of biological feature identification.

Background

The human face is the most important and most direct carrier for human emotion expression and communication, and information such as race, region, identity, status and the like of a person can be deduced through the human face, so that the human face also becomes the most obvious part for distinguishing different points between people. The human face image is acquired without contacting with the detected person, the acquisition can be carried out under the condition of not frightening the detected person, the counter feeling of the detected person is not caused, and the human face recognition can be completed only by adopting the embedded cameras of the common camera, the digital camera and the mobile phone, so that the human face recognition system is very convenient to use, and has great significance in the research of the human face recognition. With the advent and development of high-speed and high-performance computers, the scientific community has studied human faces from many disciplines such as computer graphics, image processing, computer vision and anthropology. Currently, many researchers have made a lot of research on face recognition, and many practical recognition methods are proposed, which mainly include:

(1) in the patent of face recognition method and system (publication number CN103955681A), a filtering module is used to filter an input face image to obtain a face image to be recognized. Searching a template image matched with the face image to be recognized in an image database by using a nearest neighbor classification module to obtain a matched template image, wherein the template image is obtained by filtering an original template image by using the filtering module, and the type of the matched template image is determined as the type of the face image.

The disadvantages are as follows: because the method directly carries out filtering feature transformation on the image, the face recognition accuracy is sacrificed for improving the face image recognition efficiency, especially under the condition that the image is lost or shielded.

(2) In the patent of face recognition method and system (publication number CN105095829A), face features to be recognized are extracted from an acquired face image, and according to the face features to be recognized, whether a blocking object exists in the acquired face image is detected, if yes, the blocking object is extracted from the face image, whether a matching reference face image exists in a face image library is judged, if yes, face recognition is successful, otherwise, face recognition is failed.

The disadvantages of the above technique: although information of an occlusion is taken into consideration, the image may lose information of an occlusion part, which may reduce recognition accuracy, especially when a key part is occluded.

Disclosure of Invention

The invention aims to provide an identification method based on convolutional network feature extraction in a missing face image. Firstly, performing matrix recovery operation on an input missing face image by using a truncation kernel to enable the image to recover most of missing information, then performing low-rank information extraction on the recovered image, performing convolution on the image converted from the low-rank information to extract a feature map, performing binarization and histogram statistics on the feature map to obtain final features, and finally, recognizing the face image by using an SVM (support vector machine).

In order to achieve the purpose, the invention provides the following technical scheme:

the invention provides a incomplete face recognition method based on a convolutional neural network, which comprises the following steps:

the method comprises the following steps: incomplete face recovery

(1.1) taking N original incomplete face images as input, firstly judging whether the images are gray images or color images, if so, directly selecting the images as missing matrixes, and if so, respectively selecting the images from R, G, B three color channels of the images as the missing matrixes to operate, wherein the missing matrixes are marked as I_i,i＝1,...,N；

(1.2) for each selected deletion matrix I, a method of truncating the kernel range is utilized_iPerforming matrix recovery operation to obtain an image matrix, namely a recovery matrix X_i；

Step two: face feature extraction

(2.1) for the resulting recovery matrix X_iExtracting according to blocks, and extracting a low-rank matrix after arranging in sequence;

(2.2) for the obtained low-rank matrix, converting the low-rank matrix into L after extracting the characteristic vector₁A convolution kernel in the form of a matrix, for the recovery matrix X_iPerforming convolution operation to obtain N × L₁Repeating the previous step (2.1) by using a feature map, and performing comparison on the N × L₁Extracting corresponding L from the facial feature map₂A convolution kernel, and usingThe L is₂The convolution kernel convolves the characteristic map to obtain a new characteristic map;

(2.3) carrying out binarization processing on each obtained new characteristic map, and then obtaining a value range in which a value range is positioned after weighting processingThe final feature map of (a);

(2.4) performing histogram statistics on the obtained final feature map according to blocks to obtain the vector form features of each input image;

step three: face recognition classification

And (3.1) training and classifying the obtained features by utilizing an SVM.

In the step one (1.2): the process of matrix recovery is to the missing matrix I_iOptimizing the rank so that the recovered matrix X_iThe rank is minimum; the truncation kernel-norm algorithm used is an optimization problem of solving rank:

$\begin{array}{ccc}\underset{X_i}{min}\left|\right|X_i|{|}_r& s.t.& P_{\mathrm{\Ω}}\left(X_i\right)=P_{\mathrm{\Ω}}\left(I_i\right)\end{array};$

recovery matrix X_iThe dimensions of (a) are m, n, r are an empirically determined constant, in the above formula,σ_j(X_i) Representation matrix X_iThe jth maximum singular value of (a); p_Ω(X_i) Representing a set of pixel positions, P, not missing in the original image_Ω(I_i) A set of pixel values representing all locations in the original image.

In the second step (2.1): for the obtained recovery matrix X_iBy block extraction, i.e. to the recovery matrix X_iWith the dimension k₁×k₂And extracting matrix blocks according to blocks in a mode that the step length is 1 pixel, then performing mean value removing operation on pixel values in each matrix block, and arranging each matrix block in a large matrix P in sequence.

In the second step (2.1): performing low-rank matrix extraction on the arranged large matrix P, and extracting information by using an RPCA (resilient packet access) low-rank matrix decomposition algorithm, wherein the extraction process is a solution optimization problem:

$\underset{X,E}{min}\left|\right|X|{|}_{*}+\mathrm{\λ}|\left|E\right|{|}_1$

s.t.P＝X+E

where X is the low rank matrix of the large matrix P and E is its sparse matrix. | X | non-conducting phosphor_*Representing a kernel norm operation taking matrix X, | E | | luminance₁Representing l of the taking matrix E₁Norm operation, i.e.Where m, n denote the number of rows and columns, respectively, of the matrix E, E_ijRepresenting the element in matrix E located in row i and column j.

In the second step (2.2): extracting L for low rank matrix X₁The eigenvectors corresponding to the maximum eigenvalues can be directly obtained in the RPCA solving process, and then the vectors are rearranged into k-size vectors according to the columns₁×k₂Form L of₁A convolution kernelL₁One convolution kernel and N recovery matrices X_iPerforming convolution operation to obtain N × L₁Zhang feature atlas

Repeat step (2.1) for this N × L₁Zhang feature atlasExtracting corresponding L₂A convolution kernelAnd use the L₂The convolution kernel convolves the feature mapObtaining a new characteristic map

In the second step (2.3), the new feature map is subjected to binarization treatment, and the formula is as follows:

$H\left(y\right)=\left\{\begin{array}{cc}1,& ify>0\\ 0,& otherwise\end{array}\right.,$

wherein y is a novel characteristic mapEach pixel value of (a).

In the second step (2.3), weighting processing is carried out on the new feature map,

wherein,the final characteristic map has a value range of

In the second step (2.4), the final characteristic map is processedPerforming histogram statistics according to blocks; for each input image there is L₁Final feature atlasEach sheet is putDividing the image into B image blocks; making histogram statistics in each image block, each image block hasItem, pulling B block image blocks into vectors in sequence to obtainHere, theThe vector dimension obtained by the representation isMaintaining;

handle L₁AnPutting the image into a vector to obtain the final characteristics of each input image

In the third step (3.1),

by the final features of each figureTaking the class labels corresponding to the class labels as input, and training the corresponding relation by using an SVM (support vector machine) to obtain a classifier;

and for a new test sample, obtaining the final characteristics of the input image through the first step and the second step, inputting the final characteristics into the SVM, and comparing the final characteristics with the classifier to identify the specific class label corresponding to the new sample, thereby realizing the classification and identification of the face image.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention allows to achieve a better recognition result in case of missing input images.

2. Compared with the existing traditional convolutional neural network identification technology, the method has the advantage of high speed.

Drawings

FIG. 1 is an overall block diagram of the face recognition method based on convolutional network feature extraction of the present invention;

FIG. 2 is a fragmentary face image;

fig. 3 is a restored face image.

Detailed Description

The present invention will be further illustrated with reference to the following examples.

A face recognition method based on convolutional network feature extraction is disclosed, the specific process is shown in figure 1, and the method comprises the following steps:

the method comprises the following steps: incomplete face recovery

(1.1) N original incomplete face images I_iIf the image is a gray image, the image is directly selected as a missing matrix (since the image is a matrix in a computer, the two matrixes are equivalent); if the image is a color image, the image is respectively selected from R, G, B color channels of the image to be used as a missing matrix for operation.

(1.2) for each selected deletion matrix I, a method of truncating the kernel range is utilized_iPerforming matrix recovery operation to obtain an image matrix, namely a recovery matrix X_i. Recovery matrix X_iThe deletion matrix I is reserved_iMost of the information in (1). The process of matrix recovery being missing matrix I to the input_iOptimizing the rank so that the matrix X is recovered_iIs the smallest. The truncation kernel-norm algorithm used is as follows:

$\begin{array}{ccc}\underset{X_i}{min}\left|\right|X_i|{|}_r& s.t.& P_{\mathrm{\Ω}}\left(X_i\right)=P_{\mathrm{\Ω}}\left(I_i\right)\end{array}.$

recovery matrix X_iRespectively m, n, r is an empirically determined constant (generally selected from one of 6, 7, 8, 9, 10, 11, 12), in the above formula,σ_j(X_i) Representation recovery matrix X_iThe jth maximum singular value of (a). P_Ω(X_i) A set of pixel values representing positions in the original image that are not missing. P_Ω(I_i) A set of pixel values representing all locations in the original image. Expression P_Ω(X_i)＝P_Ω(I_i) The mathematical meaning of (1) is: the pixel values of the positions not missing in the image are kept unchanged, and then the pixel values of the missing positions are restored.

The mathematical principle of truncating the kernel is as follows:

to optimize the rank of a matrix (direct optimization is difficult to solve, so the rank is approximated by other methods), approximation by using a kernel (namely the sum of all singular values of the matrix) is inaccurate, because the singular value has a certain size, and the rank is only related to the number of the first r non-zero singular values of the matrix, the optimization target is converted into min (m, n) -r singular values with the minimum matrix, so that a better effect can be achieved.

Step two: face feature extraction

(2.1) pair recovery matrix X_iPerforming extraction operation according to block (image block), wherein the specific operation is to restore the matrix X_iWith the dimension k₁×k₂Step size of 1 pixel (pixel)The method of (1) performs extraction of matrix blocks by block, and then performs a mean value removal operation on pixel values in each matrix block. Arranged in a large matrix P in order for each matrix block.

And (3) performing low-rank information extraction on the obtained large matrix P, and extracting information by using an RPCA (resilient packet access) low-rank matrix decomposition algorithm, wherein the extraction process is a solution optimization problem:

$\underset{X,E}{min}\left|\right|X|{|}_{*}+\mathrm{\λ}|\left|E\right|{|}_1$

s.t.P＝X+E

where X is the low rank matrix of the large matrix P and E is its sparse matrix. | X | non-conducting phosphor_*Representing the operation of taking the kernel norm of the matrix X (to ensure the low rank of the matrix X), | E | survival₁Representing l of the taking matrix E₁Norm operation (to ensure sparsity of matrix E), i.e.Where m, n denote the number of rows and columns, respectively, of the matrix E, E_ijRepresenting the element in matrix E located in row i and column j.

The mathematical principle of applying RPCA is as follows:

a matrix can be decomposed into a superimposed combination of a low-rank matrix representing most of the information of the original matrix and a sparse matrix representing noise and some other minor details of the original matrix. The feature extraction is performed by using the low-rank matrix, so that the final feature is more concise and effective.

(2.2) then extract L for the low rank matrix X₁The eigenvectors (directly obtained in the RPCA solving process) corresponding to the maximum eigenvalue are rearranged into k size according to the column₁×k₂Form L of₁A convolution kernelL₁One convolution kernel and N recovery matrices X_iPerforming convolution operation to obtain N × L₁Zhang feature atlas

Repeat step (2.1) for this N × L₁Zhang feature atlasExtracting corresponding L₂A convolution kernelAnd use the L₂A convolution check feature mapConvolution is carried out to obtain a new characteristic map

(2.3) carrying out binarization processing on each obtained new characteristic map, wherein a specific formula is as follows:where y is a new feature mapEach pixel value of (a).

Then, through weighting processing on the binarized image:obtain a value range inFinal feature map of

(2.4) to the final feature mapHistogram statistics are made on a block basis. For each input image there is L₁Final feature atlasEach final feature mapAnd dividing the image into B image blocks. Making histogram statistics in each image block, each image block hasItem, pulling B block image blocks into vectors in sequence to obtainHere, theThe vector dimension obtained by the representation isAnd (5) maintaining.

Handle L₁AnPutting the image into a vector to obtain the final characteristics of each input image

Step three: face recognition classification

(3.1) Final features in each figureAnd training the corresponding relation by using the SVM by taking the class label corresponding to the classifier as input to obtain a classifier.

And for a new test sample, obtaining the final characteristics of the input image through the first step and the second step, inputting the final characteristics into the SVM, and comparing the final characteristics with the classifier to identify the specific class label corresponding to the new sample, thereby realizing the classification and identification of the face image.

Claims (9)

1. A incomplete face recognition method based on a convolutional neural network is characterized in that: the method comprises the following steps:

the method comprises the following steps: incomplete face recovery

(1.1) taking N original incomplete face images as input, firstly judging whether the images are gray images or color images, if so, directly selecting the images as missing matrixes, and if so, respectively selecting the images from R, G, B three color channels of the images as the missing matrixes to operate, wherein the missing matrixes are marked as I_i,i＝1,...,N；

(1.2) for each selected deletion matrix I, a method of truncating the kernel range is utilized_iPerforming matrix recovery operation to obtain an image matrix, namely a recovery matrix X_i；

Step two: face feature extraction

(2.1) for the resulting recovery matrix X_iExtracting according to blocks, and extracting a low-rank matrix after arranging in sequence;

(2.2) for the obtained low-rank matrix, converting the low-rank matrix into L after extracting the characteristic vector₁A convolution kernel in the form of a matrix, for the recovery matrix X_iPerforming convolution operation to obtain N × L₁Repeating the previous step (2.1) by using a feature map, and performing comparison on the N × L₁Extracting corresponding L from the facial feature map₂A convolution kernel and using the L₂The convolution kernel convolves the characteristic map to obtain a new characteristic map;

(2.3) carrying out binarization processing on each obtained new characteristic map, and then obtaining a value range in which a value range is positioned after weighting processingThe final feature map of (a);

(2.4) performing histogram statistics on the obtained final feature map according to blocks to obtain the vector form features of each input image;

step three: face recognition classification

And (3.1) training and classifying the obtained features by utilizing an SVM.

2. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the step one (1.2): the process of matrix recovery is to the missing matrix I_iOptimizing the rank so that the recovered matrix X_iThe rank is minimum; the truncation kernel-norm algorithm used is an optimization problem of solving rank:

$\begin{array}{ccc}\underset{X_i}{min}\left|\right|X_i|{|}_r& s.t.& P_{\mathrm{\Ω}}\left(X_i\right)=P_{\mathrm{\Ω}}\left(I_i\right)\end{array};$

recovery matrix X_iThe dimensions of (a) are m, n, r are an empirically determined constant, in the above formula,σ_j(X_i) Representation matrix X_iThe jth maximum singular value of (a); p_Ω(X_i) Representing a set of pixel positions, P, not missing in the original image_Ω(I_i) A set of pixel values representing all locations in the original image.

3. The convolution-based nerve of claim 1The incomplete face recognition method of the network is characterized in that: in the second step (2.1): for the obtained recovery matrix X_iBy block extraction, i.e. to the recovery matrix X_iWith the dimension k₁×k₂And extracting matrix blocks according to blocks in a mode that the step length is 1 pixel, then performing mean value removing operation on pixel values in each matrix block, and arranging each matrix block in a large matrix P in sequence.

4. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the second step (2.1): performing low-rank matrix extraction on the arranged large matrix P, and extracting information by using an RPCA (resilient packet access) low-rank matrix decomposition algorithm, wherein the extraction process is a solution optimization problem:

$\begin{array}{cc}\underset{X,E}{min}& \left|\right|X|{|}_{*}+\mathrm{\λ}|\left|E\right|{|}_1\end{array}$

s.t.P＝X+E

where X is the low rank matrix of the large matrix P and E is its sparse matrix. | X | non-conducting phosphor_*Representing a kernel norm operation taking matrix X, | E | | luminance₁Representing l of the taking matrix E₁Norm operation, i.e.Where m, n denote the number of rows and columns, respectively, of the matrix E, E_ijRepresenting the element in matrix E located in row i and column j.

5. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the second step (2.2): extracting L for low rank matrix X₁The eigenvectors corresponding to the maximum eigenvalues can be directly obtained in the RPCA solving process, and then the vectors are rearranged into k-size vectors according to the columns₁×k₂Form L of₁A convolution kernelL₁One convolution kernel and N recovery matrices X_iPerforming convolution operation to obtain N × L₁Zhang feature atlas

Repeat step (2.1) for this N × L₁Zhang feature atlasExtracting corresponding L₂A convolution kernelAnd use the L₂The convolution kernel convolves the feature map to obtain a new feature map

6. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the second step (2.3), the new feature map is subjected to binarization treatment, and the formula is as follows:

$H\left(y\right)=\left\{\begin{array}{cc}1,& ify>0\\ 0,& otherwise\end{array}\right.,$

wherein y is a novel characteristic mapEach pixel value of (a).

7. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the second step (2.3), weighting processing is carried out on the new feature map,

wherein,the final characteristic map has a value range of

8. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the second step (2.4), the final characteristic map is processedPerforming histogram statistics according to blocks; for each input image there is L₁Final feature atlasEach sheet is putDividing the image into B image blocks; making histogram statistics in each image block, each image block hasItem, pulling B block image blocks into vectors in sequence to obtainHere, theThe vector dimension obtained by the representation isB dimension;

handle L₁AnPutting the image into a vector to obtain the final characteristics of each input image

9. The incomplete face recognition method based on the convolutional neural network as claimed in claim 1, wherein: in the third step (3.1),

by the final features of each figureTaking the class labels corresponding to the class labels as input, and training the corresponding relation by using an SVM (support vector machine) to obtain a classifier;

and for a new test sample, obtaining the final characteristics of the input image through the first step and the second step, inputting the final characteristics into the SVM, and comparing the final characteristics with the classifier to identify the specific class label corresponding to the new sample, thereby realizing the classification and identification of the face image.