CN111611877B - Anti-age-interference face recognition method based on multi-time-space information fusion - Google Patents

Anti-age-interference face recognition method based on multi-time-space information fusion Download PDF

Info

Publication number
CN111611877B
CN111611877B CN202010363324.7A CN202010363324A CN111611877B CN 111611877 B CN111611877 B CN 111611877B CN 202010363324 A CN202010363324 A CN 202010363324A CN 111611877 B CN111611877 B CN 111611877B
Authority
CN
China
Prior art keywords
face
age
features
identity
space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010363324.7A
Other languages
Chinese (zh)
Other versions
CN111611877A (en
Inventor
颜成钢
孟利选
殷建
孙垚棋
张继勇
张勇东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202010363324.7A priority Critical patent/CN111611877B/en
Publication of CN111611877A publication Critical patent/CN111611877A/en
Application granted granted Critical
Publication of CN111611877B publication Critical patent/CN111611877B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/179—Human faces, e.g. facial parts, sketches or expressions metadata assisted face recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Collating Specific Patterns (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an anti-age-interference face recognition method based on multi-time-space information fusion. The initial face features extracted in the invention are obtained by extracting face features from time dimension and space dimension of face images of the same person at different ages in each training sample and then carrying out information fusion, so that the invention not only can fully represent the identity features of each person, but also has certain robustness to age interference. In addition, similarity measurement is further carried out after the identity features and the age projection features are obtained, the correlation between the extracted identity features and the age features of each person is gradually reduced through the constraint of a similarity loss function, and the smaller the correlation is, the less the identity features are easily affected by age factors, and the higher the accuracy of identifying faces across ages is.

Description

Anti-age-interference face recognition method based on multi-time-space information fusion
Technical Field
The invention relates to the technical field of face recognition, in particular to an age-interference-resistant face recognition method based on multi-time-space information fusion.
Background
The face recognition technology is widely applied to the fields of security and protection, traffic, smart cities and the like for identity recognition. The face recognition technology is used as a biological recognition technology, key features of the face can be effectively extracted by combining multiple field knowledge such as the neuroscience field, the computer field and mathematics field, and the accurate recognition of the face is realized by comparing the similarity of the features of the two faces. Generally, the face recognition method includes the following four steps:
step one: and acquiring a face image and related data thereof. In general, factors such as the size and quality of the face dataset used for model training directly affect the performance of the model. It is therefore necessary to acquire a large number of face images suitable for model training from the internet, and add corresponding labels required by the model, such as identity labels, gender labels, age labels, etc., to each image. After the face image and the corresponding label data are acquired, the data set is divided into a corresponding training set, a corresponding verification set and a corresponding test set for later models.
Step two: preprocessing a face image. And processing each face image in the data set according to the characteristics of the model, wherein the processing comprises the operations of face detection and alignment, face image cutting, face overturning and the like.
Step three: and (5) extracting face features. And designing a corresponding model algorithm according to the specific requirements of face recognition, and inputting the preprocessed face image into the designed model algorithm to obtain the corresponding characteristics of the face image.
Step four: and (5) comparing and identifying the human faces. And (3) comparing the face features extracted in the step (III) with the face features subjected to verification and test to calculate the similarity, thereby realizing face recognition.
However, for some specific application scenarios, such as searching for missing children, identifying the identity of an evading person, identifying the face of an outbound passport, etc., there is a higher requirement on the face identification technology, and it is required that the accuracy of face identification can be ensured even though the two faces are far apart. Referring to fig. 1, since an increase in age may affect a face change of a person, the similarity of facial features of different ages of the same person may be greater than the similarity of facial features of the same ages of different persons. Therefore, the invention of the face recognition technology for resisting the age interference is particularly important for the application in certain fields. The face recognition method for solving the problem of age interference at present mainly comprises the following steps: discriminant methods and generative methods.
The discriminant method is a face recognition method for preventing age interference, which is to reduce age information as much as possible from the extracted features, namely, reduce the correlation between the identity features and the age features of the face, because the features of the face extracted in the step three generally comprise the age features. The generating method is to synthesize the face to the face of the age stage of the face to be recognized based on strong assumption, then compare the synthesized face with the face to be recognized to calculate the similarity, and further achieve the face recognition effect not affected by the age.
Disclosure of Invention
The invention aims to provide an anti-age-interference face recognition method based on multi-time-space information fusion, which aims to effectively overcome the defect that faces at intervals for many years cannot be accurately recognized due to age factor interference and improve the accuracy of recognizing faces across ages.
In order to achieve the above objects, the present invention provides an anti-age-interference face recognition method based on multi-time-space information fusion, which comprises the following parts:
step 1, preprocessing a data set;
the data set comprises a training set, a verification set and a test set;
step 2, loading batch data:
each training sample comprises face images of the same person at different ages, and age tags and identity tags corresponding to each image; the age label is age or out-of-body year, and the identity label is a unique name or identity card number;
the loaded batch data thus comprises a plurality of training samples, each training sample comprising a plurality of face images of the same person;
step 3, extracting multi-time space face features:
in order to acquire the characteristics capable of fully expressing the face identity of each person, extracting the face initial characteristics from the face image corresponding to each person in two aspects of time dimension and space dimension, wherein the extraction of the multi-space-time face characteristics comprises the following two parts: extracting face features in space dimension and fusing face features in time dimension;
step 4, decorrelation of face features:
based on the extraction of the space dimension face features and the fusion of the time dimension face features, obtaining initial multi-space face features;
the decorrelation of the face features refers to removing age features related to identity features from multi-time space face features; the face feature decorrelation module comprises two parts: an age characteristic projection module and an identity characteristic extraction module;
step 5, loss function of face feature extraction
Along with the training of the model, the training loss of the model is reduced through a loss function, and the parameters of the model are continuously updated; the loss function of face feature extraction mainly comprises the following three parts: identity loss function, age loss function, and relevance loss function.
Further, the step 1 is specifically implemented as follows:
1-1. Acquisition of data sets:
selecting an age-crossing face recognition public data set, and dividing the age-crossing face recognition public data set into a training set, a verification set and a test set;
1-2, preprocessing face images in a data set:
1-2-1, detecting and aligning face images in a data set by using MTCNN;
1-2-2, converting the aligned RGB face image into a single-channel gray scale image;
1-2-3, resetting the resolution of the gray level map based on given parameters to obtain a face image I;
1-2-4, randomly cutting the face image I according to a given size, wherein the size of the face image I obtained after cutting is required to match with the input size of the model;
and 1-2-5, normalizing the face image obtained after cutting, namely subtracting the mean value and dividing the mean value by the standard deviation.
Further, the step 3 is specifically implemented as follows:
the face features of the space dimension are extracted as follows:
inputting the preprocessed face image into a face feature extraction model based on the preprocessing in the step 1 and the mode of loading the batch data in the step 2, and extracting the face features of the single face image from the space dimension; the face feature extraction model mainly comprises a convolutional neural network and a residual error network ResNet, lightCNN model;
the human face features of the time dimension are fused:
after face features based on space dimension are extracted, the face features of the space dimension of a plurality of face images of different ages of the same person are input into a fusion model, so that the features are fused in the time dimension; the fusion model takes a cyclic neural network as a main part, and takes the space dimension face characteristics of a plurality of face images of different ages of the same person in each training sample as input, so that the face characteristics output at the previous moment are taken as the input of the fusion model at the next moment, namely the output of the fusion model network is not only dependent on the face characteristics of the corresponding ages at the current moment as the input, but also is related to the face characteristics of the corresponding ages output at the previous moment; the fusion model is used for memorizing the face features of the same person at different ages and outputting the multi-time space face features of the same person; finally, the information fusion of the face features in the time dimension is realized; the fusion model comprises a long-term memory network LSTM and a GRU.
Further, the step 4 is specifically implemented as follows:
based on the extraction of the space dimension face features and the fusion of the time dimension face features, obtaining initial multi-space face features;
the decorrelation of the face features refers to removing age features related to identity features from multi-time space face features; the face feature decorrelation module comprises two parts: an age characteristic projection module and an identity characteristic extraction module;
the age characteristic projection module is used for extracting preliminary age characteristics based on the extracted multi-time-space face characteristics of each person, then calculating the projection of the extracted age characteristics on the multi-time-space face characteristics, and representing the age projection characteristics; specifically, based on corresponding multi-time space face features of facesX time-space And age characteristic X age Calculating an age projection characteristic X age-projection The formula of (2) is as follows:
Figure SMS_1
x obtained based on the above formula (1) age-projection The age projection characteristic of the age characteristic on the multi-time space face characteristic is obtained;
the identity feature extraction module refers to the multi-time space face feature X corresponding to each person based on extraction time-space And age projection feature X age-projection Performing linear transformation of the characteristics to finally obtain corresponding identity characteristics of each person; specifically, based on the extracted multi-space-time face features X time-space And age projection feature X age-projection Calculating identity feature X identity The formula of (2) is as follows:
X identity ＝X time-space -X age-projection (2)
the identity X obtained based on the above formula (2) identity The identity after decorrelation is the identity feature.
Further, the step 5 is specifically implemented as follows:
the loss function of face feature extraction mainly comprises the following three parts: identity loss function, age loss function, and relevance loss function; specifically, each corresponding loss function implementation of face feature extraction is as follows:
said identity loss function L identity ：
Selecting CosFace as a specific identity loss function L identity The loss function formula is as follows:
Figure SMS_2
in the above formula (3), N is the number of training samples, s is the radius hyper-parameter of the hyper-sphere, m is the margin hyper-parameter, θ yi Of the yi-th classThe included angle between the weight vector and the true value characteristic vector;
said age loss function L age Using the loss function of the classification problem, selecting a cross entropy loss function as the loss function of age discrimination;
the similarity loss function L s ：
Based on the extracted age projection features and identity features corresponding to the face, the decorrelation loss of the identity features is measured by calculating the similarity of the extracted age projection features and the identity features after decorrelation; the correlation between the age projection features and the identity features is made smaller and smaller by the constraint of the correlation loss function, the parameter is updated and optimized continuously, and finally, the minimum correlation between the age projection features and the identity features of the face is realized, namely, the extraction and the identification of the identity features of the face resisting the age interference are realized; specifically, the characteristic X is projected based on the respective ages of each person age-projection And identity feature X identity The similarity calculation formula of (2) is as follows:
Figure SMS_3
L s ＝exp(|P|)
wherein M in the above formula (4) age-projection And M identity Respectively representing the statistical mean of the age projection characteristics and the statistical mean of the identity characteristics, V age-projection And V identity The variance of the projection characteristic of the age and the variance of the identity characteristic are respectively represented, epsilon represents a very small constant and is used for guaranteeing the nonnegativity of denominator;
based on the identity loss function, age loss function, and similarity loss function, the total training loss function L is a linear combination of the three, specifically, as shown in the following formula:
L＝L identity +αL age +βL s (5)
where α and β are training hyper-parameters that are used to balance the weights of the three loss functions.
The invention has the following advantages and innovations:
the invention provides an anti-age-interference face recognition method based on multi-time-space information fusion. The multi-space-time face features not only consider the identity features of a face image in a potential high-dimensional space, but also consider the fact that the potential identity features of images of different ages are subjected to information fusion in the time dimension along with the continuous growth of the ages, so that the obtained initial multi-space-time face features not only can fully represent the identity features of each person, but also have certain robustness to age interference. And secondly, in order to extract the identity features resisting the interference of the age factors, the age features which are primarily extracted are projected on the multi-time space human face features in the age feature projection module to obtain age projection features, and linear transformation is carried out on the basis of the multi-time space human face features and the age projection features to obtain the corresponding identity features of the human face. Finally, in order to continuously reduce the correlation between the extracted face identity features and the age projection features, the correlation between the face identity features and the age projection features in the training data is calculated, and the correlation between the face identity features and the age projection features is gradually reduced along with the training of the model through a correlation loss function.
Drawings
Fig. 1 is a face image of two persons of different ages in a face data set FGNET;
FIG. 2 is an overall flow chart of the model training of the present invention;
FIG. 3 is a schematic diagram of a multi-time-space face feature extraction module according to the present invention;
FIG. 4 is a schematic diagram of a facial feature decorrelation module according to the present invention;
fig. 5 is a model training diagram of the same person using a plurality of face images of different ages as input in an embodiment of the present invention.
Detailed Description
In order to more clearly describe the objects, technical contents and advantages of the present invention, further description will be made with reference to the specific invention embodiments and the related drawings. Note that the described embodiments of the invention are not limited thereto, and that the described examples are only some, but not all, examples of the present application. Based on the teachings of this disclosure, one of ordinary skill in the art would be able to contemplate other implementations of the present invention without performing any inventive effort. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention.
The existing face recognition technology for solving the problem of age interference is mainly divided into a generation type method and a discriminant type method, and the age interference resistance face recognition method based on multi-time space information fusion is also a discriminant type method. Different from other discriminant methods in the prior art, the initial face features extracted in the invention are obtained by extracting face features from time dimension and space dimension by face images of the same person at different ages in each training sample and then carrying out information fusion, so that the invention not only can fully represent the identity features of each person, but also has certain robustness to age interference. In addition, similarity measurement is further carried out after the identity features and the age projection features are obtained, the correlation between the extracted identity features and the age features of each person is gradually reduced through the constraint of a similarity loss function, and the smaller the correlation is, the less the identity features are easily affected by age factors, and the higher the accuracy of identifying faces across ages is.
The initial multi-time space face features extracted by the method are obtained by extracting face features from time dimension and space dimension of face images of the same person at different ages in each training sample and then carrying out information fusion; based on the extracted multi-time-space face features of each person, extracting preliminary age features, and then calculating the projection of the extracted age features on the multi-time-space face features to represent age projection features; then, carrying out linear transformation of the features based on the extracted multi-time space face features and age projection features corresponding to each person, and finally obtaining corresponding identity features of each person; removing age characteristics related to the identity characteristics from the multi-time space face characteristics; and the similarity measurement is further carried out after the identity features and the age projection features are obtained, the correlation between the extracted identity features and the age features of each person is gradually reduced through the constraint of the similarity loss function, and the accuracy of identifying the faces across ages is improved.
With reference to the flowchart of the model training method shown in fig. 2, each step of the model training method provided in the embodiment of the present application is described in detail below.
Step 1, preprocessing of data set
Preprocessing of the data set refers to processing of the data set before training data is sent to the network for training, and comprises the following parts:
1-1. Acquisition of data sets: selecting an age-crossing face recognition public data set, and dividing the age-crossing face recognition public data set into a training set, a verification set and a test set;
the age-span face recognition public data set can be provided for scientific researchers to train models, and comprises a CACD, FGNET, MORPH, LFW data set;
meanwhile, the training set, the verification set and the test set are divided according to the characteristics of the model;
1-2, preprocessing face images in a data set:
1-2-1, detecting and aligning face images in a data set by using MTCNN;
1-2-2, converting the aligned RGB face image into a single-channel gray scale image;
1-2-3, resetting the resolution of the gray level map based on given parameters to obtain a face image I;
1-2-4, randomly cutting the face image I according to a given size, wherein the size of the face image I obtained after cutting is required to match with the input size of the model;
and 1-2-5, normalizing the face image obtained after cutting, namely subtracting the mean value and dividing the mean value by the standard deviation.
The data set comprises a training set, a verification set and a test set.
Step 2, loading batch data:
to speed up model training, the model training is parallelized, and typically the training data loaded is not a single training sample, but is batch data made up of multiple training samples.
In this embodiment, each training sample includes face images of the same person at different ages, and age tags and identity tags corresponding to each image; the age label is age or out-of-body year, and the identity label is a unique name or identity card number;
the loaded batch data thus comprises a plurality of training samples, each training sample comprising a plurality of face images of the same person, in preparation for information fusion in time and space dimensions for a subsequent model.
Step 3, extracting multi-time space face features:
in order to acquire the characteristics capable of fully expressing the face identity of each person, the face initial characteristics of the face image corresponding to each person are extracted from two aspects of time dimension and space dimension. As shown in fig. 3, the extraction of the multi-space-time face features includes the following two parts: face feature extraction in the space dimension and face feature fusion in the time dimension.
The face features of the spatial dimension are extracted as follows:
inputting the preprocessed face image into a face feature extraction model based on the preprocessing in the step 1 and the mode of loading the batch data in the step 2, and extracting the face features of the single face image from the space dimension;
the face feature extraction model mainly comprises a convolutional neural network (Convolutional Neural Network), and features of a face image are extracted from a bottom network layer to a deep network layer of the model gradually, so that the effect of extracting texture features, edge features and deep contour features of the face is achieved. As a specific example of the present application, the face feature extraction model optionally includes a model such as a residual network ResNet, lightCNN, for example, residual network res net may implement extraction of face features in a spatial dimension by jumping forward shallow network layer features of a face image to deep network layers.
Face feature fusion in time dimension:
after face features based on the space dimension are extracted, the face features of the space dimension of a plurality of face images of different ages of the same person are input into a fusion model, so that the features are fused in the time dimension.
The fusion model takes a cyclic neural network (Recurrent Neural Network) as a main part, and takes the space dimension face characteristics of a plurality of face images of different ages of the same person in each training sample as input, so that the face characteristics output at the previous moment are taken as the input of the fusion model at the next moment, namely, the output of the fusion model network is not only dependent on the face characteristics of the corresponding ages at the current moment as input, but also is related to the face characteristics of the corresponding ages output at the previous moment. The fusion model is used for memorizing the face features of the same person at different ages and outputting the multi-time space face features of the same person; and finally, the information fusion of the face features in the time dimension is realized.
As a specific example of the application, the fusion model optionally comprises a Long Short-Term Memory network LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), weighting of features and the like. Based on a model of feature fusion in the elapsed time dimension, the extracted face features more fully represent the corresponding identity features of each person on one hand, and on the other hand, the extracted face features are more robust to different ages.
Step 4, decorrelation of face features:
and obtaining the initial multi-time space face features based on the extraction of the space dimension face features and the fusion of the time dimension face features.
The decorrelation of the face features refers to removing age features related to identity features from multi-time space face features. As shown in fig. 4, the face feature decorrelation module includes two parts: an age characteristic projection module and an identity characteristic extraction module.
Age characteristic projection module:
the age characteristic projection module is used for extracting preliminary age characteristics based on the extracted multi-time-space face characteristics of each person, then calculating the projection of the extracted age characteristics on the multi-time-space face characteristics, and representing the age projection characteristics; specifically, based on corresponding multi-time space face characteristics X of face time-space And age characteristic X age Calculating an age projection characteristic X age-projection The formula of (2) is as follows:
Figure SMS_4
x obtained based on the above formula (1) age-projection And the age projection characteristic of the age characteristic on the multi-time space face characteristic is obtained.
The identity feature extraction module:
the identity feature extraction module refers to the multi-time space face feature X corresponding to each person based on extraction time-space And age projection feature X age-projection And performing linear transformation of the characteristics to finally obtain the corresponding identity characteristics of each person. Specifically, based on the extracted multi-space-time face features X time-space And age projection feature X age-projection Calculating identity feature X identity The formula of (2) is as follows:
X identity ＝X time-space -X age-projection (2)
the identity X obtained based on the above formula (2) identity The identity after decorrelation is the identity feature.
Based on the embodiment of extracting the multi-time space face features, the age features related to the face corresponding identity features can be greatly reduced through the decorrelation module of the face features, and the problem that the extracted face identity features are easily interfered by ages can be effectively solved on the basis of fully extracting the corresponding potential unchanged identity features of each person.
Step 5, loss function of face feature extraction
Based on the above embodiments of the partial models, as the model is trained, the training loss of the model needs to be reduced by a loss function, and the parameters of the model are continuously updated.
The loss function of face feature extraction mainly comprises the following three parts: identity loss function, age loss function, and relevance loss function.
Specifically, each corresponding loss function implementation of face feature extraction is as follows:
identity loss function L identity ：
The identity loss function may be selected from Center loss, triplet loss, sphereFace, cosFace, arcFace, and the like as the loss function. As an example of the present solution, cosFace is selected as a specific identity loss function, specifically, the loss function formula is as follows:
Figure SMS_5
in the above formula (3), N is the number of training samples, s is the radius hyper-parameter of the hyper-sphere, m is the margin hyper-parameter, θ yi Is the included angle between the weight vector of the yi-th class and the true value characteristic vector.
Age loss function L age ：
The age-loss function uses a loss function of the classification problem. As a specific embodiment of the present approach, alternatively, the cross entropy loss function (Cross Entropy Loss) may be a loss function for age discrimination.
Similarity loss function L s ：
Based on the implementation scheme for extracting the corresponding age projection features and identity features of the face, the decorrelation loss of the identity features is measured by calculating the similarity of the extracted age projection features and identity features after decorrelation. The smaller the calculated similarity between the age projection feature and the identity feature, the better the decorrelation effect of the identity feature is, and the smaller the decorrelation loss is naturally. Thus making years by constraint of relevance loss functionThe correlation between the age projection features and the identity features is smaller and smaller, the parameters are updated and optimized continuously, and finally the minimum correlation between the age projection features and the identity features of the face is realized, namely the extraction and the identification of the identity features of the face with age interference resistance are realized. Specifically, the characteristic X is projected based on the respective ages of each person age-projection And identity feature X identity The similarity calculation formula of (2) is as follows:
Figure SMS_6
L s ＝exp(|P|)
wherein M in the above formula (4) age-projection And M identity Respectively representing the statistical mean of the age projection characteristics and the statistical mean of the identity characteristics, V age-projection And V identity Representing the variance of the age projection feature and the variance of the identity feature, respectively, epsilon represents a very small constant, e.g., 0.001, for ensuring non-negativity of the denominator.
Based on the implementation of the identity loss function, age loss function, and similarity loss function described above, the total training loss function L of this example is a linear combination of the three, specifically, as shown in the following formula:
L＝L identity +αL age +βL s (5)
where α and β are training hyper-parameters that are used to balance the weights of the three loss functions.
Based on the scheme setting of each part of the age-interference-resistant face recognition method with multi-time-space information fusion, the model is optimized by using a random gradient descent algorithm (Stochastic gradient descent, SGD), and finally the model meeting the requirements of the invention target is obtained.
Therefore, the anti-age-interference face recognition method based on multi-time-space information fusion, which uses a plurality of face images of the same person with different ages as input, has the following obvious advantages:
(1) The extraction of the face features from face images of the same person at different ages is realized by utilizing a multi-space-time face feature extraction technology. The face feature extraction based on the space dimension and the face feature fusion based on the time dimension fully extract the potential identity features which are not interfered by the ages in the face images of the same person at all ages, thereby further preparing for the decorrelation of the subsequent face identity features.
(2) Further, the decorrelation module of the face identity features of the invention extracts preliminary age features of the face, calculates projection features, namely age projection features, of the age features on the multi-temporal-spatial face features extracted in the earlier stage, realizes the extraction of the features interfered by the age, and finally obtains final identity features irrelevant to the age through the linear transformation of the multi-temporal-spatial face features and the age projection features.
In this specification, the invention has been described with reference to specific embodiments thereof. It will be apparent, however, that various modifications and changes may be made without departing from the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (2)

1. The anti-age-interference face recognition method based on multi-time-space information fusion is characterized in that the initial multi-time-space face features extracted by the method are obtained by extracting face features from time dimension and space dimension of face images of the same person at different ages in each training sample and then carrying out information fusion; based on the extracted multi-time-space face features of each person, extracting preliminary age features, and then calculating the projection of the extracted age features on the multi-time-space face features to represent age projection features; then, carrying out linear transformation of the features based on the extracted multi-time space face features and age projection features corresponding to each person, and finally obtaining corresponding identity features of each person; removing age characteristics related to the identity characteristics from the multi-time space face characteristics; further carrying out similarity measurement after obtaining the identity features and the age projection features, gradually reducing the correlation between the extracted identity features and the age features of each person through the constraint of a similarity loss function, and improving the accuracy of identifying the faces across ages;
the method comprises the following specific implementation steps:
step 1, preprocessing a data set;
the data set comprises a training set, a verification set and a test set;
step 2, loading batch data:
each training sample comprises face images of the same person at different ages, and age tags and identity tags corresponding to each image; the age label is age or out-of-body year, and the identity label is a unique name or identity card number;
the loaded batch data thus comprises a plurality of training samples, each training sample comprising a plurality of face images of the same person;
step 3, extracting multi-time space face features:
in order to acquire the characteristics capable of fully expressing the face identity of each person, extracting the face initial characteristics from the face image corresponding to each person in two aspects of time dimension and space dimension, wherein the extraction of the multi-space-time face characteristics comprises the following two parts: extracting face features in space dimension and fusing face features in time dimension;
step 4, decorrelation of face features:
based on the extraction of the space dimension face features and the fusion of the time dimension face features, obtaining initial multi-space face features;
the decorrelation of the face features refers to removing age features related to identity features from multi-time space face features; the face feature decorrelation module comprises two parts: an age characteristic projection module and an identity characteristic extraction module;
step 5, loss function of face feature extraction
Along with the training of the model, the training loss of the model is reduced through a loss function, and the parameters of the model are continuously updated; the loss function of face feature extraction mainly comprises the following three parts: identity loss function, age loss function, and relevance loss function;
the step 3 is specifically realized as follows:
the face features of the space dimension are extracted as follows:
inputting the preprocessed face image into a face feature extraction model based on the preprocessing in the step 1 and the mode of loading the batch data in the step 2, and extracting the face features of the single face image from the space dimension; the face feature extraction model mainly comprises a convolutional neural network and a residual error network ResNet, lightCNN model;
the human face features of the time dimension are fused:
after face features based on space dimension are extracted, the face features of the space dimension of a plurality of face images of different ages of the same person are input into a fusion model, so that the features are fused in the time dimension; the fusion model takes a cyclic neural network as a main part, and takes the space dimension face characteristics of a plurality of face images of different ages of the same person in each training sample as input, so that the face characteristics output at the previous moment are taken as the input of the fusion model at the next moment, namely the output of the fusion model network is not only dependent on the face characteristics of the corresponding ages at the current moment as the input, but also is related to the face characteristics of the corresponding ages output at the previous moment; the fusion model is used for memorizing the face features of the same person at different ages and outputting the multi-time space face features of the same person; finally, the information fusion of the face features in the time dimension is realized; the fusion model comprises a long-term memory network LSTM and a GRU;
the step 4 is specifically realized as follows:
based on the extraction of the space dimension face features and the fusion of the time dimension face features, obtaining initial multi-space face features;
the decorrelation of the face features refers to removing age features related to identity features from multi-time space face features; the face feature decorrelation module comprises two parts: an age characteristic projection module and an identity characteristic extraction module;
the age characteristic projection module is used for extracting preliminary age characteristics based on the extracted multi-time-space face characteristics of each person, and then calculating the extracted age characteristics of the multi-time-space personProjection on the face feature, representing an age projection feature; specifically, based on corresponding multi-time space face characteristics X of face time-space And age characteristic X age Calculating an age projection characteristic X age-projection The formula of (2) is as follows:
Figure FDA0004091750420000031
x obtained based on the above formula (1) age-projection The age projection characteristic of the age characteristic on the multi-time space face characteristic is obtained;
the identity feature extraction module refers to the multi-time space face feature X corresponding to each person based on extraction time-space And age projection feature X age-projection Performing linear transformation of the characteristics to finally obtain corresponding identity characteristics of each person; specifically, based on the extracted multi-space-time face features X time-space And age projection feature X age-projection Calculating identity feature X identity The formula of (2) is as follows:
X identity ＝X time-space -X age-projection (2)
the identity X obtained based on the above formula (2) identity Namely, the identity characteristics after decorrelation;
the step 5 is specifically realized as follows:
the loss function of face feature extraction mainly comprises the following three parts: identity loss function, age loss function, and relevance loss function; specifically, each corresponding loss function implementation of face feature extraction is as follows:
said identity loss function L identity ：
Selecting CosFace as a specific identity loss function L identity The loss function formula is as follows:
Figure FDA0004091750420000032
in the above formula (3), N is the number of training samples, s is the radius hyper-parameter of the hyper-sphere, m is the margin hyper-parameter, θ yi An included angle between the weight vector of the yi-th class and the true value characteristic vector;
said age loss function L age Using the loss function of the classification problem, selecting a cross entropy loss function as the loss function of age discrimination;
the similarity loss function L s ：
Based on the extracted age projection features and identity features corresponding to the face, the decorrelation loss of the identity features is measured by calculating the similarity of the extracted age projection features and the identity features after decorrelation; the correlation between the age projection features and the identity features is made smaller and smaller by the constraint of the correlation loss function, the parameter is updated and optimized continuously, and finally, the minimum correlation between the age projection features and the identity features of the face is realized, namely, the extraction and the identification of the identity features of the face resisting the age interference are realized; specifically, the characteristic X is projected based on the respective ages of each person age-projection And identity feature X identity The similarity calculation formula of (2) is as follows:
Figure FDA0004091750420000041
L s ＝exp(|P|)
wherein M in the above formula (4) age-projection And M identity Respectively representing the statistical mean of the age projection characteristics and the statistical mean of the identity characteristics, V age-projection And V identity The variance of the projection characteristic of the age and the variance of the identity characteristic are respectively represented, epsilon represents a very small constant and is used for guaranteeing the nonnegativity of denominator;
based on the identity loss function, age loss function, and similarity loss function, the total training loss function L is a linear combination of the three, specifically, as shown in the following formula:
L＝L identity +αL age +βL s (5)
where α and β are training hyper-parameters that are used to balance the weights of the three loss functions.
2. The anti-age-interference face recognition method based on multi-time-space information fusion of claim 1, wherein the step 1 is specifically implemented as follows:
1-1. Acquisition of data sets:
selecting an age-crossing face recognition public data set, and dividing the age-crossing face recognition public data set into a training set, a verification set and a test set;
1-2, preprocessing face images in a data set:
1-2-1, detecting and aligning face images in a data set by using MTCNN;
1-2-2, converting the aligned RGB face image into a single-channel gray scale image;
1-2-3, resetting the resolution of the gray level map based on given parameters to obtain a face image I;
1-2-4, randomly cutting the face image I according to a given size, wherein the size of the face image I obtained after cutting is required to match with the input size of the model;
and 1-2-5, normalizing the face image obtained after cutting, namely subtracting the mean value and dividing the mean value by the standard deviation.
CN202010363324.7A 2020-04-30 2020-04-30 Anti-age-interference face recognition method based on multi-time-space information fusion Active CN111611877B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010363324.7A CN111611877B (en) 2020-04-30 2020-04-30 Anti-age-interference face recognition method based on multi-time-space information fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010363324.7A CN111611877B (en) 2020-04-30 2020-04-30 Anti-age-interference face recognition method based on multi-time-space information fusion

Publications (2)

Publication Number Publication Date
CN111611877A CN111611877A (en) 2020-09-01
CN111611877B true CN111611877B (en) 2023-05-02

Family

ID=72197652

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010363324.7A Active CN111611877B (en) 2020-04-30 2020-04-30 Anti-age-interference face recognition method based on multi-time-space information fusion

Country Status (1)

Country Link
CN (1) CN111611877B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112070011A (en) * 2020-09-08 2020-12-11 安徽兰臣信息科技有限公司 Noninductive face recognition camera shooting snapshot machine for finding lost children
CN112149530A (en) * 2020-09-08 2020-12-29 安徽兰臣信息科技有限公司 It is alert with wearing glasses based on children cross age bracket noninductive face identification technique
CN111920375B (en) * 2020-09-30 2021-01-15 平安科技（深圳）有限公司 Vascular Endothelial Growth Factor (VEGF) resistance curative effect prediction device and method
BR102020026136A2 (en) * 2020-12-18 2022-07-05 Neuralmind Inteligencia Artificial Sa BIOMETRIC AUTHENTICATION SYSTEM AND PROCESS OF FINGER BLOOD VESSEL PATTERNS FOR USER RECOGNITION
CN113221660B (en) * 2021-04-14 2022-08-30 河海大学 Cross-age face recognition method based on feature fusion
CN113505698A (en) * 2021-07-09 2021-10-15 平安信托有限责任公司 Face recognition method, device, equipment and medium based on counterstudy
CN113920573B (en) * 2021-11-22 2022-05-13 河海大学 Face change decoupling relativity relationship verification method based on counterstudy

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523463A (en) * 2018-11-20 2019-03-26 中山大学 A kind of face aging method generating confrontation network based on condition

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI427541B (en) * 2009-03-18 2014-02-21 Ind Tech Res Inst System and method for performing rapid facial recognition
CN102663413B (en) * 2012-03-09 2013-11-27 中盾信安科技(江苏)有限公司 Multi-gesture and cross-age oriented face image authentication method
US10579860B2 (en) * 2016-06-06 2020-03-03 Samsung Electronics Co., Ltd. Learning model for salient facial region detection
CN106650653B (en) * 2016-12-14 2020-09-15 广东顺德中山大学卡内基梅隆大学国际联合研究院 Construction method of human face recognition and age synthesis combined model based on deep learning
US10614623B2 (en) * 2017-03-21 2020-04-07 Canfield Scientific, Incorporated Methods and apparatuses for age appearance simulation
US10671840B2 (en) * 2017-05-04 2020-06-02 Intel Corporation Method and apparatus for person recognition using continuous self-learning
CN107330412B (en) * 2017-07-06 2021-03-26 湖北科技学院 Face age estimation method based on depth sparse representation
CN107977629A (en) * 2017-12-04 2018-05-01 电子科技大学 A kind of facial image aging synthetic method of feature based separation confrontation network
CN109977887A (en) * 2019-01-30 2019-07-05 四川电科维云信息技术有限公司 A kind of face identification method of anti-age interference
CN109993125B (en) * 2019-04-03 2022-12-23 腾讯科技（深圳）有限公司 Model training method, face recognition device, face recognition equipment and storage medium

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523463A (en) * 2018-11-20 2019-03-26 中山大学 A kind of face aging method generating confrontation network based on condition

Also Published As

Publication number Publication date
CN111611877A (en) 2020-09-01

Similar Documents

Publication Publication Date Title
CN111611877B (en) Anti-age-interference face recognition method based on multi-time-space information fusion
Liu et al. Attribute-aware face aging with wavelet-based generative adversarial networks
Zhao et al. Finger vein recognition based on lightweight CNN combining center loss and dynamic regularization
CN109948425B (en) Pedestrian searching method and device for structure-aware self-attention and online instance aggregation matching
CN104866829B (en) A kind of across age face verification method based on feature learning
CN105469034B (en) Face identification method based on Weighting type distinctive sparse constraint Non-negative Matrix Factorization
CN101419671B (en) Face gender identification method based on fuzzy support vector machine
CN104123560B (en) Fuzzy facial image verification method based on phase code feature and more metric learnings
CN113076927B (en) Finger vein identification method and system based on multi-source domain migration
CN110969073B (en) Facial expression recognition method based on feature fusion and BP neural network
CN114998220B (en) Tongue image detection and positioning method based on improved Tiny-YOLO v4 natural environment
CN113569554B (en) Entity pair matching method and device in database, electronic equipment and storage medium
CN109255289A (en) A kind of across aging face identification method generating model based on unified formula
CN112560710B (en) Method for constructing finger vein recognition system and finger vein recognition system
CN106529377A (en) Age estimating method, age estimating device and age estimating system based on image
CN111931630B (en) Dynamic expression recognition method based on facial feature point data enhancement
CN105718898B (en) Face age estimation method, system based on sparse undirected probability graph model
Zuobin et al. Feature regrouping for cca-based feature fusion and extraction through normalized cut
CN111144462B (en) Unknown individual identification method and device for radar signals
CN115690549A (en) Target detection method for realizing multi-dimensional feature fusion based on parallel interaction architecture model
CN112598055B (en) Helmet wearing detection method, computer-readable storage medium and electronic device
CN114998960B (en) Expression recognition method based on positive and negative sample contrast learning
Wei et al. Shape-based object recognition via Evidence Accumulation Inference
CN115471908A (en) Attitude processing method, apparatus, system and storage medium
Chun-Rong Research on face recognition technology based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant