CN108009528B - Triple Loss-based face authentication method and device, computer equipment and storage medium - Google Patents
Triple Loss-based face authentication method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN108009528B CN108009528B CN201711436879.4A CN201711436879A CN108009528B CN 108009528 B CN108009528 B CN 108009528B CN 201711436879 A CN201711436879 A CN 201711436879A CN 108009528 B CN108009528 B CN 108009528B
- Authority
- CN
- China
- Prior art keywords
- training
- face
- sample
- scene
- certificate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Abstract
The invention relates to a triple Loss-based face authentication method, a triple Loss-based face authentication device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring a certificate photo and a scene photo of a person based on the face authentication request; respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo; inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image output by the convolutional neural network model and a second feature vector corresponding to the certificate face image; calculating cosine distances of the first eigenvector and the second eigenvector; and comparing the cosine distance with a preset threshold value, and determining a face authentication result according to the comparison result. The method improves the reliability of the face authentication.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a triple Loss-based face authentication method, a triple Loss-based face authentication device, computer equipment and a storage medium.
Background
The face authentication means comparing a scene picture of a person collected on site with a certificate picture in identity information and judging whether the person is the same person. The key technology of face authentication is face recognition.
With the rise of deep learning technology, the related problems of face recognition continuously break through the traditional technical bottleneck, and the performance level is greatly improved. In the research work of applying deep learning to solve the problem of face recognition, there are two main methods: a method based on classification learning and a method based on metric learning. The method based on classification learning is mainly characterized in that after the deep convolutional network extracts features, classification loss functions (such as softmax loss, center loss and related variants) of samples are calculated to optimize the network, the last layer of the network is a full connection layer for classification, the number of output nodes of the network is always consistent with the total class number of a training data set, the method is suitable for the condition that training samples are more, particularly training samples of the same class are rich, and the network can obtain better training effect and generalization capability. However, when the number of classes reaches hundreds of thousands or higher, the parameters of the final classification layer (full connection layer) of the network increase linearly and become quite large, which makes the network difficult to train.
The other method is a metric learning-based method, the method organizes training samples (such as binary pair or triple triplet) in a tuple mode, the network is optimized by directly calculating metric loss (such as coherent loss, triplet loss and the like) among samples based on convolution characteristic vectors without passing through a classification layer after the deep convolution network, and the method does not need the training classification layer, so that the network parameters are not influenced by the increase of the class number, the class number of a training data set is indefinite, and only the similar or dissimilar samples are selected to construct proper tuples according to corresponding strategies. Compared with a classification learning method, the metric learning method is more suitable for the conditions that the training data is wide in breadth but insufficient in depth (the number of sample types is large, but the number of similar samples is small), quite abundant tuple data can be constructed for training through different combinations among samples, meanwhile, the metric learning mode focuses more on the tuple internal relation, and the method has inherent advantages on the problem that the 1:1 face verification type judgment is yes or not.
In practical applications, many organizations require real-name registration, such as bank account opening, cell phone number registration, financial account opening, and the like. The real-name system registration requires that a user carries an identity card to a specified place, and after a worker verifies that the identity card corresponds to a photo of the identity card, the account can be opened successfully. With the development of internet technology, more and more organizations promote convenience services, and no longer force customers to reach specified network points. The geographic position of the user is not limited, the identity card is uploaded, the image acquisition device of the mobile terminal is used for acquiring the scene picture of the person on site, the system carries out face authentication, and after the face authentication is passed, the account can be opened successfully. While the traditional learning method based on measurement measures the similarity between samples by using Euclidean distance, the Euclidean distance measures the absolute distance of each point in space, and is directly related to the position coordinates of each point, which does not accord with the distribution attribute of the face feature space, so that the reliability of face recognition is low.
Disclosure of Invention
Based on this, it is necessary to provide a TripletLoss-based face authentication method, apparatus, computer device and storage medium for solving the problem of low reliability of the conventional face authentication method.
A triple Loss-based face authentication method comprises the following steps:
acquiring a certificate photo and a scene photo of a person based on the face authentication request;
respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo;
inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image and a second feature vector corresponding to the certificate face image, which are output by the convolutional neural network model; wherein the convolutional neural network model is obtained based on supervised training of a triplet loss function;
calculating cosine distances of the first feature vector and the second feature vector;
and comparing the cosine distance with a preset threshold value, and determining a face authentication result according to the comparison result.
In one embodiment, the method further comprises:
acquiring a training sample with marks, wherein the training sample comprises a certificate face image and at least one scene face image which are marked to belong to each marked object;
training a convolutional neural network module according to the training samples, and generating triple elements corresponding to the training samples through an OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample;
training the convolutional neural network model based on the supervision of a triple loss function according to the triple elements of each training sample; the triple loss function takes cosine distance as a measurement mode and optimizes model parameters through a random gradient descent algorithm;
and inputting the verification set data into the convolutional neural network model, and obtaining the trained convolutional neural network model for face authentication when the training end condition is reached.
In another embodiment, the step of training a convolutional neural network model according to the training samples and generating a triplet of elements corresponding to each training sample by OHEM includes:
randomly selecting an image as a reference sample, and selecting an image which belongs to the same label object and is different from the reference sample as a positive sample;
according to an OHEM strategy, cosine distances among features are extracted by using a convolutional neural network model trained currently, and for each reference sample, an image which has the smallest distance and belongs to a different class from the reference sample is selected from other images which do not belong to the label object and serves as a negative sample of the reference sample.
In another embodiment, the triplet loss function includes a definition of cosine distances for homogeneous samples and a definition of cosine distances for heterogeneous samples.
In another embodiment, the triplet loss function is:
wherein cos (·) represents the cosine distance, which is calculated by
N is the number of the triples,
a feature vector representing a reference sample,
a feature vector representing a homogeneous positive sample,
feature vectors representing heterogeneous negative examples [ ·]+The meanings of (A) are as follows:
α1is an inter-class spacing parameter, α2Is an intra-class interval parameter.
In another embodiment, the method further comprises: initializing by using basic model parameters trained on the basis of mass open source face data, and adding a normalization layer and a triple loss function layer behind a feature output layer to obtain a convolutional neural network model to be trained.
A triple Loss-based face authentication device comprises: the system comprises an image acquisition module, an image preprocessing module, a feature acquisition module, a calculation module and an authentication module;
the image acquisition module is used for acquiring a certificate photo and a scene photo of a person based on the face authentication request;
the image preprocessing module is used for respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo;
the feature acquisition module is used for inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image and a second feature vector corresponding to the certificate face image, which are output by the convolutional neural network model; wherein the convolutional neural network model is obtained based on supervised training of a triplet loss function;
the calculating module is used for calculating cosine distances of the first feature vector and the second feature vector;
and the authentication module is used for comparing the cosine distance with a preset threshold value and determining a face authentication result according to the comparison result.
In another embodiment, the apparatus further comprises: the system comprises a sample acquisition module, a triple acquisition module, a training module and a verification module;
the sample acquisition module is used for acquiring a training sample with marks, wherein the training sample comprises a certificate face image and at least one scene face image which are marked to belong to each marked object;
the triple acquisition module is used for training a convolutional neural network model according to the training samples and generating triple elements corresponding to the training samples through an OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample;
the training module is used for training the convolutional neural network model according to the triplet elements of each training sample and the supervision of a basis triplet loss function; the triple loss function takes cosine distance as a measurement mode and optimizes model parameters through a random gradient descent algorithm;
and the verification module is used for inputting verification set data into the convolutional neural network model to obtain the trained convolutional neural network model for face authentication when a training ending condition is reached.
A computer device, comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor implements the steps of the triple Loss-based face authentication method when executing the computer program.
A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the triple Loss-based face authentication method described above.
The human face authentication method, the human face authentication device, the computer equipment and the storage medium based on triple Loss utilize a pre-trained convolutional neural network to carry out human face authentication, because a convolutional neural network model is obtained based on the supervised training of a triple Loss function, the similarity of a scene human face image and a certificate human face image is obtained by calculating the cosine distance of a first feature vector corresponding to the scene human face image and a second feature vector corresponding to the certificate human face image, the cosine distance is measured by the included angle of space vectors, the directional difference is reflected better, the distribution attribute of a human face feature space is better met, and the reliability of the human face authentication is improved.
Drawings
Fig. 1 is a schematic structural diagram of a triple Loss-based face authentication system according to an embodiment;
fig. 2 is a flowchart of a triple Loss based face authentication method according to an embodiment;
FIG. 3 is a flowchart of the steps in training a convolutional neural network model for face authentication in one embodiment;
FIG. 4 is a schematic diagram illustrating the probability of a sample being misclassified when the inter-class intervals are consistent and the intra-class variance is large;
FIG. 5 is a schematic diagram illustrating the probability of a sample being misclassified when the inter-class intervals are consistent and the intra-class variance is small;
FIG. 6 is a diagram illustrating a transfer learning process of triple Loss based face authentication in one embodiment;
FIG. 7 is a diagram illustrating a convolutional neural network model for face authentication in one embodiment;
fig. 8 is a schematic flow chart of a human face authentication method based on triple Loss in an embodiment;
fig. 9 is a block diagram of a triple Loss based face authentication device according to an embodiment;
fig. 10 is a block diagram of another embodiment of a triple Loss-based face authentication device.
Detailed Description
Fig. 1 is a schematic structural diagram of a triple Loss-based face authentication system according to an embodiment. As shown in fig. 1, the face authentication system includes a server 101 and an image capture device 102. The server 101 is connected to the image capturing apparatus 102 via a network. The image capturing device 102 captures a real-time scene photo of the user to be authenticated and a certificate photo, and transmits the captured real-time scene photo and certificate photo to the server 101. The server 101 determines whether the person in the scene photo and the person in the identification photo are the same person, and authenticates the identity of the user to be authenticated. Based on the specific application scenario, the image capturing apparatus 102 may be a camera or a user terminal with a camera function. Taking the account opening site as an example, the image acquisition device 102 may be a camera; taking account opening via internet as an example, the image capturing device 102 may be a mobile terminal with a camera function.
In other embodiments, the face authentication system may further include a card reader for reading a certificate photo in a certificate (e.g., identification card, etc.) chip.
Fig. 2 is a flowchart of a triple Loss-based face authentication method in an embodiment. As shown in fig. 2, the method includes:
s202, acquiring a certificate photo and a scene photo of a person based on the face authentication request.
The certificate photo refers to a photo corresponding to a certificate capable of proving the identity of a person, such as a certificate photo printed on an identity card or a certificate photo in a chip. The certificate photo can be obtained by photographing the certificate, and the certificate photo stored in the certificate chip can also be read by the card reader. The certificate in this embodiment may be an identity card, a driving license, a social security card, or the like.
The scene photo of the person refers to a photo of the user to be authenticated in the field environment, wherein the photo is collected during authentication of the user to be authenticated. The field environment refers to the environment where the user is located when taking a picture, and the field environment is not limited. The scene photo may be acquired by using a mobile terminal having a camera function to capture the scene photo and send the scene photo to the server.
The face authentication means comparing a scene picture of a person collected on site with a certificate picture in identity information and judging whether the person is the same person. The face authentication request is triggered based on actual application operations, for example, based on an account opening request of a user. And the application program prompts the user to collect the photo on a display interface of the user terminal, and sends the collected photo to the server for face authentication after the photo collection is finished.
And S204, respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo.
Face detection refers to recognizing a photo and acquiring a face region in the photo.
And the key point positioning means that the positions of the key points of the human face in each picture are obtained for the human face area detected in the picture. The key points of the human face comprise eyes, nose tips, mouth corner tips, eyebrows and contour points of all parts of the human face.
In this embodiment, a cascade convolutional neural network MTCNN method based on multitask joint learning may be used to simultaneously complete face detection and face key point detection, and a face detection method based on LBP features and a face key point detection method based on shape regression may also be used.
The image preprocessing refers to performing portrait alignment and shearing processing according to the position of the detected face key point in each picture, so as to obtain a scene face image with normalized size and a certificate face image. The scene face image is a face image obtained by performing face detection, key point positioning and image preprocessing on a scene photo, and the certificate face image is a face image obtained by performing face detection, key point positioning and image preprocessing on a certificate photo.
And S206, inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image output by the convolutional neural network model and a second feature vector corresponding to the certificate face image.
The convolutional neural network model is trained in advance according to training samples based on the supervision of a triplet loss function. The convolutional neural network comprises a convolutional layer, a pooling layer, an activation function layer and a full-connection layer, and each neuron parameter of each layer is determined through training. And acquiring a first feature vector of the scene face image output by the full connection layer of the convolutional neural network model and a second feature vector corresponding to the certificate face image by utilizing the trained convolutional neural network and through network forward propagation.
The triple (triplet) refers to randomly selecting a sample from a training data set, wherein the sample is called a reference sample, then randomly selecting a sample which belongs to the same person as the reference sample as a positive sample, and selecting a sample which does not belong to the same person as a negative sample, thereby forming a triple (the reference sample, the positive sample and the negative sample). Because the testimony comparison is mainly based on the comparison between the certificate photo and the scene photo, but not the comparison between the certificate photo and the certificate photo or between the scene photo and the scene photo, the triple modes mainly have two combinations: when the identification photo image is taken as a reference sample, the positive sample and the negative sample are both scene photos; when the scene photo image is taken as a reference sample, the positive sample and the negative sample are both certificate photos.
And training a parameter-shared network aiming at each sample in the triples to obtain the feature expression of the three elements. The objective of improving the triple loss is to learn that the distance between the feature expressions of the reference sample and the positive sample is as small as possible, the distance between the feature expressions of the reference sample and the negative sample is as large as possible, and the distance between the feature expressions of the reference sample and the positive sample and the distance between the feature expressions of the reference sample and the negative sample are at a minimum interval.
S208, calculating the cosine distance between the first characteristic vector and the second characteristic vector.
Cosine distance, also called cosine similarity, is a measure of the magnitude of the difference between two individuals using the cosine value of the angle between two vectors in a vector space. The larger the cosine distance between the first characteristic vector and the second characteristic vector is, the larger the similarity between the scene face image and the certificate face image is, and the smaller the cosine distance between the first characteristic vector and the second characteristic vector is, the smaller the similarity between the scene face image and the certificate face image is. When the cosine distance between the scene face image and the certificate face image is more received at 1, the probability that the two images belong to the same person is higher, and when the cosine distance between the scene face image and the certificate face image is smaller, the probability that the two images belong to the same person is lower.
In the conventional triple loss (triplet loss) method, euclidean distances are used to measure the similarity between samples. The Euclidean distance measurement is that the absolute distance of each point in the space is directly related to the position coordinate of each point, and the Euclidean distance measurement does not accord with the distribution attribute of the face feature space. In this embodiment, the distribution attribute of the face feature space and the actual application scene are considered, and the cosine distance is used to measure the similarity between samples. Cosine distance measures the included angle of the space vector, and the difference in direction is reflected rather than the position, so that the distribution attribute of the face feature space is better met.
Specifically, the formula for calculating the cosine distance is:
where x represents the first feature vector and y represents the second feature vector.
And S210, comparing the cosine distance with a preset threshold value, and determining a face authentication result according to the comparison result.
The authentication result comprises authentication passing, namely the certificate photo and the scene photo belong to the same person. The authentication result also includes authentication failure, i.e. the certificate photo and the scene photo do not belong to the same person.
Specifically, the cosine distance is compared with a preset threshold, when the cosine distance is greater than the preset threshold, the similarity between the certificate photo and the scene photo is greater than the preset threshold, the authentication is successful, and when the cosine distance is less than the preset threshold, the similarity between the certificate photo and the scene photo is less than the preset threshold, the authentication is failed.
The triple Loss-based face authentication method utilizes a pre-trained convolutional neural network to perform face authentication, the convolutional neural network model is obtained based on the supervised training of a triple Loss function, the similarity of the scene face image and the certificate face image is obtained by calculating the cosine distance of a first feature vector corresponding to the scene face image and a second feature vector corresponding to the certificate face image, and the cosine distance is the included angle of the space vectors and reflects the difference in direction better, so that the triple Loss-based face authentication method is more in line with the distribution attribute of the face feature space, and the reliability of the face authentication is improved.
In another embodiment, the face authentication method further comprises the step of training a convolutional neural network model for face authentication. FIG. 3 is a flow diagram of the steps in training a convolutional neural network model for face authentication in one embodiment. As shown in fig. 3, this step includes:
s302, obtaining a training sample with marks, wherein the training sample comprises a certificate face image and at least one scene face image which are marked to belong to each marked object.
In this embodiment, the object, i.e., the person, is marked, and the training sample is marked with the scene face image and the certificate face image belonging to the same person in human units. Specifically, the scene face image and the certificate face image can be obtained by carrying out face detection, key point positioning and image preprocessing on the marked scene photo and certificate photo.
Face detection refers to recognizing a photo and acquiring a face region in the photo.
And the key point positioning means that the positions of the key points of the human face in each picture are obtained for the human face area detected in the picture. The key points of the human face comprise eyes, nose tips, mouth corner tips, eyebrows and contour points of all parts of the human face.
In this embodiment, a cascade convolutional neural network MTCNN method based on multitask joint learning may be used to simultaneously complete face detection and face key, and a face detection method based on LBP features and a face key point detection method based on shape regression may also be used.
The image preprocessing refers to performing portrait alignment and shearing processing according to the position of the detected face key point in each picture, so as to obtain a size-normalized scene face image and a certificate face image. The scene face image is a face image obtained by performing face detection, key point positioning and image preprocessing on a scene photo, and the certificate face image is a face image obtained by performing face detection, key point positioning and image preprocessing on a certificate photo.
S304, training a convolutional neural network model according to the training samples, and generating triple elements corresponding to the training samples through an OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample.
There are two combinations of triplets: when the identification photo image is taken as a reference sample, the positive sample and the negative sample are both scene photo images; when the scene photo image is taken as a reference sample, the positive sample and the negative sample are both certificate photo images.
Specifically, taking a certificate photo as an example of a reference image, randomly selecting a certificate photo sample of one person from a training data set, wherein the sample is called a reference sample, then randomly selecting a scene photo sample belonging to the same person as the reference sample as a positive sample, and selecting a scene photo sample not belonging to the same person as a negative sample, thereby forming a (reference sample, positive sample, negative sample) triplet.
That is, the positive sample and the reference sample are the same type of sample, that is, belong to the same person image. Negative examples are heterogeneous examples of reference examples, i.e. images that do not belong to the same person. In the training process of the convolutional neural network, an OHEM (Online Hard sample mining) strategy is adopted for constructing a triplet on line by a negative sample, namely, in the process of each iterative optimization of the network, the current network is used for carrying out forward calculation on a candidate triplet, an image which does not belong to the same user as the reference sample in the training sample and has the closest cosine distance is selected as the negative sample, and thus the triplet element corresponding to each training sample is obtained.
In one embodiment, the step of training the convolutional neural network according to the training samples and generating the triplet elements corresponding to each training sample includes the following steps S1 and S2:
s1: one image is randomly selected as a reference sample, and images which belong to the same label object and have different types from the reference sample are selected as positive samples.
The category refers to the image type to which the training sample belongs, and in this embodiment, the category of the training sample includes a scene face image and a certificate face image. Because the face authentication mainly refers to the comparison between the certificate photo and the scene photo, the reference sample and the positive sample belong to different categories, and if the reference sample is the scene face image, the positive sample is the certificate face image; if the reference sample is the certificate face image, the positive sample is the scene face image.
S2: according to an OHEM strategy, extracting cosine distances among features by using a convolutional neural network model trained currently, and selecting an image which has the minimum distance and belongs to a different class from a reference sample as a negative sample of the reference sample from other images which do not belong to the same label object for each reference sample.
The negative sample is selected from face images of labels which are not of the same person as the reference sample, specifically, the negative sample adopts an OHEM strategy to construct a triple on line in the training process of the convolutional neural network, namely, in the process of each iterative optimization of the network, the current network is used for carrying out forward calculation on a candidate triple, and an image which is not of the same user as the reference sample in the training sample, has the closest cosine distance and does not belong to the same category as the reference sample is selected as the negative sample. That is, the negative examples are of a different class than the reference examples. It can be considered that if the identity photo is taken as a reference sample in the triplet, both the positive sample and the negative sample are scene photos; otherwise, if the scene photo is taken as the reference sample, the positive sample and the negative sample are both certificate photos.
S306, training a convolutional neural network model based on supervision of a triple loss function according to the triple elements of each training sample, wherein the triple loss function optimizes model parameters by a random gradient descent algorithm by taking cosine distance as a measurement mode.
The personal identification verification terminal verifies the identity of the user by comparing whether the chip photo of the user certificate is consistent with the scene photo, the data collected by the background is only two pictures of a sample of a single person, namely the certificate photo and the scene photo captured at the moment of comparison, but the number of different individuals can be thousands. If the data with a large number of categories and few similar samples are trained by a classification-based method, the parameters of the classification layer are too large to lead to the fact that the network is very difficult to learn, so that the method for metric learning is considered to solve the problem. The typical technique of metric learning is to use a triple loss (triplet loss) method to learn an effective feature mapping by constructing image triplets, and the feature distance of the similar sample is smaller than that of the dissimilar sample under the mapping, so as to achieve the purpose of correct comparison.
The objective of triple loss is to learn that the distance between the feature expressions of the reference sample and the positive sample is as small as possible, the distance between the feature expressions of the reference sample and the negative sample is as large as possible, and the distance between the feature expressions of the reference sample and the positive sample and the distance between the feature expressions of the reference sample and the negative sample are at a minimum interval.
In another embodiment, the triplet loss function includes a definition of cosine distances for homogeneous samples and a definition of cosine distances for heterogeneous samples.
Wherein, the homogeneous sample refers to the reference sample and the positive sample, and the heterogeneous sample refers to the reference sample and the negative sample. The cosine distance of the homogeneous sample refers to the cosine distance between the reference sample and the positive sample, and the cosine distance of the heterogeneous sample refers to the cosine distance between the reference sample and the negative sample.
On one hand, the original triplet loss method only considers the difference between classes but not the difference in classes, and if the distribution in the classes is not converged enough, the generalization capability of the network is weakened, and the adaptability to the scene is reduced. On the other hand, the original triplet loss method adopts Euclidean distance to measure the similarity between samples, and actually, the cosine distance is more adopted to measure in a feature comparison link after the face model is deployed. The Euclidean distance measurement is the absolute distance of each point in the space, and is directly related to the position coordinates of each point; the cosine distance measures the included angle of the space vector, and the difference in direction is reflected rather than the position, so that the distribution attribute of the face feature space is better met.
The basic idea of the original triplet loss is to make the distance between the reference sample and the positive sample smaller than the distance between the reference sample and the negative sample by metric learning and the difference between the distances larger than a minimum spacing parameter α, so the original triplet loss function is as follows:
where N is the number of triplets,
representing the feature vector of the reference sample (anchor),
a feature vector representing homogeneous positive samples (positive),
a feature vector representing a heterogeneous negative sample (negative).
Representing the L2 paradigm, the euclidean distance. [. the]+The meanings of (A) are as follows:
it can be seen from the above equation that the original triplet loss function only defines the distance between the similar sample (anchor) and the heterogeneous sample (anchor), i.e. the inter-class distance is increased as much as possible by the interval parameter α, and the intra-class distance is not defined, i.e. the distance between the similar samples is not constrained.
In order to solve the problems, the invention provides an improved triplet loss method, which retains the definition of the inter-class distance in the original method and increases the constraint term of the intra-class distance, so that the intra-class distance is converged as much as possible. The loss function expression is as follows:
wherein cos (·) represents the cosine distance, which is calculated by
N is the number of the triples,
a feature vector representing a reference sample,
a feature vector representing a homogeneous positive sample,
feature vectors representing heterogeneous negative examples [ ·]+The meanings of (A) are as follows:
α1is an inter-class spacing parameter, α2Is an intra-class interval parameter.
Compared to the original triThe method comprises the steps of using a simple loss function, changing Euclidean distance into cosine distance in a measurement mode of the improved triple loss function, keeping consistency of the measurement mode in a training stage and a deployment stage, and improving continuity of feature learning, wherein a first item of the new triple loss function is consistent with an original triple loss function and is used for increasing inter-class difference, and a second item adds distance constraint on similar sample pairs (positive tuples) and is used for reducing intra-class difference α1Is interval parameter between classes, the value range is 0-0.2, α2The interval parameter is an intra-class interval parameter, and the value range is 0.8-1.0. It should be noted that, since the measurement is performed in a cosine manner, the obtained measurement value corresponds to the similarity between two samples, so that the difference between the two samples is measured
In the expression, only the cosine similarity of the negative tuple is α1Samples within the range that are greater than the sine-tuple cosine similarity will actually participate in the training.
The model is trained based on the improved triple loss function, and the model is subjected to back propagation optimization training through the joint constraint of the inter-class loss and the intra-class loss, so that the similar samples are as close as possible in the feature space, and the heterogeneous samples are as far away from the feature space, the recognition ability of the model is improved, and the reliability of the face authentication is improved.
And S308, inputting the verification set data into the convolutional neural network, and obtaining the trained convolutional neural network for face authentication when the training end condition is reached.
Specifically, 90% of the data from the pool of witness image data was taken as the training set, and the remaining 10% was taken as the validation set. And calculating an improved triplet loss value based on the formula, and feeding the triplet loss value back to the convolutional neural network for iterative optimization. And simultaneously observing the performance of the model in the verification set, and when the verification performance is not increased any more, the model reaches a convergence state and the training stage is terminated.
According to the face authentication method, on one hand, the constraint on the intra-class sample distance is added in the loss function of the original triplet loss, so that the intra-class difference is reduced while the inter-class difference is increased, and the generalization capability of the model is improved; on the other hand, the original metric mode of the triplet loss is changed from the Euclidean distance to the cosine distance, the consistency of the metrics of training and deployment is kept, and the continuity of feature learning is improved.
In another embodiment, the step of training the convolutional neural network further comprises: initializing by using basic model parameters trained on the basis of massive open source face data, and adding a normalization layer and an improved triple loss function layer behind a feature output layer to obtain a convolutional neural network to be trained.
Specifically, when the problem of testimony unification is solved by deep learning, the testimony comparison application performance of a conventional deep face recognition model obtained by training based on mass internet face data in a specific scene is greatly reduced, testimony data sources in the specific application scene are limited, and training results are not ideal due to insufficient samples in direct learning, so that a method for effectively carrying out extended training on scene data of a small data set is greatly needed to be developed, the accuracy of the face recognition model in the specific application scene is improved, and market application requirements are met.
The deep learning algorithm usually depends on mass data training, in the application of testimony and testimony combination, the problem of heterogeneous sample comparison of certificate photo and scene photo comparison is solved, and the performance of a conventional deep face recognition model obtained based on mass internet face data training is greatly reduced in testimony and testimony comparison application. However, the source of the testimony data is limited (the testimony data is required to be provided with the identity card image of the same person and the corresponding scene image at the same time), the data volume which can be used for training is small, the training effect is not ideal due to insufficient samples in the direct training, therefore, when the testimony-integrated model training is carried out by deep learning, the idea of transfer learning is often utilized, a basic model with reliable performance on an open source test set is firstly trained on the basis of massive internet face data, then secondary extension training is carried out on the limited testimony data, so that the model can automatically learn the feature representation of a specific mode, and the model performance is improved. This process is illustrated in fig. 6.
In the process of secondary training, the whole network is initialized by using the pre-trained basic model parameters, then an L2 normalization layer and an improved triplet loss layer are added after the feature output layer of the network, and the structural diagram of the convolutional neural network to be trained is shown in fig. 7.
In an embodiment, a flow diagram of a face authentication method is shown in fig. 8, and includes three stages, namely a data acquisition and preprocessing stage, a training stage, and a deployment stage.
And in the data acquisition and preprocessing stage, a card reader module of the people and identity card verification terminal equipment reads a certificate chip photo, a front camera captures a scene photo, and a certificate face image and a scene face image with normalized sizes are obtained after passing through a face detector, a key point detector and a face alignment and shearing module.
And in the training stage, 90% of data is taken from the testimonial image data pool as a training set, and the rest 10% is taken as a verification set. Because the testimonial comparison mainly refers to the comparison between the certificate photo and the scene photo, if the certificate photo is taken as a reference image (anchor) in the triple, the other two images are both the scene photo; otherwise, if the scene photo is taken as a reference picture, the other two pictures are both certificate photos. And adopting an OHEM online triple construction strategy, namely in the process of each iterative optimization of the network, utilizing the current network to perform forward calculation on the candidate triple, screening effective triples meeting the conditions, calculating an improved tripletloss value according to the formula, and feeding the tripletloss value back to the network for iterative optimization. And simultaneously observing the performance of the model in the verification set, and when the verification performance is not increased any more, the model reaches a convergence state and the training stage is terminated.
And a deployment stage, wherein when the trained model is deployed to a human authentication terminal for use, the images acquired by the equipment are subjected to the same preprocessing program as the training stage, then the feature vector of each human face image is obtained through network forward calculation, the similarity of the two images is obtained through calculating the cosine distance, then judgment is carried out according to a preset threshold value, the image larger than the preset threshold value is the same person, and the image larger than the preset threshold value is different persons.
According to the face authentication method, the original triplet loss function only limits the learning relationship of the inter-class distance, the original triplet loss function is improved, the constraint term of the intra-class distance is increased, the intra-class difference can be reduced as far as possible while the inter-class difference of the network is increased in the training process, and therefore the generalization capability of the network is improved, and the scene adaptability of the model is improved. In addition, the cosine distance replaces the Euclidean distance measurement mode in the original triplet loss, the distribution attribute of the face feature space is better met, the consistency of the measurement mode in the training stage and the deployment stage is kept, and the comparison result is more reliable.
In one embodiment, there is provided a face authentication apparatus, as shown in fig. 9, including: an image acquisition module 902, an image pre-processing module 904, a feature acquisition module 906, a calculation module 908, and an authentication module 910.
And an image obtaining module 902, configured to obtain a certificate photo and a scene photo of a person based on the face authentication request.
And the image preprocessing module 904 is configured to perform face detection, key point positioning, and image preprocessing on the scene photo and the certificate photo respectively to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo.
The feature acquisition module 906 is configured to input the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquire a first feature vector corresponding to the scene face image output by the convolutional neural network model and a second feature vector corresponding to the certificate face image; the convolutional neural network model is obtained based on supervised training of a triplet loss function.
A calculating module 908 for calculating cosine distances of the first eigenvector and the second eigenvector.
And the authentication module 910 is configured to compare the cosine distance with a preset threshold, and determine a face authentication result according to the comparison result.
The face authentication device utilizes the pre-trained convolutional neural network to carry out face authentication, the convolutional neural network model is obtained based on the supervised training of the improved triple loss function, the similarity of the scene face image and the certificate face image is obtained by calculating the cosine distance of the first characteristic vector corresponding to the scene face image and the second characteristic vector corresponding to the certificate face image, the cosine distance is the included angle of the space vectors, the difference in direction is reflected rather than the position, the distribution attribute of the face characteristic space is met better, and the reliability of the face authentication is improved.
As shown in fig. 9, in another embodiment, the face authentication apparatus further includes: a sample acquisition module 912, a triplet acquisition module 914, a training module 916, and a validation module 918.
A sample acquiring module 912, configured to acquire a labeled training sample, where the training sample includes one certificate face image and at least one scene face image labeled to each labeled object.
The triple obtaining module 914 is used for training the convolutional neural network model according to the training samples and generating triple elements corresponding to the training samples through the OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample.
Specifically, the triple obtaining module 914 is configured to randomly select an image as a reference sample, select an image that belongs to the same label object and is different from the reference sample in category as a positive sample, and further extract a cosine distance between features according to an OHEM policy by using a currently trained convolutional neural network model, and for each reference sample, select an image that has the smallest distance and is different from the reference sample in category as a negative sample of the reference sample from other face images that do not belong to the same label object.
Specifically, when the certificate photo is taken as a reference sample, the positive sample and the negative sample are both scene photos; when the scene photo is taken as a reference sample, the positive sample and the negative sample are both certificate photos.
The training module 916 is configured to train a convolutional neural network model based on supervision of a triplet loss function according to triplet elements of each training sample, where the triplet loss function optimizes model parameters by a random gradient descent algorithm with cosine distances as a measurement mode.
Specifically, the modified triplet loss function includes a definition of cosine distances for homogeneous samples and a definition of cosine distances for heterogeneous samples.
The modified triplet loss function is:
wherein cos (·) represents the cosine distance, which is calculated by
N is the number of the triples,
a feature vector representing a reference sample,
a feature vector representing a homogeneous positive sample,
feature vectors representing heterogeneous negative examples [ ·]+The meanings of (A) are as follows:
α1is an inter-class spacing parameter, α2Is an intra-class interval parameter.
And the verification module 918 is used for inputting the verification set data into the convolutional neural network model to obtain the trained convolutional neural network model for face authentication when the training end condition is reached.
In another embodiment, the face authentication apparatus further includes a model initialization module 920, configured to initialize the trained basic model parameters based on the massive open-source face data, and add a normalization layer and a triple loss function layer after the feature output layer to obtain a convolutional neural network to be trained. On one hand, the human face authentication device increases the constraint on the intra-class sample distance in the loss function of the original tripletloss, so that the intra-class difference is reduced while the inter-class difference is increased, and the generalization capability of the model is improved; on the other hand, the original metric mode of the triplet loss is changed from the Euclidean distance to the cosine distance, the consistency of the metrics of training and deployment is kept, and the continuity of feature learning is improved.
A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the steps of the face authentication method of the above embodiments are implemented.
A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the face authentication method of each of the above embodiments.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A triple Loss-based face authentication method comprises the following steps:
acquiring a certificate photo and a scene photo of a person based on the face authentication request;
respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo;
inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image and a second feature vector corresponding to the certificate face image, which are output by the convolutional neural network model; wherein the convolutional neural network model is obtained based on supervised training of a triplet loss function; the triplet loss function comprises the definition of cosine distances of homogeneous samples and the definition of cosine distances of heterogeneous samples;
calculating cosine distances of the first feature vector and the second feature vector;
comparing the cosine distance with a preset threshold value, and determining a face authentication result according to a comparison result;
the triplet loss function is:
wherein cos (·) represents the cosine distance, which is calculated by
N is the number of the triples,
a feature vector representing a reference sample,
a feature vector representing a homogeneous positive sample,
feature vectors representing heterogeneous negative examples [ ·]+The meanings of (A) are as follows:
α1is an inter-class spacing parameter, α2Is an intra-class interval parameter; the value range of the interval parameter between the classes is 0-0.2, and the value range of the interval parameter in the classes is 0.8-1.0.
2. The method of claim 1, further comprising:
acquiring a training sample with marks, wherein the training sample comprises a certificate face image and at least one scene face image which are marked to belong to each marked object;
training a convolutional neural network model according to the training samples, and generating triple elements corresponding to the training samples through an OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample;
training the convolutional neural network model based on the supervision of a triple loss function according to the triple elements of each training sample; the triple loss function takes cosine distance as a measurement mode and optimizes model parameters through a random gradient descent algorithm;
and inputting the verification set data into the convolutional neural network model, and obtaining the trained convolutional neural network model for face authentication when the training end condition is reached.
3. The method of claim 2, wherein the step of training a convolutional neural network model based on the training samples, and generating a triplet of elements corresponding to each training sample by OHEM comprises:
randomly selecting an image as a reference sample, and selecting an image which belongs to the same label object and is different from the reference sample as a positive sample;
according to an OHEM strategy, extracting cosine distances among features by using a convolutional neural network model trained currently, and selecting an image which has the minimum distance and belongs to a different class from the reference sample as a negative sample of the reference sample from other images which do not belong to the same label object for each reference sample.
4. The method of claim 2, wherein the homogeneous samples comprise the reference sample and the positive samples, and wherein the heterogeneous samples comprise the reference sample and the negative samples.
5. The method of claim 2, further comprising: initializing by using basic model parameters trained on the basis of massive open source face data, and adding a normalization layer and an improved triple loss function layer behind a feature output layer to obtain a convolutional neural network to be trained.
6. The method of claim 2, further comprising: initializing by using basic model parameters trained on the basis of mass open source face data, and adding a normalization layer and a triple loss function layer behind a feature output layer to obtain a convolutional neural network model to be trained.
7. A triple Loss-based face authentication device comprises: the system comprises an image acquisition module, an image preprocessing module, a feature acquisition module, a calculation module and an authentication module;
the image acquisition module is used for acquiring a certificate photo and a scene photo of a person based on the face authentication request;
the image preprocessing module is used for respectively carrying out face detection, key point positioning and image preprocessing on the scene photo and the certificate photo to obtain a scene face image corresponding to the scene photo and a certificate face image corresponding to the certificate photo;
the feature acquisition module is used for inputting the scene face image and the certificate face image into a pre-trained convolutional neural network model for face authentication, and acquiring a first feature vector corresponding to the scene face image and a second feature vector corresponding to the certificate face image, which are output by the convolutional neural network model; wherein the convolutional neural network model is obtained based on supervised training of a triplet loss function; the triplet loss function comprises the definition of cosine distances of homogeneous samples and the definition of cosine distances of heterogeneous samples;
the calculating module is used for calculating cosine distances of the first feature vector and the second feature vector;
the authentication module is used for comparing the cosine distance with a preset threshold value and determining a face authentication result according to the comparison result;
the triplet loss function is:
wherein cos (·) represents the cosine distance, which is calculated by
N is the number of the triples,
a feature vector representing a reference sample,
a feature vector representing a homogeneous positive sample,
feature vectors representing heterogeneous negative examples [ ·]+The meanings of (A) are as follows:
α1is an inter-class spacing parameter, α2Is an intra-class interval parameter; the value range of the interval parameter between the classes is 0-0.2, and the value range of the interval parameter in the classes is 0.8-1.0.
8. The apparatus of claim 7, further comprising: the system comprises a sample acquisition module, a triple acquisition module, a training module and a verification module;
the sample acquisition module is used for acquiring a training sample with marks, wherein the training sample comprises a certificate face image and at least one scene face image which are marked to belong to each marked object;
the triple acquisition module is used for training a convolutional neural network model according to the training samples and generating triple elements corresponding to the training samples through an OHEM; the triplet elements include a reference sample, a positive sample, and a negative sample;
the training module is used for training the convolutional neural network model based on the supervision of a triplet loss function according to the triplet elements of each training sample; the triple loss function takes cosine distance as a measurement mode and optimizes model parameters through a random gradient descent algorithm;
and the verification module is used for inputting verification set data into the convolutional neural network model to obtain the trained convolutional neural network model for face authentication when a training ending condition is reached.
9. The device of claim 7, further comprising a model initialization module, configured to initialize the model with basic model parameters trained based on massive open-source face data, and add a normalization layer and a triplet loss function layer after a feature output layer to obtain a convolutional neural network to be trained.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor when executing the computer program implements the steps of the triple Loss based face authentication method according to any one of claims 1 to 6.
11. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the triple Loss based face authentication method according to any one of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711436879.4A CN108009528B (en) | 2017-12-26 | 2017-12-26 | Triple Loss-based face authentication method and device, computer equipment and storage medium |
| PCT/CN2018/109169 WO2019128367A1 (en) | 2017-12-26 | 2018-09-30 | Face verification method and apparatus based on triplet loss, and computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711436879.4A CN108009528B (en) | 2017-12-26 | 2017-12-26 | Triple Loss-based face authentication method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108009528A CN108009528A (en) | 2018-05-08 |
| CN108009528B true CN108009528B (en) | 2020-04-07 |
Family
ID=62061566
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711436879.4A Active CN108009528B (en) | 2017-12-26 | 2017-12-26 | Triple Loss-based face authentication method and device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108009528B (en) |
| WO (1) | WO2019128367A1 (en) |
Families Citing this family (122)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108009528B (en) * | 2017-12-26 | 2020-04-07 | 广州广电运通金融电子股份有限公司 | Triple Loss-based face authentication method and device, computer equipment and storage medium |
| CN108922542B (en) * | 2018-06-01 | 2023-04-28 | 平安科技（深圳）有限公司 | Sample triplet acquisition method and device, computer equipment and storage medium |
| CN108921033A (en) * | 2018-06-04 | 2018-11-30 | 北京京东金融科技控股有限公司 | Face picture comparison method, device, medium and electronic equipment |
| CN110598840B (en) * | 2018-06-13 | 2023-04-18 | 富士通株式会社 | Knowledge migration method, information processing apparatus, and storage medium |
| CN109145704B (en) * | 2018-06-14 | 2022-02-22 | 西安电子科技大学 | Face portrait recognition method based on face attributes |
| CN108921952B (en) * | 2018-06-15 | 2022-09-06 | 深圳大学 | Object functionality prediction method, device, computer equipment and storage medium |
| CN110738071A (en) * | 2018-07-18 | 2020-01-31 | 浙江中正智能科技有限公司 | face algorithm model training method based on deep learning and transfer learning |
| CN109145956B (en) * | 2018-07-26 | 2021-12-14 | 上海慧子视听科技有限公司 | Scoring method, scoring device, computer equipment and storage medium |
| CN108960342B (en) * | 2018-08-01 | 2021-09-14 | 中国计量大学 | Image similarity calculation method based on improved Soft-Max loss function |
| CN108960209B (en) * | 2018-08-09 | 2023-07-21 | 腾讯科技（深圳）有限公司 | Identity recognition method, identity recognition device and computer readable storage medium |
| CN109165589B (en) * | 2018-08-14 | 2021-02-23 | 北京颂泽科技有限公司 | Vehicle weight recognition method and device based on deep learning |
| CN109271877A (en) * | 2018-08-24 | 2019-01-25 | 北京智芯原动科技有限公司 | A kind of human figure identification method and device |
| CN109145991B (en) * | 2018-08-24 | 2020-07-31 | 北京地平线机器人技术研发有限公司 | Image group generation method, image group generation device and electronic equipment |
| CN110874602A (en) * | 2018-08-30 | 2020-03-10 | 北京嘀嘀无限科技发展有限公司 | Image identification method and device |
| CN109344740A (en) * | 2018-09-12 | 2019-02-15 | 上海了物网络科技有限公司 | Face identification system, method and computer readable storage medium |
| CN109359541A (en) * | 2018-09-17 | 2019-02-19 | 南京邮电大学 | A kind of sketch face identification method based on depth migration study |
| CN109214361A (en) * | 2018-10-18 | 2019-01-15 | 康明飞（北京）科技有限公司 | A kind of face identification method and device and ticket verification method and device |
| CN109543524A (en) * | 2018-10-18 | 2019-03-29 | 同盾控股有限公司 | A kind of image-recognizing method, device |
| CN109492583A (en) * | 2018-11-09 | 2019-03-19 | 安徽大学 | A kind of recognition methods again of the vehicle based on deep learning |
| CN109685106A (en) * | 2018-11-19 | 2019-04-26 | 深圳博为教育科技有限公司 | A kind of image-recognizing method, face Work attendance method, device and system |
| CN109522850B (en) * | 2018-11-22 | 2023-03-10 | 中山大学 | Action similarity evaluation method based on small sample learning |
| CN109685121B (en) * | 2018-12-11 | 2023-07-18 | 中国科学院苏州纳米技术与纳米仿生研究所 | Training method of image retrieval model, image retrieval method and computer equipment |
| CN111325223B (en) * | 2018-12-13 | 2023-10-24 | 中国电信股份有限公司 | Training method and device for deep learning model and computer readable storage medium |
| CN109711443A (en) * | 2018-12-14 | 2019-05-03 | 平安城市建设科技(深圳)有限公司 | Floor plan recognition methods, device, equipment and storage medium neural network based |
| CN109815801A (en) * | 2018-12-18 | 2019-05-28 | 北京英索科技发展有限公司 | Face identification method and device based on deep learning |
| CN109657792A (en) * | 2018-12-19 | 2019-04-19 | 北京世纪好未来教育科技有限公司 | Construct the method, apparatus and computer-readable medium of neural network |
| CN109711358B (en) * | 2018-12-28 | 2020-09-04 | 北京远鉴信息技术有限公司 | Neural network training method, face recognition system and storage medium |
| CN109871762B (en) * | 2019-01-16 | 2023-08-08 | 平安科技（深圳）有限公司 | Face recognition model evaluation method and device |
| CN111461152B (en) * | 2019-01-21 | 2024-04-05 | 同方威视技术股份有限公司 | Cargo detection method and device, electronic equipment and computer readable medium |
| CN109886186A (en) * | 2019-02-18 | 2019-06-14 | 上海骏聿数码科技有限公司 | A kind of face identification method and device |
| US10885385B2 (en) * | 2019-03-19 | 2021-01-05 | Sap Se | Image search and training system |
| CN109948568A (en) * | 2019-03-26 | 2019-06-28 | 东华大学 | Embedded human face identifying system based on ARM microprocessor and deep learning |
| CN110147833B (en) * | 2019-05-09 | 2021-10-12 | 北京迈格威科技有限公司 | Portrait processing method, device, system and readable storage medium |
| CN110213660B (en) * | 2019-05-27 | 2021-08-20 | 广州荔支网络技术有限公司 | Program distribution method, system, computer device and storage medium |
| CN110516533B (en) * | 2019-07-11 | 2023-06-02 | 同济大学 | Pedestrian re-identification method based on depth measurement |
| CN110414431B (en) * | 2019-07-29 | 2022-12-27 | 广州像素数据技术股份有限公司 | Face recognition method and system based on elastic context relation loss function |
| CN110647880A (en) * | 2019-08-12 | 2020-01-03 | 深圳市华付信息技术有限公司 | Mobile terminal identity card image shielding judgment method |
| CN110458233B (en) * | 2019-08-13 | 2024-02-13 | 腾讯云计算（北京）有限责任公司 | Mixed granularity object recognition model training and recognition method, device and storage medium |
| CN110674688B (en) * | 2019-08-19 | 2023-10-31 | 深圳力维智联技术有限公司 | Face recognition model acquisition method, system and medium for video monitoring scene |
| CN110705357A (en) * | 2019-09-02 | 2020-01-17 | 深圳中兴网信科技有限公司 | Face recognition method and face recognition device |
| CN110555478B (en) * | 2019-09-05 | 2023-02-03 | 东北大学 | Fan multi-fault diagnosis method based on depth measurement network of difficult sample mining |
| CN111008550A (en) * | 2019-09-06 | 2020-04-14 | 上海芯灵科技有限公司 | Identification method for finger vein authentication identity based on Multiple loss function |
| CN110674637B (en) * | 2019-09-06 | 2023-07-11 | 腾讯科技（深圳）有限公司 | Character relationship recognition model training method, device, equipment and medium |
| CN110705393B (en) * | 2019-09-17 | 2023-02-03 | 中国计量大学 | Method for improving face recognition performance of community population |
| CN110647938B (en) * | 2019-09-24 | 2022-07-15 | 北京市商汤科技开发有限公司 | Image processing method and related device |
| CN112580406A (en) * | 2019-09-30 | 2021-03-30 | 北京中关村科金技术有限公司 | Face comparison method and device and storage medium |
| CN112733574B (en) * | 2019-10-14 | 2023-04-07 | 中移(苏州)软件技术有限公司 | Face recognition method and device and computer readable storage medium |
| CN111104846B (en) * | 2019-10-16 | 2022-08-30 | 平安科技（深圳）有限公司 | Data detection method and device, computer equipment and storage medium |
| CN110796057A (en) * | 2019-10-22 | 2020-02-14 | 上海交通大学 | Pedestrian re-identification method and device and computer equipment |
| CN110765933A (en) * | 2019-10-22 | 2020-02-07 | 山西省信息产业技术研究院有限公司 | Dynamic portrait sensing comparison method applied to driver identity authentication system |
| CN110852367B (en) * | 2019-11-05 | 2023-10-31 | 上海联影智能医疗科技有限公司 | Image classification method, computer device, and storage medium |
| CN110956098B (en) * | 2019-11-13 | 2023-05-12 | 深圳数联天下智能科技有限公司 | Image processing method and related equipment |
| CN111126360B (en) * | 2019-11-15 | 2023-03-24 | 西安电子科技大学 | Cross-domain pedestrian re-identification method based on unsupervised combined multi-loss model |
| CN111079566B (en) * | 2019-11-28 | 2023-05-02 | 深圳市信义科技有限公司 | Large-scale face recognition model optimization system |
| CN110929099B (en) * | 2019-11-28 | 2023-07-21 | 杭州小影创新科技股份有限公司 | Short video frame semantic extraction method and system based on multi-task learning |
| CN111222411B (en) * | 2019-11-28 | 2023-09-01 | 中国船舶重工集团公司第七一三研究所 | Laser emission safety rapid alarm method and device |
| CN111144240B (en) * | 2019-12-12 | 2023-02-07 | 深圳数联天下智能科技有限公司 | Image processing method and related equipment |
| CN111062430B (en) * | 2019-12-12 | 2023-05-09 | 易诚高科（大连）科技有限公司 | Pedestrian re-identification evaluation method based on probability density function |
| CN111091089B (en) * | 2019-12-12 | 2022-07-29 | 新华三大数据技术有限公司 | Face image processing method and device, electronic equipment and storage medium |
| CN111062338B (en) * | 2019-12-19 | 2023-11-17 | 厦门商集网络科技有限责任公司 | License and portrait consistency comparison method and system |
| CN111126240B (en) * | 2019-12-19 | 2023-04-07 | 西安工程大学 | Three-channel feature fusion face recognition method |
| CN111191563A (en) * | 2019-12-26 | 2020-05-22 | 三盟科技股份有限公司 | Face recognition method and system based on data sample and test data set training |
| CN111178249A (en) * | 2019-12-27 | 2020-05-19 | 杭州艾芯智能科技有限公司 | Face comparison method and device, computer equipment and storage medium |
| CN111241925B (en) * | 2019-12-30 | 2023-08-18 | 新大陆数字技术股份有限公司 | Face quality assessment method, system, electronic equipment and readable storage medium |
| CN111209839B (en) * | 2019-12-31 | 2023-05-23 | 上海涛润医疗科技有限公司 | Face recognition method |
| CN111198964B (en) * | 2020-01-10 | 2023-04-25 | 中国科学院自动化研究所 | Image retrieval method and system |
| CN111274946B (en) * | 2020-01-19 | 2023-05-05 | 杭州涂鸦信息技术有限公司 | Face recognition method, system and equipment |
| CN111368644B (en) * | 2020-02-14 | 2024-01-05 | 深圳市商汤科技有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN113362096A (en) * | 2020-03-04 | 2021-09-07 | 驰众信息技术（上海）有限公司 | Frame advertisement image matching method based on deep learning |
| CN111368766B (en) * | 2020-03-09 | 2023-08-18 | 云南安华防灾减灾科技有限责任公司 | Deep learning-based cow face detection and recognition method |
| CN111539247B (en) * | 2020-03-10 | 2023-02-10 | 西安电子科技大学 | Hyper-spectrum face recognition method and device, electronic equipment and storage medium thereof |
| CN111401257B (en) * | 2020-03-17 | 2022-10-04 | 天津理工大学 | Face recognition method based on cosine loss under non-constraint condition |
| CN111429414B (en) * | 2020-03-18 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Artificial intelligence-based focus image sample determination method and related device |
| CN111414862B (en) * | 2020-03-22 | 2023-03-24 | 西安电子科技大学 | Expression recognition method based on neural network fusion key point angle change |
| CN113538075A (en) * | 2020-04-14 | 2021-10-22 | 阿里巴巴集团控股有限公司 | Data processing method, model training method, device and equipment |
| CN111553399A (en) * | 2020-04-21 | 2020-08-18 | 佳都新太科技股份有限公司 | Feature model training method, device, equipment and storage medium |
| CN111507289A (en) * | 2020-04-22 | 2020-08-07 | 上海眼控科技股份有限公司 | Video matching method, computer device and storage medium |
| CN111582107B (en) * | 2020-04-28 | 2023-09-29 | 浙江大华技术股份有限公司 | Training method and recognition method of target re-recognition model, electronic equipment and device |
| CN111639535B (en) * | 2020-04-29 | 2023-08-22 | 深圳英飞拓智能技术有限公司 | Face recognition method and device based on deep learning |
| CN111709313B (en) * | 2020-05-27 | 2022-07-29 | 杭州电子科技大学 | Pedestrian re-identification method based on local and channel combination characteristics |
| CN111626212B (en) * | 2020-05-27 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Method and device for identifying object in picture, storage medium and electronic device |
| CN111738157B (en) * | 2020-06-23 | 2023-07-21 | 平安科技（深圳）有限公司 | Face action unit data set construction method and device and computer equipment |
| CN112257738A (en) * | 2020-07-31 | 2021-01-22 | 北京京东尚科信息技术有限公司 | Training method and device of machine learning model and classification method and device of image |
| CN111988614B (en) * | 2020-08-14 | 2022-09-13 | 深圳前海微众银行股份有限公司 | Hash coding optimization method and device and readable storage medium |
| CN112069993B (en) * | 2020-09-04 | 2024-02-13 | 西安西图之光智能科技有限公司 | Dense face detection method and system based on five-sense organ mask constraint and storage medium |
| CN112084956A (en) * | 2020-09-11 | 2020-12-15 | 上海交通大学烟台信息技术研究院 | Special face crowd screening system based on small sample learning prototype network |
| CN112052821B (en) * | 2020-09-15 | 2023-07-07 | 浙江智慧视频安防创新中心有限公司 | Fire-fighting channel safety detection method, device, equipment and storage medium |
| CN112287765A (en) * | 2020-09-30 | 2021-01-29 | 新大陆数字技术股份有限公司 | Face living body detection method, device and equipment and readable storage medium |
| CN112307968B (en) * | 2020-10-30 | 2022-11-08 | 天地伟业技术有限公司 | Face recognition feature compression method |
| CN112328786A (en) * | 2020-11-03 | 2021-02-05 | 平安科技（深圳）有限公司 | Text classification method and device based on BERT, computer equipment and storage medium |
| GB2600922B (en) * | 2020-11-05 | 2024-04-10 | Thales Holdings Uk Plc | One shot learning for identifying data items similar to a query data item |
| CN112836566A (en) * | 2020-12-01 | 2021-05-25 | 北京智云视图科技有限公司 | Multitask neural network face key point detection method for edge equipment |
| CN112492383A (en) * | 2020-12-03 | 2021-03-12 | 珠海格力电器股份有限公司 | Video frame generation method and device, storage medium and electronic equipment |
| CN112836719B (en) * | 2020-12-11 | 2024-01-05 | 南京富岛信息工程有限公司 | Indicator diagram similarity detection method integrating two classifications and triplets |
| CN112580011B (en) * | 2020-12-25 | 2022-05-24 | 华南理工大学 | Portrait encryption and decryption system facing biological feature privacy protection |
| CN112861626B (en) * | 2021-01-04 | 2024-03-08 | 西北工业大学 | Fine granularity expression classification method based on small sample learning |
| CN113762019B (en) * | 2021-01-22 | 2024-04-09 | 北京沃东天骏信息技术有限公司 | Training method of feature extraction network, face recognition method and device |
| CN112836629B (en) * | 2021-02-01 | 2024-03-08 | 清华大学深圳国际研究生院 | Image classification method |
| CN112966724B (en) * | 2021-02-07 | 2024-04-09 | 惠州市博实结科技有限公司 | Method and device for classifying image single categories |
| CN112766237A (en) * | 2021-03-12 | 2021-05-07 | 东北林业大学 | Unsupervised pedestrian re-identification method based on cluster feature point clustering |
| CN113065495B (en) * | 2021-04-13 | 2023-07-14 | 深圳技术大学 | Image similarity calculation method, target object re-recognition method and system |
| CN113157956B (en) * | 2021-04-23 | 2022-08-05 | 雅马哈发动机（厦门）信息系统有限公司 | Picture searching method, system, mobile terminal and storage medium |
| CN113344031B (en) * | 2021-05-13 | 2022-12-27 | 清华大学 | Text classification method |
| CN113344875A (en) * | 2021-06-07 | 2021-09-03 | 武汉象点科技有限公司 | Abnormal image detection method based on self-supervision learning |
| CN113269155A (en) * | 2021-06-28 | 2021-08-17 | 苏州市科远软件技术开发有限公司 | End-to-end face recognition method, device, equipment and storage medium |
| CN113486804B (en) * | 2021-07-07 | 2024-02-20 | 科大讯飞股份有限公司 | Object identification method, device, equipment and storage medium |
| CN113435545A (en) * | 2021-08-14 | 2021-09-24 | 北京达佳互联信息技术有限公司 | Training method and device of image processing model |
| CN113642481A (en) * | 2021-08-17 | 2021-11-12 | 百度在线网络技术（北京）有限公司 | Recognition method, training method, device, electronic equipment and storage medium |
| CN113569991A (en) * | 2021-08-26 | 2021-10-29 | 深圳市捷顺科技实业股份有限公司 | Testimony comparison model training method, computer equipment and computer storage medium |
| CN113688793A (en) * | 2021-09-22 | 2021-11-23 | 万章敏 | Training method of face model and face recognition system |
| CN113780461B (en) * | 2021-09-23 | 2022-08-05 | 中国人民解放军国防科技大学 | Robust neural network training method based on feature matching |
| CN113887653B (en) * | 2021-10-20 | 2024-02-06 | 西安交通大学 | Positioning method and system for tight coupling weak supervision learning based on ternary network |
| CN114049479A (en) * | 2021-11-10 | 2022-02-15 | 苏州魔视智能科技有限公司 | Self-supervision fisheye camera image feature point extraction method and device and storage medium |
| CN116188256A (en) * | 2021-11-25 | 2023-05-30 | 北京字跳网络技术有限公司 | Super-resolution image processing method, device, equipment and medium |
| CN114387457A (en) * | 2021-12-27 | 2022-04-22 | 腾晖科技建筑智能(深圳)有限公司 | Face intra-class interval optimization method based on parameter adjustment |
| CN114882558B (en) * | 2022-04-29 | 2024-02-23 | 陕西师范大学 | Learning scene real-time identity authentication method based on face recognition technology |
| CN114663965B (en) * | 2022-05-24 | 2022-10-21 | 之江实验室 | Testimony comparison method and device based on two-stage alternative learning |
| CN114926445B (en) * | 2022-05-31 | 2024-03-26 | 哈尔滨工业大学 | Small sample crop disease image identification method and system based on twin network |
| CN114817888A (en) * | 2022-06-27 | 2022-07-29 | 中国信息通信研究院 | Certificate registering and issuing method, device and storage medium |
| CN116127298B (en) * | 2023-02-22 | 2024-03-19 | 北京邮电大学 | Small sample radio frequency fingerprint identification method based on triplet loss |
| CN116206355A (en) * | 2023-04-25 | 2023-06-02 | 鹏城实验室 | Face recognition model training, image registration and face recognition method and device |
| CN116977461B (en) * | 2023-06-30 | 2024-03-08 | 北京开普云信息科技有限公司 | Portrait generation method, device, storage medium and equipment for specific scene |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599827A (en) * | 2016-12-09 | 2017-04-26 | 浙江工商大学 | Small target rapid detection method based on deep convolution neural network |
| CN107194341A (en) * | 2017-05-16 | 2017-09-22 | 西安电子科技大学 | The many convolution neural network fusion face identification methods of Maxout and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9129216B1 (en) * | 2013-07-15 | 2015-09-08 | Xdroid Kft. | System, method and apparatus for computer aided association of relevant images with text |
| CN107423690B (en) * | 2017-06-26 | 2020-11-13 | 广东工业大学 | Face recognition method and device |
| CN108009528B (en) * | 2017-12-26 | 2020-04-07 | 广州广电运通金融电子股份有限公司 | Triple Loss-based face authentication method and device, computer equipment and storage medium |
-
2017
- 2017-12-26 CN CN201711436879.4A patent/CN108009528B/en active Active
-
2018
- 2018-09-30 WO PCT/CN2018/109169 patent/WO2019128367A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599827A (en) * | 2016-12-09 | 2017-04-26 | 浙江工商大学 | Small target rapid detection method based on deep convolution neural network |
| CN107194341A (en) * | 2017-05-16 | 2017-09-22 | 西安电子科技大学 | The many convolution neural network fusion face identification methods of Maxout and system |
Non-Patent Citations (1)
| Title |
|---|
| OpenFace:A general-purpose face recognition library with mobile applications;BRANDON，Amos等;《CMU School of Computer Science，Tech. Rep.》;20160630;第16-17页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108009528A (en) | 2018-05-08 |
| WO2019128367A1 (en) | 2019-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108009528B (en) | Triple Loss-based face authentication method and device, computer equipment and storage medium | |
| US11288504B2 (en) | Iris liveness detection for mobile devices | |
| US10650261B2 (en) | System and method for identifying re-photographed images | |
| CN106203242B (en) | Similar image identification method and equipment | |
| US7492943B2 (en) | Open set recognition using transduction | |
| US20170262472A1 (en) | Systems and methods for recognition of faces e.g. from mobile-device-generated images of faces | |
| CN105740779B (en) | Method and device for detecting living human face | |
| WO2022206319A1 (en) | Image processing method and apparatus, and device, storage medium and computer program product | |
| Zhang et al. | A novel serial multimodal biometrics framework based on semisupervised learning techniques | |
| CN109376604B (en) | Age identification method and device based on human body posture | |
| Smith-Creasey et al. | Continuous face authentication scheme for mobile devices with tracking and liveness detection | |
| Peter et al. | Improving ATM security via face recognition | |
| CN113033519B (en) | Living body detection method, estimation network processing method, device and computer equipment | |
| Riazi et al. | SynFi: Automatic synthetic fingerprint generation | |
| CN111582155A (en) | Living body detection method, living body detection device, computer equipment and storage medium | |
| Abate et al. | Smartphone enabled person authentication based on ear biometrics and arm gesture | |
| Goud et al. | Smart attendance notification system using SMTP with face recognition | |
| KR102215535B1 (en) | Partial face image based identity authentication method using neural network and system for the method | |
| Valehi et al. | A graph matching algorithm for user authentication in data networks using image-based physical unclonable functions | |
| CN111582027A (en) | Identity authentication method and device, computer equipment and storage medium | |
| CN116612538A (en) | Online confirmation method of electronic contract content | |
| Fegade et al. | Residential security system based on facial recognition | |
| Andrews et al. | Multiple-identity image attacks against face-based identity verification | |
| KR101441106B1 (en) | Method for extracting and verifying face and apparatus thereof | |
| Shukla et al. | A Hybrid Model of Multimodal Biometrics System using Fingerprint and Face as Traits |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Face authentication method, device, computer device and storage medium based on triplet loss Effective date of registration: 20210621 Granted publication date: 20200407 Pledgee: Bank of China Limited by Share Ltd. Guangzhou Tianhe branch Pledgor: GRG Banking Equipment Co.,Ltd. Registration number: Y2021980004993 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20230302 Granted publication date: 20200407 Pledgee: Bank of China Limited by Share Ltd. Guangzhou Tianhe branch Pledgor: GRG BANKING EQUIPMENT Co.,Ltd. Registration number: Y2021980004993 |