CN114708625A - Face recognition method and device - Google Patents

Abstract

The present disclosure relates to the technical field of artificial intelligence, and provides a face recognition method and device. The method includes: extracting the first feature, second feature and third feature of each sample in the input image through a neural network model, and performing feature fusion processing on the above three features to obtain the fusion feature of each sample, and then calculating Obtain the first quality score of each sample and the second quality score of the class center corresponding to each class; calculate the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs through the neural network model; according to The first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs are trained through the loss function Neural network model; use the trained neural network model for face recognition.

Description

Face recognition method and device

technical field

The present disclosure relates to the technical field of artificial intelligence, and in particular, to a face recognition method and device.

Background technique

The general practice of existing face recognition algorithms is to treat all training samples equally. In the training of face recognition models, the training problems caused by different sample quality and different quality class centers are not considered. Different sample quality and different quality class centers actually involve image feature extraction and image feature quality estimation. In the existing face recognition technology, the feature extraction task of the face image and the quality estimation task about the feature are independent of each other, and the feature extraction task and the quality estimation task are not related and cannot be related, which leads to In the training of face recognition model, image feature extraction and image feature quality estimation cannot promote each other.

In the process of realizing the concept of the present disclosure, the inventor found that there are at least the following technical problems in the related art: in the training of the face recognition model, the problem that image feature extraction and image feature quality estimation cannot promote each other.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present disclosure provide a face recognition method and device to solve the problem in the prior art that image feature extraction and image feature quality estimation cannot promote each other in the training of a face recognition model.

A first aspect of the embodiments of the present disclosure provides a face recognition method, including: acquiring an input image, and extracting a first feature, a second feature, and a third feature of each sample in the input image by using a neural network model, and using a neural network model The neural network model calculates the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs, wherein the input image has multiple classes, each class corresponds to a class center, and each class has multiple sample; perform feature fusion processing on the first feature, second feature and third feature of each sample to obtain the fusion feature of each sample; perform interactive calculation on at least one of the following features of each sample: the first feature, the third feature Three features and fusion features to obtain the first quality score of each sample; according to the quality scores of multiple samples in each class in the input image, calculate the second quality score of the class center corresponding to each class; The first quality score of the sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs, train the neural network through the loss function Model; use the trained neural network model for face recognition.

In a second aspect of the embodiments of the present disclosure, there is provided a face recognition device, including: a first computing module, configured to acquire an input image, and extract the first feature and the second feature of each sample in the input image through a neural network model feature and the third feature, the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs is calculated by the neural network model, wherein, the input image has multiple classes, and each class corresponds to a class center, There are multiple samples in each class; the feature fusion module is used to perform feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain the fusion feature of each sample; the second calculation module, using Perform interactive calculation on at least one of the following features of each sample: the first feature, the third feature and the fusion feature to obtain the first quality score of each sample; the third calculation module is used for each sample in the input image. The quality scores of multiple samples in the class are used to calculate the second quality score of the class center corresponding to each class; the model training module is used for the first quality score of each sample and the class center corresponding to the class to which each sample belongs. The second quality score and the cosine value of the deviation angle of the class center corresponding to each sample and the class to which each sample belongs, the neural network model is trained through the loss function; the face recognition module is used to use the trained neural network model for face recognition.

In a third aspect of the embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when the processor executes the computer program.

A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above method are implemented.

Compared with the prior art, the embodiment of the present disclosure has the following beneficial effects: because the embodiment of the present disclosure extracts the first feature, the second feature and the third feature of each sample in the input image through the neural network model, and compares the above three features The features are subjected to feature fusion processing to obtain the fusion features of each sample, and then the first quality score of each sample and the second quality score of the class center corresponding to each class are obtained by calculation; The cosine value of the deviation angle of the class center corresponding to the class to which each sample belongs; according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the The cosine value of the deviation angle of the class center corresponding to the class of , trains the neural network model through the loss function; and then the trained neural network model can be used for face recognition. Therefore, the above technical means can be used to solve the problem in the prior art. In the training of face recognition model, image feature extraction and image feature quality estimation cannot promote each other, thereby improving the accuracy of face recognition model for face recognition.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only for the present disclosure. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure;

2 is a schematic flowchart of a face recognition method provided by an embodiment of the present disclosure;

3 is a schematic flowchart of a method for performing face recognition using a trained neural network model provided by an embodiment of the present disclosure;

4 is a schematic structural diagram of a face recognition device provided by an embodiment of the present disclosure;

5 is a schematic structural diagram of a face recognition module provided by an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In the following description, specific details, such as specific system structures, techniques, etc., are set forth for the purpose of explanation rather than limitation, in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure with unnecessary detail.

A method and apparatus for face recognition according to embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include terminal devices 1, 2 and 3, server 4 and network 5.

Terminal devices 1, 2 and 3 may be hardware or software. When the terminal devices 1, 2 and 3 are hardware, they can be various electronic devices having a display screen and supporting communication with the server 4, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.; When the terminal devices 1, 2 and 3 are software, they can be installed in the electronic device as above. The terminal devices 1, 2, and 3 may be implemented as multiple software or software modules, or may be implemented as a single software or software module, which is not limited in this embodiment of the present disclosure. Further, various applications may be installed on the terminal devices 1, 2 and 3, such as data processing applications, instant communication tools, social platform software, search applications, shopping applications and the like.

The server 4 can be a server that provides various services, for example, a background server that receives requests sent by the terminal device that establishes a communication connection with it, and the background server can receive and analyze the requests sent by the terminal device. result. The server 4 may be one server, or a server cluster composed of several servers, or may also be a cloud computing service center, which is not limited in this embodiment of the present disclosure.

It should be noted that the server 4 may be hardware or software. When the server 4 is hardware, it may be various electronic devices that provide various services to the terminal devices 1, 2 and 3. When the server 4 is software, it can be multiple software or software modules that provide various services for the terminal devices 1, 2 and 3, or can be a single software or software that provides various services to the terminal devices 1, 2 and 3 module, which is not limited in this embodiment of the present disclosure.

The network 5 can be a wired network connected by coaxial cables, twisted pairs and optical fibers, or a wireless network that can realize interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication, NFC), infrared (Infrared), etc., which are not limited in this embodiment of the present disclosure.

The user can establish a communication connection with the server 4 via the network 5 through the terminal devices 1, 2 and 3 to receive or send information and the like. It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual requirements of the application scenarios, which are not limited in this embodiment of the present disclosure.

FIG. 2 is a schematic flowchart of a face recognition method provided by an embodiment of the present disclosure. The face recognition method of FIG. 2 may be executed by the terminal device or the server of FIG. 1 . As shown in Figure 2, the face recognition method includes:

S201: Obtain an input image, extract the first feature, the second feature and the third feature of each sample in the input image through a neural network model, and use the neural network model to calculate the class center corresponding to each sample and the class to which each sample belongs The cosine of the deviation angle, where the input image has multiple classes, each class corresponds to a class center, and there are multiple samples in each class;

S202, performing feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain the fusion feature of each sample;

S203, do interactive calculation to at least one of the following features of each sample: the first feature, the third feature and the fusion feature, to obtain the first quality score of each sample;

S204, according to the quality points of multiple samples in each class in the input image, calculate the second quality points of the class center corresponding to each class;

S205, according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs, pass The loss function trains the neural network model;

S206, using the trained neural network model to perform face recognition.

It should be noted that the input image obtained in the embodiment of the present disclosure is used for training the neural network model, and the input image includes multiple classes, each class corresponds to a positive class center and\or negative class center, and each class has multiple classes. samples. In the field of face recognition, a class can be a person, and a sample can be a picture of a person. The class center corresponding to a class can be understood as the average value of the features of all pictures of a person, the positive class center corresponding to a class can be understood as the average value of the features of a person's positive sample pictures, and the negative class center corresponding to a class can be It is understood as the average value of the features of a person's negative sample pictures. The samples whose features are greater than the preset threshold for face recognition are positive samples, and the samples whose features are less than the preset threshold for face recognition are negative samples.

According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure extracts the first feature, the second feature and the third feature of each sample in the input image through the neural network model, and performs feature fusion processing on the above three features, Obtain the fusion feature of each sample, and then obtain the first quality score of each sample and the second quality score of the class center corresponding to each class through calculation; calculate the corresponding class of each sample and each sample through the neural network model The cosine value of the deviation angle of the class center of the The cosine value of the deviation angle is used to train the neural network model through the loss function; then the trained neural network model can be used for face recognition. Therefore, the above technical means can solve the problem of the existing technology in the face recognition model. During training, image feature extraction and image feature quality estimation cannot promote each other, thereby improving the face recognition accuracy of the face recognition model.

In step S202, feature fusion processing is performed on the first feature, the second feature and the third feature of each sample to obtain the fused feature of each sample, including: inputting the input image into the neural network model, respectively through the neural network model The second stage, the third stage and the fourth stage output the first feature, the second feature and the third feature of each sample in the input image, wherein the neural network model has four stages; input the first feature of each sample The first preset convolution layer outputs the fourth feature of each sample; the second feature of each sample is input into the second preset convolution layer, and the fifth feature of each sample is output; the third feature of each sample is input. The feature is input into the third preset convolution layer, and the sixth feature of each sample is output; the fourth feature, the fifth feature and the sixth feature of each sample are subjected to feature splicing processing to obtain the splicing feature of each sample, wherein, The feature fusion processing includes feature splicing processing; the splicing features of each sample are input into the fourth preset convolution layer, and the fusion features of each sample are output.

The neural network model has four stages, which are well known to those skilled in the art, and will not be repeated here.

Specifically, the first preset convolution layer performs a convolution kernel of the first preset size on the input first feature, the step size of the downsampling is the first preset step size, and the number of channels is the first preset number of volumes. product, the fourth feature whose matrix dimension is the first preset dimension is obtained; for example, the first feature is convolved with a convolution kernel of 3x3, a downsampling step size of 2 and a channel number of 128 to obtain the fourth feature , the dimension of the fourth feature or the matrix corresponding to the fourth feature is (14, 14, 128). The second preset convolution layer performs convolution on the input second feature with the convolution kernel being the second preset size and the number of channels being the second preset number to obtain the fifth feature whose matrix dimension is the second preset dimension For example, the second feature is convolved with a convolution kernel of 3x3 and a channel of 128 to obtain a fifth feature, and the dimension of the fifth feature or the dimension of the matrix corresponding to the fifth feature is (14, 14, 128). The third preset convolution layer performs convolution on the input third feature with the convolution kernel being the third preset size and the number of channels being the third preset number, and then the third preset convolution layer will do the convolution result. The linear interpolation upsampling processing of preset multiples obtains the sixth feature whose matrix dimension is the third preset dimension; for example, the convolution kernel is 3×3 and the convolution channel is 128 for the third feature, and then the convolution is performed. The result is upsampled by 2 times linear interpolation to obtain the sixth feature. The dimension of the sixth feature or the dimension of the matrix corresponding to the sixth feature is (14, 14, 128). The fourth preset convolution layer performs convolution on the input fourth feature whose convolution kernel is the fourth preset size and the number of channels is the fourth preset number, and obtains the fusion feature whose matrix dimension is the fourth preset dimension; For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fourth feature to obtain a convolution calculation result, and the convolution calculation result is used as a fusion feature. It should be noted that the fourth preset convolution layer can further normalize the convolution calculation result, and use the normalized result as a fusion feature.

According to the technical solutions provided by the embodiments of the present disclosure, the first feature, the second feature, and the third feature are processed through different preset convolution layers, respectively, to obtain the fourth feature, the fifth feature, and the sixth feature, and then according to the fourth feature , the fifth feature and the sixth feature to obtain a fusion feature, which improves the similarity between the fusion feature and the first feature, the second feature and the third feature.

In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fusion feature to obtain the first quality score of each sample, including: The feature and the fusion feature are interactively calculated to obtain the shallow quality score of each sample, wherein the first quality score includes the shallow quality score, including: inputting the first feature into the fifth preset convolution layer and outputting the seventh feature ; Perform dimension leveling processing on the first matrix corresponding to the seventh feature and the second matrix corresponding to the fusion feature respectively, and use the first matrix after the dimension leveling process to multiply the second matrix after the dimension leveling process. Transpose to obtain the first product; use the normalized exponential function to calculate the first product to obtain the first calculation result, and use the first calculation result to multiply the second matrix to obtain the second product; multiply the second product by The first parameter matrix is multiplied by the second parameter matrix to obtain the third product; the sigmoid function is used to calculate the third product to obtain the shallow quality score.

The feature of the sample in the embodiment of the present disclosure may be a feature map, because the feature map may correspond to a matrix, therefore, for ease of understanding, each feature in the embodiment of the present disclosure may be directly understood as a matrix.

Specifically, the fifth preset convolution layer performs convolution on the input first feature with a convolution kernel of a fifth preset size and a channel number of a fifth preset number to obtain the seventh feature. For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fifth feature to obtain the seventh feature. Then flatten the dimension of the seventh feature to matrix A of (28x28, 128), and flatten the dimension of the first matrix to matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel The number of ; A multiplied by the transpose of B; Perform the normalized exponential function softmax operation on the product result, that is, the first product; Multiply the operation result by B; then the product result, that is, the external parameter of the second product The matrix W ₁₁ (128, 64) and the matrix W ₁₂ (64, 1) are used to obtain the third product; the sigmoid function is used to calculate the third product to obtain the shallow quality score. It should be noted that, after the third product is calculated using the sigmoid function, the calculation result may be averaged, and the average value may be used as the shallow quality score. The external parameter matrix W ₁₁ is the first parameter matrix, the external parameter matrix W ₁₂ is the second parameter matrix, and the first parameter matrix and the second parameter matrix may be preset.

The above steps can be understood as the following formula:

为浅层质量分，T为转置运算的符号。W₁₁为第一参数矩阵，W₁₂为第二 参数矩阵，mean为求均值函数。

is the shallow quality score, and T is the symbol of the transpose operation. W ₁₁ is the first parameter matrix, W ₁₂ is the second parameter matrix, and mean is the mean value function.

Because the shallow quality score is output by the second stage of the neural network model, the shallow quality score can be understood as the weight value of the influence of the sample on the class center corresponding to the sample class.

In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fusion feature to obtain the first quality score of each sample, including: The feature and the fusion feature are interactively calculated to obtain the middle-level quality score of each sample, wherein the first quality score includes the middle-level quality score, including: inputting the third feature into the fifth preset convolutional layer, and outputting the eighth feature; respectively; Perform dimension leveling processing on the third matrix corresponding to the eighth feature and the second matrix corresponding to the fusion feature, and use the third matrix after dimension leveling processing to multiply the transpose of the second matrix after dimension leveling processing , obtain the fourth product; use the normalized exponential function to calculate the fourth product to obtain the second calculation result, and use the second calculation result to multiply the second matrix to obtain the fifth product; multiply the fifth product by the third The parameter matrix is then multiplied by the fourth parameter matrix to obtain the sixth product; the sigmoid function is used to calculate the sixth product to obtain the mid-level quality score.

For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the third feature to obtain the eighth feature. Then flatten the dimension of the eighth feature to a matrix C of (7x7, 128), and flatten the dimension of the first matrix to a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel the number of ; multiply C by the transpose of B; perform the normalized exponential function softmax operation on the result of the product, which is the fourth product; multiply the result of the operation by B; and then the result of the product, which is the fifth product The external parameter matrix W ₂₁ (128, 64) and the matrix W ₂₂ (64, 1) are used to obtain the sixth product; the sigmoid function is used to calculate the sixth product to obtain the mid-level quality score. It should be noted that, after the sixth product is calculated using the sigmoid function, the calculation result may be averaged, and the average value may be used as the mid-level quality score. The external parameter matrix W ₂₁ is the third parameter matrix, the external parameter matrix W ₂₂ is the fourth parameter matrix, and the third parameter matrix and the fourth parameter matrix may be preset.

The above steps can be understood as the following formula:

为中层质量分，W₂₁为第三参数矩阵，W₂₂为第四参数矩阵。

is the mid-level quality score, W ₂₁ is the third parameter matrix, and W ₂₂ is the fourth parameter matrix.

In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fusion feature to obtain the first quality score of each sample, including: the fusion feature of each sample Do interactive calculation to obtain the high-level quality score of each sample, where the first quality score includes the high-level quality score, including: inputting the fusion feature into the fifth preset convolutional layer, and outputting the ninth feature; corresponding to the ninth feature respectively The fourth matrix and the second matrix corresponding to the fusion feature are dimension-leveled, and the fourth matrix after dimension-leveling is used to multiply the transpose of the second matrix after dimension-leveling to obtain the seventh product. ; use the normalized exponential function to calculate the seventh product to obtain the third calculation result, and use the third calculation result to multiply the second matrix to obtain the eighth product; multiply the eighth product by the fifth parameter matrix and then multiply by The sixth parameter matrix, the ninth product is obtained; the sigmoid function is used to calculate the ninth product to obtain the high-level quality score.

For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fusion feature to obtain the ninth feature. Then flatten the dimension of the ninth feature to matrix D of (14x14, 128), and flatten the dimension of the first matrix to matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel the number of ; multiply D by the transpose of B; perform the normalized exponential function softmax operation on the result of the product, that is, the seventh product; then multiply the result of the operation by B; then the result of the product, which is the eighth product The external parameter matrix W ₃₁ (128, 64) and the matrix W ₃₂ (64, 1) are used to obtain the ninth product; the sigmoid function is used to calculate the ninth product to obtain the high-level quality score. It should be noted that after using the sigmoid function to calculate the ninth product, the calculation result can be averaged, and the average value can be used as the high-level quality score. The external parameter matrix W ₃₁ is the fifth parameter matrix, the external parameter matrix W ₃₂ is the sixth parameter matrix, and the fifth parameter matrix and the sixth parameter matrix may be preset.

The above steps can be understood as the following formula:

为高层质量分，W₃₁为第三参数矩阵，W₃₂为第四参数矩阵。

is the high-level quality score, W ₃₁ is the third parameter matrix, and W ₃₂ is the fourth parameter matrix.

In step S204, calculating the second quality score of the class center corresponding to each class according to the quality scores of the multiple samples in each class in the input image, including calculating the quality score of each class according to the middle-level quality score and the high-level quality score of each sample The first cohort scores corresponding to each sample, wherein the first quality score includes: the shallow layer quality score, the middle layer quality score and the high layer quality score; the second cohort score corresponding to each sample is calculated according to the shallow layer quality score of each sample ; Calculate the second quality score of the class center corresponding to each class according to the first cohort score and the second cohort score corresponding to the multiple samples in each class.

Specifically, the first queue score qi _,1 is calculated according to the following formula:

Calculate the second cohort score qi _,2 according to the following formula:

Calculate the second mass score γ of the class center corresponding to each class according to the following formula:

γAccording to as

i is the serial number of the samples in a class, R is the number of samples in a class, α is the adjustment parameter of the neural network model, which can be set to 0.2, max(R) represents the most samples in all classes The number of samples for the class.

In step S205, according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs value, training the neural network model through the loss function, including: training the neural network through the cross entropy loss function according to the first quality score of each sample and the cosine value of the deviation angle of the class center corresponding to each sample and the class to which each sample belongs A network model, wherein the loss function includes a cross-entropy loss function; wherein, the first quality score includes: shallow layer quality score, middle layer quality score, and high-level quality score; wherein, the class center includes: positive class center and negative class center.

Specifically, the cross _- entropy loss function L1 is:

s is the magnification factor, which can be set to 64, θ _yi represents the angle between the sample and the center of the positive class, θ _j represents the angle between the sample and the center of the negative class, i is the serial number of the positive samples in a class, and j is the number of positive samples in a class. The serial number of the negative samples, m ₀ can take 0.35, m ₁ can take 0.25, N is the total number of all samples, n is the total number of all negative samples.

In step S205, according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs value, training the neural network model through the loss function, including: training the neural network model through the nearest neighbor optimization loss function, wherein the loss function includes the nearest neighbor optimization loss function, including: calculating the inverse of the quality score of the shallow layer and the quality score of the middle layer of each sample. The first sum of the reciprocal and the inverse of the high-level quality score, where the first quality score includes: the shallow-level quality score, the middle-level quality score, and the high-level quality score; according to the second quality score of the class center corresponding to the class to which each sample belongs And the cosine value of the deviation angle of the negative class center corresponding to each sample and the class to which each sample belongs, use the nearest neighbor optimization function to calculate the nearest neighbor optimization result, where the class center includes: positive class center and negative class center; calculate each The second sum of the shallow, middle and high quality scores of the samples, and the second sum corresponding to each sample is multiplied by the nearest neighbor optimization result corresponding to each sample to obtain the tenth product; each sample corresponds to The first sum of , and the tenth product corresponding to each sample are added to obtain the third sum; the neural network model is trained through the third sum corresponding to each sample.

Specifically, the nearest neighbor optimization loss function L2 is _:

t can be set to 0.2, ∑ _top10 cos(θ _j -γt) is the nearest neighbor optimization result calculated by the nearest neighbor optimization function, z is the preset number of optimizations, assuming z is 10, ∑ _top10 cos(θ _j -γt) indicates that in a class , the neural network model is optimized using the top 10 negative samples with the highest similarity to the negative class center corresponding to the class.

In an optional embodiment, when training the neural network model, it is also necessary to update or modify the cosine value cos θ _p of the deviation angle of the positive class center corresponding to each sample and the class to which each sample belongs. Update the cosine value cosθ _p by the following formula:

或

or

或

or

或

or

Like m ₀ , m ₂ and m ₁ are parameters that can be set in advance.

The value of the cosine value cosθ _p is related to the neural network model determining whether the sample is a positive sample or a negative sample. Input a sample into the neural network model, and the neural network model can judge by itself whether it is a positive sample or a negative sample.

3 is a schematic flowchart of a method for performing face recognition using a trained neural network model provided by an embodiment of the present disclosure. The face recognition method of FIG. 3 may be executed by the terminal device or the server of FIG. 1 .

As shown in Figure 3, the face recognition method includes:

S301, after detecting that the target user enters the preset area, obtain the face image of the target user through the image acquisition device, and obtain the face prototype map corresponding to the face image from the face detection database;

S302, extract respectively the tenth feature and the eleventh feature corresponding to the face prototype figure and the face image by the neural network model, and calculate the first score and the second score corresponding to the tenth feature and the eleventh feature respectively;

S303, calculate the Euclidean distance between the tenth feature and the eleventh feature;

S304, calculate the Euclidean transformation distance according to the first score, the second score and the Euclidean distance;

S305, in the case that the Euclidean transformation distance is greater than the preset threshold, confirm that the face recognition is successful.

According to the technical solutions provided by the embodiments of the present disclosure, the tenth feature and the eleventh feature corresponding to the face prototype image and the face image can be extracted respectively through the neural network model, and the tenth feature and the tenth feature can be calculated respectively. The first score and the second score corresponding to a feature, and then calculate the Euclidean transformation distance. When the Euclidean transformation distance is greater than the preset threshold, it is confirmed that the face recognition is successful. Therefore, the above technical means can be used to solve the problem in the prior art. , in face recognition, the threshold of face recognition is a fixed problem, and then a face recognition scheme based on dynamic threshold is provided.

Specifically, the Euclidean transformation distance is:

D'=D+Euclidean Transform(s ₁ ,s ₂ )

f1 and f2 represent the tenth feature and the eleventh feature, respectively, s1 and s2 represent the first score and the second score, respectively, D is the Euclidean distance between f1 and f2, D' is the Euclidean transformation distance, β can be adjusted, generally take 1.2, f() can take the min() function.

It can be seen from the above formula that the Euclidean transform distance changes according to the face image of the target user acquired or detected by the image acquisition device, so the threshold value of the face recognition in the embodiment of the present disclosure is dynamically changed.

All the above-mentioned optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be repeated here.

The following are the apparatus embodiments of the present disclosure, which can be used to execute the method embodiments of the present disclosure. For details not disclosed in the device embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

FIG. 4 is a schematic diagram of a face recognition apparatus provided by an embodiment of the present disclosure. As shown in Figure 4, the face recognition device includes:

The first calculation module 401 is configured to obtain the input image, and extract the first feature, the second feature and the third feature of each sample in the input image through the neural network model, and calculate each sample and each sample through the neural network model. The cosine value of the deviation angle of the class center corresponding to the class to which it belongs, where the input image has multiple classes, each class corresponds to a class center, and there are multiple samples in each class;

The feature fusion module 402 is configured to carry out feature fusion processing to the first feature, the second feature and the third feature of each sample to obtain the fusion feature of each sample;

The second calculation module 403 is configured to perform interactive calculation on at least one of the following features of each sample: the first feature, the third feature and the fusion feature to obtain the first quality score of each sample;

The third calculation module 404 is configured to calculate the second quality score of the class center corresponding to each class according to the quality scores of the multiple samples in each class in the input image;

The model training module 405 is configured according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the deviation between each sample and the class center corresponding to the class to which each sample belongs The cosine value of the angle is used to train the neural network model through the loss function;

The face recognition module 406 is configured to use the trained neural network model to perform face recognition.

It should be noted that the input image obtained in the embodiment of the present disclosure is used for training the neural network model, and the input image includes multiple classes, each class corresponds to a positive class center and\or negative class center, and each class has multiple classes. samples. In the field of face recognition, a class can be a person, and a sample can be a picture of a person. The class center corresponding to a class can be understood as the average value of the features of all pictures of a person, the positive class center corresponding to a class can be understood as the average value of the features of a person's positive sample pictures, and the negative class center corresponding to a class can be It is understood as the average value of the features of a person's negative sample pictures. The samples whose features are greater than the preset threshold for face recognition are positive samples, and the samples whose features are less than the preset threshold for face recognition are negative samples.

According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure extracts the first feature, the second feature and the third feature of each sample in the input image through the neural network model, and performs feature fusion processing on the above three features, Obtain the fusion feature of each sample, and then obtain the first quality score of each sample and the second quality score of the class center corresponding to each class through calculation; calculate the corresponding class of each sample and each sample through the neural network model The cosine value of the deviation angle of the class center of the The cosine value of the deviation angle is used to train the neural network model through the loss function; then the trained neural network model can be used for face recognition. Therefore, the above technical means can solve the problem of the existing technology in the face recognition model. During training, image feature extraction and image feature quality estimation cannot promote each other, thereby improving the face recognition accuracy of the face recognition model.

Optionally, the feature fusion module 402 is also configured to input the input image into the neural network model, and output the first feature and the first feature of each sample in the input image through the second stage, the third stage and the fourth stage of the neural network model respectively. The second feature and the third feature, where the neural network model has four stages; the first feature of each sample is input into the first preset convolutional layer, and the fourth feature of each sample is output; the second feature of each sample is input The feature is input into the second preset convolution layer, and the fifth feature of each sample is output; the third feature of each sample is input into the third preset convolution layer, and the sixth feature of each sample is output; The fourth feature, the fifth feature and the sixth feature are subjected to feature splicing processing to obtain the splicing features of each sample, wherein the feature fusion processing includes feature splicing processing; the splicing features of each sample are input into the fourth preset convolution layer, Output the fused features for each sample.

The neural network model has four stages, which are well known to those skilled in the art, and will not be repeated here.

Specifically, the first preset convolution layer performs a convolution kernel of the first preset size on the input first feature, the step size of the downsampling is the first preset step size, and the number of channels is the first preset number of volumes. product, the fourth feature whose matrix dimension is the first preset dimension is obtained; for example, the first feature is convolved with a convolution kernel of 3x3, a downsampling step size of 2 and a channel number of 128 to obtain the fourth feature , the dimension of the fourth feature or the matrix corresponding to the fourth feature is (14, 14, 128). The second preset convolution layer performs convolution on the input second feature with the convolution kernel being the second preset size and the number of channels being the second preset number to obtain the fifth feature whose matrix dimension is the second preset dimension For example, the second feature is convolved with a convolution kernel of 3x3 and a channel of 128 to obtain a fifth feature, and the dimension of the fifth feature or the dimension of the matrix corresponding to the fifth feature is (14, 14, 128). The third preset convolution layer performs convolution on the input third feature with the convolution kernel being the third preset size and the number of channels being the third preset number, and then the third preset convolution layer will do the convolution result. The linear interpolation upsampling processing of preset multiples obtains the sixth feature whose matrix dimension is the third preset dimension; for example, the convolution kernel is 3×3 and the convolution channel is 128 for the third feature, and then the convolution is performed. The result is upsampled by 2 times linear interpolation to obtain the sixth feature. The dimension of the sixth feature or the dimension of the matrix corresponding to the sixth feature is (14, 14, 128). The fourth preset convolution layer performs convolution on the input fourth feature whose convolution kernel is the fourth preset size and the number of channels is the fourth preset number, and obtains the fusion feature whose matrix dimension is the fourth preset dimension; For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fourth feature to obtain a convolution calculation result, and the convolution calculation result is used as a fusion feature. It should be noted that the fourth preset convolution layer can further normalize the convolution calculation result, and use the normalized result as a fusion feature.

According to the technical solutions provided by the embodiments of the present disclosure, the first feature, the second feature, and the third feature are processed through different preset convolution layers, respectively, to obtain the fourth feature, the fifth feature, and the sixth feature, and then according to the fourth feature , the fifth feature and the sixth feature to obtain a fusion feature, which improves the similarity between the fusion feature and the first feature, the second feature and the third feature.

Optionally, the second calculation module 403 is further configured to perform interactive calculation on the first feature of each sample and the fusion feature, so as to obtain the shallow quality score of each sample, wherein the first quality score includes the shallow quality score. , including: inputting the first feature into the fifth preset convolution layer, and outputting the seventh feature; performing dimension leveling processing on the first matrix corresponding to the seventh feature and the second matrix corresponding to the fusion feature respectively, and using the The first matrix after flattening is multiplied by the transpose of the second matrix after dimension flattening to obtain the first product; the normalized exponential function is used to calculate the first product to obtain the first calculation result, and the 1. Multiply the calculation result by the second matrix to obtain the second product; multiply the second product by the first parameter matrix and then the second parameter matrix to obtain the third product; use the sigmoid function to calculate the third product to obtain the shallow layer quality score.

The feature of the sample in the embodiment of the present disclosure may be a feature map, because the feature map may correspond to a matrix, therefore, for ease of understanding, each feature in the embodiment of the present disclosure may be directly understood as a matrix.

Specifically, the fifth preset convolution layer performs convolution on the input first feature with a convolution kernel of a fifth preset size and a channel number of a fifth preset number to obtain the seventh feature. For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fifth feature to obtain the seventh feature. Then flatten the dimension of the seventh feature to matrix A of (28x28, 128), and flatten the dimension of the first matrix to matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel The number of ; A multiplied by the transpose of B; Perform the normalized exponential function softmax operation on the product result, that is, the first product; Multiply the operation result by B; then the product result, that is, the external parameter of the second product The matrix W ₁₁ (128, 64) and the matrix W ₁₂ (64, 1) are used to obtain the third product; the sigmoid function is used to calculate the third product to obtain the shallow quality score. It should be noted that, after the third product is calculated using the sigmoid function, the calculation result may be averaged, and the average value may be used as the shallow quality score.

The above steps can be understood as the following formula:

为浅层质量分，T为转置运算的符号。W₁₁为第一参数矩阵，W₁₂为第二 参数矩阵，mean为求均值函数。

is the shallow quality score, and T is the symbol of the transpose operation. W ₁₁ is the first parameter matrix, W ₁₂ is the second parameter matrix, and mean is the mean value function.

Because the shallow quality score is output by the second stage of the neural network model, the shallow quality score can be understood as the weight value of the influence of the sample on the class center corresponding to the sample class.

Optionally, the second calculation module 403 is further configured to perform interactive calculation on the third feature of each sample and the fusion feature to obtain the middle-level quality score of each sample, wherein the first quality score includes the middle-level quality score, including : Input the third feature into the fifth preset convolution layer, and output the eighth feature; perform dimension leveling processing on the third matrix corresponding to the eighth feature and the second matrix corresponding to the fusion feature respectively, and use the dimension leveling process Multiply the resulting third matrix by the transpose of the second matrix after dimension leveling to obtain the fourth product; use the normalized exponential function to calculate the fourth product to obtain the second calculation result, and use the second calculation Multiply the result by the second matrix to obtain the fifth product; multiply the fifth product by the third parameter matrix and then the fourth parameter matrix to obtain the sixth product; use the sigmoid function to calculate the sixth product to obtain the mid-level quality score.

For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the third feature to obtain the eighth feature. Then flatten the dimension of the eighth feature to a matrix C of (7x7, 128), and flatten the dimension of the first matrix to a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel the number of ; multiply C by the transpose of B; perform the normalized exponential function softmax operation on the result of the product, which is the fourth product; multiply the result of the operation by B; and then the result of the product, which is the fifth product The external parameter matrix W ₂₁ (128, 64) and the matrix W ₂₂ (64, 1) are used to obtain the sixth product; the sigmoid function is used to calculate the sixth product to obtain the mid-level quality score. It should be noted that, after the sixth product is calculated using the sigmoid function, the calculation result may be averaged, and the average value may be used as the mid-level quality score.

The above steps can be understood as the following formula:

为中层质量分，W₂₁为第三参数矩阵，W₂₂为第四参数矩阵。

is the mid-level quality score, W ₂₁ is the third parameter matrix, and W ₂₂ is the fourth parameter matrix.

Optionally, the second calculation module 403 is further configured to perform interactive calculation on the fusion features of each sample to obtain a high-level quality score of each sample, wherein the first quality score includes a high-level quality score, including: combining the fusion features Input the fifth preset convolution layer and output the ninth feature; perform dimension leveling processing on the fourth matrix corresponding to the ninth feature and the second matrix corresponding to the fusion feature, and use the fourth matrix after dimension leveling processing Multiply the transpose of the second matrix after dimension leveling to obtain the seventh product; use the normalized exponential function to calculate the seventh product to obtain the third calculation result, and use the third calculation result to multiply the second matrix to obtain the eighth product; multiply the eighth product by the fifth parameter matrix and then the sixth parameter matrix to obtain the ninth product; use the sigmoid function to calculate the ninth product to obtain the high-level quality score.

For example, a convolution calculation with a convolution kernel of 1×1 and a channel number of 128 is performed on the fusion feature to obtain the ninth feature. Then flatten the dimension of the ninth feature to matrix D of (14x14, 128), and flatten the dimension of the first matrix to matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the channel the number of ; multiply D by the transpose of B; perform the normalized exponential function softmax operation on the result of the product, that is, the seventh product; then multiply the result of the operation by B; then the result of the product, which is the eighth product The external parameter matrix W ₃₁ (128, 64) and the matrix W ₃₂ (64, 1) are used to obtain the ninth product; the sigmoid function is used to calculate the ninth product to obtain the high-level quality score. It should be noted that after using the sigmoid function to calculate the ninth product, the calculation result can be averaged, and the average value can be used as the high-level quality score.

The above steps can be understood as the following formula:

为高层质量分，W₃₁为第三参数矩阵，W₃₂为第四参数矩阵。

is the high-level quality score, W ₃₁ is the third parameter matrix, and W ₃₂ is the fourth parameter matrix.

Optionally, the third calculation module 404 is further configured to calculate the first queue score corresponding to each sample according to the middle layer quality score and the high layer quality score of each sample, wherein the first quality score includes: shallow layer quality score, The middle-level quality score and the high-level quality score; the second cohort score corresponding to each sample is calculated according to the shallow quality score of each sample; the first cohort score and the second cohort score corresponding to multiple samples in each class are calculated. The second quality score of the class center corresponding to each class.

Specifically, the first queue score qi _,1 is calculated according to the following formula:

Calculate the second cohort score qi _,2 according to the following formula:

q _i,2 =norm(φ ₁ )

Calculate the second mass score γ of the class center corresponding to each class according to the following formula:

i is the serial number of the samples in a class, R is the number of samples in a class, α is the adjustment parameter of the neural network model, which can be set to 0.2, max(R) represents the most samples in all classes The number of samples for the class.

Optionally, the model training module 405 is further configured to train the neural network through the cross entropy loss function according to the first quality score of each sample and the cosine value of the deviation angle of the class center corresponding to each sample and the class to which each sample belongs. A network model, wherein the loss function includes a cross-entropy loss function; wherein, the first quality score includes: shallow layer quality score, middle layer quality score, and high-level quality score; wherein, the class center includes: positive class center and negative class center.

Specifically, the cross _- entropy loss function L1 is:

s is the magnification factor, which can be set to 64, θ _yi represents the angle between the sample and the center of the positive class, θ _j represents the angle between the sample and the center of the negative class, i is the serial number of the positive samples in a class, and j is the number of positive samples in a class. The serial number of the negative samples, m ₀ can take 0.35, m ₁ can take 0.25, N is the total number of all samples, n is the total number of all negative samples.

Optionally, the model training module 405 is further configured to calculate the first sum of the reciprocal of the shallow quality score, the inverse of the middle quality score, and the inverse of the high quality score of each sample, where the first quality score includes: Layer quality score, middle layer quality score and high layer quality score; according to the second quality score of the class center corresponding to the class to which each sample belongs and the cosine value of the deviation angle of the negative class center corresponding to each sample and the class to which each sample belongs , use the nearest neighbor optimization function to calculate the nearest neighbor optimization result, where the class center includes: positive class center and negative class center; The second sum corresponding to each sample is multiplied by the nearest neighbor optimization result corresponding to each sample to obtain the tenth product; the first sum corresponding to each sample is added to the tenth product corresponding to each sample to obtain the third sum; The samples correspond to the third and train the neural network model.

Specifically, the nearest neighbor optimization loss function L2 is _:

t can be set to 0.2, ∑ _top10 cos(θ _j -γt) is the nearest neighbor optimization result calculated by the nearest neighbor optimization function, z is the preset number of optimizations, assuming z is 10, ∑ _top10 cos(θ _j -γt) indicates that in a class , the neural network model is optimized using the top 10 negative samples with the highest similarity to the negative class center corresponding to the class.

Optionally, the model training module 405 is further configured to update or modify the cosine value cos θ _p of the deviation angle of the positive class center corresponding to each sample and the class to which each sample belongs when training the neural network model. , this step can update the cosine value cosθ _p by the following formula:

或

or

或

or

或

or

Like m ₀ , m ₂ and m ₁ are parameters that can be set in advance.

The value of the cosine value cosθ _p is related to the neural network model determining whether the sample is a positive sample or a negative sample. Input a sample into the neural network model, and the neural network model can judge by itself whether it is a positive sample or a negative sample.

FIG. 5 is a schematic structural diagram of a face recognition module provided by an embodiment of the present disclosure. As shown in Figure 5, the face recognition module includes:

The detection unit 501 is configured to detect that the target user enters the preset area, obtains the face image of the target user by the image acquisition device, and obtains the face prototype diagram corresponding to the face image from the face detection database;

The first computing unit 502 is configured to extract the tenth feature and the eleventh feature corresponding to the face prototype image and the face image respectively through the neural network model, and calculate the first score corresponding to the tenth feature and the eleventh feature respectively. and the second fraction;

The second calculation unit 503 is configured to calculate the Euclidean distance of the tenth feature and the eleventh feature;

The third calculation unit 504 is configured to calculate the Euclidean transform distance according to the first score, the second score and the Euclidean distance;

The confirmation unit 505 is configured to confirm that the face recognition is successful when the Euclidean transform distance is greater than the preset threshold.

According to the technical solutions provided by the embodiments of the present disclosure, the tenth feature and the eleventh feature corresponding to the face prototype image and the face image can be extracted respectively through the neural network model, and the tenth feature and the tenth feature can be calculated respectively. The first score and the second score corresponding to a feature, and then calculate the Euclidean transformation distance. When the Euclidean transformation distance is greater than the preset threshold, it is confirmed that the face recognition is successful. Therefore, the above technical means can be used to solve the problem in the prior art. , in face recognition, the threshold of face recognition is a fixed problem, and then a face recognition scheme based on dynamic threshold is provided.

Specifically, the Euclidean transformation distance is:

D'=D+Euclidean Transform(s ₁ ,s ₂ )

f1 and f2 represent the tenth feature and the eleventh feature respectively, s1 and s2 represent the first score and the second score respectively, D is the Euclidean distance between f1 and f2, D' is the Euclidean transformation distance, β can be adjusted, Generally take 1.2, f can take the min() function.

It can be seen from the above formula that the Euclidean transform distance changes according to the face image of the target user acquired or detected by the image acquisition device, so the threshold value of the face recognition in the embodiment of the present disclosure is dynamically changed.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure.

FIG. 6 is a schematic diagram of an electronic device 6 provided by an embodiment of the present disclosure. As shown in FIG. 6 , the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, the steps in each of the foregoing method embodiments are implemented. Alternatively, when the processor 601 executes the computer program 603, the functions of each module/unit in the above-mentioned various apparatus embodiments are realized.

Illustratively, the computer program 603 may be divided into one or more modules/units, which are stored in the memory 602 and executed by the processor 601 to complete the present disclosure. One or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 603 in the electronic device 6.

The electronic device 6 may be an electronic device such as a desktop computer, a notebook computer, a palmtop computer, and a cloud server. Electronic device 6 may include, but is not limited to, processor 601 and memory 602 . Those skilled in the art can understand that FIG. 6 is only an example of the electronic device 6, and does not constitute a limitation on the electronic device 6, and may include more or less components than shown, or combine some components, or different components For example, the electronic device may also include input and output devices, network access devices, buses, and the like.

The processor 601 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-available processor Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 602 may be an internal storage unit of the electronic device 6, for example, a hard disk or a memory of the electronic device 6. The memory 602 may also be an external storage device of the electronic device 6, for example, a pluggable hard disk, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, a flash memory card ( Flash Card), etc. Further, the memory 602 may also include both an internal storage unit of the electronic device 6 and an external storage device. Memory 602 is used to store computer programs and other programs and data required by the electronic device. The memory 602 may also be used to temporarily store data that has been or will be output.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above system, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described herein again.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this disclosure.

In the embodiments provided in the present disclosure, it should be understood that the disclosed apparatus/electronic device and method may be implemented in other manners. For example, the apparatus/electronic device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. Multiple units or components may be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, and can also be implemented in the form of software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer-readable storage medium. Based on this understanding, the present disclosure realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program is processed when the When the device is executed, the steps of the foregoing method embodiments may be implemented. A computer program may include computer program code, which may be in source code form, object code form, an executable or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, Read-Only Memory (ROM), random access memory Memory (Random Access Memory, RAM), electric carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the content contained in computer-readable media may be modified as appropriate in accordance with the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not be Including electrical carrier signals and telecommunication signals.

The above embodiments are only used to illustrate the technical solutions of the present disclosure, but not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the present disclosure. within the scope of protection.

Claims (10)

1. A face recognition method, comprising:

acquiring an input image, extracting a first feature, a second feature and a third feature of each sample in the input image through a neural network model, and calculating a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has a plurality of classes, each class corresponds to one class center, and each class has a plurality of samples;

performing feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain a fusion feature of each sample;

performing interactive calculation on at least one characteristic of each sample, wherein the characteristic comprises the following characteristics: the first feature, the third feature, and the fused feature to obtain a first mass fraction of the each sample;

calculating a second quality score of the class center corresponding to each class according to the first quality scores of the samples in each class in the input image;

training the neural network model through a loss function according to the first mass score of each sample, the second mass score of a class center corresponding to a class to which each sample belongs, and the cosine values of the deviation angles of the class centers corresponding to the class to which each sample and each sample belong;

and performing face recognition by using the trained neural network model.

2. The method according to claim 1, wherein the performing a feature fusion process on the first feature, the second feature and the third feature of each sample to obtain a fused feature of each sample comprises:

inputting the input image into the neural network model, and outputting the first feature, the second feature and the third feature of each sample in the input image through a second stage, a third stage and a fourth stage of the neural network model respectively, wherein the neural network model has four stages;

inputting the first characteristic of each sample into a first preset convolution layer, and outputting a fourth characteristic of each sample;

inputting the second characteristic of each sample into a second preset convolution layer, and outputting a fifth characteristic of each sample;

inputting the third feature of each sample into a third preset convolution layer, and outputting a sixth feature of each sample;

performing feature splicing processing on the fourth feature, the fifth feature and the sixth feature of each sample to obtain a spliced feature of each sample, wherein the feature fusion processing comprises the feature splicing processing;

inputting the splicing feature of each sample into a fourth preset convolution layer, and outputting the fusion feature of each sample.

3. The method of claim 1, wherein the interactive computation is performed on at least one of the following characteristics of each sample: the first feature, the third feature, and the fused feature to obtain a first mass fraction of the each sample, comprising:

performing the interactive calculation on the first feature and the fused feature of each sample to obtain a shallow mass fraction of each sample, wherein the first mass fraction includes the shallow mass fraction, and the method includes:

inputting the first characteristic into a fifth preset convolution layer and outputting a seventh characteristic;

respectively carrying out dimensionality leveling processing on a first matrix corresponding to the seventh feature and a second matrix corresponding to the fusion feature, and multiplying the first matrix subjected to the dimensionality leveling processing by the transposition of the second matrix subjected to the dimensionality leveling processing to obtain a first product;

calculating the first product by using a normalized exponential function to obtain a first calculation result, and multiplying the first calculation result by the second matrix to obtain a second product;

multiplying the second product by the first parameter matrix and then by the second parameter matrix to obtain a third product;

and calculating the third product by using a sigmoid function to obtain the shallow layer mass score.

4. The method of claim 1, wherein the interactive computation is performed on at least one of the following characteristics of each sample: the first feature, the third feature, and the fused feature to obtain a first mass fraction of the each sample, comprising:

performing the interactive calculation on the third feature and the fusion feature of each sample to obtain a middle-layer quality score of each sample, wherein the first quality score includes the middle-layer quality score, and the method includes:

inputting the third characteristic into a fifth preset convolution layer and outputting an eighth characteristic;

respectively carrying out dimensionality leveling processing on a third matrix corresponding to the eighth feature and a second matrix corresponding to the fusion feature, and multiplying the third matrix subjected to the dimensionality leveling processing by the transpose of the second matrix subjected to the dimensionality leveling processing to obtain a fourth product;

calculating the fourth product by using a normalized exponential function to obtain a second calculation result, and multiplying the second calculation result by the second matrix to obtain a fifth product;

multiplying the fifth product by a third parameter matrix and then by a fourth parameter matrix to obtain a sixth product;

and calculating the sixth product by using a sigmoid function to obtain the middle-layer quality score.

5. The method of claim 1, wherein the interactive computation is performed on at least one of the following characteristics of each sample: the first feature, the third feature, and the fused feature to obtain a first mass fraction of the each sample, comprising:

performing the interactive calculation on the fusion features of each sample to obtain a high-level quality score of each sample, wherein the first quality score comprises the high-level quality score, and the method comprises the following steps:

inputting the fusion feature into a fifth preset convolution layer and outputting a ninth feature;

respectively carrying out dimensionality leveling processing on a fourth matrix corresponding to the ninth feature and a second matrix corresponding to the fusion feature, and multiplying the fourth matrix subjected to the dimensionality leveling processing by the transpose of the second matrix subjected to the dimensionality leveling processing to obtain a seventh product;

calculating the seventh product by using a normalized exponential function to obtain a third calculation result, and multiplying the third calculation result by the second matrix to obtain an eighth product;

multiplying the eighth product by the fifth parameter matrix and then by the sixth parameter matrix to obtain a ninth product;

and calculating the ninth product by using a sigmoid function to obtain the high-level quality score.

6. The method of claim 1, wherein calculating a second quality score for the class center corresponding to each class from first quality scores of a plurality of samples in the input image in the each class comprises:

calculating a first queue score corresponding to each sample according to the middle-layer quality score and the high-layer quality score of each sample, wherein the first quality score comprises: a superficial layer mass fraction, the middle layer mass fraction and the high layer mass fraction;

calculating a second queue score corresponding to each sample according to the shallow mass score of each sample;

calculating the second quality score of the class center corresponding to each class according to the first queue score and the second queue score corresponding to a plurality of samples in each class.

7. The method of claim 1, wherein the training the neural network model with a loss function according to the first quality score of the each sample, the second quality score of a class center corresponding to a class to which the each sample belongs, and the cosine values of the deviation angles of the class centers of the each sample and the class to which the each sample belongs comprises:

training the neural network model through a cross entropy loss function according to the first quality score of each sample and the cosine value of the deviation angle of the class center corresponding to the class to which each sample and each sample belong, wherein the loss function comprises the cross entropy loss function;

wherein the first mass fraction includes: superficial layer mass fraction, middle layer mass fraction and high layer mass fraction;

wherein, the class center comprises: a positive class center and a negative class center.

8. The method of claim 1, wherein the training the neural network model with a loss function according to the first mass score of the each sample, the second mass score of the class center corresponding to the class to which the each sample belongs, and the cosine values of the deviation angles of the each sample and the class center corresponding to the class to which the each sample belongs comprises:

training the neural network model with a neighbor optimization loss function, wherein the loss function comprises the neighbor optimization loss function, comprising:

calculating a first sum of an inverse of the superficial mass fraction, an inverse of the medial mass fraction, and an inverse of the high mass fraction for each sample, wherein the first mass fraction comprises: superficial layer mass fraction, middle layer mass fraction and high layer mass fraction;

calculating a neighbor optimization result using a neighbor optimization function according to the second quality score of the class center corresponding to the class to which the each sample belongs and the cosine value of the deviation angle of the negative class center corresponding to the class to which the each sample and the class to which the each sample belongs, wherein the class center comprises: a positive class center and a negative class center;

calculating a second sum of the shallow layer quality score, the middle layer quality score and the high layer quality score of each sample, and multiplying the second sum corresponding to each sample by the neighbor optimization result corresponding to each sample to obtain a tenth product;

adding the first sum corresponding to each sample to the tenth product corresponding to each sample to obtain a third sum;

and training the neural network model by corresponding to the third sum through each sample.

9. The method of claim 1, wherein the using the trained neural network model for face recognition comprises:

when a target user is detected to enter a preset area, acquiring a face image of the target user through image acquisition equipment, and acquiring a face prototype graph corresponding to the face image from a face detection database;

respectively extracting tenth features and eleventh features corresponding to the face prototype graph and the face image through the neural network model, and respectively calculating first scores and second scores corresponding to the tenth features and the eleventh features;

calculating Euclidean distances of the tenth feature and the eleventh feature;

calculating the Euclidean transformation distance according to the first fraction, the second fraction and the Euclidean distance;

and confirming that the face recognition is successful under the condition that the Euclidean transformation distance is greater than a preset threshold value.

10. A face recognition apparatus, comprising:

the first calculation module is configured to acquire an input image, extract a first feature, a second feature and a third feature of each sample in the input image through a neural network model, and calculate a cosine value of a deviation angle of a class center corresponding to each sample and a class to which each sample belongs through the neural network model, wherein the input image has a plurality of classes, each class corresponds to a class center, and each class has a plurality of samples;

a feature fusion module configured to perform feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain a fused feature of each sample;

a second computing module configured to interactively compute at least one of the following characteristics of each sample: the first feature, the third feature, and the fused feature to obtain a first mass fraction of the each sample;

a third calculating module configured to calculate a second quality score of the class center corresponding to each class according to the first quality scores of the plurality of samples in each class in the input image;

a model training module configured to train the neural network model by a loss function according to the first mass score of the each sample, the second mass score of a class center corresponding to a class to which the each sample belongs, and the cosine value of the deviation angle of the each sample and the class center corresponding to the class to which the each sample belongs;

and the face recognition module is configured to perform face recognition by using the trained neural network model.

Images