CN115775406A

CN115775406A - Attribute editing-based face anti-attack sample generation method and system

Info

Publication number: CN115775406A
Application number: CN202211477245.4A
Authority: CN
Inventors: 曹娟; 徐榆; 王子昂; 方凌飞; 唐胜
Original assignee: Institute of Computing Technology of CAS
Current assignee: Institute of Computing Technology of CAS
Priority date: 2022-11-23
Filing date: 2022-11-23
Publication date: 2023-03-10

Abstract

The invention provides a face anti-attack sample generation method and system based on attribute editing, which aim at an input real face image and a target image to be recognized to carry out hidden space mapping to low-dimensional manifold internal representation, and simultaneously improve the similarity of face identity information by a feature layer fusion method. In the attribute editing step, a group of attribute editing parameters are designed, the editing intensity is optimized according to the optimal editing attribute obtained in the attribute selection step, and meanwhile, the control of various optimized image generation effects is introduced, so that the edited image effect and the identity information identified by naked eyes are kept unchanged while the optimal model attack success rate is achieved. The robustness of the face recognition model is detected, and the face recognition model is finely adjusted, so that the recognition accuracy is improved.

Description

Attribute editing-based face anti-attack sample generation method and system

Technical Field

The method belongs to the technical field of evaluation of robustness, safety and robustness of an artificial intelligence algorithm, and particularly relates to a face recognition anti-attack sample generation method and system based on attribute editing.

Background

The face recognition is used as a biological feature recognition technology for identity recognition by using visual features of faces, and has the advantages of noninvasiveness, non-contact, friendliness, convenience and the like. With the development of artificial intelligence technology, the ability of face recognition technology is further improved by extracting face features based on Convolutional Neural Network (CNN), and the face recognition technology is widely applied to daily life, such as: face recognition access control, face brushing unlocking, payment and the like. The counterattack (adaptive Attack) is to add some noise that cannot be perceived by human beings to input data, so that the model makes an erroneous judgment on the input data to achieve the purpose of confusing or fooling an intelligent model, the added noise is called counterdisturbance (adaptive Perturbation), and a sample obtained after adding the noise is called a countersample (adaptive sample). The face recognition system is also very easy to receive the threat of resisting the attack. The threats of face recognition against attacks can be divided into two categories according to different purposes:

● Impersonation attack: after adding the counter disturbance, the face recognition model recognizes the input face image as a specified person.

● Avoiding attack: after adding the counter disturbance, the face recognition model recognizes the input face image as a wrong recognition (as any person other than the real person).

The method can evaluate the Robustness of the face recognition system in practical application by using a means of resisting attack (adaptive Robustness). The robustness of the evaluation model can be measured by using the success rate of the attack fight, for example, 100 fight samples are used, the target model is input, the result is output through the model, and the 'number of successfully attacked samples/total sample amount' is used as the evaluation result of the robustness fight. A lower attack success rate indicates a more robust model to such attack methods and vice versa.

Based on the confrontation sample and the anti-robustness evaluation result, the robustness and the safety of the model can be further improved. For example, the generated confrontation sample is used as training data of the model to perform fine adjustment on the model, or the generated confrontation sample is used for training a two-classifier to distinguish whether the input sample is the confrontation sample.

The digital world face recognition anti-attack method mainly generates an anti-sample through an optimization process, and can be divided into an image domain-based anti-attack method and a hidden space-based anti-attack method according to the space added by anti-disturbance. An image domain-based anti-attack method generally limits the pixel value of each pixel of a disturbed image and the visual effect of an anti-sample generated by the change size constraint of the pixel value, and includes approximately converting a non-convex optimization problem for constructing the anti-attack image into a linear form by using a fast gradient notation method (FGSM), but the algorithm cannot guarantee the success rate of the attack, and particularly has low success rate in the directional attack. Later, the latter proposed to improve the fast gradient notation through multiple iterations, but at the cost of increased computational effort. The following proposes a rapid gradient symbolic method (random gradient symbolic method), and the main idea is to add random disturbance to an input sample before gradient calculation, and jump out a large curvature near a data point, so that a countermeasure sample generated by the rapid gradient symbolic method has higher generalization capability. The latter proposes another countermeasure sample generation technique based on decision hyperplane, which utilizes an iterative linear representation of the target model to generate the countermeasure sample. Since this method is a hyperplane problem under a finite neural network, the application to a non-neural network model is quite limited, the algorithm is less versatile and requires a lot of time to analyze the model characteristics. The method has higher iteration speed and simple realization, but the disturbance added in a pixel domain can be transformed through simple images, such as: filtering and the like, and migration capability and visual effect are difficult to guarantee. The anti-attack method based on the hidden space converts the image with the spatial domain into the hidden space by using an Encoder (Encoder) to add the anti-disturbance to the hidden space (characteristic layer), and obtains the anti-disturbance Code (adaptive tension Code) by adding the anti-disturbance to the characteristic (tension Code) of the hidden space of the image, and uses a generator, such as: the countermeasure generation network (GAN) remaps the image from the hidden space back to the spatial domain, resulting in the final countermeasure sample. The migration capability of the resisting sample is very easily influenced by initialization and is locally optimal, the migration capability of the resisting sample is related to a search space, when the search space is continuously increased, the migration capability of the resisting sample is increased and then reduced, and when the search space is very large, the migration capability of the resisting sample is obviously reduced due to the fact that the resisting sample is over-fitted to a substitution model. By optimizing the countermeasure disturbance on the manifold learned based on the face data and the trained generation model, on one hand, a sufficient search space is provided, on the other hand, the manifold learned by the generation model provides better regularization, and meanwhile, the characteristics (faces) related to the countermeasure task semantics can be generated, so that the substitution model and the target model can be effectively linked, and the migration capability of the countermeasure sample is effectively improved.

In recent years, deep learning techniques have been developed and widely used in the field of computer vision. On one hand, deep learning technology leads to a new round of artificial intelligence surge, but on the other hand, a series of safety problems caused by deep learning also draw more and more attention. At present, image and video recognition technology based on deep learning is widely applied to aspects of people's life, such as intelligent supervision of network content, automatic video monitoring analysis, access control systems based on face recognition, face brushing payment and the like. In these critical application areas, the reliability and security of information and data should be valued and guaranteed. Some false images and videos generated based on the deep forgery (also called deep fake) technology have attracted a lot of attention on the internet since 2017, and especially when the deep forgery is used on a person with a huge influence, the false image and the video tend to have a larger influence by the influence of the person. In addition, a great deal of 'one-click' face changing software makes the acquisition of forged videos simpler and simpler. False images and videos have become one of the most significant information data security risks at present, and detection and supervision of false images and videos face huge challenges.

The artificial face synthesized by AI has a great threat, and can create a video of false impression that a target person does or says something with a facial expression and a body action close to the reality, thereby subverting the cognition of people on the concept of 'seeing the eyes as real'. An effective technique for detecting false face images or videos in a network environment is urgently needed in the industry, but the difficulty is great, mainly because the forged regions of the face forged images are weak and exist locally. The detection is very susceptible to image noise. In addition, the forged regions are often unpredictable, and the regions targeted by each forging method are different, so that how to correctly detect and classify the forged regions is very difficult.

The image for depth forgery detection can be roughly divided into: the method is based on image forgery defects, network structure improvement, multi-feature fusion and other tasks.

The method is a mainstream method, and attempts to detect an inconsistent phenomenon caused by operations such as scaling, rotation, distortion and the like in the counterfeiting process of a human face. A CNN-based detection method detects a low resolution image plane portion due to falsification by detecting a face region and a face surrounding region. A face X-ray method is characterized in that a face contour mask is designed, a model is guided to pay attention to a face contour region where a fake region easily appears, and a good generalization effect is achieved.

Methods for improving network structure such methods are directed to improving the effect of models on true and false classification by modifying or improving classification networks. Two different networks were proposed to focus on the mesoscopic properties of the images, based on the methods of mesoscopic and steganographic analysis features, being the Meso-4 network and variants of the Meso-4 network incorporating the inclusion module, respectively. A capsule network (CapsuleNetworks) based counterfeit detection system that has fewer parameters than conventional CNNs.

Methods of multi-feature fusion that attempt to capture more features from an image that are useful for authentication. The method comprises the steps of performing counterfeiting detection by using frequency domain characteristics, finding counterfeit traces of a forged image in a frequency domain, and classifying by using classifiers such as SVM (support vector machine) on the basis of classical frequency domain analysis, so that a good detection effect is obtained in a small amount of labeled training samples. Aiming at the compression problem of forged face images, the compression problem can be solved by excavating a forged mode in a frequency domain signal, and the human face forging network (F3-Net) is proposed to deeply excavate the forged mode through a two-stage collaborative learning framework. Finally, the method is obviously superior to other methods on a compressed FaceForensics + + data set. The face after face change is distinguished from the context of the face, and then a two-branched network is constructed, one is a classification network with facial semantics segmented as input, and the other is a classification network with facial context (e.g., hair, ears, neck) as input. The method utilizes the characteristics of two branch networks to detect forgery

Methods by means of other tasks this type of method attempts to improve the detection by means of other tasks or directly to make a forgery detection. And (3) positioning a fake area at a pixel level while performing true and false classification on the video by using a multi-task learning mode. The authors use a Y-decoder and three loss functions to constrain the network in the hope that valuable features can be shared among multiple tasks. The image segmentation task defining the counterfeiting detection problem as a pixel level adopts a full convolution network to extract features, and binaryzation is carried out on a segmentation result so as to mark a counterfeiting area in the image.

The detection method based on intra-frame falsification traces has the advantages that the training and detection speed is very high, and the method is particularly effective when single-frame image falsification traces are obvious. But the disadvantage is that in the single frame method, the local forged area is concerned in a self-adaptive manner, and the suspicious area is found.

In summary, various anti-attack algorithms aiming at face recognition are proposed at present, but the existing method has the following two technical problems: (1) The attack success rate of the anti-attack algorithm is not high when the anti-attack algorithm attacks the black box model (namely, information such as network structure, parameters and the like of an attacker to the face recognition model is unknown), and the attack effect of the attack algorithm obtained by training the substituted model is poor when the attack algorithm attacks other models, namely, the migration capability of the algorithm is weak. (2) The existing method often causes the generated human face to have poor visual angle effect on resisting the attack image, which is particularly represented as poor authenticity of the image perceived by naked eyes and changes the identity of the observed human face.

Disclosure of Invention

The invention provides a face recognition anti-attack method based on attribute editing, and provides three attribute attack modules which are named as a hidden space mapping module, a double-flow attribute editing module and a gradient integrated manifold internal attack module.

The attribute editing-based face anti-attack method is mainly used for attacking a face recognition algorithm based on a deep learning technology, and after the original face image is subjected to tiny attribute editing (such as skin color, expression and the like), the face recognition algorithm can wrongly recognize the original face image as another specified face identity, and meanwhile, the naked eye recognition identity of the edited image is kept unchanged.

The hidden space mapping module provided by the invention performs hidden space mapping to low-dimensional manifold internal representation aiming at the input real face image and the target image to be recognized, and simultaneously improves the similarity of face identity information by a feature layer fusion method.

The double-flow attribute editing module provided by the invention comprises two steps of attribute selection and attribute editing, wherein in the attribute selection step, a group of attribute selection parameters are designed, and different attack attributes are automatically selected according to different original images and target images so as to improve the attack efficiency of the model. In the attribute editing step, a group of attribute editing parameters are designed, the editing intensity is optimized according to the optimal editing attribute obtained in the attribute selection step, and meanwhile, the control of various optimized image generation effects is introduced, so that the edited image effect and identity information recognized by naked eyes are kept unchanged while the optimal model attack success rate is achieved.

The invention also provides a face anti-attack sample generation method based on attribute editing, which comprises the following steps:

step 1, taking an original face image x ^src And target identity image x ^tgt Mixing the hidden space vectors to obtain a hidden space vector z ^mix ；

Step 2, in the hidden space vector z ^mix Overlay edit attribute vector ω _s Multiplying the attribute editing direction B to obtain an attribute-selected hidden space vector, and generating a selected image x based on the attribute-selected hidden space vector ^selected ；

Step 3, in the hidden space vector z ^mix Overlay edit attribute vector ω _s And a weight matrix omega for controlling the magnitude of the property edit _e To obtain an attribute-edited hidden space vector, and generating an edited image x based on the attribute-edited hidden space vector ^edited ；

Step 4, respectively according to the edited images x ^edited And selecting image x ^selected Both and the target identity image x ^tgt Constructing a first loss function and a second loss function according to the similarity; from the generated face image and the target identity image x ^tgt Constructing a third loss function according to the similarity; and limiting the weight matrix omega by a fourth loss function _e Standard deviation of (d);

step 5, using the total loss function composed of the first loss function to the fourth loss function to train and adjust the editing attribute vector omega by the total loss function _s And the weight matrix omega _e Until the total loss function converges, saving the current edited image x ^edited Is a confrontation sample, and the character identity label of the confrontation sample is the character identity of the original facial image.

The method for generating the face attack resisting sample based on attribute editing comprises the following steps of

And a second loss function

Respectively as follows:

wherein f (.) represents a feature extractor of the picture, D (.) represents cosine similarity between two features, and the feature extractor is a pre-trained face recognition model.

The method for generating the face anti-attack sample based on attribute editing comprises the following steps: and taking the recognition accuracy of the face recognition model to be subjected to robustness evaluation on the confrontation sample as a confrontation robustness evaluation result of the face recognition model.

The method for generating the face anti-attack sample based on attribute editing comprises the following steps: and fine-tuning the target face recognition model by using the confrontation sample so as to improve the recognition accuracy of the target face recognition model.

The invention also provides a face anti-attack sample generation system based on attribute editing, which comprises the following steps:

a hidden space mapping module for mapping the original face image x ^src And target identity image x ^tgt Mixing the hidden space vectors to obtain a hidden space vector z ^mix ；

A dual-stream attribute editing module for editing the vector z in the hidden space ^mix Overlay edit attribute vector ω _s Multiplying with the attribute editing direction B to obtain the attribute selectionThe selected hidden space vector is generated based on the attribute to generate a selected image x ^selected (ii) a For implicit space vector z ^mix Overlay edit attribute vector ω _s And a weight matrix omega for controlling the editing amplitude of the attribute _e To obtain an attribute-edited hidden space vector, and generating an edited image x based on the attribute-edited hidden space vector ^edited ；

A manifold internal attack module for respectively editing the image x ^edited And selecting image x ^selected Both and the target identity image x ^tgt Constructing a first loss function and a second loss function according to the similarity; from the generated face image and the target identity image x ^tgt Constructing a third loss function according to the similarity; and limiting the weight matrix omega by a fourth loss function _e Standard deviation of (d); using the total loss function composed of the first loss function to the fourth loss function to train and adjust the editing attribute vector omega _s And a weight matrix omega _e Until the total loss function converges, the current edited image x is saved ^edited Is a confrontation sample, and the character identity label of the confrontation sample is the character identity of the original face image.

The system for generating the face anti-attack sample based on attribute editing comprises a first loss function

And a second loss function

Respectively as follows:

The system for generating the face anti-attack sample based on attribute editing comprises a manifold internal attack module, a manifold internal attack module and a data processing module, wherein the manifold internal attack module is used for: and taking the recognition accuracy of the face recognition model to be subjected to robustness evaluation on the confrontation sample as a confrontation robustness evaluation result of the face recognition model.

The system for generating the face anti-attack sample based on attribute editing is characterized in that the manifold internal attack module is used for: and fine-tuning the target face recognition model by using the confrontation sample so as to improve the recognition accuracy of the target face recognition model.

The invention also provides a storage medium for storing a program for executing the method for generating the human face anti-attack sample based on attribute editing.

The invention also provides a client used for the arbitrary attribute editing-based face anti-attack sample generation system.

The gradient integrated manifold internal attack module provided by the invention utilizes various face recognition models as discrimination models, obtains the best face counterattack sample by learning attribute selection parameters and attribute editing parameters in the manifold, improves the migration effect of the face counterattack, and can achieve better attack effect aiming at unknown black box models.

Drawings

FIG. 1 is a schematic diagram of a process for countering attacks based on attribute editing;

fig. 2 is a diagram of the result of a qualitative visual quality test of an image.

Detailed Description

The invention provides a face anti-attack method based on attribute editing, which utilizes face attribute information to resist attack, automatically learns the optimal attribute category and editing strength according to an original image and a target image which needs to be identified as an appointed figure identity by a face identification system, thereby keeping the quality of the original image and the identity information identified by naked eyes unchanged while the attack is successful. More specifically, the method comprises the following steps:

(1) The method for hidden space mapping and hidden vector fusion can map a real image to a low-dimensional representation and then edit the real image, and meanwhile improves the similarity between the original image face and the target face information.

(2) According to the face attribute editing method, the attribute selection parameters and the attribute editing parameters are introduced, so that the attribute type and the attribute editing strength which are optimal for attack can be automatically learned, and meanwhile, the generated image quality and identity information observed by naked eyes can be ensured to be unchanged due to the introduced standard deviation loss and LPIPS loss.

(2) The method disclosed by the invention carries out attack in the manifold integrated by the gradient, optimizes the representation of the image in the manifold by integrating the gradients of various face recognition models in the training process, and can improve the transfer capability against the attack and the image generation effect.

The experiment result proves that the attribute editing anti-attack method provided by the invention has higher attack success rate on a plurality of black box models and commercial API, and simultaneously achieves the optimal effect on the test index of the image generation effect, thereby verifying the effectiveness of the method.

In order to make the aforementioned features and effects of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.

In order to solve the technical problems in the prior art, a face recognition attack resisting method based on attribute editing is provided, as shown in fig. 1, a face recognition system is mistakenly recognized as a specified face identity while high-quality face image generation is maintained through a double-flow attribute editing method. The various modules of the network are described below.

(1) Hidden space mapping module

Original face image x ^src And target identity image x ^tgt Mapping to an image hidden space. Obtaining an original face image x ^src And target identity image x ^tgt After the respective hidden space vectors are mixed, a new hidden space vector z is obtained ^mix The mixed vector z ^mix Further connection on identity informationNear target identity and closer to the original face in visual effect. Wherein z is ^mix And x ^src And x ^tgt Are all vectors of 18 layers, z ^mix The first n layers of (A) are from x ^src Front n layers of (2), z ^mix The rear (18-n) layer of (A) is derived from x ^tgt The rear (18-n) layer of (b).

(2) Attribute editing module for double stream

By making a pair of z ^mix And editing to obtain the corresponding attribute editing effect of the image. Specifically, based on the attribute editing direction B, two steps of attribute selection and attribute editing are designed: the attribute selection step utilizes an N-dimensional vector ω representing the edit attribute _s Learning discrete attribute editing direction, wherein the editing attribute is the object to be edited, such as eyes, glasses, hair, expression, etc., by calculating the z-coordinate ^mix Upper superposition of omega _s Obtaining the hidden space vector after attribute selection by multiplying B, and finally generating the image x after attribute selection by a pre-trained generator ^selected . The attribute selection is to prepare for subsequent counterattack, and x generated by combining the subsequent counterattack ^selected The image has a change in appearance of the attribute while including the countermeasure information.

The attribute editing step is similar to the attribute selecting step, the editing range is based on the editing direction, and the ideal editing effect can be achieved only by editing along a certain direction. Using a weight matrix omega of the same dimension as B _e Controlling the edition amplitude of the attribute, and finally generating an image x with the edited attribute through a generator ^edited . The above two steps can be summarized as follows: the purpose of attribute selection is to learn the most effective attribute editing direction B for the attack target face recognition algorithm, and the purpose of attribute editing is to learn the editing amplitude for resisting the attack.

(3) Gradient integrated manifold internal countermeasure attack

For omega _s And ω _e The countermeasure loss for optimizing parameters, attribute selection and attribute editing by applying the gradient descent method, respectively, is represented by equations 1 and 2, respectively:

wherein f (.) represents a feature extractor of the picture, D (.) represents cosine similarity between two features, and the countermeasure loss of the formula 1 and the formula 2 is reduced by optimizing parameter updating through a gradient descent algorithm.

Meanwhile, the integrated training strategy is utilized to improve the migration capability against the attack. Specifically, we use a plurality of pre-trained face recognition models as white-box models (the rightmost FR in FIG. 1) ₁ 、FR ₂ 、...FR _n All information of the network structure, parameters and the like of the model is mastered) to extract and fuse the facial image features.

To ensure the generated face image x ^edited Quality, construction of loss function L using image similarity metric LPIPS _LPIPS To constrain x ^edited Is close to x ^src Represented by formula 3:

L _LPIPS ＝LPIPS(x ^edited ,x ^src ) Formula 3

In addition, in order to restrict the direction in the face attribute editing process, a standard deviation loss is designed to limit omega _e Is represented by formula 4

L _std ＝STD(ω _e ) Formula 4

Finally, the overall training loss function against attacks is represented by equation 5:

wherein Λ ^T Indicating a hyperparameter between these losses.

In practical experiments, images are generated by a pre-trained generator such as Karres and the like in a model, and IRSE50, facenet, IR152 and Mobileface are selected as training and testing face recognition models (three of the four models are selected)As a training model, one left as a test model). In addition, the anti-attack effect is tested on the API of the open commercial Face recognition of Aliyun and Face + +. In the test indexes, an Attack Success Rate (ASR) and a face similarity confidence coefficient (Conf.) are mainly used as methods for measuring the attack effect of the algorithm. Using pixel level L ₂ The distance, the LPIPS distance and the MS-SSIM distance are used as methods for measuring the image quality generated by the algorithm. In contrast, the optimal face recognition anti-attack methods in recent years, such as PGD, MI-FGSM, TIP-IM, AMT-GAN and Semantic-adv, are tested.

Results of the experiment

In order to verify the effectiveness of the method, the CelebA-HQ data set is selected to be tested, and the CelebA-HQ data set is a widely used face data set and provides a large amount of real high-definition face image data.

Attack success rate test

The success rate of the feeding was first tested under different black box models, and the results are shown in table 1:

TABLE 1 comparison of success rates of attacks under different models

Aiming at different black box models, the method is superior to the current best model under the average attack success rate of the IRSE50 model, the IR152 model, the MobileFace model and the four models.

The attack success rate under Aliyun and Face + + is tested at the same time, two similar attack methods SemanticADV and AMT-GAN based on changing the Face attribute are tested, and the error acceptance rates (FAR) are respectively 10 ^-5 、10 ^-4 、10 ^-3 As a threshold for judging the success of the attack, the results are shown in table 2:

TABLE 2 comparison of attack success rates under different commercial APIs

Test results on the black box model and the API show that compared with other advanced face anti-attack methods, the method obtains the best attack success rate, the black box attack success rate is improved by 24.54% compared with the best method, and huge improvement is obtained on the attack of the API.

Image effect quality testing

Secondly, quantitative and qualitative tests of the generated image quality are carried out aiming at similar methods for editing the human face attributes, namely SemanticADV and AMT-GAN. Quantitative experiments we have adopted L ₂ LPIPS, MS-SSIM, wherein L ₂ And the lower the LPIPS index is, the better the generated image quality effect is, and the higher the MS-SSIM index is, the better the generated image quality effect is. The quantitative test results are shown in Table 3, wherein Ours-w/o attributes represents the comparative experiment of the method without the attribute editing module, and Ours-w/omixing represents the comparative experiment of the method without the hidden vector fusion module.

TABLE 3 comparison of different visual quality indicators

Experiments show that the quantitative index of the image quality of the method is superior to AMT-GAN, and the value of the method is that SemanticAdv is superior to the method in index, but the method has extremely low attack success rate and is lack of practical usability as shown in tables 1 and 2. Meanwhile, a comparison experiment with the method of the self-body also shows that the attribute editing module of the method can effectively improve the image quality, and meanwhile, the hidden vector mixing module slightly reduces the image quality but greatly improves the attack success rate.

The qualitative test result is shown in fig. 2, wherein a first column in fig. 2 is an original image, a second column is an effect of a sematic adv algorithm, a third column is an effect of an AMT-GAN algorithm, a fourth column is a corresponding countermeasure sample image, a numerical value below the image represents a confidence coefficient for judging that two images belong to the same Face identity under the Face + + test, and a higher confidence coefficient indicates that the Face recognition algorithm has a higher probability of considering that two faces belong to the same identity:

observing the figure 2 shows that the image generation reality degree of the method is superior to AMT-GAN, the anti-attack effect is much higher than Semantivadv, and the balance between the attack success rate and the image quality is achieved.

The following are system examples corresponding to the above method examples, and this embodiment can be implemented in cooperation with the above embodiments. The related technical details mentioned in the above embodiments are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the above-described embodiments.

a hidden space mapping module for mapping the original face image x ^src And target identity image x ^tgt Mixing respective hidden space vectors to obtain a hidden space vector z ^mix ；

A dual-stream attribute editing module for editing the vector z in the hidden space ^mix Overlay edit attribute vector ω _s Multiplying the attribute editing direction B to obtain an attribute-selected hidden space vector, and generating a selected image x based on the attribute-selected hidden space vector ^selected (ii) a For the implicit spatial vector z ^mix Overlay edit attribute vector ω _s And a weight matrix omega for controlling the editing amplitude of the attribute _e To obtain an attribute-edited hidden space vector, and generating an edited image x based on the attribute-edited hidden space vector ^edited ；

A manifold internal attack module for respectively editing the image x ^edited And selecting image x ^selected Both and the target identity image x ^tgt Constructing a first loss function and a second loss function according to the similarity; from the generated face image and the target identity image x ^tgt Constructing a third loss function according to the similarity; and limiting the weight matrix omega by a fourth loss function _e Standard deviation of (d); formed by the first to fourth loss functionsA total loss function, and training and adjusting the edit attribute vector omega by the total loss function _s And a weight matrix omega _e Until the total loss function converges, saving the current edited image x ^edited Is a confrontation sample, and the character identity label of the confrontation sample is the character identity of the original facial image.

And a second loss function

Respectively as follows:

The invention also provides a storage medium for storing a program for executing the arbitrary attribute editing-based face counterattack sample generation method.

The invention also provides a client used for the system for generating any face anti-attack sample based on attribute editing.

Claims

1. A face anti-attack sample generation method based on attribute editing is characterized by comprising the following steps:

Step 2, in the hidden space vector z ^mix Overlay edit attribute vector ω _s Multiplying the attribute editing direction B to obtain a hidden space vector after attribute selection, and generating a selected image x based on the hidden space vector after attribute selection ^selected ；

Step 3, in the hidden space vector z ^mix Overlay edit attribute vector ω _s And a weight matrix omega for controlling the editing amplitude of the attribute _e To obtain an attribute-edited hidden space vector, and generating an edited image x based on the attribute-edited hidden space vector ^edited ；

Step 4, respectively according to the edited images x ^edited And selecting image x ^selected Both and the target identity image x ^tgt Constructing a first loss function and a second loss function according to the similarity; from the generated face image and the target identity image x ^tgt Constructing a third loss function according to the similarity; and limiting the weight matrix omega by a fourth loss function _e The standard deviation of (a);

step 5, using the total loss function composed of the first loss function to the fourth loss function to train and adjust the editing attribute vector omega by the total loss function _s And the weight matrix omega _e Until the total loss function converges, the current edited image x is saved ^edited Is a confrontation sample, and the character identity label of the confrontation sample is the character identity of the original face image.

2. The method for generating human face samples against attacks based on attribute editing as claimed in claim 1Characterized by the first loss function

And a second loss function

Respectively as follows:

3. The method for generating samples of human face attack resistance based on attribute editing as claimed in claim 1, wherein the step 5 comprises: and taking the recognition accuracy of the face recognition model to be subjected to robustness evaluation on the confrontation sample as a confrontation robustness evaluation result of the face recognition model.

4. The method for generating samples of human face attack resistance based on attribute editing as claimed in claim 1, wherein the step 5 comprises: and fine-tuning the target face recognition model by using the confrontation sample so as to improve the recognition accuracy of the target face recognition model.

5. A face anti-attack sample generation system based on attribute editing is characterized by comprising:

A dual-stream attribute editing module for editing the vector z in the hidden space ^mix Overlay edit attribute vector ω _s Multiplying the attribute editing direction B to obtain a hidden space vector after attribute selection, and generating a selected image x based on the hidden space vector after attribute selection ^selected (ii) a For the implicit spatial vector z ^mix Overlay edit attribute vector ω _s And a weight matrix omega for controlling the magnitude of the property edit _e To obtain an attribute-edited hidden space vector, and generating an edited image x based on the attribute-edited hidden space vector ^edited ；

A manifold internal attack module for respectively editing the image x ^edited And selecting image x ^selected Both and the target identity image x ^tgt Constructing a first loss function and a second loss function according to the similarity; from the generated face image and the target identity image x ^tgt Constructing a third loss function according to the similarity; and limiting the weight matrix omega by a fourth loss function _e Standard deviation of (d); a total loss function composed of the first to fourth loss functions for training and adjusting the edit attribute vector omega _s And a weight matrix omega _e Until the total loss function converges, saving the current edited image x ^edited Is a confrontation sample, and the character identity label of the confrontation sample is the character identity of the original facial image.

6. The system of claim 5, wherein the first loss function is a function of the face's face-to-attack sample

And a second loss function

Respectively as follows:

7. The system of claim 5, wherein the manifold attack module is configured to: and taking the recognition accuracy of the face recognition model to be subjected to robustness evaluation on the confrontation sample as a confrontation robustness evaluation result of the face recognition model.

8. The system of claim 5, wherein the manifold attack module is configured to: and fine-tuning the target face recognition model by using the confrontation sample so as to improve the recognition accuracy of the target face recognition model.

9. A storage medium for storing a program for executing the arbitrary attribute-editing-based face counterattack sample generation method.

10. A client is used for the human face anti-attack sample generation system based on attribute editing.