CN115798056A - Face confrontation sample generation method, device and system and storage medium - Google Patents
Face confrontation sample generation method, device and system and storage medium Download PDFInfo
- Publication number
- CN115798056A CN115798056A CN202211287314.5A CN202211287314A CN115798056A CN 115798056 A CN115798056 A CN 115798056A CN 202211287314 A CN202211287314 A CN 202211287314A CN 115798056 A CN115798056 A CN 115798056A
- Authority
- CN
- China
- Prior art keywords
- face
- picture
- original
- attack
- attacked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a method, a device, a system and a storage medium for generating a face confrontation sample, wherein the method comprises the following steps: acquiring an original face picture, wherein the original face picture comprises: an original attacked face picture and an original attacked face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. The scheme reduces the implementation complexity of the face anti-attack method. Compared with the traditional scheme, disturbance is often added to the whole picture, the time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
Description
Technical Field
The invention relates to the technical field of face recognition, in particular to a face confrontation sample generation method, a face confrontation sample generation device, a face confrontation sample generation system and a storage medium.
Background
Face recognition technology is widely used in various industries, and there are various recognition methods. However, various attack modes aiming at the face recognition technology are also emerging, and the safety of face recognition calculation is designed at risk. Therefore, people hope to improve the security of the face recognition algorithm by researching the attack mode, and the face attack resistance is one of the hot topics.
Although there are many methods for human face to fight against attacks in the industry at present, many methods achieve good effects in white box attacks (all information of models are known), and generally represent the white box attacks (the components and specific parameters of the models cannot be obtained) in scenes. In addition, most face anti-attack methods add disturbance to the whole picture in a query mode to improve the attack success rate, but introduce higher computational complexity.
Disclosure of Invention
The invention mainly aims to provide a method, a device and a system for generating a face confrontation sample and a storage medium, and aims to reduce the implementation complexity of the face confrontation attack method.
In order to achieve the above object, the present invention provides a face confrontation sample generation method, which comprises the following steps:
obtaining an original face picture, wherein the original face picture comprises: an original attacked face picture and an original attacking face picture;
based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode;
and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
Optionally, the step of performing fusion feature learning on the original attacked face picture and the original attacked face picture based on the pre-trained face recognition model, and generating the disturbance picture in a gradient iteration manner by combining a pre-generated mask with a preset area size includes:
inputting the original attacked face picture into a plurality of pre-trained face recognition models for feature extraction, and fusing the extracted features of the plurality of attacked faces to obtain a first fusion feature of the original attacked face picture;
inputting the original attack face picture into a plurality of pre-trained face recognition models for feature extraction to obtain a gradient in model parameters of the original attack face picture and a plurality of extracted attack face features, and fusing the extracted attack face features to obtain a second fusion feature of the original attack face picture;
computing a loss function based on the first and second fused features;
and generating a disturbance picture by adopting a gradient iteration mode according to the loss function and the gradient and combining a pre-generated mask with a preset area size.
Optionally, the step of generating a perturbation picture in a gradient iteration manner according to the loss function and the gradient and by combining a pre-generated mask with a preset area size includes:
calculating the mean value of the absolute values of the gradients of the original attack face pictures in all RGB channels;
calculating an average gradient according to the mean value;
updating the original attack face picture according to the average gradient and by combining a pre-generated mask with a preset area size to obtain an updated attack face picture;
carrying out normalization processing on the updated attack face picture;
updating the gradient of the updated attack face picture according to the loss function; and entering next iteration until the iteration times reach the preset times, and taking the finally updated attack face picture as a disturbance picture.
Optionally, before the step of inputting the original attack face image into a plurality of pre-trained face recognition models for feature extraction, the method further includes:
preprocessing the original attack face picture by adopting at least one of the following modes:
carrying out random transformation on the original attack face picture by adopting input diversity;
adding randomly generated tiny noise to the original attack face picture;
and carrying out affine transformation on the original attack face picture.
Optionally, the step of obtaining an original face picture includes:
obtaining an original character picture, wherein the original character picture comprises: an attacked figure picture and an attacking figure picture;
and carrying out face detection, alignment operation and normalization processing on the original figure picture to obtain an original face picture.
Optionally, the step of performing fusion feature learning on the original attacked face image and the original attacked face image based on the pre-trained face recognition model, and generating the disturbing image in a gradient iteration manner by combining a pre-generated mask with a preset area size further includes:
acquiring a plurality of face recognition models, and performing feature recognition on the face sample pictures by the face recognition models to obtain feature thermodynamic diagrams of the face recognition models;
analyzing the feature thermodynamic diagrams of the face recognition models to determine corresponding face feature sensitive areas;
and generating a mask with the size of a preset area region based on the human face feature sensitive region and a preset attack scene.
Optionally, the method further comprises:
and testing and verifying the face anti-attack sample through the test model to obtain a test result.
In addition, an embodiment of the present invention further provides a face confrontation sample generation apparatus, where the apparatus includes:
an obtaining module, configured to obtain an original face picture, where the original face picture includes: an original attacked face picture and an original attacked face picture;
the disturbance generation module is used for performing fusion feature learning on the original attacked face picture and the original attacked face picture based on a pre-trained face recognition model, and generating a disturbance picture in a gradient iteration mode by combining a pre-generated mask with a preset area size;
and the adding module is used for adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
In addition, the embodiment of the present invention further provides a face confrontation sample generation system, where the face confrontation sample generation system includes: a memory, a processor and a face confrontation sample generation program stored on the memory and executable on the processor, the face confrontation sample generation program when executed by the processor implementing the steps of the face confrontation sample generation method as described above.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where a face confrontation sample generation program is stored, and when executed by a processor, the face confrontation sample generation program implements the steps of the face confrontation sample generation method as described above.
The embodiment of the invention provides a method, a device, a system and a storage medium for generating a face confrontation sample, wherein an original face picture is obtained, and the original face picture comprises the following steps: an original attacked face picture and an original attacked face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. According to the scheme, the mask with the preset area size is generated in advance, the face counterattack can be completed by adding disturbance to the small area, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and therefore the implementation complexity of the face counterattack method is reduced. Compared with the traditional scheme, disturbance is often added to the whole picture, the time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
Drawings
Fig. 1 is a schematic view of functional modules of a terminal device to which a face confrontation sample generation apparatus of the present invention belongs;
FIG. 2 is a flowchart illustrating a first embodiment of a face confrontation sample generating method according to the invention;
FIG. 3 is a flowchart illustrating a second embodiment of a face countermeasure sample generation method according to the present invention;
FIG. 4 is a mask diagram of a face confrontation sample generation method according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a third embodiment of a face countermeasure sample generation method according to the present invention;
fig. 6 is a schematic overall flow chart of the embodiment of the face countermeasure sample generation method of the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The main solution of the embodiment of the invention is as follows: by obtaining an original face picture, the original face picture includes: an original attacked face picture and an original attacked face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. According to the scheme, the mask with the preset area size is generated in advance, the face counterattack can be completed by adding disturbance to the small area, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and therefore the implementation complexity of the face counterattack method is reduced. Compared with the traditional scheme, the method usually needs to add disturbance to the whole picture, time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
The embodiment of the invention considers that: although there are many methods for human face to fight against attacks in the industry at present, most of them achieve good effect in white box attack (knowing all information of the model), and generally represent in black box attack (unable to obtain the composition and specific parameters of the model). In addition, most face anti-attack methods add disturbance to the whole picture in a query mode to improve the attack success rate, but introduce higher computational complexity.
Based on this, the embodiment of the invention provides a solution, which can reduce the implementation complexity of the face anti-attack method.
Specifically, referring to fig. 1, fig. 1 is a schematic diagram of functional modules of a terminal device to which the face confrontation sample generating device of the present invention belongs. The face confrontation sample generation device can be a device independent from the terminal device, and can be borne on the terminal device or the system in a hardware or software mode. The terminal device can be an intelligent mobile terminal such as a mobile phone and a tablet personal computer, and can also be a network device such as a server.
In this embodiment, the terminal device to which the face countermeasure sample generation apparatus belongs at least includes an output module 110, a processor 120, a memory 130 and a communication module 140.
The memory 130 stores an operating system and a face countermeasure sample generation program; the output module 110 may be a display screen, a speaker, etc. The communication module 140 may include a WIFI module, a mobile communication module, a bluetooth module, and the like, and communicates with an external device or a server through the communication module 140.
As an embodiment, the face countermeasure sample generation program in the memory 130 implements the following steps when executed by the processor:
obtaining an original face picture, wherein the original face picture comprises: an original attacked face picture and an original attacked face picture;
based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode;
and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
Further, the face countermeasure sample generation program in the memory 130 when executed by the processor further implements the steps of:
inputting the original attacked face picture into a plurality of pre-trained face recognition models for feature extraction, and fusing the extracted features of the plurality of attacked faces to obtain a first fusion feature of the original attacked face picture;
inputting the original attack face picture into a plurality of pre-trained face recognition models for feature extraction to obtain a gradient in model parameters of the original attack face picture and a plurality of extracted attack face features, and fusing the extracted attack face features to obtain a second fusion feature of the original attack face picture;
computing a loss function based on the first and second fused features;
and generating a disturbance picture by adopting a gradient iteration mode according to the loss function and the gradient and by combining a pre-generated mask with a preset area size.
Further, the face countermeasure sample generation program in the memory 130 when executed by the processor further implements the steps of:
calculating the mean value of the absolute values of the gradients of the original attack face pictures in all RGB channels;
calculating an average gradient according to the mean value;
updating the original attack face picture according to the average gradient and by combining a pre-generated mask with a preset area size to obtain an updated attack face picture;
carrying out normalization processing on the updated attack face picture;
updating the gradient of the updated attack face picture according to the loss function; and entering next iteration until the iteration times reach the preset times.
Further, the face confrontation sample generation program in the memory 130 when executed by the processor further implements the steps of:
preprocessing the original attack face picture by adopting at least one of the following modes:
randomly transforming the original attack face picture by adopting input diversity;
adding small noise generated randomly to the original attack face picture;
and carrying out affine transformation on the original attack face picture.
Further, the face confrontation sample generation program in the memory 130 when executed by the processor further implements the steps of:
obtaining an original character picture, wherein the original character picture comprises: an attacked figure picture and an attacking figure picture;
and carrying out face detection, alignment operation and normalization processing on the original figure picture to obtain an original face picture.
Further, the face confrontation sample generation program in the memory 130 when executed by the processor further implements the steps of:
acquiring a plurality of face recognition models, and performing feature recognition on the face sample pictures by the face recognition models to obtain feature thermodynamic diagrams of the face recognition models;
analyzing the characteristic thermodynamic diagrams of the plurality of face recognition models, and determining corresponding face characteristic sensitive areas;
and generating a mask with the size of a preset area region based on the human face feature sensitive region and a preset attack scene.
Further, the face confrontation sample generation program in the memory 130 when executed by the processor further implements the steps of:
and testing and verifying the face anti-attack sample through the test model to obtain a test result.
By the above scheme, in this embodiment, an original face picture is specifically obtained, where the original face picture includes: an original attacked face picture and an original attacked face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. According to the scheme, the mask with the preset area size is generated in advance, the face counterattack can be completed by adding disturbance to the small area, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and therefore the implementation complexity of the face counterattack method is reduced. Compared with the traditional scheme, the method usually needs to add disturbance to the whole picture, time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
Based on the above terminal device architecture, but not limited to the above architecture, the method embodiment of the present invention is proposed.
The execution subject of the method of this embodiment may be a face confrontation sample generation apparatus, which may be an apparatus independent from the terminal device, and may be borne on the terminal device or system in the form of hardware or software. The terminal equipment can be an intelligent mobile terminal such as a mobile phone and a tablet personal computer, and can also be network equipment such as a server.
The embodiment of the face confrontation sample generation method can be applied to face confrontation attack methods of various face recognition scenes so as to reduce the implementation complexity of the face confrontation attack method and improve the safety of a face recognition algorithm through the face confrontation attack.
Referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a face confrontation sample generating method according to the invention.
As shown in fig. 2, a method for generating a face confrontation sample according to an embodiment of the present invention includes the following steps:
step S101, obtaining an original face picture, wherein the original face picture comprises: an original attacked face picture and an original attacked face picture;
the original face picture is a face picture obtained by pre-collecting and processing, and can be extracted from a figure picture, or can be obtained from each network platform or various face recognition scenes, or can be obtained from each face recognition scene.
The original face picture comprises: an original attacked face picture and an original attacking face picture. And generating a face countermeasure sample by using the original attacked face picture and adopting a preset countermeasure generation strategy.
Specifically, as an embodiment, the step of acquiring an original face picture may include:
obtaining an original character picture, wherein the original character picture comprises: an attacked figure picture and an attacking figure picture;
and carrying out face detection, alignment operation and normalization processing on the original figure picture to obtain an original face picture.
Specifically, after the original character picture is obtained, data preprocessing may be performed on the original character picture.
The data preprocessing is an indispensable composition step for improving the network training effect, and the accuracy of processing the face confrontation sample is further improved.
In this embodiment, the entire person image is input, so that face detection and face alignment operations are usually required. The face detection can be performed by adopting Retinaface in the embodiment. Meanwhile, the image data is normalized to improve the network convergence speed.
In addition, for a black box attack scene, the present embodiment may adopt input diversity (input diversity) to perform random transformation on the input picture at each iteration, so as to reduce overfitting and improve generalization of the attack. In addition, the following processing scheme is also adopted in the embodiment: under a certain theory, the input picture is added with small noise generated randomly, and affine transformation such as translation and rotation is carried out on the input picture.
In the preprocessing scheme, input diversity (input diversity) is mainly used for an attack character picture, random transformation is performed on the picture input in each iteration, fine noise generated randomly is added to the input picture, and affine transformation such as translation and rotation is performed on the input picture.
Step S102, based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacked face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode;
in the scheme of the embodiment, an artificial intelligence deep learning technology is mainly utilized, fusion feature learning is carried out through a pre-trained face recognition model, a disturbance picture with the same size as a mask is obtained through continuous updating of gradients, and then the disturbance picture can be added to an attacker picture to generate a counterattack picture, namely a face counterattack sample.
In the embodiment, the disturbance adding manner is performed by adopting a gradient iteration manner and a fusion attack manner, wherein the fusion attack is mainly performed by fusing features of the face sample image recognition by using a plurality of face recognition models (such as facenet, mobility, and the like) so as to improve the generalization of the black box attack.
Specifically, as an implementation manner, firstly, inputting the original attacked face picture into a plurality of pre-trained face recognition models for feature extraction, and fusing the extracted features of the plurality of attacked faces to obtain a first fusion feature of the original attacked face picture;
inputting the original attack face picture into a plurality of pre-trained face recognition models for feature extraction to obtain a gradient in model parameters of the original attack face picture and a plurality of extracted attack face features, and fusing the extracted attack face features to obtain a second fusion feature of the original attack face picture;
computing a loss function based on the first and second fused features;
and generating a disturbance picture by adopting a gradient iteration mode according to the loss function and the gradient and combining a pre-generated mask with a preset area size.
The step of generating the perturbation image in a gradient iteration manner according to the loss function and the gradient and by combining a pre-generated mask with a preset area size may include:
calculating the mean value of the absolute values of the gradients of the original attack face pictures in all RGB channels;
calculating an average gradient according to the mean value;
updating the original attack face picture according to the average gradient and by combining a pre-generated mask with a preset area size to obtain an updated attack face picture;
normalizing the updated attack face picture;
updating the gradient of the updated attack face picture according to the loss function; and entering next iteration until the iteration times reach the preset times. And obtaining a disturbance picture based on the finally updated attack face picture.
More specifically, as an embodiment, the step of gradient iteration generated by perturbation may be as follows:
graph input: attacked face picture I vic Attack face picture I att Mask I, mask I mask And fusing the model M.
Step 1, initializing an input picture and obtaining an attacked face picture I vic Input fusion model M to obtain fusion feature f 1 The fusion model M comprises a plurality of pre-trained face recognition models, features of an input picture are extracted through the face recognition models, and the features extracted by the face recognition models are fused to obtain fused features;
step 2, starting circulation;
step 3, attack face picture I att Adding input diversity (input diversity) and random transformations;
step 4, processing the attack face picture I att Inputting into a fusion model M to obtain I att Gradient g and fusion feature f of 2 ；
Step 5, calculating a loss function L =1-F cos (f 1 ,f 2 ) In which F cos (f 1 ,f 2 ) Representing cosine similarity of the two;
step 6, solving the absolute value of each RGB channel and the mean value to obtain g', then calculating the average gradient
Step 7, updating the disturbance attack graph
Wherein F (×) represents a custom sign function, λ =1.0/255;
step 8, adding I att Is mapped to [ -1,1 ] values]Namely, carrying out picture normalization processing;
step 9, updating I according to the loss function att And (4) entering next iteration until the iteration times reach the preset times.
And finally, obtaining an updated attack face picture, thereby generating a disturbance picture.
For example, as shown in fig. 6, the model 1, the model 2, the model 3, and the model 4 may be respectively: IRSE50, IR101, facenet, mobileface: the different face recognition models that are commonly used,
and step S103, adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
And after obtaining the disturbance picture, adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
According to the scheme, the embodiment specifically obtains the original face picture, and the original face picture includes: an original attacked face picture and an original attacking face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. According to the scheme, the mask with the size of the pre-generated preset area region is adopted, the face confrontation attack can be completed by adding disturbance to the small region, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and therefore the implementation complexity of the face confrontation attack method is reduced. Compared with the traditional scheme, the method usually needs to add disturbance to the whole picture, time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
Referring to fig. 3, fig. 3 is a flowchart illustrating a second embodiment of the face countermeasure sample generation method according to the present invention.
As shown in fig. 3, based on the embodiment shown in fig. 2, in the step S102, before performing fusion feature learning on the original attacked face image and the original attacked face image based on the pre-trained face recognition model, and generating a perturbed image by using a gradient iteration method in combination with a pre-generated mask with a preset area size, the embodiment of the present invention further includes:
step S80, obtaining a plurality of face recognition models, and carrying out feature recognition on face sample pictures to obtain feature thermodynamic diagrams of the plurality of face recognition models;
s90, analyzing the characteristic thermodynamic diagrams of the plurality of face recognition models to determine corresponding face characteristic sensitive areas;
and S100, generating a mask with a preset area region size based on the human face feature sensitive region and a preset attack scene.
Compared with the embodiment shown in fig. 2, the present embodiment further includes a scheme of generating a mask with a preset area size.
Specifically, the mask is used as an important basis for perturbation attack and is mainly obtained by analyzing a face sensitive area, that is, analyzing a feature thermodynamic diagram of a plurality of face recognition models to obtain the mask.
The human face feature sensitive regions are mainly three regions of human eyes, nose and mouth, as shown in fig. 4. Through a large number of experimental comparisons, the area weight ratio influencing the face feature information is as follows: eyes > nose > mouth. That is, for smaller disturbance, the effect of using the eye region as a mask is slightly better than that of using the nose region as a mask, and fig. 4 a, b, c and d respectively show the schematic effects of the combined disturbance area ratios of four different regions.
In addition, only the upper and lower regions of the eye region are reserved, and similar disturbance effects can still be reserved by removing the middle part. The disturbance effect is basically consistent as in the graph a and the graph d in fig. 4, but the disturbance area occupation ratio of the graph d is obviously smaller.
The size of the area region of the mask may be determined according to an actual attack scenario, for example: in a white-box attack scenario, the mask may be scaled down appropriately due to knowledge of the model parameters. And the black box attacks the scene, and the attack difficulty is higher, then the mask area needs to be increased properly.
The scheme of the embodiment can generate the face confrontation sample based on a small-region perturbation patch mode (namely a small-region mask perturbation patch mode). Compared with the traditional scheme, disturbance is often required to be added to the whole picture, the time complexity of the generation method is high, and the method for realizing the attack only through small-area disturbance is provided in the embodiment.
The overall flow of this embodiment can be seen with reference to fig. 6. In the embodiment, an artificial intelligence deep learning technology is utilized, fusion feature learning is carried out through a pre-trained face recognition model, a disturbance picture with the same size as a small-area mask is obtained by continuously updating a gradient, the disturbance is added to an attacker picture, and a disturbance attack picture sample is generated. As shown in fig. 6, the core of the scheme of the present embodiment can be divided into three parts, namely data preprocessing, mask generation and disturbance generation.
Therefore, the mask with the preset area region size is adopted, the face confrontation attack can be completed by adding disturbance to the small region, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and the realization complexity of the face confrontation attack method is further reduced. Compared with the traditional scheme, disturbance is often added to the whole picture, the time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
Referring to fig. 5, fig. 5 is a flowchart illustrating a second embodiment of the face countermeasure sample generation method according to the present invention.
As shown in fig. 5, based on the embodiment shown in fig. 3, the method further includes:
and step S104, testing and verifying the face anti-attack sample through the test model to obtain a test result.
Compared with the embodiment shown in fig. 3, the embodiment further includes a scheme of performing test verification on the generated human face anti-attack sample.
Specifically, referring to fig. 6, a corresponding test model, such as model 5, may be configured, and the face is subjected to test verification against the attack sample through the test model. The test model may specifically employ IR50.
When the test model tests and verifies the face anti-attack sample, the comparison test can be carried out by combining the original attacked picture.
Specifically, a face counterattack sample and an original attacked picture are respectively input into a test model, the face counterattack sample is tested and verified through the test model, and compared with the output of the original attacked picture to obtain a loss value (Lcos), which can be obtained through cosine similarity of the face counterattack sample and the original attacked picture, so that the generalization and robustness of the face counterattack sample can be detected through testing and verifying the face counterattack sample, and further, relevant parameters in a face counterattack sample generation algorithm can be updated according to the test and verification result, and the accuracy, the generalization and the robustness of the face counterattack sample are improved.
It should be noted that, the above embodiments may be reasonably combined according to actual situations, and are not described in detail herein.
In addition, an embodiment of the present invention further provides a face confrontation sample generation apparatus, where the apparatus includes:
an obtaining module, configured to obtain an original face picture, where the original face picture includes: an original attacked face picture and an original attacking face picture;
the disturbance generation module is used for performing fusion feature learning on the original attacked face picture and the original attacked face picture based on a pre-trained face recognition model, and generating a disturbance picture in a gradient iteration mode by combining a pre-generated mask with a preset area size;
and the adding module is used for adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
The embodiment of the present invention may refer to the above embodiments, and details are not described herein.
In addition, the embodiment of the present invention further provides a face confrontation sample generation system, which is characterized in that the face confrontation sample generation system includes: the system comprises a memory, a processor and a face confrontation sample generation program stored on the memory and capable of running on the processor, wherein the face confrontation sample generation program realizes the face confrontation sample generation method according to the embodiment when being executed by the processor.
The principle of generating the face confrontation sample according to the embodiment of the present invention can refer to the above embodiments, and will not be described herein again.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where a face confrontation sample generation program is stored on the computer-readable storage medium, and when executed by a processor, the face confrontation sample generation program implements the face confrontation sample generation method according to the foregoing embodiment.
The embodiment of the present invention may refer to the above embodiments, and details are not described herein.
Compared with the prior art, the method, the device, the system and the storage medium for generating the face confrontation sample provided by the embodiment of the invention have the advantages that the original face picture is obtained, and the original face picture comprises the following steps: an original attacked face picture and an original attacking face picture; based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode; and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample. According to the scheme, the mask with the preset area size is generated in advance, the face counterattack can be completed by adding disturbance to the small area, the disturbance added to the whole picture is avoided, the scheme is suitable for white box attack and black box attack at the same time, query and model migration are not needed, and therefore the implementation complexity of the face counterattack method is reduced. Compared with the traditional scheme, disturbance is often added to the whole picture, the time complexity is high, and the method for generating the face confrontation sample can realize the attack method only through small-area disturbance.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a recommendation effect evaluation system (e.g., a mobile phone, a computer, a server, a controlled terminal, or a network device) to execute the method of each embodiment of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A face confrontation sample generation method is characterized by comprising the following steps:
obtaining an original face picture, wherein the original face picture comprises: an original attacked face picture and an original attacked face picture;
based on a pre-trained face recognition model, performing fusion feature learning on the original attacked face picture and the original attacking face picture, and generating a disturbance picture by combining a pre-generated mask with a preset area size in a gradient iteration mode;
and adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
2. The method according to claim 1, wherein the step of performing fusion feature learning on the original attacked face picture and the original attacked face picture based on the pre-trained face recognition model, and generating the disturbing picture in a gradient iteration mode by combining a pre-generated mask with a preset area size comprises:
inputting the original attacked face picture into a plurality of pre-trained face recognition models for feature extraction, and fusing the extracted features of the plurality of attacked faces to obtain a first fusion feature of the original attacked face picture;
inputting the original attack face picture into a plurality of pre-trained face recognition models for feature extraction to obtain gradients in model parameters of the original attack face picture and a plurality of extracted attack face features, and fusing the extracted attack face features to obtain a second fusion feature of the original attack face picture;
computing a loss function based on the first and second fused features;
and generating a disturbance picture by adopting a gradient iteration mode according to the loss function and the gradient and combining a pre-generated mask with a preset area size.
3. The method according to claim 2, wherein the step of generating the perturbation image by a gradient iteration method according to the loss function and the gradient and combining a pre-generated mask with a preset area size comprises:
calculating the mean value of the absolute values of the gradients of the original attack face pictures in all RGB channels;
calculating an average gradient according to the mean value;
updating the original attack face picture according to the average gradient and by combining a pre-generated mask with a preset area size to obtain an updated attack face picture;
normalizing the updated attack face picture;
updating the gradient of the updated attack face picture according to the loss function; and entering next iteration until the iteration times reach the preset times, and taking the finally updated attack face picture as a disturbance picture.
4. The method of claim 2, wherein the step of inputting the original attack face picture into a plurality of pre-trained face recognition models for feature extraction is preceded by the step of:
preprocessing the original attack face picture by adopting at least one of the following modes:
carrying out random transformation on the original attack face picture by adopting input diversity;
adding small noise generated randomly to the original attack face picture;
and carrying out affine transformation on the original attack face picture.
5. The method of claim 1, wherein the step of obtaining the original face picture comprises:
obtaining an original character picture, wherein the original character picture comprises: an attacked figure picture and an attacking figure picture;
and carrying out face detection, alignment operation and normalization processing on the original figure picture to obtain an original face picture.
6. The method according to any one of claims 1 to 5, wherein the step of performing fusion feature learning on the original attacked face picture and the original attacked face picture based on the pre-trained face recognition model, and generating the disturbed picture in a gradient iteration manner by combining a pre-generated mask with a preset area size further comprises:
acquiring a plurality of face recognition models, and performing feature recognition on the face sample pictures by the face recognition models to obtain feature thermodynamic diagrams of the face recognition models;
analyzing the characteristic thermodynamic diagrams of the plurality of face recognition models, and determining corresponding face characteristic sensitive areas;
and generating a mask with the size of a preset area region based on the human face feature sensitive region and a preset attack scene.
7. The method of claim 6, further comprising:
and testing and verifying the face anti-attack sample through the test model to obtain a test result.
8. A face confrontation sample generation apparatus, wherein the apparatus comprises:
an obtaining module, configured to obtain an original face picture, where the original face picture includes: an original attacked face picture and an original attacked face picture;
the disturbance generation module is used for performing fusion feature learning on the original attacked face picture and the original attacked face picture based on a pre-trained face recognition model, and generating a disturbance picture in a gradient iteration mode by combining a pre-generated mask with a preset area size;
and the adding module is used for adding the disturbance picture into the original attack face picture to generate a face anti-attack sample.
9. A face confrontation sample generation system, characterized in that the face confrontation sample generation system comprises: a memory, a processor and a face confrontation sample generation program stored on the memory and executable on the processor, the face confrontation sample generation program when executed by the processor implementing the steps of the face confrontation sample generation method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that a face confrontation sample generation program is stored on the computer-readable storage medium, and when executed by a processor, the face confrontation sample generation program implements the steps of the face confrontation sample generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211287314.5A CN115798056A (en) | 2022-10-20 | 2022-10-20 | Face confrontation sample generation method, device and system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211287314.5A CN115798056A (en) | 2022-10-20 | 2022-10-20 | Face confrontation sample generation method, device and system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115798056A true CN115798056A (en) | 2023-03-14 |
Family
ID=85433316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211287314.5A Pending CN115798056A (en) | 2022-10-20 | 2022-10-20 | Face confrontation sample generation method, device and system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115798056A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116259098A (en) * | 2023-05-10 | 2023-06-13 | 南京理工大学 | Feature attention-based migration face recognition attack resistance method and device |
| CN117496582A (en) * | 2023-12-28 | 2024-02-02 | 苏州元脑智能科技有限公司 | Face recognition model training method and device, electronic equipment and storage medium |
-
2022
- 2022-10-20 CN CN202211287314.5A patent/CN115798056A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116259098A (en) * | 2023-05-10 | 2023-06-13 | 南京理工大学 | Feature attention-based migration face recognition attack resistance method and device |
| CN116259098B (en) * | 2023-05-10 | 2023-07-25 | 南京理工大学 | Feature attention-based migration face recognition attack resistance method and device |
| CN117496582A (en) * | 2023-12-28 | 2024-02-02 | 苏州元脑智能科技有限公司 | Face recognition model training method and device, electronic equipment and storage medium |
| CN117496582B (en) * | 2023-12-28 | 2024-04-16 | 苏州元脑智能科技有限公司 | Face recognition model training method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111340008B (en) | Method and system for generation of counterpatch, training of detection model and defense of counterpatch | |
| US11995155B2 (en) | Adversarial image generation method, computer device, and computer-readable storage medium | |
| CN115798056A (en) | Face confrontation sample generation method, device and system and storage medium | |
| US10275672B2 (en) | Method and apparatus for authenticating liveness face, and computer program product thereof | |
| CN111626925B (en) | Method and device for generating counterwork patch | |
| WO2019152983A2 (en) | System and apparatus for face anti-spoofing via auxiliary supervision | |
| CN112364745B (en) | Method and device for generating countermeasure sample and electronic equipment | |
| CN106650615B (en) | A kind of image processing method and terminal | |
| CN113420731B (en) | Model training method, electronic device and computer-readable storage medium | |
| CN111931153B (en) | Identity verification method and device based on artificial intelligence and computer equipment | |
| CN112001285B (en) | Method, device, terminal and medium for processing beauty images | |
| CN111723707A (en) | Method and device for estimating fixation point based on visual saliency | |
| CN111178146A (en) | Method and device for identifying anchor based on face features | |
| CN113435264A (en) | Face recognition attack resisting method and device based on black box substitution model searching | |
| EP3842990A1 (en) | Face recognition method and device | |
| Sabeena et al. | Digital image forensic using deep flower pollination with adaptive Harris hawk optimization | |
| CN113255575A (en) | Neural network training method and device, computer equipment and storage medium | |
| Stamm et al. | Anti-forensic attacks using generative adversarial networks | |
| CN117975519A (en) | Model training and image generating method and device, electronic equipment and storage medium | |
| CN116958306A (en) | Image synthesis method and device, storage medium and electronic equipment | |
| CN115410257A (en) | Image protection method and related equipment | |
| CN114373215A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN114038030A (en) | Image tampering identification method, device and computer storage medium | |
| CN114331791A (en) | Model watermark generation method, model infringement identification method, model watermark generation device, model infringement identification device and computer equipment | |
| Lau et al. | Attribute-guided encryption with facial texture masking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |