CN114882323B

CN114882323B - Countermeasure sample generation method and device, electronic equipment and storage medium

Info

Publication number: CN114882323B
Application number: CN202210797001.8A
Authority: CN
Inventors: 王瑶; 刘闯; 胡峻毅; 叶雨桐; 陈诗昱
Original assignee: Sixth Mirror Technology Beijing Group Co ltd
Current assignee: Hebei Sixth Mirror Intelligent Technology Co.,Ltd.
Priority date: 2022-07-08
Filing date: 2022-07-08
Publication date: 2022-10-14
Anticipated expiration: 2042-07-08
Also published as: CN114882323A

Abstract

The invention relates to the technical field of artificial intelligence, and provides a method and a device for generating a confrontation sample, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an original image; disturbance noise generated based on multi-dimensional Gaussian distribution is added to the original image to obtain a countermeasure sample of the original image, the countermeasure sample provides unexpected output of the preset image recognition model, and the original image can provide correct output of the preset image recognition model. According to the method, the countermeasure sample of the original image is obtained by adding the disturbance noise generated based on the multidimensional Gaussian distribution to the original image, the countermeasure sample can be obtained more efficiently and more quickly, and the robustness verification efficiency of the preset image recognition model is further improved.

Description

Confrontation sample generation method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a method and a device for generating a confrontation sample, electronic equipment and a storage medium.

Background

The neural network like the black box has a high-precision prediction function and stable network performance, and is widely applied to various fields, for example, in an image recognition model, the neural network has excellent performance and can recognize a specified target image with high precision.

With the wide application of image recognition models, a series of security and privacy information problems are brought about. For example, an image recognition model is subjected to a confounding attack that results in the target not matching the gallery, or an impersonation attack that matches a non-target gallery.

How to guarantee the robustness of the image recognition model and effectively resist various attacks on the image recognition model is an urgent problem to be solved by technical personnel in the field.

Disclosure of Invention

The invention aims to provide a method and a device for generating a countermeasure sample, electronic equipment and a storage medium, wherein the countermeasure sample of an original image is obtained by adding disturbance noise generated based on multi-dimensional Gaussian distribution to the original image, so that the robustness of a preset image recognition model is verified by utilizing the countermeasure sample.

In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:

in a first aspect, an embodiment of the present invention provides a countermeasure sample generation method, where the method includes: acquiring an original image; and adding disturbance noise generated based on multi-dimensional Gaussian distribution to the original image to obtain an antagonistic sample of the original image, wherein the antagonistic sample provides unexpected output of a preset image recognition model, and the original image can provide correct output of the preset image recognition model.

Optionally, the step of adding disturbance noise generated based on a multidimensional gaussian distribution to the original image includes:

generating disturbance noise data satisfying the multi-dimensional Gaussian distribution;

adding disturbance noise to the original image by using the disturbance noise data to obtain a primary selection sample;

if the initially selected sample does not meet a preset optimization condition, returning to the step of generating disturbance noise data meeting the multidimensional Gaussian distribution, wherein the preset optimization condition is determined based on an initial temperature;

if the initially selected sample meets the preset optimization condition and meets a preset termination condition, taking the initially selected sample as the confrontation sample;

and if the initially selected sample meets the preset optimization condition and does not meet the preset termination condition, adjusting the initial temperature, and returning to the step of generating disturbance noise data based on the multidimensional Gaussian distribution until the initially selected sample meets the preset termination condition to obtain the countermeasure sample.

Optionally, the disturbance noise data is a disturbance noise matrix of M1 × N1, the original image is an image of M2 × N2, and M1< M2 and/or N1< N2, and the step of adding disturbance noise to the original image by using the disturbance noise data to obtain an initial selection sample includes:

adjusting the disturbance noise matrix by using a double-line interpolation value to enable M1= M2 and N1= N2;

and correspondingly superposing each element in the adjusted disturbance noise matrix on the pixel value of each pixel point in the original image to obtain the primary selection sample.

Optionally, the method further comprises:

for the primary selection sample of the ith time, obtaining a first similarity of the ith-1 time, wherein the first similarity is the similarity between the primary selection sample of the ith-1 time and the original image, and i is an integer greater than 0;

calculating a second similarity between the primary selection sample of the ith time and the original image;

judging whether the initially selected sample meets an acceptance condition or not according to the first similarity and the second similarity;

if the primary selection sample meets the acceptance condition and i is less than a preset number of times, or the primary selection sample does not meet the acceptance condition, judging that the primary selection sample does not meet the preset optimization condition;

and if the primary selection sample meets the acceptance condition and i is not less than the preset times, judging that the primary selection sample meets the preset optimization condition.

Optionally, the step of judging whether the preliminary selection sample meets an acceptance condition according to the first similarity and the second similarity includes:

if the second similarity is smaller than the first similarity, judging that the initially selected sample meets the acceptance condition;

and if the second similarity is not less than the first similarity, determining whether the initially selected sample meets the acceptance condition or not according to the first similarity, the second similarity, the initial temperature and a temperature control factor.

Optionally, the step of determining whether the initially selected sample satisfies the acceptance condition according to the first similarity, the second similarity, the initial temperature and a temperature control factor includes:

generating random numbers which meet the uniform distribution, and taking the random numbers as reference probabilities;

calculating the acceptance probability for representing the receptivity of the initially selected sample according to the first similarity, the second similarity, the initial temperature and the temperature control factor;

if the acceptance probability is not less than the reference probability, judging that the initially selected sample meets the acceptance condition;

and if the acceptance probability is smaller than the reference probability, judging that the initially selected sample does not meet the acceptance condition.

Optionally, the adjusting the initial temperature includes:

if the initial temperature is larger than a preset threshold value, adjusting the initial temperature according to a first adjusting factor;

if the initial temperature is not larger than the preset threshold, adjusting the initial temperature according to a second adjustment factor, wherein the second adjustment factor is smaller than the first adjustment factor.

In a second aspect, an embodiment of the present invention provides a challenge sample generation apparatus, including:

the acquisition module is used for acquiring an original image; the generation module is used for adding disturbance noise generated based on multi-dimensional Gaussian distribution to the original image to obtain an antagonistic sample of the original image, the antagonistic sample provides unexpected output of a preset image recognition model, and the original image can provide correct output of the preset image recognition model.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory; the memory is used for storing programs; the processor is configured to implement the confrontational sample generation method of the first aspect when executing the program.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the confrontation sample generation method according to the first aspect.

Compared with the prior art, the method, the device, the electronic equipment and the storage medium for generating the countermeasure sample provided by the embodiment of the invention have the advantages that the countermeasure sample of the original image is obtained by adding the disturbance noise generated based on the multidimensional Gaussian distribution to the original image, the countermeasure sample provides unexpected output of the preset image recognition model, and the original image can provide correct output of the preset image recognition model, so that the robustness of the preset image recognition model can be verified by using the countermeasure sample.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a first flowchart illustrating a method for generating a challenge sample according to an embodiment of the present invention.

Fig. 2 is a flowchart of a challenge sample generation method according to an embodiment of the present invention.

Fig. 3 is an exemplary diagram of cosine similarity change of an confusion attack Sphereface model under a given number of queries according to an embodiment of the present invention.

Fig. 4 is an exemplary diagram of cosine similarity variation of an impersonation attack sphere model for a given number of queries according to the embodiment of the present invention.

Fig. 5 is an exemplary diagram of cosine similarity variation of a confusion attack Cosface model under a given number of queries according to an embodiment of the present invention.

Fig. 6 is an exemplary diagram of cosine similarity variation of an impersonation attack Cosface model given query times according to an embodiment of the present invention.

Fig. 7 is an exemplary diagram of cosine similarity variation of an Arcface model for confusion attack at a given number of queries according to an embodiment of the present invention.

Fig. 8 is an exemplary diagram of cosine similarity variation of an arcfacce model for impersonation attack at a given number of queries according to an embodiment of the present invention.

Fig. 9 is a block diagram of a challenge sample generation device provided by an embodiment of the present invention.

Fig. 10 shows a block schematic diagram of an electronic device provided by an embodiment of the present invention.

An icon: 10-an electronic device; 11-a processor; 12-a memory; 13-a bus; 100-challenge sample generation means; 110-an obtaining module; 120-generating module.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

In the description of the present invention, it should be noted that, if the terms "upper", "lower", "inner", "outer", etc. are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which the product of the present invention is used to usually place, it is only for convenience of description and simplification of the description, but it is not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.

Furthermore, the appearances of the terms "first," "second," and the like, if any, are only used to distinguish one description from another and are not to be construed as indicating or implying relative importance.

It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.

In order to improve the robustness of the target identification system, there are two main types of methods for attacking the target identification system: white box attacks and black box attacks.

The white box attack is a clear target model for an attacker, and information such as gradient, network weight, parameters and the like can be used for optimizing a target function or adding tiny malicious disturbance to attack in the attack process, so that the state of knowing one another and being a hundred of victory is achieved. However, since the attack needs to have a deeper understanding of the target model, the attack is mostly used for the robustness research of the early target recognition system, and is difficult to be applied to the real situation.

In order to overcome the defect of white box attack, a black box attack mode is provided, and the precondition of the black box attack is as follows: the attacker has no knowledge of the target model and thus is more in line with the real scene.

A common implementation of black box attacks is: by training a proxy model that functions similarly to the target model, i.e., under the same input query, a prediction similar to the target model is obtained. Such an approach typically enables learning of an agent model from a target model, attempts a white-box attack on it, and then can transfer to the target model. But the training of one agent model has good performance, and meanwhile, the attack of the transferable agent model needs large-scale and diversified training data to ensure the attack success rate.

Another common implementation of black box attacks is: by simulating the internal structure of the target model, construction as a gradient is a common approach. For example, a pseudo gradient is constructed by using symmetric difference, an objective function is expressed as a continuous optimization problem based on an optimization method, and the gradient is estimated based on the continuous optimization problem, but the method needs a large amount of calculation consumption in an attack process.

In order to overcome the defect that the robustness verification of a target model is not efficient due to a large amount of calculation in the two black box attack implementation modes, embodiments of the present invention provide a method, an apparatus, an electronic device, and a storage medium for generating a countermeasure sample.

The inventor finds through research that the image changed in this way is not perceptible to human eyes by adding carefully designed slight malicious disturbance to the original clean sample or changing the structure of the input sample. Specifically, it can be expressed as: for example, the original clean sample is x, and the neural network with good training effect is obtained

Adding as little as possible, imperceptible to the human eye, perturbations on x

So that

Then the challenge sample that generated the original clean sample can be described as the following optimization problem:

namely at the satisfaction of

Under the constraint of (2) such that

If so, the process of generating the challenge sample also translates into a process of searching for an acceptable optimal solution.

Referring to fig. 1, fig. 1 is a first flowchart illustrating a method for generating a challenge sample according to an embodiment of the present invention, the method including the following steps:

step S101, an original image is acquired.

In this embodiment, the original image may be an image including a target to be recognized, and an image correctly output by a preset image recognition model may be provided, for example, the target to be recognized is a human face, the original image is a human face image, and the preset image recognition model is a preset human face recognition model, then the countermeasure sample of the generated human face image may verify robustness of the preset human face recognition model, and of course, the target to be recognized may also be another object, and correspondingly, the preset image recognition model is a preset object recognition model.

Step S102, disturbance noise generated based on multi-dimensional Gaussian distribution is added to the original image to obtain a confrontation sample of the original image, the confrontation sample provides unexpected output of the preset image recognition model, and the original image can provide correct output of the preset image recognition model.

In this embodiment, the countermeasure sample may be obtained by attempting to add disturbance noise to the original image, where the number of attempts may be 1 or more, and finally the countermeasure sample of the original image that meets the preset condition is obtained, where the number of attempts is multiple, the disturbance noise that is added in each attempt may be different, but the disturbance noise that is added in each attempt is generated based on a multidimensional gaussian distribution, and the disturbance noise that is added in each time may be disturbance noise of one multidimensional gaussian distribution, or superimposed on the basis of the disturbance noise of the last time, or obtained by performing range-limited preprocessing on the disturbance noise of one multidimensional gaussian distribution, or obtained by performing range-limited preprocessing before or after the superimposition on the basis of the disturbance noise of the last time. Therefore, the process of obtaining the countermeasure sample can be understood as a process of searching for an optimal solution based on the disturbance noise generated by the multidimensional gaussian distribution, and the optimal solution is the countermeasure sample.

According to the method provided by the embodiment of the invention, the countermeasure sample of the original image is generated by adding the disturbance noise generated based on the multi-dimensional Gaussian distribution into the original image, so that the specific structure of the preset image recognition model is not required to be known, any parameter of the preset image recognition model is not required to be accessed, a large amount of calculation required by training is avoided, the countermeasure sample can be obtained more efficiently and more quickly, and the robustness verification efficiency of the preset image recognition model is further improved.

On the basis of fig. 1, an embodiment of the present invention further provides a specific implementation manner of adding disturbance noise, please refer to fig. 2, fig. 2 is a second flowchart of a method for generating a countermeasure sample according to an embodiment of the present invention, and step S102 includes the following sub-steps:

in sub-step S1021, disturbance noise data satisfying a multidimensional gaussian distribution is generated.

In this embodiment, as a specific implementation, the noise data is disturbed

May be obeyed with a mean of 0 and covariance as an identity matrix

Is given by the multidimensional gaussian normal distribution Ν (0,

) In addition, in order to control the degree of disturbance of the disturbance noise data to the original image, the size of the generated disturbance noise data may be limited, for example, -u ≦

U, u may be set as desired, e.g. u =0.12, representing a vector

Each element in (b) has a value of [ -u, u]In the meantime. As a specific implementation manner, the preset coefficients can also be adopted simultaneously

The size of the disturbing noise data is limited, for example,

at this time, the disturbance noise data can be used

Thus obtaining the product.

And a substep S1022, adding disturbance noise to the original image by using the disturbance noise data to obtain a primary selection sample.

In this embodiment, adding the disturbance noise to the original image may be implemented by superimposing disturbance noise data on pixel values of pixels of the original image, and as a specific implementation manner, the disturbance noise data may be a matrix of M1 × N1, the original image is an image of M2 × N2, M1 may be less than or equal to M2, and N1 may be less than or equal to N2.

In order to reduce the amount of processed data and speed up the processing process, M1 may be set to be smaller than M2, and N1 may be set to be smaller than N2, in which case, the implementation manner of adding disturbance noise to the original image may be:

first, the disturbance noise matrix is adjusted by a two-line interpolation so that M1= M2 and N1= N2.

Secondly, correspondingly superposing each element in the adjusted disturbance noise matrix on the pixel value of each pixel point in the original image to obtain a primary selection sample.

In this embodiment, each element in the disturbance noise matrix corresponds to each pixel of the original image one to one, and the element value of each element and the pixel value of the corresponding pixel are superimposed. As a specific implementation, in order to define the size of each element value in the disturbance noise matrix, each element value may be multiplied byA predetermined coefficient

And then, overlapping the pixel values with the corresponding pixel points to obtain a primary selection sample.

And a substep S1023 of returning to the step of generating disturbance noise data meeting multidimensional Gaussian distribution if the primary selected sample does not meet a preset optimization condition, wherein the preset optimization condition is determined based on the initial temperature.

In this embodiment, the initial temperature is used for better controlling a balance relationship between search strength and solution diversity when the countermeasure sample is generated, and the initial temperature is introduced by using the idea of a simulated annealing algorithm, so that the number of search times is reduced as much as possible under the condition of meeting the diversity requirement, and finally the solution is not trapped in local optimum and efficiency is not reduced due to too many search times.

In this embodiment, an initial value is preset before the first-selected sample is generated, and in the subsequent process of generating the first-selected sample, the initial temperature may be gradually decreased to balance the search strength and the solution diversity, so that the initial temperature is not decreased indefinitely and becomes unreasonable before obtaining the countermeasure sample, and therefore, in general, a termination value is also set for the initial temperature. For example, the initial value of the initial temperature is set to 1 and the end value is

。

In this embodiment, iteration is usually performed for multiple times at the same initial temperature, and the initial temperature is adjusted only when the iteration number is slow enough, and multiple iterations are performed based on the adjusted initial temperature, and since the diversity of solutions is ensured, the iteration number may be preset, for example, the iteration number is set to 16 times, that is, at least 16 iterations are performed at the same initial temperature, and the initial temperature can be adjusted. Therefore, the preset condition is used for representing that the iteration number requirement is met and the initially selected sample meets the preset acceptable condition.

In this embodiment, if the primary selected sample does not satisfy the predetermined optimization condition, the method returns to substep S1021, and continues to perform substeps S1021 to substep S1025 until the primary selected sample satisfies the predetermined optimization condition and the predetermined termination condition.

And a substep S1024, if the primary selection sample meets a preset optimization condition and meets a preset termination condition, taking the primary selection sample as a confrontation sample.

In this embodiment, the preset termination condition may be set as needed, for example, the preset termination condition is set such that the similarity between the primary selection sample and the original image is less than or equal to a preset threshold.

And in the substep S1025, if the initially selected sample meets the preset optimization condition and does not meet the preset termination condition, adjusting the initial temperature, and returning to the step of generating disturbance noise data based on multi-dimensional Gaussian distribution until the initially selected sample meets the preset termination condition to obtain the confrontation sample.

In this embodiment, it can be understood that there is no strict execution sequence between the sub-step S1023 to the sub-step S1025, and only branches in three different situations, and the finally obtained confrontation sample is the initially selected sample meeting the preset termination condition.

In the embodiment, in order to ensure the diversity of the solutions, when the temperature is high, the worse degradation solution can be accepted, as the temperature is reduced, only the better degradation solution can be accepted, when the temperature approaches 0, the degradation solution is not accepted, so that the local optimum can be jumped out to a certain extent, the global optimum is shifted to, and simultaneously, the simplicity and the universality of the processing process are ensured. In order to adjust the initial temperature more reasonably and effectively reduce the number of iterations, this embodiment further provides a specific implementation manner for adjusting the initial temperature:

if the initial temperature is larger than the preset threshold value, adjusting the initial temperature according to a first adjusting factor; and if the initial temperature is not greater than the preset threshold, adjusting the initial temperature according to a second adjustment factor, wherein the second adjustment factor is greater than the first adjustment factor.

In this embodiment, the second adjustment factor is greater than the first adjustment factor, so that the initial temperature may decrease to the preset threshold at a faster speed when being greater than the preset threshold, and decrease slowly at a slower speed when being less than the preset threshold. The first adjustment factor, the second adjustment factor, and the preset threshold may all be set as required, and the first adjustment factor and the second adjustment factor are usually between 0.8 and 0.99, for example, the first adjustment factor is set to 0.9, the second adjustment factor is set to 0.98, and the preset threshold is set to 0.006.

In this embodiment, in order to determine whether the primary selection sample satisfies the preset optimization condition, an embodiment of the present invention further provides a specific implementation manner, since the sub-step S1021 to the sub-step S1025 may be executed once or multiple times before the confrontation sample is determined, which is described in detail below with the ith time as an example, and for the primary selection sample generated at the ith time, a manner whether the primary selection sample satisfies the preset optimization condition may be:

firstly, for the ith primary selection sample, obtaining a first similarity of the ith-1 th time, wherein the first similarity is the similarity between the ith-1 st primary selection sample and the original image, and i is an integer greater than 0.

In this embodiment, for the 1 st primary sample, the first similarity may be set as a preset similarity, and the preset similarity may be set as 1.

In this embodiment, as a specific implementation manner, the first similarity may be characterized by a cosine value between the feature vector of the initially selected sample at the i-1 st time and the feature vector of the original image, and a specific calculation manner is as follows:

inputting the initial selection sample of the (i-1) th time into a preset image recognition model to obtain a first feature vector of the initial selection sample of the (i-1) th time;

inputting an original image into a preset image recognition model to obtain a second feature vector of the original image;

and calculating a cosine value through the first feature vector and the second feature vector, and taking the calculated cosine value as a first similarity.

Second, a second similarity between the initial selected sample of the ith time and the original image is calculated.

In this embodiment, the second similarity and the first similarity are calculated in the same manner, and are not described herein again.

Thirdly, judging whether the initially selected sample meets the receiving condition or not according to the first similarity and the second similarity.

In this embodiment, whether the preliminary selection sample satisfies the acceptance condition at least includes the following two cases: (1) the second similarity is less than the first similarity; (2) the second similarity is not less than the first similarity; the method specifically comprises the following steps:

and if the second similarity is smaller than the first similarity, judging that the initially selected sample meets the receiving condition.

In this embodiment, as a specific implementation manner, in order to continue to superimpose the disturbance noise when i is less than the preset number of times under the condition that the second similarity is less than the first similarity, an initial value of the disturbance noise is introduced in the embodiment of the present invention, so as to use the initial value of the disturbance noise

It is shown that the process of the present invention,

all 0, for the ith primary sample, if the second similarity is smaller than the first similarity, then

+

，

+

Show that

+

The value of each element in the vector is constrained to be between (-u, u), meaning that

The value of each element in the vector is limited to (-u, u), values greater than u are limited to u, and values less than-u are limited to-u.

And if the second similarity is not less than the first similarity, determining whether the initially selected sample meets the acceptance condition according to the first similarity, the second similarity, the initial temperature and the temperature control factor.

In this embodiment, the second similarity is not less than the first similarity, the preliminary selection sample needs to be accepted or not accepted according to a certain probability, the degradation solution can be accepted according to the Metropolis criterion with a certain probability, the idea of the Metropolis criterion is to accept the degradation solution according to the probability instead of using a completely determined rule, the calculation amount can be effectively reduced, and the specific implementation manner of determining whether the preliminary selection sample meets the acceptance condition according to the first similarity, the second similarity, the initial temperature and the temperature control factor is as follows:

(1) Random numbers satisfying uniform distribution are generated, and the random numbers are used as reference probabilities.

In this embodiment, g represents a random number, and the random number satisfying uniform distribution may be represented as:

the random number takes on values between (0, 1).

(2) And calculating the acceptance probability for representing the acceptability of the initially selected sample according to the first similarity, the second similarity, the initial temperature and the temperature control factor.

In this embodiment, as a specific implementation manner, the following formula may be adopted to calculate the reception probability:

wherein the content of the first and second substances,

to receive the probability, i is the ith iteration,

is the initial temperature at the ith iteration,

in order to obtain the second degree of similarity,

is a first degree of similarity, wherein,

satisfies the following conditions:

n is the total number of iterations before the termination condition is met,

is the temperature control factor at the ith iteration,

satisfies the following conditions:

is shown in (10) ^-5 ，

Is shown in (10) ^-3 。

(3) And if the acceptance probability is not less than the reference probability, judging that the initially selected sample meets the acceptance condition.

(4) And if the acceptance probability is smaller than the reference probability, judging that the primary selection sample does not meet the acceptance condition.

And fourthly, if the primary selection sample meets the acceptance condition and i is less than the preset times or the primary selection sample does not meet the acceptance condition, judging that the primary selection sample does not meet the preset optimization condition.

And fifthly, if the primary selected sample meets the acceptance condition and i is not less than the preset times, judging that the primary selected sample meets the preset optimization condition.

It should be noted that, during the initial temperature adjustment, i needs to be reset to 0, so as to ensure that at least a preset number of iterations are performed at the same initial temperature.

In order to illustrate the technical effects provided by this embodiment, the inventor and the prior art perform a comparison test, and use a face image in a face database LFW (laboratory Faces in the Wild, LFW) as a test object, where a face recognition target model includes three models, namely, a sphere, a Cosface, and an Arcface, and the prior art adopts a boundry, optimization, and evolution 3 method, and performs a confusion attack and an impersonation attack on the three different models respectively by using the prior art and the method of this embodiment to obtain the query times and the mean square error of an optimal countermeasure sample, where the query times are the times of performing feature recognition on the face recognition target model in the process of determining the countermeasure sample, and the mean square error is used to calculate the proximity between the original sample image and the countermeasure sample, and for comparison, an average value of the mean square error is obtained by using the results of a given 500 face data set in the comparison test. The test results are shown in table 1.

TABLE 1

As can be seen from the experimental data results in table 1, compared with the bounderiy method and the Optimization method, the bounderiy method and the Optimization method obviously cannot obtain better results for more than twice the number of queries, and the evo method can obtain fewer distortion countermeasure samples but needs more queries than the method of the present embodiment.

In order to better explain the effect brought by setting the size of the disturbance noise smaller than the original image in the method of the embodiment, the embodiment also performs a comparison test on the sizes of different disturbance noises, and in the comparison test, the sizes of the sampled disturbance noises are respectively: 20 × 20 × 3, 45 × 45 × 3, and 60 × 60 × 3, and table 2 shows the average number of queries and the mean square error when generating the best countermeasure sample according to the method of the present embodiment.

TABLE 2

The present embodiment further obtains cosine similarity changes of different disturbance noise magnitudes under the confusion attack and the impersonation attack at a given query number, fig. 3 is an exemplary graph of cosine similarity changes of a confusion attack sphere model under the given query number, provided by the embodiment of the present invention, fig. 4 is an exemplary graph of cosine similarity changes of the impersonation attack sphere model under the given query number, provided by the embodiment of the present invention, fig. 5 is an exemplary graph of cosine similarity changes of a confusion attack Cosface model under the given query number, provided by the embodiment of the present invention, fig. 6 is an exemplary graph of cosine similarity changes of the impersonation attack Cosface model under the given query number, provided by the embodiment of the present invention, fig. 7 is an exemplary graph of cosine similarity changes of an obfuscation attack Arcface model under the given query number, provided by the embodiment of the present invention, and fig. 8 is an exemplary graph of cosine similarity changes of the impersonation attack arace model under the given query number, provided by the embodiment of the present invention. Similarly, the cosine similarity in fig. 3-8 is an average given the results of 500 face data sets.

From the experimental results, for the Cosface model,

the corresponding results produce the confrontational samples faster, i.e., fewer queries, with the best results being less distortion of the confrontational samples. For the sphere model, with the same distortion value,

fewer query coefficients are required, whereas for the Arcface model,

and

the effect is almost the same.

In order to prove the practicability of the method, the method is also used for attacking any commercialized face recognition model, including confusion attack and impersonation attack, and the experiment sets the size of disturbance noise

The rest of the conditions were the same as in the above test. Table 3 shows the experimental result data of the LFW data set and the Megaface data set, 500 average query times and mean square error of the image, which prove the feasibility of the present technique.

TABLE 3

As can be seen from table 3, no matter the LFW data set or the Megaface data set is, the query times and the mean square error of the method of the present embodiment do not fluctuate too much, so the method of the present embodiment has a certain universality.

In addition, the method of the embodiment has simple algorithm and strong transferability, and can be used for expanding and applying to an image classifier with good attack training. To illustrate the effect, this example performed experiments using Imagenet data sets, and 500 Imagenet verification sets were selected to be correctly classifiedImages of classes as experimental data sets, image input size

. The experiment is divided into a target attack and a non-target attack, and for the target attack, the target object is set to be the softmax second object in the embodiment. The target models of the attack of the embodiment comprise a Resnet50 model, a Densenet121 model and an inceptionv3 model. In the task of attacking image classification, since the size of the input image is large, the present embodiment sets the noise size to be a little larger,

，

other experimental conditions were kept consistent with the experimental condition settings described above. Table 4 shows the average query times and mean square error when the image classifier is attacked by the method of the present embodiment, and the target attack and the non-target attack are performed.

TABLE 4

As can be seen from table 4, no matter the LFW data set or the Megaface data set is, no matter the Resnet50 model, the densenert 121 model, or the inclusion v3 model is, the query times and the mean square error of the method of the present embodiment do not fluctuate too much, so the method of the present embodiment has better transferability.

In order to perform the corresponding steps in the above-described embodiments and various possible implementations, an implementation of the countermeasure sample generation apparatus 100 is given below. Referring to fig. 9, fig. 9 is a block diagram illustrating a countermeasure sample generation apparatus 100 according to an embodiment of the invention. It should be noted that the basic principle and the resulting technical effects of the countermeasure sample generation apparatus 100 provided in the present embodiment are the same as those of the above embodiments, and for the sake of brief description, no reference is made to this embodiment.

The confrontation sample generation apparatus 100 includes an acquisition module 110 and a generation module 120.

An obtaining module 110, configured to obtain an original image.

The generating module 120 is configured to add disturbance noise generated based on multi-dimensional gaussian distribution to the original image to obtain a countermeasure sample of the original image, where the countermeasure sample provides an unexpected output of the preset image recognition model, and the original image can provide a correct output of the preset image recognition model.

Optionally, the generating module 120 is specifically configured to: generating disturbance noise data satisfying multidimensional Gaussian distribution; adding disturbance noise to the original image by using disturbance noise data to obtain a primary selection sample; if the initially selected sample does not meet the preset optimization condition, returning to the step of generating the disturbance noise data meeting the multidimensional Gaussian distribution, wherein the preset optimization condition is determined based on the initial temperature; if the primary selection sample meets the preset optimization condition and the preset termination condition, taking the primary selection sample as a confrontation sample; and if the initially selected sample meets the preset optimization condition and does not meet the preset termination condition, adjusting the initial temperature, and returning to the step of generating disturbance noise data based on multi-dimensional Gaussian distribution until the initially selected sample meets the preset termination condition to obtain the confrontation sample.

Optionally, the disturbance noise data is a disturbance noise matrix of M1 × N1, the original image is an image of M2 × N2, M1< M2 and/or N1< N2, and the generating module 120 is specifically configured to, when the disturbance noise data is used to add disturbance noise to the original image, obtain an initial selection sample: adjusting the disturbance noise matrix by using a two-line interpolation value to enable M1= M2 and N1= N2; and correspondingly superposing each element in the adjusted disturbance noise matrix on the pixel value of each pixel point in the original image to obtain a primary selection sample.

Optionally, the generating module 120 is further configured to: for the primary selection sample of the ith time, obtaining the first similarity of the (i-1) th time, wherein the first similarity is the similarity between the primary selection sample of the (i-1) th time and the original image, and i is an integer greater than 0; calculating a second similarity between the primary selection sample of the ith time and the original image; judging whether the initially selected sample meets the acceptance condition or not according to the first similarity and the second similarity; if the primary selection sample meets the acceptance condition and i is less than the preset times, or the primary selection sample does not meet the acceptance condition, judging that the primary selection sample does not meet the preset optimization condition; and if the initial selection sample meets the acceptance condition and i is not less than the preset times, judging that the initial selection sample meets the preset optimization condition.

Optionally, the generating module 120 is specifically configured to, when determining whether the primary selection sample meets the acceptance condition according to the first similarity and the second similarity: if the second similarity is smaller than the first similarity, judging that the initially selected sample meets the acceptance condition; and if the second similarity is not less than the first similarity, determining whether the initially selected sample meets the acceptance condition or not according to the first similarity, the second similarity, the initial temperature and the temperature control factor.

Optionally, the generating module 120 is specifically configured to, when determining whether the initially selected sample meets the acceptance condition according to the first similarity, the second similarity, the initial temperature, and the temperature control factor: generating random numbers which meet the uniform distribution, and taking the random numbers as reference probability; calculating the acceptance probability for representing the acceptability of the initially selected sample according to the first similarity, the second similarity, the initial temperature and the temperature control factor; if the acceptance probability is not less than the reference probability, judging that the initially selected sample meets the acceptance condition; and if the acceptance probability is smaller than the reference probability, judging that the primary selection sample does not meet the acceptance condition.

Optionally, the generating module 120 is specifically configured to, when being configured to adjust the initial temperature: if the initial temperature is larger than the preset threshold value, adjusting the initial temperature according to a first adjusting factor; if the initial temperature is not larger than the preset threshold, the initial temperature is adjusted according to a second adjustment factor, and the second adjustment factor is smaller than the first adjustment factor.

Referring to fig. 10, fig. 10 shows a block schematic diagram of an electronic device 10 according to an embodiment of the present invention, where the electronic device 10 includes a processor 11, a memory 12, and a bus 13, and the processor 11, the memory 12, and the bus 13 are connected together.

The processor 11 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the countermeasure sample generation method may be performed by instructions in the form of hardware integrated logic circuits or software in the processor 11. The Processor 11 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

The memory 12 is used to store a program, such as the countermeasure sample generation apparatus 100 in fig. 9. The countermeasure sample generation apparatus 100 includes at least one software functional module that can be stored in the memory 12 in the form of software or firmware (firmware) or solidified in an Operating System (OS) of the electronic device 10. The processor 11, after receiving the execution instruction, executes the program to implement the countermeasure sample generation method disclosed in the above embodiment.

Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for generating a confrontational sample as described above.

In summary, embodiments of the present invention provide a countermeasure sample generation method, an apparatus, an electronic device, and a storage medium, where the method includes: acquiring an original image; disturbance noise generated based on multi-dimensional Gaussian distribution is added to the original image to obtain a countermeasure sample of the original image, the countermeasure sample provides unexpected output of the preset image recognition model, and the original image can provide correct output of the preset image recognition model. Compared with the prior art, the countermeasure sample generation method, the apparatus, the electronic device and the storage medium provided by the embodiment of the invention obtain the countermeasure sample of the original image by adding the disturbance noise generated based on the multidimensional Gaussian distribution to the original image, the countermeasure sample gives the unexpected output of the preset image recognition model, and the original image can give the correct output of the preset image recognition model, so that the robustness of the preset image recognition model can be verified by using the countermeasure sample.

The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. A method of generating a challenge sample, the method comprising:

acquiring an original image;

adding disturbance noise generated based on multi-dimensional Gaussian distribution to the original image to obtain an antagonistic sample of the original image, wherein the antagonistic sample provides unexpected output of a preset image recognition model, and the original image can provide correct output of the preset image recognition model;

the step of adding disturbance noise generated based on a multidimensional Gaussian distribution to the original image includes:

generating disturbance noise data satisfying the multidimensional Gaussian distribution;

if the initially selected sample does not meet a preset optimization condition, returning to the step of generating the disturbance noise data meeting the multidimensional Gaussian distribution, wherein the preset optimization condition is determined based on an initial temperature;

if the initially selected sample meets the preset optimization condition and does not meet the preset termination condition, adjusting the initial temperature, and returning to the step of generating the disturbance noise data based on the multidimensional Gaussian distribution until the initially selected sample meets the preset termination condition to obtain the confrontation sample;

the method further comprises the following steps:

for the primary selection sample of the ith time, obtaining a first similarity of the (i-1) th time, wherein the first similarity is the similarity between the primary selection sample of the (i-1) th time and the original image, and i is an integer greater than 0;

if the initial selection sample meets the acceptance condition and i is not less than the preset times, judging that the initial selection sample meets the preset optimization condition;

the step of judging whether the initially selected sample meets the acceptance condition according to the first similarity and the second similarity comprises the following steps:

2. The method for generating a countermeasure sample according to claim 1, wherein the disturbance noise data is a disturbance noise matrix of M1 × N1, the original image is an image of M2 × N2, and M1< M2 and/or N1< N2, and the step of adding disturbance noise to the original image by using the disturbance noise data to obtain a preliminary sample includes:

3. The method of claim 1, wherein the step of determining whether the preliminary selected sample satisfies the acceptance condition based on the first similarity, the second similarity, the initial temperature, and a temperature control factor comprises:

4. The challenge sample generation method of claim 1 wherein said step of adjusting said initial temperature comprises:

5. A challenge sample generating device, the device comprising:

the acquisition module is used for acquiring an original image;

the generation module is used for adding disturbance noise generated based on multi-dimensional Gaussian distribution to the original image to obtain an antagonistic sample of the original image, the antagonistic sample provides unexpected output of a preset image identification model, and the original image can provide correct output of the preset image identification model;

the generation module is specifically configured to: generating disturbance noise data satisfying the multi-dimensional Gaussian distribution; adding disturbance noise to the original image by using the disturbance noise data to obtain a primary selection sample; if the initially selected sample does not meet a preset optimization condition, returning to the step of generating disturbance noise data meeting the multidimensional Gaussian distribution, wherein the preset optimization condition is determined based on an initial temperature; if the initially selected sample meets the preset optimization condition and meets a preset termination condition, taking the initially selected sample as the confrontation sample; if the initially selected sample meets the preset optimization condition and does not meet the preset termination condition, adjusting the initial temperature, and returning to the step of generating disturbance noise data based on the multidimensional Gaussian distribution until the initially selected sample meets the preset termination condition to obtain the countermeasure sample;

the generation module is further configured to: for the ith primary selection sample, obtaining a first similarity of the ith-1 th time, wherein the first similarity is the similarity between the ith-1 st primary selection sample and the original image, and i is an integer greater than 0; calculating a second similarity between the primary selection sample at the ith time and the original image; judging whether the initially selected sample meets an acceptance condition or not according to the first similarity and the second similarity; if the primary selection sample meets the acceptance condition and i is less than a preset number of times, or the primary selection sample does not meet the acceptance condition, judging that the primary selection sample does not meet the preset optimization condition; if the initial selection sample meets the acceptance condition and i is not less than the preset times, judging that the initial selection sample meets the preset optimization condition;

the generating module is specifically configured to, when determining whether the preliminary selection sample meets an acceptance condition according to the first similarity and the second similarity: if the second similarity is smaller than the first similarity, judging that the initially selected sample meets the acceptance condition; and if the second similarity is not less than the first similarity, determining whether the initially selected sample meets the acceptance condition or not according to the first similarity, the second similarity, the initial temperature and a temperature control factor.

6. An electronic device comprising a processor and a memory; the memory is used for storing programs; the processor is configured to implement the countermeasure sample generation method of any of claims 1-4 when executing the program.

7. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, is adapted to carry out a method of generating a challenge sample according to any one of claims 1-4.