CN116152087A

CN116152087A - Unlimited countermeasure sample generation method, unlimited countermeasure sample generation device, electronic equipment and storage medium

Info

Publication number: CN116152087A
Application number: CN202211549873.9A
Authority: CN
Inventors: 陈鑫泉; 高希彤; 赵娟娟; 叶可江; 须成忠
Original assignee: Shenzhen Institute of Advanced Technology of CAS
Current assignee: Shenzhen Institute of Advanced Technology of CAS
Priority date: 2022-12-05
Filing date: 2022-12-05
Publication date: 2023-05-23

Abstract

The embodiment of the application provides an unlimited countermeasure sample generation method, an unlimited countermeasure sample generation device, electronic equipment and a storage medium, and relates to the technical field of countermeasure sample generation. Wherein the method comprises the following steps: obtaining a denoising image and a corresponding prediction image at the current moment based on a backward process of the diffusion model; generating a disturbance between the predicted image and a challenge sample thereof through a challenge attack on the predicted image; and transferring the disturbance to the denoising image at the current moment, and denoising the denoising image transferred at the current moment through the backward process of the diffusion model until an unlimited countermeasure sample is generated. The embodiment of the application solves the problems that the shape distortion of an unlimited countermeasure sample generated in the related technology is not in accordance with the real distribution, the semantic information of the area used for manual discrimination is fuzzy, and the discrimination is difficult.

Description

Unlimited countermeasure sample generation method, unlimited countermeasure sample generation device, electronic equipment and storage medium

Technical Field

The present application relates to the field of challenge sample generation technology, and in particular, to an unlimited challenge sample generation method, device, electronic apparatus, and storage medium.

Background

The neural network is useful in various fields including credit evaluation of banks, pedestrian recognition of automatic driving, intruder recognition of security systems, risk operation recognition in high-risk work, and the like, so it is important to ensure the safety of the neural network. At present, a mode of attacking the neural network is called as an antagonism sample, and a method for generating the antagonism sample is to antagonize the attack, and modify data before the data is input into the neural network, so that the modified picture is not greatly different from an original picture or accords with the real picture distribution, but can be recognized by the neural network to be wrong. In order to improve the robustness of neural networks, more and more work is beginning to study how challenge samples are generated and how challenge defenses are performed. The stronger and more mobile attack samples can make the model robustness after the countermeasure training stronger, and with the deep research of the countermeasure samples, the model interpretability and the research of the countermeasure sample mobility are provided with more basis guidance for the design and training of the neural network in the deep learning.

However, the prior art has the problems that the generated unlimited countermeasure sample shape is distorted, the real distribution is not met, the semantic information of the region used for manual discrimination is fuzzy, and the resolution is difficult.

From the above, the problem of how to make the generated unlimited challenge sample image high quality and preserve key semantics remains to be solved.

Disclosure of Invention

The embodiments of the present application provide a method, an apparatus, an electronic device, and a storage medium for generating unlimited countermeasure samples, which can solve the problems of distortion of the shape of the countermeasure samples, non-conforming to the actual distribution, fuzzy semantic information of the region used as manual discrimination, and difficult resolution in the related technology. The technical scheme is as follows:

according to one aspect of embodiments of the present application, a method of unlimited challenge sample generation, the method comprising: obtaining a denoising image and a corresponding prediction image at the current moment based on a backward process of the diffusion model; generating a disturbance between the predicted image and a challenge sample thereof through a challenge attack on the predicted image; and transferring the disturbance to the denoising image at the current moment, and denoising the denoising image transferred at the current moment through the backward process of the diffusion model until an unlimited countermeasure sample is generated.

According to one aspect of embodiments of the present application, an unlimited challenge sample generating device, the device comprising: the image denoising module is used for obtaining a denoising image at the current moment and a corresponding prediction image based on a backward process of the diffusion model; the disturbance generation module is used for generating disturbance between the predicted image and the countermeasure sample through the countermeasure attack on the predicted image; and the sample generation module is used for migrating the disturbance to the denoising image at the current moment, and denoising the migrated denoising image at the current moment through a backward process of the diffusion model until an unlimited countermeasure sample is generated.

According to one aspect of an embodiment of the present application, an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein the memory stores computer programs, and the processor reads the computer programs in the memory through the communication bus; the computer program, when executed by a processor, implements an unlimited challenge sample generation method as described above.

According to one aspect of embodiments of the present application, a storage medium has stored thereon a computer program which, when executed by a processor, implements an unlimited challenge sample generation method as described above.

According to one aspect of embodiments of the present application, a computer program product comprising a computer program stored in a storage medium, a processor of a computer device reading the computer program from the storage medium, the processor executing the computer program such that the computer device, when executing, implements an unlimited challenge sample generation method as described above.

The beneficial effects that this application provided technical scheme brought are:

in the technical scheme, the disturbance which does not accord with the real distribution is removed by utilizing the strong denoising capability of the diffusion model, so that the generated image has high quality, objects in the image accord with the real distribution, and the diffusion model can generate samples similar to the original image but with a certain sample diversity simultaneously by combining the hot start and the low-frequency guiding skills, and meanwhile, important features in the original image are determined by utilizing the feature visualization and are reserved in the generated antigen sample, so that the problems of low image quality and key semantic ambiguity of the unlimited antigen sample generated in the prior art are effectively solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a flow chart illustrating a method of unlimited challenge sample generation according to an example embodiment;

FIG. 2 is a schematic diagram of a diffusion model generation sample shown in accordance with an exemplary embodiment;

FIG. 3 is a flow chart of step 110, shown in accordance with an embodiment of Shi Lixin;

FIG. 4 is a system diagram illustrating a diffusion model generation sample according to an exemplary embodiment;

FIG. 5 is a predicted image at different times during a backward process of a diffusion model, according to an example embodiment;

FIG. 6 is a flow chart illustrating low frequency guidance according to an example embodiment;

FIG. 7 is a flow chart illustrating one method of preserving important features in a test sample according to an exemplary embodiment;

FIG. 8 is a schematic diagram of a class activation map CAM, according to an example embodiment;

FIG. 9 is a flowchart illustrating another method of preserving important features in a test sample according to an exemplary embodiment;

FIG. 10 is a flowchart illustrating test book generation robust against sample generation methods, according to an exemplary embodiment;

FIG. 11 is a block diagram illustrating the construction of an unlimited challenge sample generating device according to an example embodiment;

FIG. 12 is a hardware configuration diagram of an electronic device shown according to an exemplary embodiment;

fig. 13 is a block diagram of an electronic device, according to an example embodiment.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present application and are not to be construed as limiting the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.

As described above, in order to improve the robustness of the neural network, more and more efforts have been made to investigate how to generate challenge samples and how to perform challenge defenses. The stronger and more mobile attack samples can make the model robustness after the countermeasure training stronger, and with the deep research of the countermeasure samples, the model interpretability and the research of the countermeasure sample mobility are provided with more basis guidance for the design and training of the neural network in the deep learning.

The challenge sample under the Lp norm is a common challenge sample, wherein the Lp norm means that the magnitude of the limiting disturbance is within a certain range of L1, L2 and Linf, an attacker can modify the data layer after acquiring transmission information generally, and as the modified disturbance is small, naked eyes and original pictures are not the same, but the neural network can classify errors with high probability; in a real scene, an attacker can carry out unlimited attack on the picture, so long as the attack does not interfere with normal recognition, the picture can be normally classified by human beings, but the picture can be wrongly classified by a neural network, for example, means such as black and white blocks which do not influence human judgment are attached to road signs.

Since the challenge sample is discovered, the security problem of the artificial intelligence algorithm is widely paid attention to the industry and academia, the traditional method for generating the challenge sample limits the modeling of the disturbance within a certain range of Lp norm, and the real perception distance of the disturbance by human cannot be modeled, so that more and more researchers start to convert the gaze into the unrestricted challenge sample. There is no norm limitation to the disturbance range of the antigen sample without limitation, but the generated sample needs to be ensured to have the semanteme similar to the original image in the human eye. The disturbance range of the attack comprises the Lp norm, and is further perfection and supplement to the challenge sample under the Lp norm.

The unrestricted challenge samples generated by the prior art are mostly based on a codec network and a generated challenge network. The encoder network is changed in the process of generating model training, so that the output picture becomes an countermeasure sample, the method modifies the loss function of the network, and the loss of the distance between the finally generated countermeasure sample and the original picture is limited on the basis of the original loss function, so that the target attack network is classified into an error type or a target type countermeasure attack loss, and the trained generation model can directly obtain the countermeasure sample. The generation of the countermeasure network is to change the generated model in an inference stage after normal training without modification, so that the output picture becomes a countermeasure sample.

However, the above method has disadvantages that when the model based on the encoder network and the generated countermeasure network is used to generate unlimited countermeasure samples, disturbance is mostly performed on hidden variables, so that the generated samples have problems of poor image quality, semantic blurring of key objects, and bad edge or shape of the objects and real distribution.

Therefore, the problems of distortion of the shape of the generated countermeasure sample, non-conforming to the real distribution, fuzzy regional semantic information used for manual discrimination and difficult resolution still exist in the related technology.

Therefore, the unlimited countermeasure sample generation method provided by the application can effectively improve the image quality in the generated sample and reserve key semantics. Accordingly, the unrestricted challenge sample generation method is applicable to unrestricted challenge sample generation devices that may be deployed on an electronic device, which may be a computer device deploying von neumann architecture, e.g., a desktop computer, a notebook computer, a server, etc.

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Referring to fig. 1, an embodiment of the present application provides an unlimited challenge sample generation method, which is applicable to an electronic device, which may be a computer device.

In the following method embodiments, for convenience of description, the execution subject of each step of the method is described as an electronic device, but this configuration is not particularly limited.

As shown in fig. 1, the method may include the steps of:

step 110, obtaining a denoising image and a corresponding prediction image at the current moment based on a backward process of the diffusion model.

Specifically, as shown in fig. 2, a schematic diagram of a diffusion model generating an image sample is shown, wherein a backward process is from left to right; the right to left forward process. The diffusion model is a generating model, namely a model capable of generating image samples, and the core idea of the diffusion model is that a forward process q for adding noise step by step to the image under real distribution and finally changing the image into a standard Gaussian noise is defined, and then the generating task of the image is described as a series of denoising processes, namely a backward process p _θ . The diffusion model learns how to remove noise in the backward process by using a neural network through noise adding in the forward process, and further removes noise step by step from Gaussian noise to obtain an image conforming to distribution.

In one possible implementation, the denoised image is denoised by a backward process of the diffusion model to obtain a denoised image and a corresponding predicted image at the current time. The backward denoising process of the diffusion model is an iteration process, denoising images at different moments can be obtained according to different iteration times, and the predicted image is obtained after the last iteration is completed.

The inventors have found that if the predicted images can be close to a certain reference image such that their semantic information is the same, it would be advantageous to accelerate the backward process in the diffusion model. For this purpose, the present application employs a hot start to counter-guide the generation process of unrestricted counter-samples. That is, the hot start refers to a backward process of the diffusion model from a certain point in the forward process, instead of starting the backward process of the diffusion model from the standard gaussian noise, and in one possible implementation, the step 110 further includes the following steps as shown in fig. 3:

at step 1110, test samples are obtained from the test dataset.

The test sample may be considered as a reference image for providing the prediction image proximity, or may be understood as an original image.

And 1130, denoising the test sample through the forward process of the diffusion model to obtain a denoised image at the target moment.

In step 1150, denoising the denoising image at the target moment through the backward process of the diffusion model, so as to obtain a denoising image at the current moment and a corresponding prediction image.

Specifically, after the test sample is obtained from the test set, the application introduces a hot start method in the forward noise adding process, that is, a certain iteration time t in the forward process starts backward, as shown in fig. 4, when the initial image as reverse noise removal is not standard gaussian noise X _T The noise image of the test sample obtained by the forward process of the diffusion model is added with noise for the t time, the value of t is generally 500 to 700 times, at the moment, the noise image not only keeps the structural information of the test sample, but also loses the detail information, the noise image is used as an initial image of the denoising process, and the finally generated prediction image can still be similar to the structure of the test sample and can accelerate the backward process.

Step 130, by the challenge attack on the predicted image, a disturbance between the predicted image and its challenge sample is generated.

In the process of generating unlimited challenge samples, in order to make the generated result as close as possible to the target tag, it is necessary to perform challenge attack on the predicted image and classify the generated predicted image using the attacked model, thereby generating challenge samples that are not perceived by human eyes but are misclassified by the attacked model.

In one possible implementation, the present application provides two different classifiers to generate predicted images and their countering disturbances between samples at different stages according to the different classifiers. The two different classifiers include a true classifier and a noise-trained natural classifier. Specifically, the true classifier refers to the classifier in the attacked model, and the natural classifier refers to the classifier trained by noise.

The first stage refers to that the predicted image predicted by the real classifier belongs to a predicted category consistent with the real category, and the second stage refers to that the predicted image predicted by the real classifier does not belong to a predicted category consistent with the real category. As shown in fig. 5, it can be seen that approximate tone information appears in the early period of t >700, after which contour information is gradually supplemented, and the contour information is approximately completed at t=300 until texture information of the completed detail is finally supplemented, whereby the second phase can be approximately regarded as being in the t >400 section, and the first phase is in the t < =400 section.

Specifically, the predicted image is input into an attacked model to conduct classified prediction, and different classifiers are selected to generate disturbance according to the recognition capability of the attacked model. The first stage, when the predicted image predicted by the true classifier belongs to a predicted category consistent with the true category, and the attacked model can still identify the predicted image containing noise, generating disturbance by using the true classifier; and in the second stage, when the predicted image predicted by the true classifier belongs to a predicted category inconsistent with the true category, and the attacked model cannot identify the predicted image containing noise, generating disturbance by using the natural classifier trained by adding noise.

In one possible implementation, the predicted image and the perturbation between the countersamples are generated from the difference between the countersamples and the predicted image. It can also be considered that the disturbance substantially reflects the difference between the predicted category to which the challenge sample is predicted to belong and the true category to which the predicted image belongs.

And 150, transferring the disturbance to the denoising image at the current moment, and denoising the denoising image transferred at the current moment through a backward process of the diffusion model until an unlimited countermeasure sample is generated.

In this embodiment, the disturbance migration is implemented according to the following calculation formula:

wherein X is _t-1 The denoised image at the current time is represented by u, the mean value of the gaussian distribution at the current time is represented by Σ, the variance of the gaussian distribution at the current time is represented by Σ, the disturbance between the predicted image obtained by optimizing the countermeasures loss function C and the countermeasures sample thereof is represented by s, and the pre-configured super parameter for controlling the intensity of the added disturbance is represented by s.

Here, C may be represented by the cross entropy loss SCE in the PGD attack, and the calculation formulas for C and SCE are as follows:

C＝SCE(y _p ,y)or-SCE(y _p ,y’)

SCE(p,q)＝-∑p _i log(g _i )

wherein y is _p The independent heat vector representing the predicted category, y represents the independent heat vector of the true category, and y' represents the independent heat vector of the erroneous determination category that is desired to be finally misdetermined when being attacked.

The inventor realizes that in the hot start process, if t is too large, the predicted image and the reference image are not necessarily ensured to be close, and if t is too small, the diversity of the challenge sample is reduced due to the lack of iteration times, and for this reason, the application adds low-frequency guidance in the generation process of the unrestricted challenge sample, so as to ensure that the predicted image is closer to the test sample. In one possible implementation, as shown in fig. 6, before the step 150, the method further includes the following steps:

step 210, determining a noisy image of the test sample at the current moment based on the forward process of the diffusion model.

And 230, comparing the low-frequency information in the image with the denoising image at the current moment to obtain a corresponding error.

And 250, according to the distribution area of the error on the test sample, filling the low-frequency information in the denoising image at the current moment so as to perform disturbance migration based on the filled denoising image.

Specifically, low-frequency guidance is added in the backward process of the diffusion model, and firstly noisy images of the test sample at different moments in the forward process are determined

In order to make the predicted image in the backward process be as close to the test sample as possible, the denoising image x at the same moment _t And noise-added image->

The comparison of the low frequency information is performed, the low frequency information is determined by the formula +.>

The result, where phi denotes the low frequency filtering (downsampling) and linear interpolation operation (upsampling), is aimed at obtaining the same low frequency information as the test image resolution. After the low-frequency information with the same resolution as the test image is obtained, the low-frequency information is added into the denoising image at the current moment, and iteration is continued to obtain a final prediction image.

The integrity of semantic information in the generated countermeasure sample is ensured as much as possible, and the similarity of the low-frequency information between the prediction image and the test sample and the diversity of the generated sample can be freely controlled by limiting the size of the filter in the low-frequency filtering process and the time period for supplementing the low-frequency information.

Further, the inventors have found that in the course of generating unlimited challenge samples, the semantics for image classification are progressively blurred, making it difficult for humans to distinguish, and in order to ensure that important information in the images is not lost, first, important areas considered by the human eye or classifier need to be determined, so that these important areas are preserved as much as possible in the generated challenge samples. Since significant regions in the generation process may be transformed, the unlimited challenge sample generation method proposed herein uses significant regions in the test sample to control the generation process.

In one possible implementation, as shown in fig. 7, the present application provides a method of preserving important features in a test sample.

Step 310, acquiring a class mapping activation map CAM corresponding to the test sample, and determining a salient region in the test sample based on pixel values of the class mapping activation map;

wherein the CAM (Class Activation Mapping, class activation map), also known as class thermodynamic diagram or saliency map. As shown in fig. 8, the left side is a test sample, the right side is a test sample+thermodynamic diagram constructed by using CAM technology, namely, a class activation map, the size of the class activation map is consistent with that of the test sample, the pixel value represents the influence degree of the corresponding region of the test sample on the prediction output, and the larger the pixel value is, the larger the influence degree is, and the more the contribution is made to the decision of the subsequent classification.

Step 330, performing countermeasure guidance on the generation process of the unrestricted countermeasure sample according to the salient region.

Specifically, a feature visualization technology is utilized to obtain a class mapping activation graph CAM of the test sample, the larger the pixel value in the CAM is, the more the pixel value contributes to the final classification decision, and the small step size is used for carrying out countermeasure guidance on the salient region to be reserved based on the pixel value of the CAM, so that the salient region is reserved as far as possible, and meanwhile, the introduction of the region which is not required to be reserved due to the use of the large step size is avoided.

In another possible implementation, as shown in fig. 9, the present application provides another method of preserving important features in a test sample.

In step 410, the denoising image at the current time is divided into a main mask region and a background mask region by semantic division.

The background mask region is considered as a mask region outside the main mask region in the denoising image.

Step 430, determining a corresponding region of the subject mask in the noisy image at the current time based on the noisy image of the test sample at the current time.

In step 450, in the denoising image at the current time, the main mask area is replaced with the corresponding area determined in the denoising image at the current time.

That is, the main mask area is a significant area in the test sample, and the background mask area is an area outside the significant area. In the forward noise adding process of the test sample, the image information in the test sample can be directly selected for the position of the salient region without any noise adding process, so that the salient region in the final predicted image is completely the same as the test sample, and the salient region in the test sample is reserved; and the background mask area is generated by using a backward process, so that the noise intensity in the backward process can be met in the morning in the main mask area and the background mask area in the solving process, and finally the main mask area and the background mask area are added, so that a final predicted image can be obtained through iteration, the finally generated predicted image is consistent with a test sample in the main part, various results can be generated in the background part, and the countermeasure sample has diversity.

Specifically, preserving important features in the test sample is achieved by the following calculation formula:

where 1mask represents a main body mask area, and 0mask represents a background mask area.

Through the process, the disturbance which does not accord with the real distribution is removed by utilizing the strong denoising capability of the diffusion model, so that the generated image is high in quality, objects in the image accord with the real distribution, and in combination with the hot start and the low-frequency guiding skills, the diffusion model can generate samples similar to the original image but with a certain sample diversity, and meanwhile, important features in the original image are determined by utilizing feature visualization, and the generated challenge samples are reserved, so that the problems of low image quality and key semantic ambiguity of unlimited challenge samples generated in the prior art are effectively solved.

Referring to fig. 10, the present application provides a method for testing the robustness of a model under unlimited challenge, which may include the steps of:

step 510, inputting the unrestricted challenge sample into the attacked model for classification prediction to obtain a classification result of the unrestricted challenge sample; the classification result is used to indicate the prediction category to which the unrestricted challenge sample belongs.

And step 530, calculating the classification accuracy of the attacked model according to the difference between the predicted category to which the unlimited countermeasure sample belongs and the real category indicated by the tag carried by the test sample.

Specifically, after the unlimited countermeasure sample is obtained by the unlimited countermeasure sample generation method provided by the application, classification prediction is performed on the unlimited countermeasure sample by using the attacked model to obtain a classification result, and the attacked model can be determined according to the neural network model in the actual application scene, and is not limited in particular. And calculating the classification accuracy of the challenge sample and the test sample in the attacked model according to the classification difference of the challenge sample and the test sample in the same attacked model.

With the cooperation of the above embodiments, the robustness assessment of the attacked model is realized. The better the robustness is, the more indirectly reflects that the challenge sample generated by the unlimited challenge sample generation method provided by the application has the advantages of high generated image quality, clear key semantics and accordance with real distribution.

The following are device embodiments of the present application that may be used to perform the unlimited challenge sample generation methods referred to herein. For details not disclosed in the device embodiments of the present application, please refer to a method embodiment of the unlimited challenge sample generation method related to the present application.

Referring to fig. 11, an unlimited challenge sample generating apparatus 800 is provided in an embodiment of the present application, including but not limited to: an image denoising module 810, a disturbance generation module 830, and a sample generation module 850.

The image denoising module 810 is configured to obtain a denoised image and a corresponding predicted image at the current time based on a backward process of the diffusion model.

The disturbance generating module 830 is configured to generate a disturbance between the predicted image and the counterattack on the predicted image.

The sample generation module 850 is configured to migrate the disturbance to the denoising image at the current time, and denoise the migrated denoising image at the current time through a backward process of the diffusion model until an unlimited countermeasure sample is generated.

It should be noted that, in the unlimited countermeasure sample generating apparatus provided in the foregoing embodiment, only the division of the functional modules is illustrated when the unlimited countermeasure sample generating apparatus performs unlimited countermeasure sample generation, and in practical applications, the above-mentioned functions may be allocated by different functional modules according to needs, that is, the internal structure of the unlimited countermeasure sample generating apparatus may be divided into different functional modules to complete all or part of the functions described above.

In addition, the unlimited countermeasure sample generating apparatus and the method of the unlimited countermeasure sample generating apparatus provided in the foregoing embodiments belong to the same concept, and a specific manner in which each module performs an operation has been described in detail in the method embodiment, which is not described herein.

Fig. 12 shows a structural schematic of an electronic device according to an exemplary embodiment.

It should be noted that the electronic device is just one example adapted to the present application, and should not be construed as providing any limitation to the scope of use of the present application. Nor should the electronic device be construed as necessarily relying on or necessarily having one or more of the components of the exemplary electronic device 2000 illustrated in fig. 12.

The hardware structure of the electronic device 2000 may vary widely depending on the configuration or performance, as shown in fig. 12, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU, central Processing Units) 270.

Specifically, the power supply 210 is configured to provide an operating voltage for each hardware device on the electronic device 2000.

The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices.

Of course, in other examples of adaptation of the present application, the interface 230 may further include at least one serial-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, as shown in fig. 12, which is not specifically limited herein.

The memory 250 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, where the resources stored include an operating system 251, application programs 253, and data 255, and the storage mode may be transient storage or permanent storage.

The operating system 251 is used for managing and controlling various hardware devices and applications 253 on the electronic device 2000, so as to implement the operation and processing of the cpu 270 on the mass data 255 in the memory 250, which may be Windows server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.

The application 253 is a computer program that performs at least one specific task based on the operating system 251, and may include at least one module (not shown in fig. 12), each of which may respectively include a computer program for the electronic device 2000. For example, the unlimited challenge sample generation method apparatus may be considered as an application 253 deployed on the electronic device 2000.

The data 255 may be a photograph, an image, or the like stored in a disk, or may be an unlimited challenge sample, or the like, and stored in the memory 250.

The central processor 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read the computer program stored in the memory 250, thereby implementing the operation and processing of the bulk data 255 in the memory 250. The unlimited challenge sample generation method is accomplished, for example, by the central processor 270 reading a series of computer programs stored in the memory 250.

Furthermore, the present application can be realized by hardware circuitry or by a combination of hardware circuitry and software, and thus, the implementation of the present application is not limited to any specific hardware circuitry, software, or combination of the two.

Referring to fig. 13, an embodiment of the present application provides an electronic device 4000, where the electronic device 4000 may be a desktop computer, a notebook computer, a server, or the like.

In fig. 13, the electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.

Wherein the processor 4001 is coupled to the memory 4003, such as via a communication bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004, the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The processor 4001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 4001 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

The communication bus 4002 may include a pathway to transfer information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 13, but not only one bus or one type of bus.

Memory 4003 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (Electrically Erasable Programmable Read Only Memory ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

The memory 4003 has stored thereon a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.

The computer program, when executed by the processor 4001, implements the unlimited challenge sample generation method in the above embodiments.

Further, in the embodiments of the present application, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the unlimited challenge sample generation method in the above embodiments.

In an embodiment of the present application, a computer program product is provided, which includes a computer program stored in a storage medium. The processor of the computer device reads the computer program from the storage medium, and the processor executes the computer program so that the computer device executes the unlimited challenge sample generation method in the above-described embodiments.

Compared with the related art, the unlimited countermeasure sample generation method based on the diffusion model removes disturbance which does not accord with real distribution by utilizing the strong denoising capability of the diffusion model, so that the image quality of the generated unlimited countermeasure sample is higher, and objects in the image accord with the real distribution; by combining hot start and low-frequency guide skills, a diffusion model can generate samples similar to an original image but with a certain sample diversity, and the generation process is controlled to enable the picture semantics to be similar to the original image, so that generated unlimited antigen samples are consistent in semantic distribution; the important features in the original image are determined by utilizing feature visualization, the important features are reserved in the generated antigen sample, the influence of disturbance on the region of interest (namely the salient region) of human eyes is reduced, and the problems that the image quality of the antigen sample is low and the key semantics are fuzzy without limitation in the prior art are effectively solved.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

The foregoing is only a partial embodiment of the present application, and it should be noted that, for a person skilled in the art, several improvements and modifications can be made without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method of unlimited challenge sample generation, the method comprising:

obtaining a denoising image and a corresponding prediction image at the current moment based on a backward process of the diffusion model;

generating a disturbance between the predicted image and a challenge sample thereof through a challenge attack on the predicted image;

and transferring the disturbance to the denoising image at the current moment, and denoising the denoising image transferred at the current moment through the backward process of the diffusion model until an unlimited countermeasure sample is generated.

2. The method of claim 1, wherein the generating the disturbance between the predicted image and its challenge sample by the challenge attack on the predicted image comprises:

inputting the predicted image into an attacked model to carry out classified prediction, and generating disturbance by using a real classifier in the attacked model in a first stage; the first stage refers to that the predicted category of the predicted image predicted by the real classifier is consistent with the real category;

and generating a disturbance between the predicted image and the countermeasure sample according to the difference between the countermeasure sample and the predicted image.

3. The method of claim 2, wherein the generating the disturbance between the predicted image and its challenge sample by the challenge attack on the predicted image comprises:

in the second stage, generating disturbance by using a natural classifier subjected to noise adding training; the second stage is that the predicted category of the predicted image predicted by the real classifier is inconsistent with the real category.

4. The method of claim 1, wherein the backward process based on the diffusion model obtains a denoised image and a corresponding predicted image at a current time, comprising:

obtaining a test sample from a test dataset;

the test sample is subjected to noise adding through the forward process of the diffusion model, and a noise adding image at the target moment is obtained;

denoising the denoising image at the target moment through the backward process of the diffusion model to obtain a denoising image at the current moment and a corresponding prediction image.

5. The method of claim 1, wherein the diffusion model-based backward procedure, after obtaining the denoised image and the corresponding predicted image at the current time, further comprises:

determining a noise-added image of the test sample at the current moment based on the forward process of the diffusion model;

comparing the low-frequency information in the image with the denoising image at the current moment to obtain a corresponding error;

and according to the distribution area of the error on the test sample, filling the low-frequency information in the denoising image at the current moment so as to carry out the disturbance migration based on the filled denoising image.

6. The method of claim 1, wherein the method further comprises:

acquiring a class activation map CAM corresponding to a test sample, and determining a significant region in the test sample based on pixel values of the class activation map;

and performing countermeasure guidance on the generation process of the unrestricted countermeasure sample according to the salient region.

7. The method of claim 1, wherein the diffusion model-based backward procedure, after obtaining the denoised image and the corresponding predicted image at the current time, further comprises:

dividing the denoising image at the current moment into a main mask area and a background mask area through semantic division;

determining a corresponding region of the main mask in the noisy image at the current moment based on the noisy image of the test sample at the current moment;

and in the denoising image at the current moment, replacing the main mask area with the corresponding area determined in the denoising image at the current moment.

8. The method of any one of claims 1 to 7, wherein after the generating the unrestricted challenge sample, the method further comprises:

inputting the unrestricted challenge sample into an attacked model for classification prediction to obtain a classification result of the unrestricted challenge sample; the classification result is used for indicating a prediction category to which the unlimited countermeasure sample belongs;

and calculating the classification accuracy of the attacked model according to the difference between the prediction category to which the unlimited countermeasure sample belongs and the real category indicated by the tag carried by the test sample.

9. An unlimited challenge sample generating device, the device comprising:

the image denoising module is used for obtaining a denoising image at the current moment and a corresponding prediction image based on a backward process of the diffusion model;

the disturbance generation module is used for generating disturbance between the predicted image and the countermeasure sample through the countermeasure attack on the predicted image;

and the sample generation module is used for migrating the disturbance to the denoising image at the current moment, and denoising the migrated denoising image at the current moment through a backward process of the diffusion model until an unlimited countermeasure sample is generated.

10. An electronic device, comprising: at least one processor, at least one memory, and at least one communication bus, wherein,

the memory stores a computer program, and the processor reads the computer program in the memory through the communication bus;

the computer program, when executed by the processor, implements the unlimited challenge sample generation method of any of claims 1 to 8.