CN113887651A - Acquisition method and device of countermeasure sample image and electronic equipment - Google Patents

Abstract

The disclosure provides a method and a device for obtaining a confrontation sample image and electronic equipment, and relates to the technical field of artificial intelligence such as image classification. The specific implementation scheme is as follows: when obtaining the confrontation sample image, respectively obtaining the proportions and the confrontation parameters of a plurality of sample images and a plurality of confrontation operators; determining a plurality of sample images to be processed for countermeasure processing from the plurality of sample images according to respective proportions of the plurality of countermeasure operators; and processing the plurality of sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the plurality of sample images to be processed to obtain a countermeasure sample image, so as to obtain the countermeasure sample image. The countermeasure parameters of a plurality of countermeasure operators are adopted to carry out countermeasure processing on the sample image, and the uniformity of the size of the disturbance norm of the countermeasure sample image can be improved.

Description

Acquisition method and device of countermeasure sample image and electronic equipment

Technical Field

The disclosure relates to the technical field of image processing, in particular to the technical field of artificial intelligence such as image classification, and specifically relates to a method and a device for acquiring a confrontation sample image and electronic equipment.

Background

The counterattack is a tiny disturbance generated according to the weight of the deep learning model, and the tiny disturbance can cause the output result of the deep learning model to obviously deviate from the normal output result, thereby causing the output result of the deep learning model to be wrong.

In order to make the deep learning model have the capability of coping with such counterattack, countertraining is usually performed on the deep learning model, which requires obtaining a countersample image for countertraining, and therefore, how to obtain the countersample image for countertraining is crucial.

Disclosure of Invention

The disclosure provides a method and a device for acquiring a countermeasure sample image and electronic equipment, which can accurately acquire the countermeasure sample image and can improve the uniformity of the size of a disturbance norm of the countermeasure sample image.

According to a first aspect of the present disclosure, there is provided a countermeasure sample image acquisition method that may include:

a plurality of sample images are acquired, and respective proportions and countermeasure parameters of a plurality of countermeasure operators are obtained.

Determining a plurality of sample images to be processed for countermeasure processing from a plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein, the plurality of sample images to be processed correspond to respective antagonistic operators.

And processing the sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the sample images to be processed to obtain the countermeasure sample images.

According to a second aspect of the present disclosure, there is provided an acquisition apparatus of a countermeasure specimen image, which may include:

the first acquisition unit is used for acquiring a plurality of sample images and respective proportions and confrontation parameters of a plurality of confrontation operators.

A first processing unit, configured to determine a plurality of to-be-processed sample images to be subjected to countermeasure processing from among the plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein, the plurality of sample images to be processed correspond to respective antagonistic operators.

And the second processing unit is used for processing the sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the sample images to be processed to obtain the countermeasure sample images.

According to a third aspect of the present disclosure, there is provided an electronic device, which may include:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of obtaining an image of a challenge sample of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method for acquiring a countermeasure sample image according to the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to execute the method for acquiring an image of a challenge sample according to the first aspect.

According to the technical scheme, the confrontation sample image can be accurately acquired, and the uniformity of the size of the disturbance norm of the confrontation sample image can be improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flow chart of a method for acquiring a countermeasure sample image provided according to a first embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a method for processing a plurality of sample images to be processed to obtain a confrontation sample image according to a second embodiment of the present disclosure;

FIG. 3 is a schematic flow chart diagram of a method for generating an image processing model based on resist sample image co-training according to a third embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a device for acquiring a countermeasure sample image provided according to a fourth embodiment of the present disclosure;

fig. 5 is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In embodiments of the present disclosure, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the access relationship of the associated object, meaning that there may be three relationships, e.g., A and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. In the description of the text of the present disclosure, the character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, in the embodiments of the present disclosure, "first", "second", "third", "fourth", "fifth", and "sixth" are only used to distinguish the contents of different objects, and have no other special meaning.

The technical scheme provided by the embodiment of the disclosure can be applied to image processing scenes, such as image classification scenes, image recognition scenes or image segmentation scenes, and can be specifically set according to actual needs. In an image processing scenario, the image may be processed using a deep learning model, but the deep learning model is vulnerable to counterattack when the image is processed.

In order to make the deep learning model have the capability of coping with such counterattack, it is common practice to perform countertraining on the deep learning model. In the conventional scheme, when the confrontation training is performed on the deep learning model, a programmer generally realizes the confrontation training on the deep learning model through a programming mode of a facies process according to respective ideas. However, the countervailing ability of the deep learning model obtained by the countervailing training is still poor.

In order to improve the countervailing capability of the deep learning model, when the deep learning model is generated through training, the countervailing sample images are added into the training sample image set, namely the training sample image set comprises the conventional sample images and the countervailing sample images, and the deep learning model is generated through co-training of the conventional sample images and the countervailing sample images, so that the deep learning model obtained through training can better process the conventional images and the countervailing images, and the countervailing capability of the deep learning model is improved.

Therefore, how to acquire the confrontation sample image is crucial to acquiring a deep learning model with good confrontation capability. In order to obtain the countermeasure sample image well, it may be considered that, among the plurality of obtained normal sample images, a part of the normal sample image is selected as a sample image to be subjected to countermeasure processing, and countermeasure parameters of a plurality of countermeasure operators are adopted to perform countermeasure processing on the sample image, so that the countermeasure sample image is obtained. The countermeasure parameters of a plurality of countermeasure operators are adopted to carry out countermeasure processing on the sample image, and the uniformity of the size of the disturbance norm of the countermeasure sample image can be improved.

Therefore, the deep learning model can be generated by training the conventional sample image and the confrontation sample image together, so that the trained deep learning model can better process the conventional image and the confrontation image, and the confrontation capacity of the deep learning model is improved.

For example, the deep learning model may be an image processing model, which may be an image classification model, an image recognition model, an image segmentation model, or the like, and may be specifically set according to actual needs.

When the image processing models are different depth learning models, the corresponding model outputs are different. For example, when the image processing model is an image classification model, the model output of the image classification model is the category to which the image belongs; when the image processing model is an image recognition model, the model output of the image recognition model is the recognition result of the image; when the image processing model is an image segmentation model, the model output of the image segmentation model is the segmentation result of the image.

Based on the technical concept, the embodiment of the present disclosure provides a method for acquiring a countermeasure sample image, and the method for acquiring a countermeasure sample image provided by the present disclosure will be described in detail through a specific embodiment. It is to be understood that the following detailed description may be combined with other embodiments, and that the same or similar concepts or processes may not be repeated in some embodiments.

Example one

Fig. 1 is a flowchart illustrating a method for acquiring a countermeasure sample image according to a first embodiment of the disclosure, where the method for acquiring the countermeasure sample image may be performed by software and/or a hardware device, for example, the hardware device may be a terminal or a server. For example, referring to fig. 1, the method for obtaining the confrontation sample image may include:

s101, obtaining a plurality of sample images and proportions and confrontation parameters of a plurality of confrontation operators.

The sample images can be understood as regular sample images, and the proportion of the countermeasure operator can be used for determining the number of sample images needing countermeasure processing in the sample images. Assuming that the number of the operators is 3, the proportion of one operator is 10%, the proportion of one operator is 15%, and the proportion of one operator is 25%, the number of the sample images to be subjected to the countermeasure processing among the plurality of sample images can be determined to be 50% of the number of the plurality of sample images according to the proportion of the 3 operators.

In the embodiment of the disclosure, by setting the proportion of the countermeasure operator, the number and the generation frequency of the sample images needing countermeasure processing in the generation process of the countermeasure sample images can be flexibly adjusted through the proportion of the countermeasure operator, and the flexibility of adjustment is improved.

The parameter information of the countermeasure operator is used for setting the preset norm and the norm boundary, so that disturbance can be controlled in the preset norm and the norm boundary through the parameter information of the countermeasure operator, the disturbance size is controllable, and the uniformity of the disturbance norm size of the countermeasure sample image can be improved. For example, the parameter information of the countermeasure operator may include an accepted norm category, a maximum perturbation size, and a single-step perturbation step size, and may also include other information, which may be specifically set according to actual needs.

For example, when a plurality of sample images are acquired, the plurality of sample images sent by other electronic devices may be received, the plurality of sample images may also be acquired from a local storage, or the plurality of sample images may also be acquired in other manners, which may be specifically set according to actual needs, and here, as for the acquiring manner of the plurality of sample images, the embodiment of the present disclosure is not particularly limited.

For example, when obtaining the respective proportions and countermeasure parameters of the plurality of countermeasure operators, the respective proportions and countermeasure parameters of the plurality of countermeasure operators may be fixedly set, may also be dynamically set according to actual needs, and may specifically be set according to actual needs.

When the sample images, the respective proportions of the antagonistic operators and the antagonistic parameters are respectively obtained, the sample images to be processed for antagonistic processing can be determined from the sample images according to the respective proportions of the antagonistic operators, that is, the following S102 is executed:

s102, determining a plurality of to-be-processed sample images to be subjected to countermeasure processing from the plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein, a plurality of sample images to be processed correspond to respective antagonistic operators.

Assuming that the number of the plurality of sample images is 1000, and the number of the plurality of antagonistic operators is 3, the sample images can be respectively marked as a first antagonistic operator, a second antagonistic operator and a third antagonistic operator; the proportion of the first antagonizing operator is 10%, the proportion of the second antagonizing operator is 15%, and the proportion of the third antagonizing operator is 25%, then 100 sample images can be randomly selected from 1000 sample images as to-be-processed sample images needing antagonizing processing according to the proportion of the first antagonizing operator of 10%; according to the 15% of the first countermeasure operator, 150 sample images can be randomly selected from the remaining 900 sample images as sample images to be processed for countermeasure processing, and 250 sample images can be randomly selected from the remaining 750 sample images as sample images to be processed for countermeasure processing, so that 500 sample images can be randomly selected from the 1000 sample images as a plurality of sample images to be processed for countermeasure processing.

It can be understood that, after a plurality of to-be-processed sample images that need to be subjected to countermeasure processing are determined from a plurality of sample images according to respective proportions of a plurality of countermeasure operators, each to-be-processed sample image in the plurality of to-be-processed sample images corresponds to one countermeasure operator. For example, if 100 sample images to be processed in 500 selected sample images are determined based on the corresponding proportion of 10% of the first antagonistic operator, the antagonistic operator corresponding to each of the 100 sample images to be processed is the first antagonistic operator; similarly, if 150 sample images to be processed are determined based on 15% of the corresponding proportion of the second competitor, the competitor corresponding to each of the 150 sample images to be processed is the second competitor; if 250 sample images to be processed are determined based on the corresponding proportion of 25% of the third antagonistic operator, the antagonistic operator corresponding to each of the 250 sample images to be processed is the third antagonistic operator.

After determining a plurality of sample images to be processed for countermeasure processing from the plurality of sample images according to respective proportions of the plurality of countermeasure operators, the plurality of sample images to be processed may be processed based on countermeasure parameters of the countermeasure operators corresponding to the plurality of sample images to be processed, that is, the following S103 is performed:

s103, processing the multiple sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the multiple sample images to be processed to obtain the countermeasure sample images.

The method has the advantages that the multiple to-be-processed sample images are processed based on the countermeasure parameters of the countermeasure operators corresponding to the multiple to-be-processed sample images, so that disturbance can be controlled in the preset norm and norm boundary through the parameter information of the countermeasure operators, the disturbance size is controllable, and the uniformity of the disturbance norm size of the countermeasure sample images can be improved.

It can be seen that, in the embodiment of the present disclosure, when obtaining a confrontation sample image, the respective proportions and confrontation parameters of a plurality of sample images and a plurality of confrontation operators may be obtained first; determining a plurality of sample images to be processed for countermeasure processing from the plurality of sample images according to respective proportions of the plurality of countermeasure operators; then processing the plurality of sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the plurality of sample images to be processed to obtain countermeasure sample images, so as to obtain the countermeasure sample images; in addition, the countermeasure parameters of a plurality of countermeasure operators are adopted to carry out countermeasure processing on the sample image, and the uniformity of the size of the disturbance norm of the countermeasure sample image can be improved.

Therefore, the image processing model can be generated by training the conventional sample image and the confrontation sample image together, so that the trained image processing model can better process the conventional image and the confrontation image, and the confrontation capacity of the image processing model is improved.

Based on the above-mentioned embodiment shown in fig. 1, in order to facilitate understanding how to process the multiple sample images to be processed based on the countermeasure parameters of the countermeasure operators corresponding to the multiple sample images to be processed in S103 shown in the first embodiment, a countermeasure sample image is obtained, and the following detailed description will be made by using a second embodiment shown in fig. 2.

Example two

Fig. 2 is a schematic flow chart of a method for processing a plurality of sample images to be processed to obtain a countermeasure sample image according to a second embodiment of the disclosure, which may also be performed by software and/or hardware devices. For example, referring to fig. 2, the method for processing a plurality of sample images to be processed to obtain a confrontation sample image may include:

s201, processing the multiple sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the multiple sample images to be processed to obtain the corresponding inverse normalized countermeasure samples of the multiple sample images to be processed.

For example, when the plurality of sample images to be processed are processed based on the countermeasure parameters of the countermeasure operators corresponding to the plurality of sample images to be processed, two cases may be included:

one situation is: in view of the fact that, in general, when the countermeasure processing is performed on the sample image, the countermeasure processing is typically performed on the sample image before normalization, and therefore, when the plurality of sample images to be processed are sample images after normalization processing, inverse normalization processing may be performed on the plurality of sample images to be processed first to obtain inverse normalized sample images corresponding to the plurality of processed sample images; and performing countermeasure processing on the inverse normalized sample images corresponding to the plurality of processed sample images based on countermeasure parameters of countermeasure operators corresponding to the plurality of sample images to be processed.

In this case, when the sample image is normalized, the sample image is normalized by using sample information corresponding to the sample image dataset, such as a mean value and a variance, and may also include other information, such as a dimension, to obtain a normalized sample image; taking the example that the sample information includes the mean value and the method, correspondingly, when the inverse normalization processing is performed on the multiple sample images to be processed, the inverse normalization processing can be performed on the sample images by using the mean value and the variance corresponding to the sample image data set, so as to obtain the inverse normalized sample images corresponding to the sample images.

The other situation is as follows: when the plurality of sample images to be processed are directly unnormalized sample images, the plurality of unnormalized sample images to be processed can be processed directly based on the countermeasure parameters of the countermeasure operators corresponding to the plurality of sample images to be processed respectively.

For example, when a plurality of sample images to be processed are processed, in addition to processing the plurality of sample images to be processed according to the countermeasure parameters of the countermeasure operator, the plurality of sample images to be processed may be processed together in combination with the countermeasure target of the countermeasure operator, such as target countermeasure or no target countermeasure; here, the embodiment of the present disclosure is only described by taking as an example that a plurality of sample images to be processed are processed based on a countermeasure parameter of a countermeasure operator corresponding to each of the plurality of sample images to be processed, but the embodiment of the present disclosure is not limited thereto.

In the method, a countercheck parameter of a countercheck operator corresponding to each of a plurality of sample images to be processed is used to process the plurality of sample images to be processed, so as to obtain an inverse normalized countercheck sample corresponding to each of the plurality of sample images to be processed, and in view of that, in a normal case, an input of an image processing model is a normalized sample image, so that, in order to meet an input requirement of the image processing model, the obtained inverse normalized countercheck sample may be normalized again, that is, the following S202 is performed:

s202, normalization processing is carried out on the inverse normalization countermeasure samples corresponding to the multiple to-be-processed sample images, and the normalization countermeasure samples corresponding to the multiple to-be-processed sample images are obtained.

For example, when performing normalization on the inverse normalized countermeasure samples corresponding to the multiple sample images to be processed, see the description in S201 above, the mean and the variance corresponding to the sample image data set may also be used to perform normalization on the sample images to obtain the normalized countermeasure samples corresponding to the multiple sample images to be processed.

Thus, after obtaining the normalized countermeasure samples corresponding to the plurality of to-be-processed sample images, the countermeasure sample image can be further obtained according to the plurality of to-be-processed sample images and the normalized countermeasure samples corresponding to the plurality of to-be-processed sample images, that is, the following S203 is executed:

s203, obtaining a confrontation sample image according to the plurality of sample images to be processed and the plurality of normalized confrontation samples corresponding to the sample images to be processed.

For example, when obtaining a countermeasure sample image according to a plurality of sample images to be processed and normalization countermeasure samples corresponding to the plurality of sample images to be processed, the normalization countermeasure samples corresponding to the plurality of sample images to be processed may be input into an image processing model to obtain label information corresponding to each normalization countermeasure sample; matching label information corresponding to each of the plurality of sample images to be processed with label information corresponding to the corresponding normalized countermeasure sample, and determining that generation of the countermeasure sample image fails if the label information corresponding to the sample images to be processed is different from the label information corresponding to the corresponding normalized countermeasure sample image; on the contrary, if the label information corresponding to the sample image to be processed is the same as the label information corresponding to the corresponding normalized confrontation sample image, which indicates that the confrontation sample image is successfully generated, the normalized confrontation sample image is determined as the confrontation sample image, and the confrontation sample image is acquired.

For example, the above-described predicted classification information of the inverse normalized confrontation sample image, the normalized confrontation sample image, and the normalized confrontation sample image may be stored in a finite state machine.

It can be seen that, in the embodiment of the present disclosure, when the multiple sample images to be processed are processed based on the countermeasure parameters of the corresponding countermeasure operators of the multiple sample images to be processed to obtain the countermeasure sample images, the multiple sample images to be processed may be processed based on the countermeasure parameters of the corresponding countermeasure operators of the multiple sample images to be processed to obtain the inverse normalized countermeasure samples corresponding to the multiple sample images to be processed; carrying out normalization processing on the inverse normalization countermeasure samples corresponding to the multiple sample images to be processed respectively to obtain normalization countermeasure samples corresponding to the multiple sample images to be processed respectively; and obtaining a countermeasure sample image according to the plurality of to-be-processed sample images and the plurality of normalized countermeasure samples corresponding to the to-be-processed sample images, so as to obtain the countermeasure sample image.

Based on any embodiment, after the sample images to be processed are processed based on the sample information and the countermeasure parameters of the countermeasure operators corresponding to the sample images to be processed respectively to obtain the countermeasure sample images, the image processing model can be generated based on the countermeasure sample images, so that the image processing model obtained through training can better process the conventional images and can better process the countermeasure images, and the countermeasure capability of the image processing model is improved. Next, how to generate an image processing model based on the confrontation sample image co-training will be described in detail by the following third embodiment shown in fig. 3.

EXAMPLE III

Fig. 3 is a flowchart of a method for generating an image processing model based on the concurrent training of countermeasure sample images, which may also be performed by software and/or hardware devices, according to a third embodiment of the present disclosure. For example, referring to fig. 3, the method for generating an image processing model based on the antagonistic sample image co-training may include:

s301, acquiring a training sample image set.

The training sample image set comprises a plurality of sample images which are not subjected to countermeasure processing in the plurality of sample images, a plurality of countermeasure sample images and label information corresponding to each target sample image, and the target sample image is any one of the plurality of sample images and the plurality of countermeasure sample images.

It can be understood that, in order to ensure that the multiple antagonistic sample images are normalized antagonistic sample images, before processing based on the multiple sample images and the multiple antagonistic sample images, it may be determined whether the multiple sample images are normalized sample images, and if the multiple sample images are non-normalized sample images, the multiple sample images may be normalized by using the mean and variance in the embodiment shown in fig. 2 to obtain normalized sample images, and each target sample image is input into the image processing model; if the plurality of sample images are normalized sample images, the target sample images are directly input into the image processing model, that is, the following S302 is executed to train the image processing model.

And S302, respectively inputting each target sample image into the image processing model to obtain the corresponding prediction label information of each target sample image.

The prediction tag information is tag information obtained by inputting each target sample image into the image processing model, and in order to distinguish tag information corresponding to each target sample image, the tag information obtained by the image processing model may be referred to as prediction tag information here.

For example, the image processing model may be an image classification model, an image recognition model, an image segmentation model, or the like, and may be specifically set according to actual needs.

When the image processing models are different depth learning models, the corresponding model outputs are different. For example, when the image processing model is an image classification model, the model output of the image classification model is the category to which the image belongs; when the image processing model is an image recognition model, the model output of the image recognition model is the recognition result of the image; when the image processing model is an image segmentation model, the model output of the image segmentation model is the segmentation result of the image.

After the prediction tag information and the tag information corresponding to each target sample image are acquired through the above S301 and S302, the following S303 may be executed:

and S303, updating the network parameters of the image processing model according to the corresponding predicted label information and label information of each target sample image.

It can be understood that, in order to make the trained image processing model better process both the conventional image and the confrontation image, the natural accuracy and the confrontation accuracy of the image processing model need to be trained, which can be realized by a loss function. The natural precision can be understood as the processing precision when the image processing model processes the conventional image, and the countermeasure precision can be understood as the processing precision when the image processing model processes the countermeasure image.

For example, when updating the network parameters of the image processing model according to the prediction tag information and the tag information corresponding to each target sample image in the plurality of sample images and the plurality of antagonistic sample images, for each target sample image, the loss function corresponding to the target sample image may be calculated and determined according to the prediction tag information and the tag information corresponding to the target sample image, so as to obtain the loss function corresponding to each target sample image, and then the network parameters of the image processing model may be updated according to the loss function corresponding to each target sample image.

For example, when updating the network parameters of the image processing model according to the loss function corresponding to each target sample image, considering that the plurality of sample images and the plurality of confrontation sample images included in the training sample image set are the same batch of sample images for performing one training operation, an average loss function corresponding to the loss function corresponding to each target sample image may be determined first; updating network parameters of the image processing model according to the average loss function, and directly determining the updated image processing model as a finally trained image processing model if the updated image processing model is converged; if the updated image processing model is not converged, the steps are executed again until the updated image processing model is converged, and the image processing model during convergence is determined as the finally trained image processing model, so that the final image processing model is obtained, the image processing model obtained through training can better process the conventional image and the counterimage, and the counterability of the image processing model is improved.

With the above description, after the image processing model is obtained through training, the image processing model can be used for processing the image to be processed in the application process, so that the accuracy of the image processing result is effectively improved.

Example four

Fig. 4 is a schematic structural diagram of a countermeasure sample image acquisition device 40 according to a fourth embodiment of the disclosure, and for example, referring to fig. 4, the countermeasure sample image acquisition device 40 may include:

a first obtaining unit 401, configured to obtain a plurality of sample images, and respective proportions and confrontation parameters of a plurality of confrontation operators.

A first processing unit 402, configured to determine a plurality of to-be-processed sample images to be subjected to countermeasure processing from the plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein, a plurality of sample images to be processed correspond to respective antagonistic operators.

The second processing unit 403 is configured to process the multiple sample images to be processed based on the countermeasure parameters of the countermeasure operators corresponding to the multiple sample images to be processed, so as to obtain a countermeasure sample image.

Optionally, the second processing unit 403 includes a first processing module, a second processing module, and a third processing module.

The first processing module is used for processing the plurality of sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the plurality of sample images to be processed to obtain the corresponding inverse normalized countermeasure samples of the plurality of sample images to be processed.

And the second processing module is used for carrying out normalization processing on the inverse normalized countermeasure samples corresponding to the multiple to-be-processed sample images to obtain the normalized countermeasure samples corresponding to the multiple to-be-processed sample images.

And the third processing module is used for obtaining a confrontation sample image according to the plurality of sample images to be processed and the plurality of normalized confrontation samples corresponding to the sample images to be processed respectively.

Optionally, the plurality of sample images to be processed are sample images after normalization processing; the first processing module comprises a first processing submodule and a second processing submodule.

And the first processing submodule is used for carrying out inverse normalization processing on the plurality of sample images to be processed to obtain the inverse normalized sample images corresponding to the plurality of processed sample images.

And the second processing submodule is used for carrying out countermeasure processing on the inverse normalized sample images respectively corresponding to the plurality of processed sample images based on the countermeasure parameters of the countermeasure operators respectively corresponding to the plurality of to-be-processed sample images to obtain the inverse normalized countermeasure samples respectively corresponding to the plurality of to-be-processed sample images.

Optionally, the third processing module includes a third processing sub-module and a fourth processing sub-module.

And the third processing submodule is used for inputting the normalized countermeasure samples corresponding to the multiple to-be-processed sample images into the image processing model to obtain the label information corresponding to the normalized countermeasure samples.

And the fourth processing submodule is used for determining the normalized countermeasure sample image as the countermeasure sample image if the label information corresponding to the sample image to be processed is the same as the label information corresponding to the normalized countermeasure sample image corresponding to the sample image to be processed.

Optionally, the device 40 for acquiring a countermeasure sample image further includes a second acquiring unit, a third processing unit, and an updating unit.

The second acquisition unit is used for acquiring a training sample image set; the training sample image set comprises a plurality of sample images which are not subjected to countermeasure processing in the plurality of sample images, a plurality of countermeasure sample images and label information corresponding to each target sample image, and the target sample image is any one of the plurality of sample images and the plurality of countermeasure sample images.

And the third processing unit is used for respectively inputting each target sample image into the image processing model to obtain the corresponding prediction label information of each target sample image.

And the updating unit is used for updating the network parameters of the image processing model according to the predicted label information and the label information corresponding to each target sample image.

Optionally, the update unit includes a first update module and a second update module.

And the first updating module is used for determining a loss function corresponding to the target sample image according to the prediction label information and the label information corresponding to the target sample image aiming at each target sample image.

And the second updating module is used for updating the network parameters of the image processing model according to the loss functions corresponding to the target sample images.

Optionally, the second update module includes a first update submodule and a second update submodule.

And the first updating submodule is used for determining the average loss function corresponding to the loss function corresponding to each target sample image.

And the second updating submodule is used for updating the network parameters of the image processing model according to the average loss function.

The device 40 for obtaining a countermeasure sample image provided in the embodiment of the present disclosure can execute the technical solution of the method for obtaining a countermeasure sample image shown in any one of the embodiments, and the implementation principle and the beneficial effect thereof are similar to those of the method for obtaining a countermeasure sample image, and reference may be made to the implementation principle and the beneficial effect of the method for obtaining a countermeasure sample image, which are not described herein again.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program, stored in a readable storage medium, from which at least one processor of the electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any of the embodiments described above.

Fig. 5 is a schematic block diagram of an electronic device 50 provided by an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 5, the apparatus 50 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data required for the operation of the device 50 can also be stored. The calculation unit 501, the ROM 502, and the RAM503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

A number of components in device 50 are connected to I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, or the like; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508, such as a magnetic disk, optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 50 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 501 may be a variety of general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 501 performs the respective methods and processes described above, such as the acquisition method of the resist sample image. For example, in some embodiments, the acquisition method of the challenge sample image may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 50 via ROM 502 and/or communication unit 509. When the computer program is loaded into the RAM503 and executed by the computing unit 501, one or more steps of the above-described acquisition method of the countermeasure sample image may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the acquisition method of the antagonistic sample image by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (17)

1. A method of obtaining a countermeasure sample image, comprising:

acquiring a plurality of sample images and respective proportions and confrontation parameters of a plurality of confrontation operators;

determining a plurality of sample images to be processed for countermeasure processing from a plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein the plurality of sample images to be processed correspond to respective antagonistic operators;

and processing the sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the sample images to be processed to obtain the countermeasure sample images.

2. The method of claim 1, wherein the processing the plurality of sample images to be processed based on the countermeasure parameters of the countermeasure operator corresponding to each of the plurality of sample images to be processed to obtain a countermeasure sample image comprises:

processing the sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the sample images to be processed to obtain corresponding inverse normalized countermeasure samples of the sample images to be processed;

carrying out normalization processing on the inverse normalization countermeasure samples corresponding to the multiple sample images to be processed to obtain the normalization countermeasure samples corresponding to the multiple sample images to be processed;

and obtaining the confrontation sample image according to the plurality of sample images to be processed and the normalized confrontation samples corresponding to the plurality of sample images to be processed respectively.

3. The method of claim 2, wherein the plurality of sample images to be processed are normalized sample images;

the processing the plurality of sample images to be processed based on the countermeasure parameters of the countermeasure operators corresponding to the plurality of sample images to be processed to obtain the inverse normalized countermeasure samples corresponding to the plurality of sample images to be processed includes:

carrying out inverse normalization processing on the plurality of sample images to be processed to obtain inverse normalized sample images corresponding to the plurality of processed sample images;

and performing countermeasure processing on the inverse normalized sample images corresponding to the plurality of processed sample images based on countermeasure parameters of countermeasure operators corresponding to the plurality of to-be-processed sample images to obtain the inverse normalized countermeasure samples corresponding to the plurality of to-be-processed sample images.

4. The method of claim 2 or 3, wherein the obtaining the confrontation sample image according to the plurality of to-be-processed sample images and the normalized confrontation sample corresponding to each of the plurality of to-be-processed sample images comprises:

inputting the normalized countermeasure samples corresponding to the sample images to be processed into an image processing model to obtain label information corresponding to the normalized countermeasure samples;

and if the label information corresponding to the sample image to be processed is the same as the label information corresponding to the corresponding normalized confrontation sample image, determining the normalized confrontation sample image as the confrontation sample image.

5. The method according to any one of claims 1-4, further comprising:

acquiring a training sample image set; the training sample image set comprises a plurality of sample images which are not subjected to countermeasure processing in the plurality of sample images, a plurality of countermeasure sample images and label information corresponding to each target sample image, and the target sample image is any one of the plurality of sample images and the plurality of countermeasure sample images;

inputting each target sample image into the image processing model respectively to obtain the corresponding prediction label information of each target sample image;

and updating the network parameters of the image processing model according to the predicted label information corresponding to each target sample image and the label information.

6. The method of claim 5, wherein the updating the network parameters of the image processing model according to the predicted label information and the label information corresponding to the target sample images comprises:

for each target sample image, determining a loss function corresponding to the target sample image according to the predicted label information corresponding to the target sample image and the label information;

and updating the network parameters of the image processing model according to the loss function corresponding to each target sample image.

7. The method of claim 6, wherein the updating the network parameters of the image processing model according to the loss function corresponding to each target sample image comprises:

determining an average loss function corresponding to the loss function corresponding to each target sample image;

and updating the network parameters of the image processing model according to the average loss function.

8. An acquisition apparatus of a countermeasure specimen image, comprising:

a first acquisition unit configured to acquire a plurality of sample images and respective proportions and countermeasure parameters of a plurality of countermeasure operators;

a first processing unit, configured to determine a plurality of to-be-processed sample images to be subjected to countermeasure processing from among the plurality of sample images according to respective proportions of the plurality of countermeasure operators; wherein the plurality of sample images to be processed correspond to respective antagonistic operators;

and the second processing unit is used for processing the sample images to be processed based on the countermeasure parameters of the corresponding countermeasure operators of the sample images to be processed to obtain the countermeasure sample images.

9. The apparatus of claim 8, wherein the second processing unit comprises a first processing module, a second processing module, and a third processing module;

the first processing module is configured to process the multiple sample images to be processed based on countermeasure parameters of countermeasure operators corresponding to the multiple sample images to be processed, so as to obtain inverse normalized countermeasure samples corresponding to the multiple sample images to be processed;

the second processing module is configured to perform normalization processing on the inverse normalized countermeasure samples corresponding to the multiple to-be-processed sample images, so as to obtain normalized countermeasure samples corresponding to the multiple to-be-processed sample images;

the third processing module is configured to obtain the countermeasure sample image according to the multiple to-be-processed sample images and the normalized countermeasure samples corresponding to the multiple to-be-processed sample images.

10. The apparatus according to claim 9, wherein the plurality of sample images to be processed are normalized sample images; the first processing module comprises a first processing submodule and a second processing submodule;

the first processing submodule is used for carrying out inverse normalization processing on the plurality of sample images to be processed to obtain inverse normalized sample images corresponding to the plurality of processed sample images;

the second processing submodule is configured to perform countermeasure processing on the inverse normalized sample images corresponding to the plurality of processed sample images based on countermeasure parameters of countermeasure operators corresponding to the plurality of to-be-processed sample images, so as to obtain inverse normalized countermeasure samples corresponding to the plurality of to-be-processed sample images.

11. The apparatus of claim 9 or 10, wherein the third processing module comprises a third processing sub-module and a fourth processing sub-module;

the third processing submodule is used for inputting the normalized countermeasure samples corresponding to the multiple to-be-processed sample images into the image processing model to obtain the label information corresponding to the normalized countermeasure samples;

the fourth processing submodule is configured to determine the normalized countermeasure sample image as the countermeasure sample image if the tag information corresponding to the to-be-processed sample image is the same as the tag information corresponding to the normalized countermeasure sample image.

12. The apparatus according to any of claims 8-11, further comprising a second obtaining unit, a third processing unit and an updating unit;

the second acquisition unit is used for acquiring a training sample image set; the training sample image set comprises a plurality of sample images which are not subjected to countermeasure processing in the plurality of sample images, a plurality of countermeasure sample images and label information corresponding to each target sample image, and the target sample image is any one of the plurality of sample images and the plurality of countermeasure sample images;

the third processing unit is configured to input each target sample image into the image processing model, so as to obtain prediction label information corresponding to each target sample image;

and the updating unit is used for updating the network parameters of the image processing model according to the predicted label information and the label information corresponding to each target sample image.

13. The apparatus of claim 12, wherein the update unit comprises a first update module and a second update module;

the first updating module is configured to determine, for each target sample image, a loss function corresponding to the target sample image according to the prediction tag information and the tag information corresponding to the target sample image;

and the second updating module is used for updating the network parameters of the image processing model according to the loss functions corresponding to the target sample images.

14. The apparatus of claim 13, wherein the second update module comprises a first update submodule and a second update submodule;

the first updating submodule is used for determining an average loss function corresponding to the loss function corresponding to each target sample image;

and the second updating submodule is used for updating the network parameters of the image processing model according to the average loss function.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of acquiring an image of a challenge sample of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the countermeasure sample image acquisition method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method of acquiring an image of a challenge sample according to any one of claims 1 to 7.

Images