CN114510592A

CN114510592A - Image classification method and device, electronic equipment and storage medium

Info

Publication number: CN114510592A
Application number: CN202011287253.3A
Authority: CN
Inventors: 陈思哲; 杨勇; 朱季峰
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-11-17
Filing date: 2020-11-17
Publication date: 2022-05-17

Abstract

The embodiment of the application provides an image classification method and device, electronic equipment and a storage medium, and relates to the technical field of machine learning. The method comprises the following steps: acquiring an image to be classified; inputting the images to be classified into a pre-trained image classification model to obtain a classification result of the image classification model; the image classification model is formed by training a countermeasure sample set and soft labels corresponding to the countermeasure sample set, the soft labels are generated by the target model according to the countermeasure sample set as input, and the countermeasure sample set is generated by amplifying the cross-domain image data set through at least two countermeasure sample generators. The image classification model of the embodiment of the application can basically restore the function and the test precision of the target model, so that an accurate classification result is predicted for the image to be classified, and the problem that the image classification service cannot be normally carried out due to the fact that a cloud server of the target model is down is solved.

Description

Image classification method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of machine learning technologies, and in particular, to an image classification method and apparatus, an electronic device, and a storage medium.

Background

As the depth network gradually deepens, the training cost of the network increases. A well-performing model has become an important achievement, or even property, for companies/research institutions. In order to protect the model and open the functions to the public, organizations often deploy the model in the cloud and open the API to the users, so as to implement "machine learning as a service" (MLaaS).

Machine learning as a service (MLaaS) is a set of cloud computing services that provide end-users with machine learning products and solutions for data transformation, model training, and final predictive analysis. These services typically allow users to upload tagged sharp images to meet their needs by creating specific workflows using built-in algorithms or directly using pre-trained models. For these services, the user can access the payment API interface provided by MLaaS and obtain the corresponding classification result through the selected input. In general, the user has no access to the details of the target model or parameters for optimization (i.e., black box),

because the target model is generally deployed at the cloud end by the model provider, when the cloud service is down, the user cannot call the target model, and the work progress of the user is further influenced.

Disclosure of Invention

Embodiments of the present invention provide an image classification method, apparatus, electronic device, and storage medium that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

In a first aspect, an image classification method is provided, which includes:

acquiring an image to be classified;

inputting the images to be classified into a pre-trained image classification model to obtain a classification result of the image classification model;

the image classification model is formed by training a countermeasure sample set and soft labels corresponding to the countermeasure sample set, the soft labels are generated by a target model by taking the countermeasure sample set as input, and the countermeasure sample set is generated by amplifying a cross-domain image data set through at least two countermeasure sample generators.

In one possible implementation, the method for training the image component model includes:

acquiring an image data set, inputting the image data set to at least two confrontation sample generators trained in advance for data augmentation, and acquiring the confrontation sample sets output by the at least two confrontation sample generators;

inputting the confrontation sample set into a target model, and obtaining a soft label output by the target model, wherein the soft label is used for representing the probability distribution of the classification result of the corresponding confrontation sample;

and training the initial model to be converged according to the confrontation sample set and the soft label, and using the initial model as an image classification model.

In one possible implementation, acquiring an image data set, inputting the image data set to at least two confrontation sample generators trained in advance for data augmentation, includes:

acquiring at least one image dataset, and creating a plurality of parallel processing threads according to the image dataset and the number of the confrontation sample generators, wherein the threads are used for inputting one image dataset to one confrontation sample generator and obtaining the confrontation sample set output by the confrontation sample generator;

wherein at least one of the image data set and the antagonistic sample generator processed by any two threads is different.

In one possible implementation, inputting the set of confrontation samples to the target model includes:

and if any thread obtains the countermeasure sample set at the current moment, inputting the countermeasure sample set into the target model.

In one possible implementation, training the initial model to converge according to the confrontation sample set and the soft label, as an image classification model, includes:

training an initial model of a current stage to be convergent according to a countermeasure sample set obtained by any thread at the current moment and a soft label corresponding to the countermeasure sample set, obtaining an initial model of a next stage, and taking the obtained initial model of the last stage as an image classification model;

and the initial model of the last stage is trained according to the generated last confrontation sample set and the corresponding soft label.

In one possible implementation, inputting the image data set to at least two pre-trained confrontation sample generators for data augmentation, before further comprising:

acquiring a training sample set, and training the confrontation sample generator according to the training sample set to obtain the trained confrontation sample generator;

wherein the training sample set differs from the image data set in data domain.

the data in the image data set is obtained by performing image transformation on training samples in a training sample set.

In one possible implementation, the challenge sample generator is a ResNet structure.

In a second aspect, a model recovery method is provided, including:

acquiring an image data set and at least two confrontation sample generators trained in advance, wherein the image data set is different from the data field of the training set of the confrontation sample generators;

respectively inputting the image data sets to at least two confrontation sample generators to obtain confrontation sample sets output by the at least two confrontation sample generators;

inputting the confrontation sample set into a target model to obtain a soft label output by the target model;

and training the initial model to be converged according to the confrontation sample set and the soft label, and taking the initial model as a recovered target model.

In a third aspect, an image classification apparatus is provided, including:

the image to be classified acquisition module is used for acquiring an image to be classified;

the input module is used for inputting the images to be classified into the pre-trained image classification model to obtain the classification result of the image classification model;

In one possible implementation manner, the image classification apparatus further includes: a model restoration module, the model restoration module comprising:

the image data set acquisition unit is used for acquiring an image data set, inputting the image data set to at least two confrontation sample generators trained in advance for data augmentation, and acquiring the confrontation sample sets output by the at least two confrontation sample generators;

the soft label acquisition unit is used for inputting the confrontation sample set into the target model and acquiring a soft label output by the target model, wherein the soft label is used for representing the probability distribution of the classification result of the corresponding confrontation sample;

and the training unit is used for training the initial model to be convergent according to the confrontation sample set and the soft label, and the initial model is used as an image classification model.

In one possible implementation, the image dataset acquisition unit is specifically configured to: acquiring at least one image dataset, and creating a plurality of parallel processing threads according to the image dataset and the number of the confrontation sample generators, wherein the threads are used for inputting one image dataset to one confrontation sample generator and obtaining the confrontation sample set output by the confrontation sample generator;

In one possible implementation manner, the soft tag obtaining unit is specifically configured to: and if any thread obtains the countermeasure sample set at the current moment, inputting the countermeasure sample set into the target model.

In one possible implementation, the training unit is specifically configured to: training an initial model of a current stage to be converged according to a countermeasure sample set obtained by any one thread at the current moment and a soft label corresponding to the countermeasure sample set to obtain an initial model of a next stage, and taking the obtained initial model of the last stage as an image classification model;

In one possible implementation, the model recovery module further includes:

the first generator training unit is used for acquiring a training sample set, training the confrontation sample generator according to the training sample set and acquiring the trained confrontation sample generator;

wherein, the migratability of the confrontation sample generator meets the preset condition; the training sample set differs from the data domain of the image data set.

In one possible implementation, the model recovery module further includes:

the second generator training unit is used for acquiring a training sample set, training the confrontation sample generator according to the training sample set and acquiring the confrontation sample generator after training;

In a fourth aspect, an embodiment of the present application provides a model recovery apparatus, including:

the device comprises a preparation module, a comparison module and a comparison module, wherein the preparation module is used for acquiring an image data set and at least two confrontation sample generators trained in advance, and the image data set is different from the training sample set of the confrontation sample generators in data field;

the countermeasure sample set acquisition module is used for inputting the image data sets into at least two countermeasure sample generators respectively and acquiring countermeasure sample sets output by the at least two countermeasure sample generators;

the soft label acquisition module is used for inputting the confrontation sample set into the target model to acquire a soft label output by the target model;

and the recovery module is used for training the initial model to be convergent according to the confrontation sample set and the soft label, and the initial model is used as a recovered target model.

In a fifth aspect, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of the method according to the first aspect or the second aspect.

In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the method as provided in the first or second aspect.

According to the image classification method, the device, the electronic equipment and the storage medium provided by the embodiment of the invention, the countermeasure sample set is obtained and input into the target model to obtain the soft label, the initial model is trained by utilizing the countermeasure sample set and the soft label, and the trained initial model is used as the image classification model which can basically recover the function and the test precision of the target model, so that an accurate classification result is predicted for the image to be classified, and the problem that the image classification service cannot be normally carried out due to the crash of a cloud server of the target model is avoided.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a logic diagram of a model recovery method;

FIG. 2 is a flowchart illustrating an image classification method according to an embodiment of the present application;

FIG. 3 is a schematic flowchart of recovering a target model according to an embodiment of the present disclosure;

FIG. 4 is a diagram illustrating obtaining a challenge sample set using multiple threads according to an embodiment of the present disclosure;

fig. 5 is a schematic flowchart of a model recovery method according to an embodiment of the present application;

fig. 6 is a schematic diagram of an application scenario provided in an embodiment of the present application;

fig. 7 is a schematic diagram of another application scenario provided in the embodiment of the present application;

FIG. 8 is a schematic flow chart diagram illustrating a model recovery method according to another embodiment of the present application;

fig. 9 is a schematic structural diagram of an image classification apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a model recovery module according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a model recovery apparatus according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

As can be seen from the above description, when a model deployed in a cloud goes down, a user cannot call a target model, so that a work progress is affected, and therefore, in order to solve the above problems, in the prior art, a model recovery method is usually adopted to recover the target model to the local, so that image classification is performed by using the model recovered to the local.

FIG. 1 is a logic diagram of a model recovery method, and as shown in FIG. 1, the generation of the model itself is a process requiring a lot of manpower and material resources, and first, a creator of the model first collects a lot of sample image sets P_V(X), the sample image in the sample image set may be X_iShowing that the acquired sample image needs to be labeled (annotated), that is, the label of the sample image is determined, that is, the training sample set D with the sample label can be obtained_V＝{(x_i,y_i) Denotes wherein y is_iRepresenting a sample image x_iWhen labeling, the initial structure, initial parameters, etc. of the model need to be determined, and the model can be expressed by the expression F_VThat is, the final model is obtained by training the model with the training sample having the sample label until convergence, and the expression y is F_V(x) Representing, i.e. when x is input to the model, the model output F_V(x) An opaque black box is deployed at the cloud end, and an inquiry interface is provided for the outside, so that the outside accesses the black box through the inquiry interface, and inputs images into the black box, and only the output of the model can be obtained, but the operation process of the model cannot be obtained.

The application provides an image classification method, an image classification device, an electronic device and a computer-readable storage medium, which aim to solve the above technical problems in the prior art.

To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Fig. 2 is a schematic flowchart of an image classification method according to an embodiment of the present application, and as shown in fig. 2, the method includes:

and S101, acquiring an image to be classified.

The image to be classified is the image to be classified. The images to be classified may be live images of a live broadcast platform, images of chat content, monitoring images, or images of other scenes that need to be classified, and are not specifically limited herein.

S102, inputting the image to be classified into a pre-trained image classification model, and obtaining a classification result of the image classification model.

It should be noted that, the image classification model in the embodiment of the present application is not a model deployed in the cloud (referred to as a target model for short), but a model obtained by restoring a target model to the local, and by restoring the target model to the local, the problem that the target model cannot be called due to downtime of a cloud server can be avoided.

The image classification model of the embodiment of the application is formed by training the countermeasure sample set and the soft label output by the target model on the countermeasure sample set. The countermeasure sample set is a set composed of countermeasure samples, the model gives an error output with high confidence level due to the subsequent input, and interference information is specially added to the common samples by the countermeasure samples, so that the processing result of the target model on the interference information can be analyzed by analyzing the soft label generated by the target model on the countermeasure samples, and the prediction capability of the target model can be recovered.

The challenge sample set of the embodiments of the present application is generated by augmenting a cross-domain image sample set with at least two challenge sample generators.

It should be understood that, when the image data set and the training sample set adopted by the training countermeasure sample generator have different data fields, the countermeasure sample set is generated in a cross-domain manner, so that the countermeasure sample set is generated in a cross-domain manner, and compared with when the image data set and the training sample set adopted by the training countermeasure sample generator have the same data fields, the high-quality augmentation of the countermeasure sample can be ensured.

As can be seen from the above description, in the embodiments of the present application, the confrontation samples in the confrontation sample set are used as training samples, and the target model trains the initial model to converge by using the soft labels output by the confrontation sample set as sample labels, so as to obtain the image classification model.

According to the method and the device, the countermeasure sample set is obtained, the countermeasure sample set is input into the target model to obtain the soft labels, the initial model is trained by using the countermeasure sample set and the soft labels, the trained initial model serves as the image classification model, the image classification model can basically recover the function and the test precision of the target model, accurate classification results can be predicted for the images to be classified, and the problem that the image classification business cannot be normally carried out due to downtime of a cloud server of the target model is solved. Fig. 3 is a schematic flowchart of a process of recovering a target model according to an embodiment of the present application, and as shown in fig. 3, the method includes:

s201, acquiring an image data set, inputting the image data set to at least two confrontation sample generators trained in advance for data augmentation, and obtaining the confrontation sample sets output by the at least two confrontation sample generators.

It should be understood that the data set is the basis for training and testing the neural network, and the number of classification problems solved by the image data set is not particularly limited in the embodiments of the present application, and may be, for example, an image data set that solves a binary classification problem, or an image data set that solves more categories, such as a ten-category problem. The two-classification problem, i.e. the data in the image dataset is either of type 1 or type 2, such as an image dataset consisting of apple pictures depicting either green apples or red apples, is an image dataset solving the two-classification problem.

After the image dataset is obtained, the image dataset is not directly input into the target model to be restored, but the image dataset is input into a pre-trained confrontation sample generator to generate a confrontation sample set, and as the name suggests, the confrontation sample set is a set formed by confrontation samples (adaptive extensions). The countersample is an input sample formed by deliberately adding a subtle disturbance to the data set, the input after the disturbance causing the model to give an erroneous output with high confidence.

An example generator (example generator) is a generator for generating example of an experiment, the aim of which is to generate data that makes the classification results of an initial model and a target model inconsistent, and the aim of the initial model is to imitate the output of the target model. This forms a system for gaming one over the other.

The confrontation sample generator is a multi-branch structure, each branch represents a class, and finally, a loss function controlled by the class is added to the generator to restrict each branch to generate data with specific class attributes. This has the advantage of controlling the challenge sample generator to generate training data with sufficient diversity to the initial model. If not controlled, the challenge sample generator will only generate training data with single or partial attributes to the initial model, which also causes the problem of model collapse of the conventionally generated challenge network. If the initial model has enough capacity to resist the sample generator, the attribute of the model to be restored can be effectively copied, so that the function of the target model can be restored or a series of subsequent actions can be carried out. For example, in verifying the defense ability against attacks on the target model, the countermeasure sample generator obtained in the embodiment of the present application may be used to generate a countermeasure sample, and the countermeasure sample may be migrated to the target model to complete the defense ability verification.

The data augmentation mode of the embodiment of the application can be that the same image data set is input into a plurality of different confrontation sample generators, and the outputs of the confrontation sample generators are not the same because the confrontation sample generators are different, so that a plurality of confrontation sample sets can be obtained based on one image data set. Further, before inputting the data set to a pre-trained confrontation sample generator, further comprising:

and acquiring a training sample set, and training the confrontation sample generator according to the training sample set to obtain the trained confrontation sample generator.

The training sample set of the embodiment of the present application is different from the data field of the image data set, for example, the data field in the image data set may be a cat or a dog, and the data field of the training sample set may be a face image.

It should be understood that the confrontation sample generator is trained with the confrontation sample discriminator, and the confrontation sample generator is part of a generated confrontation network (GAN), and the generated confrontation network includes at least two parts, namely a confrontation sample generator (generated Model) and a confrontation sample discriminator (discriminating Model), wherein the confrontation sample generator is used for generating the confrontation sample and making the generated picture (i.e. the confrontation sample) look "real, and the confrontation sample discriminator is used for correctly discriminating whether a picture is generated or exists, so that the mode of the generated confrontation network is: the generation model generates some pictures- > the discrimination model learns to distinguish the generated pictures from the real pictures- > the generation model improves the generation model according to the discrimination model, and new pictures- > … are generated. Specifically, the training process of the confrontation sample generator is as follows:

(1) the parameters of both networks of generator G and discriminator D are initialized.

(2) N samples are taken from the training set, the n samples are input to a generator, and n challenge samples are generated using the defined noise profile.

(3) And the fixed generator G trains the discriminator D by using n confrontation samples, so that the trainer D can distinguish true from false as much as possible.

(4) After k times of cyclic update of discriminator D, 1 time of generator G is updated.

(5) After multiple updating iterations, in an ideal state, the final discriminator D cannot distinguish whether the picture comes from a real training sample set or from a sample generated by the generator G, the discrimination probability is 0.5 at this time, and the training is completed.

According to the embodiment of the application, the effectiveness of the countermeasure sample on the model recovery is found to depend on the migration capability of the sample to a great extent through experiments, so that when the countermeasure sample generator is selected, the mobility of the countermeasure sample generator needs to be quantitatively evaluated, and the countermeasure sample generator with high mobility is used for generating the countermeasure sample.

Alternatively, the embodiment of the present application may quantitatively evaluate the migratability of the anti-sample generator through an attribution map. Given an input data, attribution of a depth model refers to calculating an importance value for each dimension of the input for a certain output of the model. This value represents the degree of importance of the input dimension to this output. The greater the importance value, the greater the impact of changing the data of the dimension on this output. Since each input dimension has a attributed value corresponding to it, the attributed graph size and the input size are the same. Current model attribution methods can be broadly divided into two categories: perturbation-based attribution and gradient-based attribution. Based on the attribution of the disturbance, the change of the output is observed by making some change to a certain dimension or a certain dimension of the input data, and the importance of the dimension is measured by the change size of the output. In estimating mobility, the model generates a set of attribution maps for a given set of probe data. Thus each model can be seen as being embedded in a model space that is made up of all dimensions of the attributed graph. The mobility between models is measured by the distance between the model's embedded points in the model space.

On the basis of the above embodiments, as an alternative embodiment, the model result of the countermeasure sample generator of the embodiment of the present application adopts a ResNet structure. ResNet is a residual network, and can effectively solve the problems of gradient dispersion/explosion and network degradation. Through verification, the confrontation sample generator adopting the ResNet structure has high mobility.

In addition, the training sample set adopted by the training confrontation sample generator in the embodiment of the application is different from the data field of the image data set, so that the high-mobility advantage of the confrontation sample generator is utilized to ensure the high-quality augmentation of the confrontation sample.

It should be noted that, when the image data set and the training sample set adopted by the training countermeasure sample generator have different data fields, which is called to generate the countermeasure sample set in a cross-domain manner, the embodiment of the present application may obtain the image data set by performing image transformation on the training sample set in addition to generating the countermeasure sample set in a cross-domain manner. For example, a set of image data sets may be obtained by performing translation processing on training samples in a training sample set, rendering the training samples in the training sample set, and then obtaining a set of image data sets, scaling the training samples in the training sample set, and similarly obtaining a set of image data sets.

S202, inputting the confrontation sample set into the target model, and obtaining the soft label output by the target model.

According to the embodiment of the application, the countermeasure samples in the countermeasure sample set are input into the target model, and the soft label output by the target model is obtained.

The soft label is to label each sample with a plurality of class labels, representing the probability distribution of the classification result. For example, when multiple annotators annotate the same sample, different annotators may label the same sample into different categories due to the confusability of the sample itself and the subjectivity of the annotators during the annotation. And the samples are labeled by adopting soft labels, so that the correlation among different classes can be described more easily. For example, if a pair of anti samples is a three-classification sample, the soft label result output by the target model may be: category one: 32%, class two: 43%, Category three: 35%, i.e. labeling the probabilities of the confrontation sample in the three classes, respectively. Compared with the acquisition of a hard tag, the acquisition of the soft tag output by the target model can acquire more details generated by the target model during prediction, thereby being more beneficial to the recovery of the model.

And S203, training the initial model to be convergent according to the confrontation sample set and the soft label, and using the initial model as an image classification model.

In the training field of neural network models, a cross entropy loss function is usually calculated according to the output of a hard tag and an initial model, and parameters of the initial model are adjusted to make the model converge, but the amount of information contained in the hard tag is too small compared with that contained in a soft tag.

According to the model recovery method, the image data set is firstly input into the countermeasure sample generator to carry out data amplification to obtain the countermeasure sample set, training data with enough diversity can be generated to be used by the initial model, the countermeasure sample set is input into the target model to obtain the soft label, compared with the method of obtaining the hard label, more prediction details of the target model can be obtained, the initial model is trained to be converged according to the countermeasure sample and the soft label, recovery information can be increased, recovery speed is higher, and feasibility can be guaranteed. Through verification, the model recovery method can achieve the excellent effects that the Test Accuracy (Test Accuracy) is higher than 75% and the model Function recovery Rate (Function recovery Rate) is higher than 91%, and is superior to the existing model recovery method.

On the basis of the above embodiments, as an alternative embodiment, the number of the confrontation sample generators is at least two.

Optionally, the training sample sets adopted by any two confrontation sample generators in the embodiment of the present application in training are different, for example, the data field of one training sample set is an image of the face recognition field, and the other training sample set is an image of the fruit recognition field. By arranging the cross-domain confrontation sample generator, a cross-domain confrontation sample set can be further obtained, the initial model is trained by utilizing the cross-domain confrontation sample set, so that the prediction capability of the finally obtained image classification model is also cross-domain, and the range and the prediction precision of an application scene are improved; in addition, the training sample sets used by any two confrontation sample generators in training can also be different training sample sets in the same data field.

On this basis, the embodiment of the present application acquires an image dataset, inputs the image dataset to at least two confrontation sample generators trained in advance, and includes:

the method comprises the steps of obtaining at least one image data set, creating a plurality of threads for parallel processing according to the image data set and the number of confrontation sample generators, enabling the threads to be used for inputting one image data set to one confrontation sample generator, and obtaining the confrontation sample set output by the confrontation sample generator.

It is to be appreciated that the image data set of the embodiments of the present application may be in the same data domain as the training samples of the confrontational sample generator, thereby facilitating the confrontational sample generator to generate the confrontational samples with higher misleading. Based on this, the number of image data sets corresponds to the number of countermeasure sample generators, and the threads to be created further correspond to the number of countermeasure sample generators.

In this case, if the number of image data sets is M and the number of challenge sample generators is N, where M is not less than 1 and N is greater than 1, M × N threads need to be constructed.

In this embodiment of the present application, the image data sets processed by any two threads are different from the confrontation sample generator, for example, the two threads may be to input the same image data set to different confrontation sample generators, or to input different image data sets to the same confrontation sample generator, and this embodiment of the present application is not limited in particular.

According to the embodiment of the application, the countermeasure sample set is generated by constructing the multiple parallel threads, so that the generation speed of the countermeasure sample set is greatly increased compared with the speed of serially generating the countermeasure sample set, and a foundation is laid for increasing the model recovery speed.

FIG. 4 is a schematic diagram of obtaining a confrontation sample set by using multiple threads according to an embodiment of the present application, and as shown in FIG. 4, 3 threads of processing are created, where thread 1 is used for processing an image data set D_s1Input to the confrontation sample generator G_adv1Obtaining a confrontation sample Generator G_adv1Outputted confrontation sample set D_st1(ii) a Wherein thread 2 is used to apply an image dataset D_s2Input to the confrontation sample generator G_adv2Obtaining a confrontation sample Generator G_adv2Outputted confrontation sample set D_st2(ii) a Wherein thread 3 is used to apply an image dataset D_s3Input to the confrontation sample generator G_adv3Obtaining a confrontation sample Generator G_adv3Outputted confrontation sample set D_st3. It is noted that the image data set D_s1～D_s3May be the same or different.

On the basis of the above embodiments, as an alternative embodiment, inputting the confrontation sample set to the target model includes:

That is to say, in the embodiment of the present application, by parallel multiple threads, after any one thread obtains a countermeasure sample set, the obtained countermeasure sample set is input to the target model without waiting for the processing results of other threads, so as to obtain the soft label output by the target model. This arrangement can shorten the period of model recovery.

Taking the embodiment shown in FIG. 4 as an example, if thread 1 first obtains the countermeasure sample set D_st1Then it will confront the sample set D_st1Inputting the target model in the confrontation sample set D_st1During the process of inputting the target model, or against the sample set D_st1After entering the target model, thread 2 obtains a set of confrontation samples D_st2Then it will confront the sample set D_st2Inputting the target model in the confrontation sample set D_st2During the process of inputting the target model, or against the sample set D_st2After entering the target model, thread 3 obtains a set of confrontation samples D_st2Then it will confront the sample set D_st3And inputting the target model. As is clear from the above description, the data (countermeasure sample set) input into the target model is actually realized by inputting data into an interface exposed to the target model, and since the interface has a function of receiving a large amount of data in real time at the time of actual application, the interface is not affected even when a plurality of countermeasure sample sets are simultaneously input into the target model.

On the basis of the above embodiments, as an alternative embodiment, training an initial model to converge according to the confrontation sample set and the soft label, as an image classification model, includes:

It should be noted that the time for obtaining the countermeasure sample set by each thread in the embodiment of the present application is not consistent, and further, the time for outputting the soft label by the target model according to the countermeasure sample set is also not consistent. For example, although the time for thread 1 to obtain the countermeasure sample set 1 is earlier than the time for thread 2 to obtain the countermeasure sample set 2, the time for the target model to obtain the soft label (set) 1 from the countermeasure sample set 1 is later than the time for the target model to obtain the soft label (set) 2 from the countermeasure sample set 2 and the soft label (set) 2, the initial model of the current stage is trained to converge according to the countermeasure sample set 2 and the soft label (set) 2 to obtain the initial model of the next stage, and then the initial model of the next stage is trained according to the countermeasure sample set 1 and the soft label (set) 1 to obtain the initial model of the last stage, ….

Fig. 5 is a flowchart illustrating a model recovery method according to an embodiment of the present application, and as shown in fig. 5, 3 threads are created, where thread 1 is used to apply an image data set D_s1Input to the confrontation sample generator G_adv1Obtaining a confrontational sample Generator G_adv1Outputted confrontational sample set D_st1(ii) a Wherein thread 2 is used to apply an image dataset D_s2Input to the confrontation sample generator G_adv2Obtaining a confrontation sample Generator G_adv2Outputted confrontation sample set D_st2(ii) a Wherein thread 3 is used to apply an image dataset D_s3Input to the confrontation sample generator G_adv3Obtaining a confrontation sample Generator G_adv3Outputted confrontation sample set D_st3(ii) a To-be-confronted sample set D_st1～D_st3Respectively input to a target model Mv to obtain corresponding soft labels L₁～L₃For convenience of description, in the embodiment shown in fig. 5, the confrontation sample set D_st1Is obtained earlier than the challenge sample set D_st2Antagonistic sample set D_st2Is obtained earlier than the challenge sample set D_st3Soft label L₁Is obtained earlier than the soft label L₂Soft label L₂Is obtained earlier than the soft label L₃So in FIG. 5, first according to the challenge sample set D_st1And a soft label L₁Training an initial model M_s0Until convergence, obtaining an initial model M of the first stage_s1Then according to the confrontation sample set D_st2And a soft label L₂Training initial model M_s1Until convergence, obtaining the initial model M of the second stage_s2According to the confrontation sample set D_st3And a soft label L₃Training initial model M_s2Until convergence, obtaining the initial model M of the third stage_s3The initial model M_s3As an image classification model.

On the basis of the above embodiments, after the initial model is trained to obtain the image classification model, the image acquisition model may also be deployed on other servers, so that when the server of the target model is down, the image classification can be performed through the image acquisition models deployed on other servers.

Referring to fig. 6, in an application scenario diagram provided in the embodiment of the present application, a server 11 and a server 12 are respectively connected to a terminal 13 through a network, for example, through a wired or wireless network connection, and the server 11 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing a cloud computing service.

The server 11 is deployed with an object model, the object model is a model with complete image classification capability and accuracy, and the server 12 is deployed with a model obtained by restoring the object model, that is, the image classification model in the above embodiments, which restores most of the image classification capability and accuracy of the object model.

The terminal 13 may include a mobile phone, a smart tv, a tablet Computer, a notebook Computer, or a Personal Computer (PC), etc. When the terminal 13 needs to perform image classification, a classification request is initiated to a target model deployed on the server 11, and the target model predicts a picture to be classified contained in the classification request to obtain a classification result. When the server 11 is down, the terminal 13 initiates a classification request to an image classification model deployed on the server 12 when image classification needs to be performed, and the image classification model predicts a to-be-classified image contained in the classification request to obtain a classification result, so that the problem that image classification work cannot be operated due to the down of the server 11 is avoided.

The execution method of the server in the embodiment of the application can be completed in a cloud computing (cloud computing) mode, and the cloud computing is a computing mode, and distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can obtain computing power, storage space and information service according to needs. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand.

As a basic capability provider of cloud computing, a cloud computing resource pool (called as an ifas (Infrastructure as a Service) platform for short is established, and multiple types of virtual resources are deployed in the resource pool and are selectively used by external clients.

According to the logic function division, a PaaS (Platform as a Service) layer can be deployed on an IaaS (Infrastructure as a Service) layer, a SaaS (Software as a Service) layer is deployed on the PaaS layer, and the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, a web container, etc. SaaS is a variety of business software, such as web portal, sms, and mass texting. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

Referring to fig. 7, in another application scenario diagram provided in the embodiment of the present application, the local database 21 and the local database 22 are respectively connected to the terminal 23 directly through a network, for example, through a wired or wireless network connection, and the server 11 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services.

An application program capable of implementing the target model is deployed on the local database 21, the target model is a model with complete image classification capability and accuracy, and the application program has a validity period of use, and a model after restoring the target model, that is, the image classification model in the above embodiments, is deployed on the local database 22, and the image classification model restores most of the image classification capability and accuracy of the target model.

The terminal 13 may include a mobile phone, a smart tv, a tablet Computer, a notebook Computer, or a Personal Computer (PC), etc. When the terminal 13 needs to perform image classification, in the valid period of the application program, classification requests may be respectively initiated to the application program deployed on the local database 21 and the image classification model deployed on the local database 22, so that the target model and the image classification model of the application program operation may respectively predict the pictures to be classified included in the classification requests, and obtain two classification results, that is, classification result 1 and classification result 2. If the two classification results are different, the image classification model is trained by using the classification result 1 obtained by the image to be classified and the target model, so that the prediction accuracy of the image classification model is improved, and thus, when the application program fails, the image classification model in the local database 22 can be used for image classification. In order to verify whether the model recovery method is effective or not, the embodiment of the application recovers the trained ResNet50 neural network on a CIFAR10(5 thousands of RGB color maps with the size of 32 × 32) based on a Keras deep learning framework, and the Test Accuracy (TA, Test Accuracy) reaches 87.91%.

Using all 120K samples of ImageNet-tiny to perform 3 times of cross-domain black box confrontation data augmentation (qz, confrontation sample generators are respectively trained on drawing, animation and natural pictures) to obtain 480K samples, using the 480K samples to query a black box to obtain soft labels, respectively training 3 models including Xceptation, DenseNet and inclusion V3 for the sample-label pairs, and performing 10 epoch iterations. Finally, the accuracy of the three models is tested by using 1 ten thousand test sets of CIFAR10, so that the TA is 81.31%, 80.80% and 75.15%, and the corresponding FRR (Function recovery Rate) is 92.5%, 91.9% and 85.5%, respectively. It can be seen that the function of the original model can be largely restored by using the embodiment of the invention.

An embodiment of the present application further provides a modeling method, see fig. 8, including:

s301, acquiring an image data set and at least two confrontation sample generators trained in advance, wherein the image data set is different from the data field of the training set of the confrontation sample generators.

According to the embodiment of the application, the data fields of the image data set and the training set of the countermeasure sample generator are different, the cross-domain image data set is used for generating the countermeasure sample set, and compared with the situation that the data fields of the image data set and the training sample set adopted by the training countermeasure sample generator are the same, high-quality augmentation of the countermeasure samples can be guaranteed.

S302, the image data sets are respectively input into at least two confrontation sample generators, and confrontation sample sets output by the at least two confrontation sample generators are obtained.

According to the embodiment of the application, the image data sets are input into the plurality of confrontation sample generators, and because the internal parameters of different confrontation sample generators are different, the confrontation sample sets output by different confrontation sample generators are also different, so that data augmentation is realized.

And S303, inputting the confrontation sample set into the target model to obtain a soft label output by the target model.

According to the embodiments, compared with the method for acquiring only the hard tag, the method for acquiring the soft tag output by the target model can acquire more details generated by the target model during prediction, so that the method is more beneficial to the recovery of the model.

And S304, training the initial model to be converged according to the confrontation sample set and the soft label, and taking the initial model as a recovered target model.

It should be understood that step S304 is the same as or similar to step S203 in the above embodiments, and is not described herein again.

The model restoration method according to the embodiment of the present application may be performed by using a plurality of threads processed in parallel as described in the above embodiment when inputting the image data set to at least two antagonistic sample generators trained in advance, thereby further increasing the speed of model restoration.

An embodiment of the present application provides an image classification apparatus, as shown in fig. 9, the apparatus may include: the image to be classified acquiring module 101 and the input module 102 specifically:

the image to be classified acquiring module 101 is used for acquiring an image to be classified;

The input module 102 is configured to input an image to be classified into a pre-trained image classification model, and obtain a classification result of the image classification model;

it should be noted that the image classification model in the embodiment of the present application is not a model deployed in the cloud (referred to as a target model for short), but a model obtained by restoring the target model to the local, and by restoring the target model to the local, the problem that the target model cannot be called due to the cloud server being down can be avoided.

It should be noted that, when the image data field and the training sample field adopted by the training confrontation sample generator are different, it is called to generate the confrontation sample set in a cross-domain manner, so that the confrontation sample set is generated in a cross-domain manner, and the high-quality augmentation of the confrontation sample is ensured.

The method and the device for image classification are applicable to black box attacks, the countermeasure samples in the countermeasure sample set are used as training samples, the soft labels output by the target model are used as sample labels to train the initial model to be convergent, and then the image classification model is obtained.

The soft label of the embodiment of the application is generated by taking a countermeasure sample set as an input of a target model, wherein the countermeasure sample set is generated by amplifying a cross-domain image data set through at least two countermeasure sample generators.

According to the method and the device for predicting the image classification model, the countermeasure sample set is obtained, the countermeasure sample set is input into the target model to obtain the soft label, the initial model is trained by using the countermeasure sample set and the soft label, the trained initial model is used as the image classification model, and the image classification model can basically recover the function and the test precision of the target model, so that an accurate classification result is predicted for the image to be classified, and the problem that the image classification service cannot be normally carried out due to downtime of a cloud server of the target model is solved. The image classification device provided in the embodiment of the present invention specifically executes the processes of the method embodiments, and details of the contents of the method embodiments of the image classification device are specifically referred to, and are not repeated herein. According to the image classification device provided by the embodiment of the invention, the countermeasure sample set is obtained and input into the target model to obtain the soft label, the initial model is trained by utilizing the countermeasure sample set and the soft label, the trained initial model is used as the image classification model, and the image classification model can basically recover the function and the test precision of the target model, so that an accurate classification result is predicted for the image to be classified, and the problem that the image classification service cannot be normally developed due to the downtime of a cloud server of the target model is avoided.

In one possible implementation manner, the image classification apparatus further includes: a model recovery module for recovering the target model, referring to fig. 10, the model recovery module includes:

an image data set obtaining unit 201, configured to obtain an image data set, input the image data set to at least two confrontation sample generators trained in advance for data augmentation, and obtain the confrontation sample sets output by the at least two confrontation sample generators;

The confrontation sample generator is a multi-branch structure, each branch represents a class, and finally, a loss function controlled by the class is added to the generator to restrict each branch to generate data with specific class attributes. This has the advantage of controlling the confrontation sample generator to generate training data with sufficient diversity to the initial model. If not controlled, the challenge sample generator will only generate training data with single or partial attributes to the initial model, which also causes the problem of model collapse of the conventionally generated challenge network. If the initial model has strong enough capacity for resisting the sample generator, the attribute of the model to be recovered can be effectively copied, so that the function of the target can be stolen or a series of subsequent attacks can be carried out. For example, in verifying the defense ability against attacks on the target model, the countermeasure sample generator obtained in the embodiment of the present application may be used to generate a countermeasure sample, and the countermeasure sample may be migrated to the target model to complete the defense ability verification.

A soft label obtaining unit 202, configured to input the confrontation sample set to the target model, and obtain a soft label output by the target model, where the soft label is used to represent a probability distribution of a classification result of the corresponding confrontation sample.

The soft label is to label each sample with a plurality of class labels, representing the probability distribution of the classification result. For example, when multiple annotators annotate the same sample, different annotators may label the same sample into different categories due to the confusability of the sample itself and the subjectivity of the annotators during the annotation. And the samples are labeled by adopting soft labels, so that the correlation among different classes can be described more easily. For example, if a pair of anti samples is a three-class sample, the soft label result output by the target model may be: category one: 32%, class two: 43%, Category three: 35%, i.e. labeling the probabilities of the confrontation sample in the three classes, respectively. Compared with the acquisition of a hard tag, the acquisition of the soft tag output by the target model can acquire more details generated by the target model during prediction, thereby being more beneficial to the recovery of the model.

And the training unit 203 is used for training the initial model to be converged according to the confrontation sample set and the soft label, so as to serve as an image classification model.

In the training field of neural network models, a cross entropy loss function is usually calculated according to the output of a hard tag and an initial model, and parameters of the initial model are adjusted to make the model converge, but the amount of information contained in a useful hard tag is too small compared with that of information contained in a soft tag.

According to the model recovery method, the image data set is firstly input into the countermeasure sample generator to obtain the countermeasure sample set, training data with enough diversity can be generated and used by the initial model, the countermeasure sample set is input into the target model to obtain the soft label, more prediction details of the target model can be obtained compared with the case of obtaining the hard label, the initial model is trained to be converged according to the countermeasure sample and the soft label, recovery information can be increased, recovery speed is higher, and feasibility can be guaranteed. Through verification, the model recovery method can achieve the excellent effects that the Test Accuracy (Test Accuracy) is higher than 75% and the model Function recovery Rate (Function recovery Rate) is higher than 91%, and is superior to the existing model recovery method.

In one possible implementation, the number of challenge sample generators is at least two.

The image dataset acquisition unit is specifically configured to: acquiring at least one image dataset, and creating a plurality of parallel processing threads according to the image dataset and the number of the confrontation sample generators, wherein the threads are used for inputting one image dataset to one confrontation sample generator and obtaining the confrontation sample set output by the confrontation sample generator;

It will be appreciated that the image data set of the embodiments of the present application is preferably in the same data domain as the training samples of the confrontation sample generator, thereby facilitating the generation of more misleading confrontation samples by the confrontation sample generator. Based on this, the number of image data sets corresponds to the number of countermeasure sample generators, and the threads to be created further correspond to the number of countermeasure sample generators.

In this embodiment of the present application, the image data set and the confrontation sample generator processed by any two threads are different from each other, for example, the two threads may input the same image data set to different confrontation sample generators, may input different image data sets to different confrontation sample generators, and may input different image data sets to the same confrontation sample generator, which is not limited in this embodiment of the present application.

That is to say, in the embodiment of the present application, by paralleling multiple threads, after any one thread obtains a countermeasure sample set, the obtained countermeasure sample set is input to the target model without waiting for the processing results of other threads, so as to obtain the soft label output by the target model. This arrangement can shorten the period of model recovery.

In one possible implementation, the training unit is specifically configured to:

In one possible implementation, the model recovery module further includes:

wherein, the migratability of the confrontation sample generator meets the preset condition; the training sample set is in a different data domain than the image data set.

According to the embodiment of the application, the effectiveness of the countermeasure sample on the model recovery is found to depend on the migration capability of the sample to a great extent through experiments, so that when the countermeasure sample generator is selected, the mobility of the countermeasure sample generator needs to be quantitatively evaluated, and the countermeasure sample generator with high mobility is used for generating the countermeasure sample. The training sample set adopted by the training confrontation sample generator in the embodiment of the application is different from the data field of the image data set, so that the high-mobility advantage of the confrontation sample generator is utilized, and the high-quality augmentation of the confrontation sample is ensured.

In one possible implementation, the model recovery module further includes:

In one possible implementation, the challenge sample generator is a ResNet structure. ResNet is a residual network, and can effectively solve the problems of gradient dispersion/explosion and network degradation. Through verification, the confrontation sample generator adopting the ResNet structure has high mobility.

An embodiment of the present application further provides a model recovery apparatus, see fig. 11, including:

a preparation module 301, configured to obtain an image data set and at least two confrontational sample generators trained in advance, where the image data set is different from a data field of a training sample set of the confrontational sample generators;

a confrontation sample set obtaining module 302, configured to obtain confrontation sample sets output by at least two confrontation sample generators by inputting the image data sets to the at least two confrontation sample generators, respectively;

a soft label obtaining module 303, configured to input the countermeasure sample set to the target model, and obtain a soft label output by the target model;

and a recovery module 304, configured to train the initial model to converge according to the confrontation sample set and the soft label, so as to serve as a recovered target model.

An embodiment of the present application provides an electronic device, which includes: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: the method comprises the steps of obtaining a countermeasure sample set, inputting the countermeasure sample set to a target model to obtain a soft label, training an initial model by using the countermeasure sample set and the soft label, and taking the trained initial model as an image classification model, wherein the image classification model can basically recover the function and the test precision of the target model, so that an accurate classification result is predicted for an image to be classified, and the problem that the image classification service cannot be normally carried out due to the fact that a cloud server of the target model is down is solved. .

In an alternative embodiment, there is provided an electronic device, as shown in fig. 12, an electronic device 4000 shown in fig. 12 including: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004. In addition, the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 4002 may include a path that carries information between the aforementioned components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 12, but this is not intended to represent only one bus or type of bus.

The Memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

The memory 4003 is used for storing application codes for executing the scheme of the present application, and the execution is controlled by the processor 4001. Processor 4001 is configured to execute application code stored in memory 4003 to implement what is shown in the foregoing method embodiments.

The present application provides a computer-readable storage medium, on which a computer program is stored, which, when running on a computer, enables the computer to execute the corresponding content in the foregoing method embodiments. Compared with the prior art, the method has the advantages that the countermeasure sample set is obtained and input into the target model to obtain the soft labels, the initial model is trained by using the countermeasure sample set and the soft labels, the trained initial model is used as the image classification model, and the image classification model can basically recover the function and the test precision of the target model, so that an accurate classification result is predicted for the images to be classified, and the problem that the image classification service cannot be normally carried out due to downtime of a cloud server of the target model is solved.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of execution is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a partial embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An image classification method, comprising:

acquiring an image to be classified;

inputting the image to be classified into a pre-trained image classification model to obtain a classification result of the image classification model;

the image classification model is formed by training an antagonistic sample set and soft labels corresponding to the antagonistic sample set, wherein the soft labels are generated by taking the antagonistic sample set as input by a target model; the challenge sample set is generated by augmenting a cross-domain image data set with at least two challenge sample generators.

2. The image classification method according to claim 1, wherein the training method of the image component model comprises:

acquiring the image data set, inputting the image data set to at least two pre-trained confrontation sample generators for data augmentation, and acquiring the confrontation sample sets output by the at least two confrontation sample generators;

inputting the confrontation sample set into the target model to obtain a soft label output by the target model;

and training an initial model to be converged according to the confrontation sample set and the soft label, and using the initial model as an image classification model.

3. The image classification method of claim 2, wherein the obtaining an image dataset and inputting the image dataset to at least two pre-trained confrontational sample generators for data augmentation comprises:

acquiring at least one image data set, and creating a plurality of threads for parallel processing according to the image data set and the number of the confrontation sample generators, wherein the threads are used for inputting one image data set to one confrontation sample generator and acquiring the confrontation sample set output by the confrontation sample generator;

4. The image classification method of claim 3, wherein the inputting the antagonistic sample set to the target model comprises:

and if any thread obtains the confrontation sample set at the current moment, inputting the confrontation sample set into the target model.

5. The image classification method according to claim 4, wherein the training of the initial model to converge according to the confrontation sample set and the soft label as the image classification model comprises:

training an initial model of a current stage to be convergent according to a countermeasure sample set obtained by any thread at the current moment and a soft label corresponding to the countermeasure sample set, obtaining an initial model of a next stage, and taking the obtained initial model of the last stage as the image classification model;

6. The image classification method according to any one of claims 2 to 5, wherein the inputting the image data set to at least two pre-trained confrontational sample generators for data augmentation further comprises:

acquiring a training sample set, and training a confrontation sample generator according to the training sample set to obtain the trained confrontation sample generator; the training sample set is different from the data domain of the image data set.

7. The image classification method according to any one of claims 2 to 5, wherein the inputting the image dataset to at least two pre-trained resist sample generators for data augmentation further comprises:

acquiring a training sample set, and training a confrontation sample generator according to the training sample set to obtain the trained confrontation sample generator;

and the data in the image data set is obtained by performing image transformation on the training samples in the training sample set.

8. The image classification method according to any of claims 2 to 7, characterized in that the confrontation sample generator is of a ResNet structure.

9. A method of model restoration, comprising:

acquiring an image data set and at least two confrontational sample generators trained in advance, wherein the data set is different from the data field of the training set of the confrontational sample generators;

inputting the image data sets to the at least two confrontation sample generators respectively, and obtaining the confrontation sample sets output by the at least two confrontation sample generators;

10. An image classification apparatus, comprising:

the input module is used for inputting the images to be classified into a pre-trained image classification model to obtain a classification result of the image classification model;

11. A model restoration apparatus, comprising:

the system comprises a preparation module, a comparison module and a comparison module, wherein the preparation module is used for acquiring an image data set and at least two confrontation sample generators trained in advance, and the image data set is different from the training sample set of the confrontation sample generators in data field;

a confrontation sample set acquisition module, configured to input the data sets to the at least two confrontation sample generators respectively, and obtain confrontation sample sets output by the at least two confrontation sample generators;

the soft label obtaining module is used for inputting the confrontation sample set into the target model and obtaining a soft label output by the target model;

12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the image classification method according to any one of claims 1 to 8 or the model restoration method according to claim 9 are implemented when the processor executes the program.

13. A computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the image classification method according to any one of claims 1 to 8 or the model restoration method according to claim 9.