CN114707572A - Deep learning sample testing method and device based on loss function sensitivity - Google Patents

Deep learning sample testing method and device based on loss function sensitivity Download PDF

Info

Publication number
CN114707572A
CN114707572A CN202210173151.1A CN202210173151A CN114707572A CN 114707572 A CN114707572 A CN 114707572A CN 202210173151 A CN202210173151 A CN 202210173151A CN 114707572 A CN114707572 A CN 114707572A
Authority
CN
China
Prior art keywords
sample
sensitivity
loss function
drift
loss
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210173151.1A
Other languages
Chinese (zh)
Inventor
陈晋音
李晓豪
金海波
郑海斌
宣琦
倪洪杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202210173151.1A priority Critical patent/CN114707572A/en
Publication of CN114707572A publication Critical patent/CN114707572A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a deep learning sample testing method and device based on loss function sensitivity, which comprises the steps of obtaining an image data set as a clean sample and constructing a deep learning network; carrying out attack resisting operation on the image data set to obtain a resisting sample; screening to obtain a total sample set, and dividing the total sample set into a training sample set and a testing sample set; setting a loss function sensitivity function; establishing a particle swarm optimization model, calculating a loss sensitivity drift value, performing iterative updating, and selecting two optimal drift sample sets of a normal sample and a confrontation sample; training by using the optimal drift sample set to obtain a loss sensitivity classifier; inputting the two optimal drift sample sets into a deep learning network for retraining to obtain a test model; inputting the test sample set into the test model for iteration, searching for an optimal drift noise sample, calculating a loss function sensitivity drift value of the optimal drift noise sample, and judging by using a loss sensitivity classifier to complete the detection of the sample.

Description

Deep learning sample testing method and device based on loss function sensitivity
Technical Field
The invention belongs to the field of safety problems facing deep learning samples, and particularly relates to a deep learning sample testing method and device based on loss function sensitivity.
Background
Deep learning is one of the directions in which artificial intelligence has attracted much attention in recent years, and as Deep Neural Networks (DNNs) have enjoyed great success in many applications, and with the mass data coming along, neural networks have enjoyed good performance in image classification tasks, for example, the ImageNet data set has 320 million artificially labeled images. Although amazon and other large platforms already label class labels of large-scale images as much as possible, some errors are inevitable in the labeling process, and the samples with wrong labels are easy to damage the performance of the model trained on the same type of samples. And the ability of the deep neural network to remember the mass data of randomly assigned labels demonstrates the sensitivity of the overfitting when training the noise samples. Therefore, a DNN algorithm robust to noise labels is needed to solve the potential problem. Further, where samples are readily available but accurate labeling is expensive, the algorithm can make datasets with more noisy labels more beneficial than datasets with fewer more accurate labels.
At present, the artificial intelligence attacks are mainly divided into poisoning attacks and counterattack attacks, and the poisoning attacks are mainly formed by marking normal samples. And then, injecting the poisoning sample into a training data set, thereby completing embedding a backdoor trigger into the deep learning model in a training stage, and triggering attack outbreak when a poison sample is input in a testing stage. The counterattack is mainly characterized in that some disturbances which cannot be perceived by human eyes are added in an original sample, different from other attacks, the counterattack mainly occurs when a counterattack sample is constructed, a machine learning model is input by using the generated counterattack sample and a normal sample, a deception recognition result is obtained, and the counterattack is divided into a white box attack and a black box attack.
However, the existing neural network testing method is focused on the identification precision of the model and the robustness of the model in a malicious sample, and in addition, the sample testing method can only be used for identifying a small number of samples or a specific sample, so that the risk of damaging the model when a plurality of types of malicious samples are applied to an edge end is ignored. Malicious samples of different types still have destructive effects in the defended neural network model. When the neural network is applied to real life, for example, in an automatic driving scene, a malicious sample formed by samples generated by different weather and pollution on targets such as pedestrians or guideboards can attack an automatic discrimination model, so that a great risk is brought to model errors, and the potential risk is not ignored.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a deep learning sample testing method and device based on loss function sensitivity.
In order to achieve the purpose, the technical scheme of the invention is as follows: 1. a deep learning sample testing method based on loss function sensitivity comprises the following steps:
(1) acquiring an image data set as a clean sample, and constructing a deep learning network;
(2) selecting an anti-attack method, and carrying out anti-attack operation on the image data set obtained in the step (1) to obtain an anti-sample; screening, and taking the screened clean sample and the confrontation sample as a total sample set; dividing the total sample set into a training sample set and a testing sample set;
(3) setting a loss function sensitivity function based on the loss function and the training sample set;
(4) establishing a particle swarm optimization model, calculating loss sensitivity drift values of all samples, performing iterative updating on the loss sensitivity drift values, and selecting two optimal drift sample sets of normal samples and confrontation samples; training by using the optimal drift sample set to obtain a loss sensitivity classifier;
(5) inputting the two types of optimal drift sample sets obtained in the step (4) into the deep learning network constructed in the step (1) for retraining to obtain a test model; inputting the test sample set obtained in the step (2) into a test model for iterative updating, finding out an optimal drift noise sample corresponding to the test sample, calculating a loss function sensitivity drift value of the optimal drift noise sample, inputting the drift value into the loss sensitivity classifier constructed in the step (4) for discrimination, and completing the detection of the sample.
Further, the image dataset is a typical image dataset such as MNIST, CIFAR10, ImageNet, GTSRB, CASIA, and the like.
Further, the deep learning network is a network such as VGG16, AlexNet, VGG11 or ResNet 34.
Further, the anti-attack method is one of FGSM, Boundary and MIFGSM.
Further, the step (3) is specifically:
3.1) constructing a loss function: setting the cross entropy function as a loss function, wherein the formula is as follows:
Figure BDA0003519298880000021
wherein C represents the number of classification results; n denotes an overview of the training samples and h is the final output of the network.
3.2) constructing a loss sensitivity function: the gradient function generated by derivation of the loss function on the sample is used as a target function, and the second-order norm is used for dimensionality reduction, and the function is as follows:
Figure BDA0003519298880000022
generating new noise samples by adding noises with different degrees on a training sample set, and simultaneously inputting the training sample set and the noise samples into a deep learning model to generate a loss function value; using the offset value of the loss function of the training sample set and the noise sample as the loss function sensitivity function:
Ls＝Lfnoise-Lfnor
wherein, LfnoiseIs the sample loss sensitivity value after adding noise, LfnorIs the loss sensitivity value for the initial non-additive noise.
Further, the step (4) specifically includes the following steps:
(4.1) adding random noise to the input sample, and initializing a particle swarm optimization model;
(4.2) sample loss sensitivity drift value calculation: calculating the loss function sensitivity of the initialized sample added with random noise through the loss function sensitivity function set in the step (3), and then making a difference with the loss function sensitivity of the input sample to obtain drift values of the loss function sensitivities of different noise samples;
(4.3) updating the historical optimal position and the global optimal position: calculating the sensitivity value of the loss function of the particle swarm, comparing the historical optimal sensitivity of the loss function and updating the historical optimal position p of each particlebestGlobal optimum position g of particle swarmbest；
(4.4) updating the speed v of the particle swarmiAnd position xi；
(4.5) selecting an optimal drift sample: continuously updating and optimizing the loss function sensitivity of the input samples to generate noise samples with different loss sensitivities, and recording the noise sample with the maximum loss function sensitivity corresponding to each sample, namely the optimal drift sample, in the iterative updating of the same input sample; different optimal drifting samples generated by the normal sample and the confrontation sample are gathered to form two optimal drifting sample sets of the normal sample and the confrontation sample.
(4.6) loss sensitivity classifier construction: selecting an optimal drift sample interval to record drift intervals of different types of samples to form a sample interval data set, wherein the data set comprises a normal sample drift interval type and an antagonistic sample drift interval type, and training the data set by using a two-classifier to train the two-classifier which can distinguish the normal sample from the antagonistic sample according to loss sensitivity; the two classifiers are used as loss sensitivity classifiers.
Further, the step (5) specifically comprises:
(5.1) sorting the two types of optimal drift sample sets obtained in the step (4) according to the magnitude of the drift values, selecting front Top-K sheet samples and labels of each type, inputting the samples and the labels into the deep learning model trained in the step (1) for retraining, and generating a test model;
(5.1) inputting the test sample set divided in the step (2) into a test model to obtain a prediction label; inputting the test sample and the prediction label into the particle group optimization model for iterative updating, finding out the optimal drift noise sample corresponding to the test sample, calculating the loss function sensitivity drift value of the optimal drift noise sample, and inputting the drift value into the step (4) for discrimination, thereby realizing the purpose of deducing whether the sample belongs to a normal sample or a confrontation sample, and completing the detection of the sample.
A second aspect of the embodiments of the present invention provides a deep learning sample testing apparatus based on a sensitivity of a loss function, including one or more processors, for implementing the above deep learning sample testing method based on a sensitivity of a loss function.
A third aspect of the embodiments of the present invention is a computer-readable storage medium, on which a program is stored, which, when being executed by a processor, is used for the above-mentioned deep learning sample testing method based on the sensitivity of the loss function.
The invention has the beneficial effects that: the method can ensure that the safety detection is carried out on the input sample before the sample is input into the deep learning model, whether the sample is a malicious sample or not is tested, and whether the judgment strategy of the deep learning model is disturbed or not is tested, so that serious consequences in practical application are avoided, and the effectiveness in the common model training and sample testing process is ensured.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a schematic view of the apparatus of the present invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The method and apparatus for testing deep learning samples based on sensitivity of loss function according to the present invention will be described in detail with reference to the accompanying drawings. The features of the following examples and embodiments may be combined with each other without conflict.
Before the sample library is taken for model training, a sample identification task needs to be carried out on the model library. Specifically, clean samples of the same type and corresponding confrontation samples are input into the model as test data sets, the loss sensitivity interval of each type of sample is found, abnormal samples which are not in the normal sample interval are removed, and therefore the effect of removing malicious samples on the basis of main task training convergence can be guaranteed. When samples of different attack types are input into the test model, the drift degrees of the loss functions of the samples are different from those of the optimized model, and the types of the samples are reversely deduced by using the difference of the drift intervals. Using clean samples XcleanAnd confrontation sample XadvThe three data sets are input into a model, and corresponding three loss function sensitivities are generated. And performing optimization sequencing on the generated loss sensitivities by using a particle swarm optimization algorithm, selecting out Top-k samples in each class, inputting the Top-k samples into the original model for retraining, and generating a new discrimination model. And inputting unknown samples to be tested into a discrimination model, and discriminating the samples by using the regenerated loss sensitivity.
Referring to fig. 1, a method for detecting a deep learning malicious sample based on a loss function sensitivity according to an embodiment of the present invention includes the following steps:
1) data set selection:
typical image datasets such as MNIST, CIFAR10, ImageNet, GTSRB, CASIA, etc. are selected, and in the embodiment of the present invention, a CIFAR10 dataset is taken as an example, and the CIFAR10 dataset is a 10-classified 6000-picture dataset with a picture size of 32 × 32.
2) Sample attack preparation:
2.1) selecting a deep learning network: networks such as VGG16, AlexNet, VGG11, ResNet34 and the like can be selected, and the VGG16 network is taken as an example in the embodiment of the present invention.
2.3) selecting an anti-attack method to generate an anti-attack sample: FGSM, Boundary, MIFGSM, etc. are selected to resist attack. In the embodiment of the invention, the MNIST takes FGSM and Boundary as examples, wherein FGSM is a white box attack, and Boundary is a black box attack. Both attacks are targetless attacks, i.e. random noise is added, and only the original label needs to be changed.
2.4) sample screening and combination: the embodiment of the invention screens the original clean samples and the counterattack samples, and firstly eliminates the malicious samples which can not be correctly classified by the model and are unsuccessful in counterattack. Secondly, the two types of samples which are screened are classified separately to form a total sample set. And (4) selecting 4000 samples from each type in the total sample set to form a training sample set. And finally, selecting 1000 samples different from the training samples from each type in the total sample set to form a test sample set for final performance test.
3) Setting a loss function sensitivity function:
3.1) constructing a loss function: ideally, there is one clean data set
Figure BDA0003519298880000051
Wherein xiIs a feature space, yiIs a class label. The function of the classifier is to find different class labels f: x → R from different feature spacesc. In the embodiment of the present invention, the output layer of the DNN model is set as the softmax layer, and the cross entropy function is set as the loss function, so the loss function is:
Figure BDA0003519298880000052
wherein C represents the number of classification results; n represents the total number of training samples and h is the final output of the network.
3.2) constructing a loss sensitivity function: firstly, in order to further improve the sensitivity of the loss function to the sample, a gradient function generated by derivation of the loss function to the sample is used as a target function, and the gradient function is reduced by using a second-order norm, wherein a function Lf of the gradient function is as follows:
Figure BDA0003519298880000061
new noise samples are generated by adding different degrees of noise to the training sample set, and the training sample set is input to the loss function values generated in the target model simultaneously with the noise samples. Using the offset value of the loss function of the training sample set and the noise sample as the loss function sensitivity function Ls:
Ls＝Lfnoise-Lfnor
wherein, LfnoiseIs the sample loss sensitivity value after adding noise, LfnorIs the loss sensitivity value for the initial non-additive noise.
When noise is added to the sample, noise addition along the gradient direction is selected, and the noise addition process is as follows:
x'＝x+s*grad
where s is the iteration step size.
When changing the pixel value of the image, the following requirements are satisfied:
Figure BDA0003519298880000062
wherein x represents an original image; x' represents an image after changing the pixel value; l0 represents the maximum number of pixels that have been altered; l ∞ represents the maximum value of the pixel modification; size (x) is the number of pixels in the image 0< x; 0< α, β < 1.
4) Particle swarm optimization model (PSO) building
4.1) initialization of particle swarm optimization model
In the embodiment of the invention, when each sample is input into the particle swarm optimization model, 20 initialization samples added with different random noises are constructed by adding random noises, and each initialization sample is taken as a particle to construct 20 particles of the PSO model. Using the RGB values of all pixel points of each input sample as the position matrix x of the particlesiThe variation matrix of RGB values is used as the velocity matrix v of the particlesi. And the current iteration number G, the maximum iteration number GkCurrent inertial weight factor ωg. Historical optimal position p of ith particlebestParticles ofGlobal optimum position g for population discoverybest。
4.2) sample loss sensitivity value calculation
And (4) calculating the loss function sensitivity of the initialized 20 initialization samples added with random noise through the loss function sensitivity function set in the step (3), and then calculating the drift values of the loss function sensitivities of different noise samples by making a difference with the loss function sensitivity of the input sample.
4.3) calculating the sensitivity value of the loss function of the particle swarm, comparing the historical optimal sensitivity of the loss function and updating the historical optimal position p of each particlebestGlobal optimum position g of particle swarmbest。
4.4) updating the speed v of the particle swarmiAnd position xiAnd an inertia factor is adopted in the updating process of the particle speed and the particle position, when the value of the inertia factor is larger, the global searching capability is stronger, and when the value of the inertia factor is smaller, the local searching capability is stronger. The calculation formula is as follows:
ω(g)＝(ωini-ωend)(Gk-g)/Gk+ωend
υi＝ω(g)×υi+c1×rand()×(pbesti-xi)+c2×rand()×(gbesti-xi)
xi＝xi+υi
wherein, ω isiniIs the initial weight factor value, ωendIs the final weight factor value, c1And c2To initialize the learning factor, rand () is a random number between (0, 1) generated by the system.
4.5) optimal Drift sample set selection
Continuously updating and optimizing the loss function sensitivity of the input samples in the step 4.4) to generate noise samples with different loss sensitivities, and recording the noise sample with the maximum loss function sensitivity corresponding to each sample, namely the optimal drift sample, in the iterative updating of the same input sample. Different optimal drift samples generated by the normal sample and the confrontation sample are collected to form two optimal drift sample sets of the normal sample and the confrontation sample.
4.6) loss sensitivity classifier construction
Selecting an optimal drift sample interval to record drift intervals of different types of samples to form a sample interval data set, wherein the data set comprises a normal sample drift interval type and an antagonistic sample drift interval type, and training the data set by using a two-classifier, thereby training the two-classifier which can distinguish the normal sample from the antagonistic sample according to loss sensitivity. The two classifiers are used as loss sensitivity classifiers.
5) Sample testing
5.1) model retraining
In order to increase the loss function discrimination of the two types of samples, the two types of optimal drift sample sets in 4.5) are sorted according to the magnitude of the drift value, the front Top-K pieces of samples and labels of each type are selected, and the samples and the labels are sent into the deep learning model (the VGG16 model in the embodiment) trained in the step (1) for retraining to generate a test model for testing a subsequent test data set.
5.2) testing of samples
Inputting each sample in the test sample set in 2.4) into the test model in 5.1), a prediction label can be obtained. Inputting the test sample and the prediction label into the particle group optimization model in 5.1) for iterative updating, finding out an optimal drift noise sample corresponding to the test sample, calculating a loss function sensitivity drift value of the optimal drift noise sample, and inputting the change drift value into the classifier constructed in the step 4.6) for discrimination, thereby realizing the purpose of deducing whether the sample belongs to a normal sample or a confrontation sample, and further completing the detection of the sample.
Corresponding to the embodiment of the deep learning sample testing method based on the loss function sensitivity, the invention also provides an embodiment of a deep learning sample testing device based on the loss function sensitivity.
Referring to fig. 2, an embodiment of the present invention provides a deep learning sample testing apparatus based on a sensitivity of a loss function, which includes one or more processors, and is configured to implement the deep learning sample testing method based on a sensitivity of a loss function in the foregoing embodiment.
The embodiment of the deep learning sample testing device based on the sensitivity of the loss function can be applied to any equipment with data processing capability, such as computers and other equipment or devices. The apparatus embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for running through the processor of any device with data processing capability. In terms of hardware, as shown in fig. 2, a hardware structure diagram of any device with data processing capability where the deep learning sample testing apparatus based on loss function sensitivity of the present invention is located is shown, except for the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 2, in the embodiment, any device with data processing capability where the apparatus is located may also include other hardware according to the actual function of the any device with data processing capability, which is not described again.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
An embodiment of the present invention further provides a computer-readable storage medium, on which a program is stored, where the program, when executed by a processor, implements the deep learning sample testing method based on the sensitivity of the loss function in the foregoing embodiments.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any data processing capability device described in any of the foregoing embodiments. The computer readable storage medium can be any device with data processing capability, such as a plug-in hard disk, a Smart Media Card (SMC), an SD Card, a Flash memory Card (Flash Card), etc. provided on the device. Further, the computer readable storage medium may include both an internal storage unit and an external storage device of any data processing capable device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing-capable device, and may also be used for temporarily storing data that has been output or is to be output.
The above embodiments are only used for illustrating the design idea and features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the content of the present invention and implement the present invention accordingly, and the protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes and modifications made in accordance with the principles and concepts disclosed herein are intended to be included within the scope of the present invention.

Claims (9)

1. A deep learning sample testing method based on loss function sensitivity is characterized by comprising the following steps:
(1) acquiring an image data set as a clean sample, and constructing a deep learning network;
(2) selecting an anti-attack method, and carrying out anti-attack operation on the image data set obtained in the step (1) to obtain an anti-sample; screening, and taking the screened clean sample and the screened confrontation sample as a total sample set; dividing the total sample set into a training sample set and a testing sample set;
(3) setting a loss function sensitivity function based on the loss function and the training sample set;
(4) establishing a particle swarm optimization model, calculating loss sensitivity drift values of all samples, performing iterative updating on the loss sensitivity drift values, and selecting two optimal drift sample sets of normal samples and confrontation samples; training by using the optimal drift sample set to obtain a loss sensitivity classifier;
(5) inputting the two types of optimal drift sample sets obtained in the step (4) into the deep learning network constructed in the step (1) for retraining to obtain a test model; inputting the test sample set obtained in the step (2) into a test model for iterative updating, finding out an optimal drift noise sample corresponding to the test sample, calculating a loss function sensitivity drift value of the optimal drift noise sample, inputting the drift value into the loss sensitivity classifier constructed in the step (4) for discrimination, and completing the detection of the sample.
2. The method for testing deep learning samples based on loss function sensitivity as claimed in claim 1, wherein the image data set is typical image data set such as MNIST, CIFAR10, ImageNet, GTSRB, CASIA, etc.
3. The method for testing deep learning samples based on loss function sensitivity as claimed in claim 1, wherein the deep learning network is a network such as VGG16, AlexNet, VGG11 or ResNet 34.
4. The method for testing deep learning samples based on loss function sensitivity as claimed in claim 1, wherein the attack countermeasure method is one of FGSM, Boundary, and MIFGSM.
5. The method for testing deep learning samples based on sensitivity to loss functions as claimed in claim 1, wherein the step (3) is specifically:
3.1) constructing a loss function: setting the cross entropy function as a loss function, and the formula is as follows:
Figure FDA0003519298870000011
wherein C represents the number of classification results; n denotes an overview of the training samples and h is the final output of the network.
3.2) constructing a loss sensitivity function: the gradient function generated by derivation of the loss function on the sample is used as a target function, and the second-order norm is used for dimensionality reduction, and the function is as follows:
Figure FDA0003519298870000012
generating new noise samples by adding noises with different degrees on a training sample set, and simultaneously inputting the training sample set and the noise samples into a deep learning model to generate a loss function value; using the offset value of the loss function of the training sample set and the noise sample as the loss function sensitivity function:
Ls＝Lfnoise-Lfnor
wherein, LfnoiseIs the sample loss sensitivity value after adding noise, LfnorIs the loss sensitivity value for the initial non-additive noise.
6. The deep learning sample testing method based on loss function sensitivity as claimed in claim 1, wherein the step (4) comprises the following steps:
(4.1) adding random noise to the input sample, and initializing a particle swarm optimization model;
(4.2) sample loss sensitivity drift value calculation: calculating the loss function sensitivity of the initialized sample added with random noise through the loss function sensitivity function set in the step (3), and then making a difference with the loss function sensitivity of the input sample to obtain drift values of the loss function sensitivities of different noise samples;
(4.3) updating the historical optimal position and the global optimal position: calculating the sensitivity value of the loss function of the particle swarm, and comparing the historical optimal sensitivity of the loss functionMeasure and update the historical optimal position p of each particlebestGlobal optimum position g of particle swarmbest；
(4.4) updating the velocity v of the particle populationiAnd position xi；
(4.5) selecting an optimal drift sample: continuously updating and optimizing the loss function sensitivity of the input samples to generate noise samples with different loss sensitivities, and recording the noise sample with the maximum loss function sensitivity corresponding to each sample, namely the optimal drift sample, in the iterative updating of the same input sample; different optimal drift samples generated by the normal sample and the confrontation sample are collected to form two optimal drift sample sets of the normal sample and the confrontation sample.
(4.6) loss sensitivity classifier construction: selecting an optimal drift sample interval to record drift intervals of different types of samples to form a sample interval data set, wherein the data set comprises a normal sample drift interval type and an antagonistic sample drift interval type, and training the data set by using a two-classifier to train the two-classifier which can distinguish the normal sample from the antagonistic sample according to loss sensitivity; the two classifiers are used as loss sensitivity classifiers.
7. The method for testing deep learning samples based on sensitivity to loss function as claimed in claim 1, wherein the step (5) is specifically as follows:
(5.1) sequencing the two types of optimal drift sample sets obtained in the step (4) according to the magnitude of the drift values, selecting front Top-K samples and labels of each type, inputting the samples and labels into the deep learning model trained in the step (1) for retraining, and generating a test model;
(5.1) inputting the test sample set divided in the step (2) into a test model to obtain a prediction label; inputting the test sample and the prediction label into the particle group optimization model for iterative updating, finding out the optimal drift noise sample corresponding to the test sample, calculating the loss function sensitivity drift value of the optimal drift noise sample, and inputting the drift value into the step (4) for discrimination, thereby realizing the purpose of deducing whether the sample belongs to a normal sample or a confrontation sample, and completing the detection of the sample.
8. A deep learning sample testing device based on loss function sensitivity, comprising one or more processors for implementing the deep learning sample testing method based on loss function sensitivity according to any one of claims 1 to 7.
9. A computer-readable storage medium, on which a program is stored, which, when being executed by a processor, is configured to carry out the method for testing a deep learning sample based on sensitivity to a loss function of any one of claims 1 to 7.
CN202210173151.1A 2022-02-24 2022-02-24 Deep learning sample testing method and device based on loss function sensitivity Pending CN114707572A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210173151.1A CN114707572A (en) 2022-02-24 2022-02-24 Deep learning sample testing method and device based on loss function sensitivity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210173151.1A CN114707572A (en) 2022-02-24 2022-02-24 Deep learning sample testing method and device based on loss function sensitivity

Publications (1)

Publication Number Publication Date
CN114707572A true CN114707572A (en) 2022-07-05

Family

ID=82166950

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210173151.1A Pending CN114707572A (en) 2022-02-24 2022-02-24 Deep learning sample testing method and device based on loss function sensitivity

Country Status (1)

Country Link
CN (1) CN114707572A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115797711A (en) * 2023-02-20 2023-03-14 泉州装备制造研究所 Confrontation sample improved classification method based on reconstruction model
CN116032557A (en) * 2022-12-09 2023-04-28 清华大学 Method and device for updating deep learning model in network security anomaly detection
WO2024178581A1 (en) * 2023-02-28 2024-09-06 华为技术有限公司 Data processing method and apparatus, and storage medium and program product

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116032557A (en) * 2022-12-09 2023-04-28 清华大学 Method and device for updating deep learning model in network security anomaly detection
CN116032557B (en) * 2022-12-09 2024-07-02 清华大学 Method and device for updating deep learning model in network security anomaly detection
CN115797711A (en) * 2023-02-20 2023-03-14 泉州装备制造研究所 Confrontation sample improved classification method based on reconstruction model
CN115797711B (en) * 2023-02-20 2023-04-21 泉州装备制造研究所 Improved classification method for countermeasure sample based on reconstruction model
WO2024178581A1 (en) * 2023-02-28 2024-09-06 华为技术有限公司 Data processing method and apparatus, and storage medium and program product

Similar Documents

Publication Publication Date Title
Li et al. Backdoor learning: A survey
Song et al. Constructing unrestricted adversarial examples with generative models
Zhong et al. Backdoor embedding in convolutional neural network models via invisible perturbation
CN112953924B (en) Network abnormal flow detection method, system, storage medium, terminal and application
CN108111489B (en) URL attack detection method and device and electronic equipment
Melis et al. Is deep learning safe for robot vision? adversarial examples against the icub humanoid
CN108960080B (en) Face recognition method based on active defense image anti-attack
CN114707572A (en) Deep learning sample testing method and device based on loss function sensitivity
CN111401407B (en) Countermeasure sample defense method based on feature remapping and application
CN107577945B (en) URL attack detection method and device and electronic equipment
Chaubey et al. Universal adversarial perturbations: A survey
CN111488879A (en) Method and apparatus for improving segmentation performance using dual-embedding configuration
Song et al. Generative adversarial examples
Liang et al. We can always catch you: Detecting adversarial patched objects with or without signature
Casper et al. Diagnostics for deep neural networks with automated copy/paste attacks
CN117372804A (en) Countermeasure sample defense method based on image gradient calculation
Xiang et al. Revealing perceptible backdoors, without the training set, via the maximum achievable misclassification fraction statistic
CN115632843A (en) Target detection-based generation method of backdoor attack defense model
CN115100391A (en) Improved domain-confrontation neural network-based tile defect detection method and system
Nami et al. Adversarial attacks and defense on deep learning models for big data and IoT
Li et al. Adversarial examples detection through the sensitivity in space mappings
CN114638356A (en) Static weight guided deep neural network back door detection method and system
Bunzel et al. Multi-class Detection for Off The Shelf transfer-based Black Box Attacks
Luo et al. Defective Convolutional Networks
Zhou et al. An Improved Method for Making CNN Immune to Backdoor Attack by Activating Clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination