CN110008987B - Method and device for testing robustness of classifier, terminal and storage medium - Google Patents
Method and device for testing robustness of classifier, terminal and storage medium Download PDFInfo
- Publication number
- CN110008987B CN110008987B CN201910126943.1A CN201910126943A CN110008987B CN 110008987 B CN110008987 B CN 110008987B CN 201910126943 A CN201910126943 A CN 201910126943A CN 110008987 B CN110008987 B CN 110008987B
- Authority
- CN
- China
- Prior art keywords
- sample
- classifier
- escape
- target test
- malicious
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
- G06F18/2193—Validation; Performance evaluation; Active pattern learning techniques based on specific statistical tests
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/285—Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
Abstract
The invention is suitable for the technical field of classifier test, and provides a method, a device, a terminal and a storage medium for testing the robustness of a classifier, wherein the method comprises the following steps: the method comprises the steps of inputting a preset test sample into a target test classifier for classification, obtaining a malicious sample in the test sample, inputting random noise into a preset sensor network, obtaining a reference characteristic value of the malicious sample through the sensor network to generate a reference sample, modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample, inputting the escape variant into the target test classifier for classification, obtaining a classification result of the target test classifier after being attacked by the escape variant, and outputting the robustness of the target test classifier according to the classification result after being attacked, so that the robustness of the classifier is tested by generating the escape variant, and the test effect and the test efficiency of the robustness of the classifier are improved.
Description
Technical Field
The invention belongs to the technical field of classifier testing, and particularly relates to a method, a device, a terminal and a storage medium for testing classifier robustness.
Background
With the progress and development of society, more and more data needs to be collected and processed by people, how to select a fast and effective tool or a method to process the data becomes a focus of attention, a machine learning algorithm can adaptively learn the characteristics of the data and analyze the data, and better classification performance is achieved in many security applications, such as applications of spam filtering, intrusion detection, malware detection systems and the like, so that people generally process the complex data fast and effectively through a classifier obtained through machine learning.
However, in the application of the classifier, there may be some misleading decisions made by the classifier by some attackers through modifying some malicious data samples, or probing the vulnerability of the classifier, so that the malicious attackers can achieve their illegal purpose through the vulnerability of the classifier. Therefore, before people process data by using the classifier, the classifier is required to be tested to test whether the classifier can achieve the expected level of correctness of classifying the data in the case of malicious data attack, namely, the robustness of the classifier is tested.
Generally, when testing the robustness of a classifier, people adopt a simulation attack method or a genetic algorithm, but the former has weak attack performance and is difficult to achieve an expected test effect, and the latter has relatively good attack effect and is relatively long in time consumption, for example, when testing the robustness of the classifier by using the genetic algorithm, 500 malicious PDF files can be generated into feasible variants (avoiding the detection of the classifier and attacking the classifier), so that the test effect and the test efficiency of the robustness of the existing test classifier are all to be improved.
Disclosure of Invention
The invention aims to provide a method, a device, a terminal and a storage medium for testing the robustness of a classifier, and aims to solve the problems that the existing classifier has unsatisfactory testing effect and low testing efficiency when testing the robustness of the classifier because the existing technology cannot provide an effective classifier testing method.
In one aspect, the present invention provides a method for testing classifier robustness, the method comprising the following steps:
inputting a preset test sample into a target test classifier for classification, and obtaining a malicious sample and a normal sample in the test sample;
inputting random noise into a preset sensor network, and acquiring a reference characteristic value of the malicious sample through the sensor network to generate a reference sample;
modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample;
inputting the escape variants into the target test classifier for classification, and obtaining a classification result of the target test classifier after being attacked by the escape variants;
and outputting the robustness of the target test classifier according to the classification result after the attack.
In another aspect, the present invention provides a device for testing classifier robustness, the device comprising:
the system comprises a sample acquisition unit, a target test classifier and a data processing unit, wherein the sample acquisition unit is used for inputting a preset test sample into the target test classifier for classification, and acquiring a malicious sample and a normal sample in the test sample;
the characteristic value acquisition unit is used for inputting random noise into a preset sensor network and acquiring a reference characteristic value of the malicious sample through the sensor network to generate a reference sample;
the characteristic value modification unit is used for modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample so as to generate an escape variant of the malicious sample;
the attack classification unit is used for inputting the escape variants into the target test classifier for classification, and obtaining a classification result of the target test classifier after being attacked by the escape variants; and
and the performance output unit is used for outputting the robustness of the target test classifier according to the classification result after the attack.
In another aspect, the present invention further provides a test terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method for testing robustness of a classifier as described above when executing the computer program.
In another aspect, the present invention further provides a computer-readable storage medium, which stores a computer program, which when executed by a processor implements the steps of the method for testing classifier robustness as described above.
The method comprises the steps of firstly inputting a preset test sample into a target test classifier for classification, obtaining a malicious sample in the test sample, inputting random noise into a preset sensor network, obtaining a reference characteristic value of the malicious sample through the sensor network to generate a reference sample, then modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample, then inputting the escape variant into the target test classifier for classification, obtaining a classification result of the target test classifier after being attacked by the escape variant, and finally outputting the robustness of the target test classifier according to the classification result after being attacked, so that the robustness of the classifier is tested by generating the escape variant, and the test effect and the test efficiency of the robustness of the classifier are improved.
Drawings
FIG. 1 is a flowchart illustrating an implementation of a method for testing classifier robustness according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating an implementation of a method for testing classifier robustness according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a testing apparatus for classifier robustness according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a testing apparatus for classifier robustness according to a fourth embodiment of the present invention; and
fig. 5 is a schematic structural diagram of a test terminal according to a fifth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1 shows an implementation flow of a method for testing classifier robustness according to an embodiment of the present invention, and for convenience of description, only the relevant parts related to the embodiment of the present invention are shown, which is detailed as follows:
in step S101, a preset test sample is input into the target test classifier for classification, and a malicious sample and a normal sample in the test sample are obtained.
The embodiment of the invention is suitable for the test terminal which can test the performance, such as robustness and the like, of the classifier. In the embodiment of the invention, the test samples consist of samples without malicious features and samples with malicious features which are preset and classified by a discriminator, the discriminator can correctly classify the samples without the malicious features and the samples with the malicious features, the malicious samples are samples which can be correctly detected by a target test classifier in the test samples and have malicious attack features, the tested classifier is called as a target test classifier, the preset test samples are firstly input into the target test classifier to be classified, the malicious samples and normal samples which can be correctly classified by the target test classifier in the test samples are obtained, and the accuracy of detecting the malicious attack features by the malicious samples is ensured.
In step S102, random noise is input into a preset sensor network, and a reference characteristic value of a malicious sample is obtained through the sensor network, so as to generate a reference sample.
In the embodiment of the invention, the preset sensor network is a preset multilayer sensor network, so that the distribution of samples generated by the multilayer sensor network is the same as that of preset test samples, random noise is input into the multilayer sensor network, sample characteristic values are obtained through the multilayer sensor network, and samples are generated according to the sample characteristic values, so that the attack of subsequent escape variants is improved.
In step S103, the feature value of the malicious sample is modified according to the reference feature value of the reference sample to generate an escape variant of the malicious sample.
In the embodiment of the invention, after the reference sample with the same distribution as the test sample is obtained, the characteristic value of the malicious sample separated from the test sample is modified according to the reference characteristic value of the reference sample, so that the malicious sample, namely the escape variant is generated, the escape variant retains partial characteristics which can enable the escape variant to attack the classifier, and the rest partial characteristics which are irrelevant to the aggressivity are modified.
In step S104, the escape variants are input to the target test classifier for classification, and a classification result of the target test classifier attacked by the escape variants is obtained.
In the embodiment of the invention, the escape variants are input into the target test classifier for classification test, and the classification result of the target test classifier after being attacked by the escape variants is obtained, so that whether the malicious sample correctly classified by the target test classifier can be correctly classified by the target test classifier after partial characteristics are modified is obtained.
In step S105, the robustness of the target test classifier is output based on the classification result after the attack.
In the embodiment of the invention, whether the robustness of the target test classifier reaches the standard or not can be obtained according to the classification result of the attacked target test classifier.
In the embodiment of the invention, a preset test sample is firstly input into a target test classifier for classification, a malicious sample in the test sample is obtained, random noise is input into a preset sensor network, a reference characteristic value of the malicious sample is obtained through the sensor network to generate the reference sample, then the characteristic value of the malicious sample is modified according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample, the escape variant is input into the target test classifier for classification, a classification result of the target test classifier after being attacked by the escape variant is obtained, and finally the robustness of the target test classifier is output according to the classification result after the attack, so that the robustness of the classifier is tested through the generated escape variant, and the test effect and the test efficiency of the robustness of the classifier are further improved.
Example two:
fig. 2 shows an implementation flow of a method for testing classifier robustness according to an embodiment of the present invention, and for convenience of description, only the relevant parts related to the embodiment of the present invention are shown, which is detailed as follows:
in step S201, a preset test sample is input into the target test classifier for classification, and a malicious sample in the test sample is obtained.
The embodiment of the invention is suitable for the test terminal which can test the performance, such as robustness and the like, of the classifier. In the embodiment of the invention, the test samples consist of samples without malicious features and samples with malicious features which are preset and classified by a discriminator, the discriminator can correctly classify the samples without the malicious features and the samples with the malicious features, the malicious samples are samples which can be correctly detected by a target test classifier in the test samples and have malicious attack features, the tested classifier is called as a target test classifier, the preset test samples are firstly input into the target test classifier to be classified, the malicious samples and normal samples which can be correctly classified by the target test classifier in the test samples are obtained, and the accuracy of detecting the malicious attack features by the malicious samples is ensured.
In step S202, random noise is input into a preset sensor network, and a reference characteristic value of a malicious sample is obtained through the sensor network, so as to generate a reference sample.
In the embodiment of the invention, the preset sensor network is a preset multilayer sensor network, so that the distribution of samples generated by the multilayer sensor network is the same as that of preset test samples, random noise is input into the multilayer sensor network, sample characteristic values are obtained through the multilayer sensor network, and samples are generated according to the sample characteristic values, so that the attack of subsequent escape variants is improved.
In step S203, the feature value of the malicious sample is modified according to the reference feature value of the reference sample to generate an escape variant of the malicious sample.
In the embodiment of the invention, after the reference sample with the same distribution as the test sample is obtained, the characteristic value of the malicious sample separated from the test sample is modified according to the reference characteristic value of the reference sample, so that the malicious sample, namely the escape variant is generated, the escape variant retains partial characteristics which can enable the escape variant to attack the classifier, and the rest partial characteristics which are irrelevant to the aggressivity are modified.
In step S204, the escape variants are input to the target test classifier for classification, and a classification result of the target test classifier attacked by the escape variants is obtained.
In the embodiment of the invention, the escape variants are input into the target test classifier for classification test, and the classification result of the target test classifier after being attacked by the escape variants is obtained, so that whether the malicious sample correctly classified by the target test classifier can be correctly classified by the target test classifier after partial characteristics are modified is obtained.
In step S205, an escape proportion of the target test classifier misclassifying the escape variants is obtained according to the classification result after the attack.
In the embodiment of the present invention, after the target test classifier classifies the escape variants, the ratios of the escape variants that can be incorrectly and correctly classified by the target test classifier can be obtained through comparison, and for convenience of the following description, the ratio of the incorrectly classified escape variants is referred to as the escape ratio.
In step S206, when the escape ratio reaches the preset ratio threshold, a second parameter of the target test classifier is adjusted.
In the embodiment of the present invention, for convenience of subsequent description, the parameter of the target test classifier is referred to as a second parameter, when the escape proportion reaches a preset proportion threshold, which may be because the parameter of the target test classifier is not optimized to be optimal, the parameter of the target test classifier needs to be adjusted to reduce the escape proportion, and when the parameter of the target test classifier is optimized to be optimal, the escape proportion still reaches the preset proportion threshold, which indicates that the robustness of the target test classifier is not qualified, then the process jumps to step S209, and outputs the test result of the robustness of the target test classifier, where the preset proportion threshold may be set to be 25%.
Preferably, when the escape proportion reaches a preset proportion threshold value, the escape proportion will be
Adjusting the second parameter as an adjustment index of the target test classifier to increase the adjustment index of the target test classifier until the escape proportion is smaller than a preset proportion threshold, wherein the larger the adjustment index is, the better the classification performance of the target test classifier is, thereby improving the accuracy of testing the target test classifier while improving the classification performance of the target test classifier, wherein x isnDenotes a normal sample, n denotes a normal sample number, xMRepresenting a malicious sample, M representing a malicious sample sequence number, z representing random noise, G (z) representing a reference sample generated by the random noise, D (x)n) Representing target test classifier versus sample xnAs a result of the classification of (a),
denotes xn、xMG (z) random samples under a distribution,
representing target test classifier pair samples
The classification result of (1), C (G (z), x)M) Representing reference samples G (z) and malicious samples xMGenerated escape sample, D (C (G (z), x)M) Represents the target test classifier pair escape sample C (G (z), x)D) As a result of the classification of (a),
to represent
In that
Gradient of (A), Pmal(x) Representing the distribution of malicious samples, Pnormal(x) Represents the distribution of normal samples, Pz(z) represents the distribution of random noise samples,
represents D (x)n) The expected value under a normal sample distribution,
denotes D (C (G (z), x)M) Expected values under random noise distribution and malicious sample distribution,
to represent
With the expectation of a random sample distribution, λ is a constant parameter.
In step S207, when the escape proportion is smaller than the preset proportion threshold, a first parameter of the sensor network is adjusted.
In the embodiment of the present invention, when the escape proportion is smaller than the preset proportion threshold, the parameter of the sensor network is adjusted, and the coverage of the reference feature generated by the sensor network is expanded, so as to further improve the accuracy of testing the target test classifier.
Preferably, when the escape proportion is smaller than the preset proportion threshold value, the escape proportion is smaller than the preset proportion threshold value
The first parameter is adjusted as an adjustment index of the adjustment sensor network, so that the adjustment index of the adjustment sensor network is reduced until the escape proportion reaches a preset proportion threshold value, and the smaller the adjustment index is, the stronger the attack strength of the escape variant attack target test classifier is, thereby further improving the test of the target test classificationThe accuracy of the device.
Further, when the first parameter or the second parameter is adjusted, random gradient descent adjustment is performed on the first parameter or the second parameter by adopting an Adam optimization algorithm, so that the optimization time of the first parameter or the second parameter is shortened, and the test efficiency of the target test classifier is improved.
In step S208, the Wassertein distance between the distribution of the escape variants and the normal sample distribution in the test sample is obtained to determine whether to continue the test on the target classifier according to the Wassertein distance.
In the embodiment of the invention, the Wassertein distance is the Earth moving distance, and the Earth-Mover distance (EM distance) is used for measuring the distance between the two distributions, when the smaller the Wassertein distance between the distribution of the escape variant and the distribution of the normal sample in the test sample is, the harder the escape variant and the normal sample are to be distinguished, the higher the attack strength of the escape sample on the target test classifier is, when the Wassertein distance is converged, the robustness of the target test classifier can be output by skipping to the step S209, otherwise, the first parameter of the sensor network can be continuously adjusted to continuously strengthen the attack strength of the escape variant.
In step S209, the robustness of the target test classifier is output based on the classification result after the attack.
In the embodiment of the invention, according to the classification result of the attacked target test classifier, if the escape proportion reaches the preset escape threshold, whether the robustness of the target test classifier reaches the standard can be obtained.
In the embodiment of the invention, a malicious sample in a test sample is obtained, a reference characteristic value of the malicious sample is obtained through a sensor network, the reference sample is generated, then the characteristic value of the malicious sample is modified according to the reference characteristic value of the reference sample to generate an escape variant, the escape variant is input into a target test classifier to be classified, a classification result attacked by the escape variant is obtained, an escape proportion of the target test classifier for wrongly classifying the escape variant is obtained according to the classification result after the attack, parameters of the target test classifier and the sensor network are adjusted according to the escape proportion, and finally the robustness of the target test classifier is output according to the classification result after the attack, so that the robustness of the classifier is tested through the generated escape variant, and the test effect and the test efficiency of the robustness of the classifier are improved.
Example three:
fig. 3 shows a structure of a testing apparatus for classifier robustness provided by a third embodiment of the present invention, and for convenience of description, only the parts related to the third embodiment of the present invention are shown, which include:
the sample obtaining unit 31 is configured to input a preset test sample into a target test classifier for classification, and obtain a malicious sample and a normal sample in the test sample;
the characteristic value obtaining unit 32 is configured to input random noise into a preset sensor network, and obtain a reference characteristic value of a malicious sample through the sensor network to generate a reference sample;
a feature value modification unit 33, configured to modify the feature value of the malicious sample according to the reference feature value of the reference sample, so as to generate an escape variant of the malicious sample;
the attack classification unit 34 is configured to input the escape variants into the target test classifier for classification, and obtain a classification result of the target test classifier after being attacked by the escape variants; and
and the performance output unit 35 is used for outputting the robustness of the target test classifier according to the classification result after the attack.
In the embodiment of the invention, a preset test sample is firstly input into a target test classifier for classification, a malicious sample in the test sample is obtained, random noise is input into a preset sensor network, a reference characteristic value of the malicious sample is obtained through the sensor network to generate the reference sample, then the characteristic value of the malicious sample is modified according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample, the escape variant is input into the target test classifier for classification, a classification result of the target test classifier after being attacked by the escape variant is obtained, and finally the robustness of the target test classifier is output according to the classification result after the attack, so that the robustness of the classifier is tested through the generated escape variant, and the test effect and the test efficiency of the robustness of the classifier are further improved.
In the embodiment of the present invention, each unit of the testing apparatus for classifier robustness may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein. The detailed implementation of each unit can refer to the description of the first embodiment, and is not repeated herein.
Example four:
fig. 4 shows a structure of a testing apparatus for classifier robustness provided by the fourth embodiment of the present invention, and for convenience of explanation, only the parts related to the fourth embodiment of the present invention are shown, which include:
the sample obtaining unit 41 is configured to input a preset test sample into the target test classifier for classification, and obtain a malicious sample and a normal sample in the test sample;
the characteristic value obtaining unit 42 is configured to input random noise into a preset sensor network, and obtain a reference characteristic value of a malicious sample through the sensor network to generate a reference sample;
a feature value modifying unit 43, configured to modify the feature value of the malicious sample according to the reference feature value of the reference sample, so as to generate an escape variant of the malicious sample;
the attack classification unit 44 is configured to input the escape variants into the target test classifier for classification, and obtain a classification result of the target test classifier after being attacked by the escape variants;
a proportion obtaining unit 45, configured to obtain, according to the classification result after the attack, an escape proportion of the misclassification escape variant of the target test classifier;
a second adjusting unit 46, configured to adjust a second parameter of the target test classifier when the escape proportion reaches a preset proportion threshold;
a first adjusting unit 47, configured to adjust a first parameter of the sensor network when the escape proportion is smaller than a preset proportion threshold;
a distance obtaining unit 48, configured to obtain a Wassertein distance between the distribution of the escape variants and the distribution of normal samples in the test sample, so as to determine whether to continue testing the target classifier according to the Wassertein distance; and
and the performance output unit 49 is used for outputting the robustness of the target test classifier according to the classification result after the attack.
In the embodiment of the invention, a malicious sample in a test sample is obtained, a reference characteristic value of the malicious sample is obtained through a sensor network, the reference sample is generated, then the characteristic value of the malicious sample is modified according to the reference characteristic value of the reference sample to generate an escape variant, the escape variant is input into a target test classifier to be classified, a classification result attacked by the escape variant is obtained, an escape proportion of the target test classifier for wrongly classifying the escape variant is obtained according to the classification result after the attack, parameters of the target test classifier and the sensor network are adjusted according to the escape proportion, and finally the robustness of the target test classifier is output according to the classification result after the attack, so that the robustness of the classifier is tested through the generated escape variant, and the test effect and the test efficiency of the robustness of the classifier are improved.
In the embodiment of the present invention, each unit of the testing apparatus for classifier robustness may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein. The detailed implementation of each unit can refer to the description of the second embodiment, and is not repeated herein.
Example five:
fig. 5 shows a structure of a test terminal according to a fifth embodiment of the present invention, and for convenience of description, only a part related to the fifth embodiment of the present invention is shown, where the structure includes:
the computing terminal 5 of an embodiment of the present invention comprises a processor 51, a memory 52 and a computer program 53 stored in the memory 52 and operable on the processor 51. The processor 51 executes the computer program 53 to implement the steps in the above-mentioned embodiments of the method for testing robustness of each classifier, for example, steps S101 to S105 shown in fig. 1 and steps S201 to S209 shown in fig. 2. Alternatively, the processor 51, when executing the computer program 53, implements the functions of the units in the above-described test apparatus embodiment of the respective classifier robustness, for example, the functions of the units 31 to 35 shown in fig. 3 and the units 41 to 49 shown in fig. 4.
In the embodiment of the invention, when the processor executes the computer program, a malicious sample in the test sample is firstly obtained, the reference characteristic value of the malicious sample is obtained through the sensor network to generate a reference sample, then modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant, inputting the escape variant into a target test classifier for classification to obtain a classification result attacked by the escape variant, obtaining the escape proportion of the error classification escape variants of the target test classifier according to the classification result after the attack, adjusting parameters of the target test classifier and the sensor network according to the escape proportion, finally outputting the robustness of the target test classifier according to the classification result after attack, therefore, the robustness of the classifier is tested by generating the escape variants, and the testing effect and the testing efficiency of the robustness of the classifier are further improved.
The steps in the embodiment of the method for testing the robustness of the classifier when the processor executes the computer program may refer to the descriptions of the first embodiment and the second embodiment, and are not described herein again.
Example six:
in an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, and the computer program, when executed by a processor, implements the steps in the above-described embodiments of the method for testing robustness of a classifier, for example, steps S101 to S105 shown in fig. 1 and steps S201 to S209 shown in fig. 2. Alternatively, the computer program may be adapted to perform the functions of the units of the test apparatus embodiment for robustness of the respective classifier described above, for example, the functions of the units 31 to 35 shown in fig. 3 and the units 41 to 49 shown in fig. 4, when the computer program is executed by the processor.
In the embodiment of the invention, after the computer program is executed by the processor, a malicious sample in the test sample is obtained, the reference characteristic value of the malicious sample is obtained through the sensor network to generate a reference sample, then modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant, inputting the escape variant into a target test classifier for classification to obtain a classification result attacked by the escape variant, obtaining the escape proportion of the error classification escape variants of the target test classifier according to the classification result after the attack, adjusting parameters of the target test classifier and the sensor network according to the escape proportion, finally outputting the robustness of the target test classifier according to the classification result after attack, therefore, the robustness of the classifier is tested by generating the escape variants, and the testing effect and the testing efficiency of the robustness of the classifier are further improved.
For the steps in the embodiment of the method for testing classifier robustness when the computer program is executed by the processor, reference may be made to the description of the first embodiment and the second embodiment, which is not repeated herein.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, storage media, e.g., ROM/RAM, magnetic disks, optical disks, flash memory, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A method for testing classifier robustness is characterized by comprising the following steps:
inputting a preset test sample into a target test classifier for classification, and obtaining a malicious sample and a normal sample in the test sample, wherein the target test classifier is a classifier of safety application;
inputting random noise into a preset sensor network, and acquiring a reference characteristic value of the malicious sample through the sensor network to generate a reference sample;
modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample to generate an escape variant of the malicious sample;
inputting the escape variants into the target test classifier for classification, and obtaining a classification result of the target test classifier after being attacked by the escape variants;
obtaining the escape proportion of the target test classifier for misclassifying the escape variants according to the classification result after the attack;
when the escape proportion is smaller than a preset proportion threshold value, adjusting a first parameter of the sensor network until the escape proportion reaches the preset proportion threshold value;
acquiring Wassertein distances between the distribution of the escape variants and the distribution of normal samples in the test samples, and judging whether to continue testing the target classifier according to the Wassertein distances;
if the target classifier is judged to be continuously tested, outputting the robustness of the target test classifier according to the classification result after the attack;
the step of adjusting a first parameter of the sensor network comprises:
will be provided with
Adjusting the first parameter as an adjustment indicator for the sensor network, wherein z represents the random noise, Pz(z) represents the distribution of the random noise samples, xMRepresenting said malicious sample, Pmal(x) Representing the distribution of said malicious samples, G (z) representing said reference sample, C (G (z), xD) Representing the reference sample G (z) and the malicious sample xDGenerated escape sample, D (C (G (z), x)D) Represents the target test classifier on the escape sample C (G (z), x)D) As a result of the classification of (a),
denotes D (C (G (z), x)D) InRandom noise distribution and expected values under malicious sample distribution.
2. The method of claim 1, wherein after the step of obtaining an escape proportion of the target test classifier that misclassifies the escape variants according to the classification result after the attack, the method further comprises, before the step of adjusting a first parameter of the sensor network until the escape proportion reaches the preset proportion threshold, the step of:
when the escape proportion reaches a preset proportion threshold value, adjusting a second parameter of the target test classifier until the escape proportion is smaller than the preset proportion threshold value;
adjusting a second parameter of the target test classifier, comprising:
will be provided with
Adjusting the second parameter as an adjustment index of the target test classifier, wherein xnRepresents the normal sample, D (x)n) Representing target test classifier versus sample xnAs a result of the classification of (a),
denotes xn、xMG (z) random samples under a distribution,
representing the target test classifier pair samples
As a result of the classification of (a),
to represent
In that
Gradient of (A), Pnormal(x) Represents the distribution of the normal samples and represents the distribution of the normal samples,
represents D (x)n) The expected value under a normal sample distribution,
to represent
With the expectation of a random sample distribution, λ is a constant parameter.
3. The method of claim 2, wherein the first parameter or the second parameter is randomly gradient-down adjusted using an Adam optimization algorithm.
4. A device for testing classifier robustness, the device comprising:
the system comprises a sample obtaining unit, a sample obtaining unit and a sample analyzing unit, wherein the sample obtaining unit is used for inputting a preset test sample into a target test classifier for classification, and obtaining a malicious sample and a normal sample in the test sample, and the target test classifier is a classifier for safety application;
the characteristic value acquisition unit is used for inputting random noise into a preset sensor network and acquiring a reference characteristic value of the malicious sample through the sensor network to generate a reference sample;
the characteristic value modification unit is used for modifying the characteristic value of the malicious sample according to the reference characteristic value of the reference sample so as to generate an escape variant of the malicious sample;
the attack classification unit is used for inputting the escape variants into the target test classifier for classification, and obtaining a classification result of the target test classifier after being attacked by the escape variants;
a proportion obtaining unit, configured to obtain, according to the classification result after the attack, an escape proportion of the escape variant wrongly classified by the target test classifier;
a first adjusting unit, configured to adjust a first parameter of the sensor network until the escape proportion reaches a preset proportion threshold when the escape proportion is smaller than the preset proportion threshold;
a distance obtaining unit, configured to obtain a Wassertein distance between the distribution of the escape variants and a normal sample distribution in the test sample, so as to determine whether to continue testing the target classifier according to the wasserein distance; and
the performance output unit is used for outputting the robustness of the target test classifier according to the classification result after the attack if the target classifier is judged to be continuously tested;
the first adjusting unit comprises a first adjusting unit for adjusting the first adjusting unit to adjust the first adjusting unit to the first adjusting unit
Means for adjusting the first parameter as an adjustment indicator for the sensor network, wherein z represents the random noise, Pz(z) represents the distribution of the random noise samples, xMRepresenting said malicious sample, Pmal(x) Representing the distribution of said malicious samples, G (z) representing said reference sample, C (G (z), xD) Representing the reference sample G (z) and the malicious sample xDGenerated escape sample, D (C (G (z), x)D) Represents the target test classifier on the escape sample C (G (z), x)D) As a result of the classification of (a),
denotes D (C (G (z), x)D) Expected values under random noise distribution and malicious sample distribution.
5. The apparatus of claim 4, wherein the apparatus further comprises:
a second adjusting unit, configured to adjust a second parameter of the target test classifier until the escape proportion is smaller than a preset proportion threshold when the escape proportion reaches the preset proportion threshold;
the second adjusting unit comprises a second adjusting unit for adjusting the second adjusting unit to adjust the second adjusting unit to the first adjusting unit
A unit for adjusting the second parameter as an adjustment index of the target test classifier, wherein x isnRepresents the normal sample, D (x)n) Representing target test classifier versus sample xnAs a result of the classification of (a),
denotes xn、xMG (z) random samples under a distribution,
representing the target test classifier pair samples
As a result of the classification of (a),
to represent
In that
Gradient of (A), Pnormal(x) Represents the distribution of the normal samples and represents the distribution of the normal samples,
represents D (x)n) The expected value under a normal sample distribution,
to represent
With the expectation of a random sample distribution, λ is a constant parameter.
6. A test terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 3 when executing the computer program.
7. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 3.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910126943.1A CN110008987B (en) | 2019-02-20 | 2019-02-20 | Method and device for testing robustness of classifier, terminal and storage medium |
| PCT/CN2019/108799 WO2020168718A1 (en) | 2019-02-20 | 2019-09-28 | Classifier robustness testing method, apparatus, terminal and storage medium |
| PCT/CN2020/072339 WO2020168874A1 (en) | 2019-02-20 | 2020-01-16 | Classifier robustness test method and device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910126943.1A CN110008987B (en) | 2019-02-20 | 2019-02-20 | Method and device for testing robustness of classifier, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008987A CN110008987A (en) | 2019-07-12 |
| CN110008987B true CN110008987B (en) | 2022-02-22 |
Family
ID=67165913
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910126943.1A Active CN110008987B (en) | 2019-02-20 | 2019-02-20 | Method and device for testing robustness of classifier, terminal and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110008987B (en) |
| WO (2) | WO2020168718A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008987B (en) * | 2019-02-20 | 2022-02-22 | 深圳大学 | Method and device for testing robustness of classifier, terminal and storage medium |
| CN111582359B (en) * | 2020-04-28 | 2023-04-07 | 新疆维吾尔自治区烟草公司 | Image identification method and device, electronic equipment and medium |
| CN112381150A (en) * | 2020-11-17 | 2021-02-19 | 上海科技大学 | Confrontation sample detection method based on sample robustness difference |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103021406A (en) * | 2012-12-18 | 2013-04-03 | 台州学院 | Robust speech emotion recognition method based on compressive sensing |
| CN105740771A (en) * | 2016-01-22 | 2016-07-06 | 张健敏 | Bulldozing device with target identification function |
| CN107241350A (en) * | 2017-07-13 | 2017-10-10 | 北京紫光恒越网络科技有限公司 | Network security defence method, device and electronic equipment |
| CN107463951A (en) * | 2017-07-19 | 2017-12-12 | 清华大学 | A kind of method and device for improving deep learning model robustness |
| CN107688829A (en) * | 2017-08-29 | 2018-02-13 | 湖南财政经济学院 | A kind of identifying system and recognition methods based on SVMs |
| CN107749859A (en) * | 2017-11-08 | 2018-03-02 | 南京邮电大学 | A kind of malice Mobile solution detection method of network-oriented encryption flow |
| CN107862270A (en) * | 2017-10-31 | 2018-03-30 | 深圳云天励飞技术有限公司 | Face classification device training method, method for detecting human face and device, electronic equipment |
| CN108108769A (en) * | 2017-12-29 | 2018-06-01 | 咪咕文化科技有限公司 | A kind of sorting technique of data, device and storage medium |
| CN108491837A (en) * | 2018-03-07 | 2018-09-04 | 浙江工业大学 | A kind of confrontation attack method improving car plate attack robust |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050033200A1 (en) * | 2003-08-05 | 2005-02-10 | Soehren Wayne A. | Human motion identification and measurement system and method |
| US20050259820A1 (en) * | 2004-05-24 | 2005-11-24 | Eastman Kodak Company | Temporally distributed watermarking for image sequences |
| US10404745B2 (en) * | 2013-08-30 | 2019-09-03 | Rakesh Verma | Automatic phishing email detection based on natural language processing techniques |
| US20150067833A1 (en) * | 2013-08-30 | 2015-03-05 | Narasimha Shashidhar | Automatic phishing email detection based on natural language processing techniques |
| CN104792530B (en) * | 2015-04-15 | 2017-03-22 | 北京航空航天大学 | Deep-learning rolling bearing fault diagnosis method based on SDA (stacked denoising autoencoder) and Softmax regression |
| CN105488413A (en) * | 2015-06-19 | 2016-04-13 | 哈尔滨安天科技股份有限公司 | Malicious code detection method and system based on information gain |
| CN105975857A (en) * | 2015-11-17 | 2016-09-28 | 武汉安天信息技术有限责任公司 | Method and system for deducing malicious code rules based on in-depth learning method |
| CN106529293B (en) * | 2016-11-09 | 2019-11-05 | 东巽科技（北京）有限公司 | A kind of sample class determination method for malware detection |
| CN107276805B (en) * | 2017-06-19 | 2020-06-05 | 北京邮电大学 | Sample prediction method and device based on intrusion detection model and electronic equipment |
| CN108615071B (en) * | 2018-05-10 | 2020-11-24 | 创新先进技术有限公司 | Model testing method and device |
| CN109359815A (en) * | 2018-09-10 | 2019-02-19 | 华北电力大学 | Based on the smart grid deep learning training sample generation method for generating confrontation network |
| CN109120652A (en) * | 2018-11-09 | 2019-01-01 | 重庆邮电大学 | It is predicted based on difference WGAN network safety situation |
| CN110008987B (en) * | 2019-02-20 | 2022-02-22 | 深圳大学 | Method and device for testing robustness of classifier, terminal and storage medium |
-
2019
- 2019-02-20 CN CN201910126943.1A patent/CN110008987B/en active Active
- 2019-09-28 WO PCT/CN2019/108799 patent/WO2020168718A1/en active Application Filing
-
2020
- 2020-01-16 WO PCT/CN2020/072339 patent/WO2020168874A1/en active Application Filing
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103021406A (en) * | 2012-12-18 | 2013-04-03 | 台州学院 | Robust speech emotion recognition method based on compressive sensing |
| CN105740771A (en) * | 2016-01-22 | 2016-07-06 | 张健敏 | Bulldozing device with target identification function |
| CN107241350A (en) * | 2017-07-13 | 2017-10-10 | 北京紫光恒越网络科技有限公司 | Network security defence method, device and electronic equipment |
| CN107463951A (en) * | 2017-07-19 | 2017-12-12 | 清华大学 | A kind of method and device for improving deep learning model robustness |
| CN107688829A (en) * | 2017-08-29 | 2018-02-13 | 湖南财政经济学院 | A kind of identifying system and recognition methods based on SVMs |
| CN107862270A (en) * | 2017-10-31 | 2018-03-30 | 深圳云天励飞技术有限公司 | Face classification device training method, method for detecting human face and device, electronic equipment |
| CN107749859A (en) * | 2017-11-08 | 2018-03-02 | 南京邮电大学 | A kind of malice Mobile solution detection method of network-oriented encryption flow |
| CN108108769A (en) * | 2017-12-29 | 2018-06-01 | 咪咕文化科技有限公司 | A kind of sorting technique of data, device and storage medium |
| CN108491837A (en) * | 2018-03-07 | 2018-09-04 | 浙江工业大学 | A kind of confrontation attack method improving car plate attack robust |
Non-Patent Citations (2)
| Title |
|---|
| A robustness testing approach for SOAP web services;Nuno Laranjeiro等;《J Internet Serv Appl》;20121231;第215-232页 * |
| 模糊分类器的鲁棒性;刘小钦;《中国优秀硕士学位论文全文数据库（信息科技辑）》;20180215(第02期);第I140-262页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020168718A1 (en) | 2020-08-27 |
| WO2020168874A1 (en) | 2020-08-27 |
| CN110008987A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tesfahun et al. | Intrusion detection using random forests classifier with SMOTE and feature reduction | |
| CN113554089B (en) | Image classification countermeasure sample defense method and system and data processing terminal | |
| CN110008987B (en) | Method and device for testing robustness of classifier, terminal and storage medium | |
| US11200318B2 (en) | Methods and apparatus to detect adversarial malware | |
| CN110351291B (en) | DDoS attack detection method and device based on multi-scale convolutional neural network | |
| CN111062036A (en) | Malicious software identification model construction method, malicious software identification medium and malicious software identification equipment | |
| JP2006079479A (en) | Time series data determination method | |
| CN112560596B (en) | Radar interference category identification method and system | |
| CN113360912A (en) | Malicious software detection method, device, equipment and storage medium | |
| CN111626367A (en) | Countermeasure sample detection method, apparatus, device and computer readable storage medium | |
| CN112738092A (en) | Log data enhancement method, classification detection method and system | |
| CN115277189B (en) | Unsupervised intrusion flow detection and identification method based on generation type countermeasure network | |
| CN111400707A (en) | File macro virus detection method, device, equipment and storage medium | |
| CN110598794A (en) | Classified countermeasure network attack detection method and system | |
| CN112001424B (en) | Malicious software open set family classification method and device based on countermeasure training | |
| CN113542252A (en) | Detection method, detection model and detection device for Web attack | |
| JP2021022316A (en) | Learning device, learning method, and learning program | |
| CN113534059B (en) | Radar active interference identification method based on deep convolutional network under open set scene | |
| CN113839963A (en) | Network security vulnerability intelligent detection method based on artificial intelligence and big data | |
| CN111209567B (en) | Method and device for judging perceptibility of improving robustness of detection model | |
| US11551137B1 (en) | Machine learning adversarial campaign mitigation on a computing device | |
| Yan et al. | $ D^ 3$: Detoxing Deep Learning Dataset | |
| Kamel et al. | AdaBoost ensemble learning technique for optimal feature subset selection | |
| CN114884755B (en) | Network security protection method and device, electronic equipment and storage medium | |
| US20230145002A1 (en) | Connecting adversarial attacks to neural network topography |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |