CN111259404A - Toxic sample generation method, device, equipment and computer readable storage medium - Google Patents
Toxic sample generation method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111259404A CN111259404A CN202010024362.XA CN202010024362A CN111259404A CN 111259404 A CN111259404 A CN 111259404A CN 202010024362 A CN202010024362 A CN 202010024362A CN 111259404 A CN111259404 A CN 111259404A
- Authority
- CN
- China
- Prior art keywords
- particle
- training
- particles
- sample
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Abstract
The invention relates to the technical field of computers, and discloses a method, a device and equipment for generating a poisoning sample and a computer readable storage medium. The toxic sample generation method comprises the following steps: acquiring a model to be detoxified and a sample set, and initializing particle parameters of particles in a particle swarm; training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result; performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively; outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached; generating a poisoned sample based on the target globally optimal extremum and the sample set. The invention can improve the attack effect of the poisoned sample.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for generating a poisoning sample.
Background
With the development of computer technology, in recent years, with the continuous maturity and rapid development of technology, a large number of enterprises make breakthrough progress in the field of machine learning. At present, machine learning systems are gradually widely applied to security-sensitive tasks, such as security monitoring, intrusion detection, malware detection, and the like. The training process is an important stage of machine learning, and the quality of training data has a direct relation to the quality of a prediction model. However, the security of this process is often overlooked, and therefore many attackers focus attacks against machine learning models on the training data. At present, the most common attack method is a poison-throwing attack, and an attacker injects some carefully forged poisoning data samples to reduce the classification or clustering precision of the trained model, so that the aim of destroying the trained model is fulfilled.
During a toxic attack, the most critical step is to generate and optimize a toxic sample. In the existing scheme, a gradient descent method is generally adopted to generate and optimize poisoning points, however, an optimization algorithm of gradient descent often falls into a local optimal solution, so that misleading effects of poisoning samples on a machine learning model are poor, namely, attack effects are poor. Therefore, how to improve the attack effect of the poisoned sample is a problem which needs to be solved urgently at present.
Disclosure of Invention
The invention mainly aims to provide a method, a device, equipment and a computer readable storage medium for generating a poisoning sample, aiming at improving the attack effect of the poisoning sample.
In order to achieve the above object, the present invention provides a method for generating a poisoning sample, including:
acquiring a model to be detoxified and a sample set, and initializing particle parameters of particles in a particle swarm;
training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result;
performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively;
outputting a target global optimal extreme value of the particle swarm by the output particles when a preset iteration stop condition is reached;
generating a poisoned sample based on the target globally optimal extremum and the sample set.
Optionally, the sample set includes a training sample set and a testing sample set, and the step of training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of each particle according to the training result includes:
training the model to be poisoned based on the initialized particles and the training sample set to obtain a trained model to be poisoned;
testing the trained model to be detoxified on the test sample set to obtain the fitness of each particle;
and updating the particle parameters of each particle according to the fitness.
Optionally, the step of updating the particle parameter of each particle according to the fitness includes:
determining the current global optimal extreme value of the particle swarm according to the fitness, and determining the current individual optimal extreme value of each particle;
and updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
Optionally, the particle parameters include particle positions and particle velocities, and the step of updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value includes:
substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particle into a first preset calculation formula for calculation to obtain an updated particle speed;
and substituting the updated particle velocity and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position.
Optionally, the first preset formula is:
vij＝αvi(j-1)+c1r1(pbestij-pi(j-1))+c2r2(gbestj-pi(j-1))
the second preset formula is as follows:
pij＝vij+pi(j-1)
wherein v isijThe particle velocity obtained by updating the particle i after j times of training is 1, 2, …, N, N is the group number of the preset particle group, α is the preset inertia weight, c is the preset particle velocity1Is a first predetermined acceleration factor, c2Is a second predetermined acceleration factor, r1Is a first predetermined random number, 0 < r1＜1，r2Is a second predetermined random number, 0 < r2＜1，vi(j-1)Particle velocity, pbest, of particle i before jth trainingijIs the current individual optimum extreme value, gbest, of the particle i after j times of trainingjIs the current global optimum extreme value, p, of the particle swarm after j times of trainingi(j-1)Is the particle position of particle i before the jth training, pijAnd updating the obtained particle position after j times of training for the particle i.
Optionally, before the step of training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of each particle according to the training result, the method further includes:
detecting whether the initialized particle parameters of the particles exceed a preset attribute range or not to obtain a detection result;
processing the particles beyond the preset attribute range according to the detection result to obtain processed particles;
the step of training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of the particles according to the training result comprises the following steps:
and training the model to be poisoned based on the processed particles and the sample set, and updating the particle parameters of the particles according to the training result.
Optionally, the step of obtaining the model to be administered and the sample set includes:
when a poisoning sample generation instruction is received, acquiring model information to be poisoned and sample information carried in the poisoning sample generation instruction;
and acquiring the model to be poisoned according to the model information to be poisoned, and acquiring a sample set according to the sample information.
Further, to achieve the above object, the present invention also provides a poisoning sample generation apparatus including:
the acquisition module is used for acquiring a model to be detoxified and a sample set and initializing particle parameters of particles in the particle swarm;
the updating module is used for training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of the particles according to the training result;
the iteration module is used for carrying out iteration training on the model to be poisoned based on the updated particles so as to update the particle parameters of the particles in an iteration mode;
the output module is used for outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
and the generating module is used for generating a poisoning sample based on the target global optimal extreme value and the sample set.
Further, to achieve the above object, the present invention also provides a poisoning sample generation apparatus including: a memory, a processor and a poisoning sample generation program stored on the memory and executable on the processor, the poisoning sample generation program when executed by the processor implementing the steps of the poisoning sample generation method as described above.
Furthermore, to achieve the above object, the present invention also provides a computer readable storage medium having a poisoning sample generation program stored thereon, which when executed by a processor implements the steps of the poisoning sample generation method as described above.
The invention provides a method, a device and equipment for generating a poisoning sample and a computer readable storage medium, wherein a model to be poisoned and a sample set are obtained, and the particle parameters of particles in a particle swarm are initialized; training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result; performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively; outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached; and generating a poisoning sample based on the target global optimal extreme value and the sample set. Compared with the prior art in which the poisoning sample is generated based on the gradient descent method, the poisoning sample is generated based on the particle swarm optimization algorithm, and partial local optimal points can be effectively avoided, so that the attack effect of the poisoning sample can be effectively improved. Meanwhile, in the embodiment of the invention, an attacker does not need to know any knowledge of the model to be poisoned, belongs to black box attack and can be applied to almost all machine learning algorithms, so that the method for generating the poisoning sample in the embodiment of the invention has universality.
Drawings
FIG. 1 is a schematic diagram of an apparatus architecture of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for generating a poisoning specimen according to a first embodiment of the present invention;
FIG. 3 is a detailed flowchart of step S20 according to the first embodiment of the present invention;
FIG. 4 is a functional block diagram of a first embodiment of the toxic sample generating device of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present invention.
In the embodiment of the present invention, the poison sample generating device may be a server, or may also be a terminal device such as a PC (personal computer), a tablet computer, or a portable computer.
As shown in fig. 1, the poisoning sample generation apparatus may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the poisoned sample generating device configuration shown in fig. 1 does not constitute a limitation of poisoned sample generating devices and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a poisoning sample generation program.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client and performing data communication with the client; and the processor 1001 may be configured to invoke the poisoning sample generation procedure stored in the memory 1005 and perform the following operations:
acquiring a model to be detoxified and a sample set, and initializing particle parameters of particles in a particle swarm;
training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result;
performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively;
outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
generating a poisoned sample based on the target globally optimal extremum and the sample set.
Further, the sample set includes a training sample set and a testing sample set, and the processor 1001 may call the poisoning sample generation program stored in the memory 1005, and further perform the following operations:
training the model to be poisoned based on the initialized particles and the training sample set to obtain a trained model to be poisoned;
testing the trained model to be detoxified on the test sample set to obtain the fitness of each particle;
and updating the particle parameters of each particle according to the fitness.
Further, the processor 1001 may call the poisoning sample generation program stored in the memory 1005, and further perform the following operations:
determining the current global optimal extreme value of the particle swarm according to the fitness, and determining the current individual optimal extreme value of each particle;
and updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
Further, the particle parameters include particle position and particle velocity, and the processor 1001 may call the poisoning sample generation program stored in the memory 1005, and further perform the following operations:
substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particle into a first preset calculation formula for calculation to obtain an updated particle speed;
and substituting the updated particle velocity and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position.
Further, the first preset formula is as follows:
vij＝αvi(j-1)+c1r1(pbestij-pi(j-1))+c2r2(gbestj-pi(j-1))
the second preset formula is as follows:
pij＝vij+pi(j-1)
wherein v isijThe particle velocity obtained by updating the particle i after j times of training is 1, 2, …, N, N is the group number of the preset particle group, α is the preset inertia weight, c is the preset particle velocity1Is a first predetermined acceleration factor, c2Is a second predetermined acceleration factor, r1Is a first predetermined random number, 0 < r1＜1，r2Is a second predetermined random number, 0 < r2＜1，vi(j-1)Particle velocity, pbest, of particle i before jth trainingijIs the current individual optimum extreme value, gbest, of the particle i after j times of trainingjIs the current global optimum extreme value, p, of the particle swarm after j times of trainingi(j-1)Is the particle position of particle i before the jth training, pijAnd updating the obtained particle position after j times of training for the particle i.
Further, the processor 1001 may call the poisoning sample generation program stored in the memory 1005, and further perform the following operations:
detecting whether the initialized particle parameters of the particles exceed a preset attribute range or not to obtain a detection result;
processing the particles beyond the preset attribute range according to the detection result to obtain processed particles;
and training the model to be poisoned based on the processed particles and the sample set, and updating the particle parameters of the particles according to the training result.
Further, the processor 1001 may call the poisoning sample generation program stored in the memory 1005, and further perform the following operations:
when a poisoning sample generation instruction is received, acquiring model information to be poisoned and sample information carried in the poisoning sample generation instruction;
and acquiring the model to be poisoned according to the model information to be poisoned, and acquiring a sample set according to the sample information.
Based on the hardware structure, various embodiments of the virus sample generation method are provided.
The invention provides a method for generating a poisoning sample.
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for generating a virus sample according to a first embodiment of the present invention.
In this embodiment, the poisoning sample generation method includes:
step S10, obtaining a model to be poisoned and a sample set, and initializing particle parameters of each particle in the particle swarm;
the method for generating the poisoning sample according to the present embodiment is implemented by a poisoning sample generation device, which is described by taking a server as an example. In this embodiment, the attacker knowledge is set as black box knowledge, that is, the attacker does not know all knowledge of the target machine learning model (that is, the machine learning model to be poisoned), and the attacker capability is the capability of injecting the sample, that is, the capability of the attacker to inject the poisoned sample into the target machine learning model during the machine learning training process.
In this embodiment, a model to be poisoned and a sample set are obtained first, and particle parameters of each example in a particle swarm are initialized.
Wherein, the step of obtaining the model to be poisoned and the sample set comprises the following steps:
step a1, when a poisoning sample generation instruction is received, acquiring model information to be poisoned and sample information carried in the poisoning sample generation instruction;
step a2, obtaining the model to be poisoned according to the model information to be poisoned, and obtaining a sample set according to the sample information.
In this embodiment, when a user (i.e., an attacker) wants to perform a poisoning attack on a machine learning model of another person, information of a model to be poisoned and information of a sample to be injected may be set first, and a poisoning sample generation instruction is triggered, at this time, when the server receives the poisoning sample generation instruction, the server obtains the model information to be poisoned and the sample information carried in the poisoning sample generation instruction; the information of the model to be poisoned is information of the machine learning model to be poisoned and attacked, and the sample information is sample information corresponding to the machine learning model to be poisoned and attacked. Wherein the sample set comprises a training sample set and a testing sample set.
It should be noted that, in the embodiments of the present invention, the obtaining of the model to be poisoned and the sample set is only limited to be used, and specific data of the model to be poisoned and the sample set cannot be obtained. That is to say, the embodiment of the present invention is applicable to a black box attack scenario, that is, an attacker does not need to know any knowledge of a model to be poisoned, so the poisoning sample generation method in the embodiment of the present invention has universality and can be applied to almost all machine learning algorithms.
Step S20, training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result;
after initializing the particle parameters of each particle in the particle swarm, training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of each particle according to the training result. Specifically, the model to be poisoned may be trained based on the initialized particles and the training sample set to obtain a trained model to be poisoned; then, testing the trained model to be poisoned on a test sample set to obtain the fitness of each particle; and finally, updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value. Wherein the particle parameters include particle position and particle velocity. The current global optimum extremum is for the entire particle swarm, i.e., the current best position among all particles of the particle swarm. As can be appreciated, the more accurate the prediction isThe lower the maximum value of the global optimal extremum, the better the corresponding perturbation effect, that is, the position of the particle corresponding to the minimum fitness value until the current training is reached. The current individual optimal extreme value of each particle is the historical best position of each particle until the training is performed, namely the position of the particle corresponding to the minimum fitness value of each particle. For convenience of illustration, the current global optimum extreme value after j times of training can be recorded as gbestjMarking the current individual optimal extreme value of the particle i after j times of training as pbestijWherein j represents j training, i represents ith particle, i is 1, 2, …, and N is the population number of the preset particle group. After the first training, the current global optimum extreme value gbest can be obtained1And the current individual optimum extremum pbesti1Obtaining the particle parameters of each particle after the first update, including the particle velocity v after the first updatei1And a first updated particle position pi1. For the specific particle parameter updating process, reference may be made to the following embodiments, which are not described herein again.
Step S30, performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of each particle iteratively;
and then, carrying out iterative training on the model to be poisoned based on the updated particles so as to iteratively update the particle parameters of the particles.
Specifically, taking the second training process as an example, the fitness of each particle obtained after the first training test is denoted as a first fitness, the fitness of each particle obtained after the second training test is denoted as a second fitness, and denoted as fit (p)i2) Recording the current global optimum extreme value after the first training as a first current global optimum extreme value and as a gbest1Recording the current individual optimal extreme value of each particle after the first training as a first current individual optimal extreme value and as pbesti1。
First, the second fitness fit (p) of each particle is comparedi2) Fitness fit (pbest) corresponding to the respective first current individual optimum extremumi1) The magnitude relationship between the two, if fit (p)i2)＜fit(pbesti1) Then determine pbesti2＝pi2(ii) a If it is (p)i2)≥fit(pbesti1) Then determine pbesti2＝pbesti1Judging whether an iteration stop condition is preset or not, and if so, outputting a target global optimal solution of the particle swarm; if not, continuing the iterative training process.
When it is judged that fit (p)i2)＜fit(pbesti1) Determining pbesti2＝pi2Thereafter, the second fitness fit (p) of each particle is further comparedi2) Fitness fit (gbest) corresponding to the first current global optimum extreme value1) The magnitude relationship between them; if fit (p) is presenti2)＜fit(gbest1) Then, determine gbest2Is the presence of fit (p)i2) P corresponding to the minimum value ofi2(ii) a If it is (p)i2)≥fit(gbest1) Then, determine gbest2＝gbest1Judging whether an iteration stop condition is preset or not, and if so, outputting a target global optimal solution of the particle swarm; if not, continuing the iterative training process.
And after the second current global optimal extreme value and the second current individual optimal extreme value are obtained, updating the particle parameters of each particle. By analogy, the updating method of the particle parameters after each training is similar to the updating method of the particle parameters after the second training.
Step S40, outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
and when the preset iteration stop condition is reached, stopping the iterative training and outputting the target global optimal extreme value of the particle swarm. Wherein, the preset iteration stop condition may include, but is not limited to: and the iteration times reach preset iteration times, or the current global optimal extreme value obtained by updating and determining is smaller than a preset threshold value.
Step S50, generating a poisoning sample based on the target global optimal extremum and the sample set.
And after the target global optimal extreme value of the particle is obtained, generating a poisoning sample based on the target global optimal extreme value and the sample set. The target global optimal extreme value is actually a disturbance, and the target global optimal extreme value is superposed on the samples in the sample set to generate the poisoning sample. The poisoning sample can be used for poisoning a target machine learning model of other people so as to enable the poisoning sample to be used.
The embodiment of the invention provides a toxic sample generation method, which comprises the steps of obtaining a model to be subjected to toxic administration and a sample set, and initializing particle parameters of particles in a particle swarm; training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result; performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively; outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached; and generating a poisoning sample based on the target global optimal extreme value and the sample set. Compared with the prior art in which the poisoning sample is generated based on the gradient descent method, the poisoning sample is generated based on the particle swarm optimization algorithm, and partial local optimal points can be effectively avoided, so that the attack effect of the poisoning sample can be effectively improved. Meanwhile, in the embodiment of the invention, an attacker does not need to know any knowledge of the model to be poisoned, belongs to black box attack and can be applied to almost all machine learning algorithms, so that the method for generating the poisoning sample in the embodiment of the invention has universality.
Further, referring to fig. 3, fig. 3 is a detailed flowchart of step S20 according to the first embodiment of the present invention.
In this embodiment, the sample set includes a training sample set and a testing sample set, and step S20 includes:
step S21, training the model to be poisoned based on the initialized particles and the training sample set to obtain a trained model to be poisoned;
in this embodiment, the sample set includes a training sample set and a testing sample set, and the model to be poisoned is trained based on the initialized particles and the training sample set to obtain the trained model to be poisoned. Specifically, the initialized particles (i.e., initial disturbance) are superimposed on a training sample set training sample, and then the model to be poisoned is trained through the superimposed training sample, so as to obtain the trained model to be poisoned. The specific training process is determined based on the model to be poisoned, and reference is made to the prior art.
Step S22, testing the trained model to be poisoned on the test sample set to obtain the fitness of each particle;
and then, testing the trained model to be poisoned on a test sample set to obtain the fitness of each particle. And testing the trained model to be poisoned on the test sample set to obtain the prediction accuracy corresponding to each particle, and taking the prediction accuracy as the fitness of the particle.
And step S23, updating the particle parameters of each particle according to the fitness.
After the fitness of each particle is obtained, the particle parameters of each particle are updated according to the fitness.
Specifically, step S23 includes:
b1, determining the current global optimal extreme value of the particle swarm according to the fitness, and determining the current individual optimal extreme value of each particle;
and b2, updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
Firstly, determining the current global optimal extreme value of the particle swarm according to the fitness, and determining the current individual optimal extreme value of each particle. The current global optimal extreme value is for the whole particle swarm, namely the current best position in all the particles of the particle swarm. It can be understood that the lower the prediction accuracy is, the better the corresponding perturbation effect is, that is, the current global optimal extreme value is the particle position of the particle corresponding to the minimum fitness value until the training is performed. The current individual optimal extreme value of each particle is the historical best position of each particle until the training is performed, namely the position of the particle corresponding to the minimum fitness value of each particle. For convenience of illustration, the current global optimum extreme value after j times of training can be recorded as gbestjThe particles after j times of training arei's current individual optimum extremum is noted as pbestijWherein j represents j training, i represents ith particle, i is 1, 2, …, and N is the population number of the preset particle group. In this embodiment, the current global optimal extreme value obtained by the first training is gbest1The current individual optimal extreme value of the particle i is pbesti1The particle with the lowest prediction accuracy (i.e. the lowest fitness) is marked as pmThe position of the particle after the first training is denoted as pi1At this time, gbest1＝pm，pbesti1＝pi1。
And then, updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
Specifically, the particle parameters include a particle position and a particle velocity, and step b2 includes:
b21, substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particle into a first preset calculation formula for calculation to obtain an updated particle speed;
the particle parameters comprise a particle position and a particle speed, wherein the updating process of the particle speed is as follows: substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particles into a first preset calculation formula to calculate to obtain an updated particle speed; wherein, the first preset formula is as follows:
vij＝αvi(j-1)+c1r1(pbestij-pi(j-1))+c2r2(gbestj-pi(j-1))
wherein v isijThe particle velocity obtained by updating the particle i after j times of training is 1, 2, …, N, N is the group number of the preset particle group, α is the preset inertia weight, c is the preset particle velocity1Is a first predetermined acceleration factor, c2Is a second predetermined acceleration factor, r1Is a first predetermined random number, 0 < r1＜1，r2Is a second predetermined random number, 0 < r2＜1，vi(j-1)Particle velocity, pbest, of particle i before jth trainingijIs the current individual optimum extreme value, gbest, of the particle i after j times of trainingjIs the current global optimum extreme value, p, of the particle swarm after j times of trainingi(j-1)Is the particle position of particle i before the jth training, pijAnd updating the obtained particle position after j times of training for the particle i.
Since the present embodiment is directed to the first training, j is 1. At this time, vi1＝αvi0+c1r1(pbesti1-pi0)+c2r2(gbest1-pi0) Wherein, gbest1＝pm，pbesti1＝pi1，vi1The particle velocity, v, of the particle i after the 1 st trainingi0The particle velocity of the particle i before the 1 st training, i.e. the initial particle velocity of the particle i after initialization, pi0The particle position of the particle i before the 1 st training is the initial particle position of the particle i after initialization.
And b22, substituting the updated particle velocity and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position.
The updating process of the particle position is as follows: and substituting the updated particle speed and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position. Wherein the second predetermined formula is:
pij＝vij+pi(j-1)
wherein p isijThe position v of the particle obtained by updating the particle i after j times of trainingijThe particle velocity obtained by updating the particle i after j times of training, i is 1, 2, …, N, N is the group number of the preset particle group, pi(j-1)The position of the particle i before the jth training.
Since the present embodiment is directed to the first training, j is 1. At this time, pi1＝vi1+pi0Wherein p isi1Are particlesi particle position after 1 st training, vi1Is the particle velocity, p, of particle i after the 1 st trainingi0The position of particle i before the 1 st training.
Through the mode, after the model to be poisoned is trained for the first time, the particle parameters of the particles are updated intelligently, so that iterative training can be conveniently carried out on the model to be poisoned based on a similar mode subsequently, the particle parameters of the particles are updated iteratively, the target global optimal extreme value of the particle swarm is output, and the poisoning sample is generated based on the target global optimal extreme value.
Further, based on the above embodiments, a second example of the virus sample generation method of the present invention is provided.
In this embodiment, before step S20, the method further includes:
step A, detecting whether the particle parameters of the initialized particles exceed a preset attribute range or not to obtain a detection result;
because the sample has specific attributes, different specific attributes have corresponding attribute ranges, for example, for a picture type sample, the RGB (Red-Green-Blue) values of each pixel point are all in the range of 0 to 255. In order to ensure that the attributes of the finally generated poisoned sample are also in the corresponding attribute range after the disturbance (i.e. the target global optimum extremum) is superimposed on the sample, it is first required to ensure that the particle parameters of each particle after initialization are all in the corresponding attribute range. Therefore, in this embodiment, after initializing the particle parameters of each particle in the particle group, it is detected whether the particle parameters of each particle after initialization exceed the preset attribute range, so as to obtain a detection result. The preset attribute range may be preset when the toxic sample generation request is triggered, or may be determined according to sample information.
B, processing the particles beyond the preset attribute range according to the detection result to obtain processed particles;
and after the detection result is obtained, processing the particles beyond the preset attribute range according to the detection result, so that the particles beyond the preset attribute range are in the preset attribute range, and thus obtaining the processed particles. Specific treatment methods may be, but are not limited to: 1) randomly initializing the particle parameters of the particles beyond the preset attribute range again, and detecting the particle parameters of the reinitialized particles until the particle parameters of the finally processed particles are in the preset attribute range; 2) and updating and replacing the particle parameters of the particles beyond the preset attribute range according to the preset attribute range.
At this time, step S20 includes:
and training the model to be poisoned based on the processed particles and the sample set, and updating the particle parameters of the particles according to the training result.
Then, training the model to be poisoned based on the processed particles and the sample set, updating the particle parameters of each particle according to the training result, and further continuing to execute the subsequent steps, and the specific execution process may refer to the above-mentioned embodiments, which is not described herein again.
In this embodiment, by detecting and processing the particle parameters of the initialized particles, the particle parameters of the particles used for model training can be ensured to be in the corresponding attribute range, so that after the disturbance (i.e., the target global optimal extremum) is superimposed on the sample, the attributes of the finally generated poisoned sample are also in the corresponding attribute range.
The invention also provides a toxic sample generating device.
Referring to fig. 4, fig. 4 is a functional block diagram of a toxic sample generating device according to a first embodiment of the present invention.
As shown in fig. 4, the poisoning sample generation apparatus includes:
the acquisition module 10 is used for acquiring a model to be poisoned and initializing particle parameters of each particle in the particle swarm;
an updating module 20, configured to train the model to be poisoned based on the initialized particles and the sample set, and update particle parameters of each particle according to a training result;
the iteration module 30 is configured to perform iterative training on the model to be poisoned based on the updated particles, so as to update the particle parameters of each particle iteratively;
the output module 40 is used for outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
a generating module 50, configured to generate a poisoning sample based on the target globally optimal extremum and the sample set.
Further, the sample set includes a training sample set and a testing sample set, and the update module 20 includes:
the training unit is used for training the model to be poisoned based on the initialized particles and the training sample set to obtain a trained model to be poisoned;
the testing unit is used for testing the trained model to be poisoned on the testing sample set to obtain the fitness of each particle;
and the updating unit is used for updating the particle parameters of the particles according to the fitness.
Further, the update unit includes:
the determining subunit is used for determining the current global optimal extreme value of the particle swarm according to the fitness and determining the current individual optimal extreme value of each particle;
and the updating subunit is used for updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
Further, the particle parameters include a particle position and a particle velocity, and the update subunit is specifically configured to:
substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particle into a first preset calculation formula for calculation to obtain an updated particle speed;
and substituting the updated particle velocity and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position.
Further, the first preset formula is as follows:
vij＝αvi(j-1)+c1r1(pbestij-pi(j-1))+c2r2(gbestj-pi(j-1))
the second preset formula is as follows:
pij＝vij+pi(j-1)
wherein v isijThe particle velocity obtained by updating the particle i after j times of training is 1, 2, …, N, N is the group number of the preset particle group, α is the preset inertia weight, c is the preset particle velocity1Is a first predetermined acceleration factor, c2Is a second predetermined acceleration factor, r1Is a first predetermined random number, 0 < r1＜1，r2Is a second predetermined random number, 0 < r2＜1，vi(j-1)Particle velocity, pbest, of particle i before jth trainingijIs the current individual optimum extreme value, gbest, of the particle i after j times of trainingjIs the current global optimum extreme value, p, of the particle swarm after j times of trainingi(j-1)Is the particle position of particle i before the jth training, pijAnd updating the obtained particle position after j times of training for the particle i.
Further, the method for generating a poisoning sample further includes:
the detection module is used for detecting whether the initialized particle parameters of the particles exceed a preset attribute range or not to obtain a detection result;
the processing module is used for processing the particles beyond the preset attribute range according to the detection result to obtain processed particles;
the update module 20 is specifically configured to: and training the model to be poisoned based on the processed particles and the sample set, and updating the particle parameters of the particles according to the training result.
Further, the obtaining module 10 includes:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring model information to be poisoned and sample information carried in a poisoning sample generation instruction when the poisoning sample generation instruction is received;
and the second acquisition unit is used for acquiring the model to be poisoned according to the model information to be poisoned and acquiring a sample set according to the sample information.
The function implementation of each module in the above poisoning sample generation apparatus corresponds to each step in the above poisoning sample generation method embodiment, and the function and implementation process thereof are not described in detail here.
The present invention also provides a computer readable storage medium having a poisoning sample generation program stored thereon, which when executed by a processor implements the steps of the poisoning sample generation method according to any one of the above embodiments.
The specific embodiment of the computer-readable storage medium of the present invention is substantially the same as the embodiments of the method for generating a poisoning sample, and therefore, the detailed description thereof is omitted here.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for generating a poisoning sample, the method comprising:
acquiring a model to be detoxified and a sample set, and initializing particle parameters of particles in a particle swarm;
training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of the particles according to the training result;
performing iterative training on the model to be poisoned based on the updated particles to update the particle parameters of the particles iteratively;
outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
generating a poisoned sample based on the target globally optimal extremum and the sample set.
2. The poisoning sample generation method of claim 1, wherein the sample set comprises a training sample set and a testing sample set, and the step of training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of each particle according to the training result comprises:
training the model to be poisoned based on the initialized particles and the training sample set to obtain a trained model to be poisoned;
testing the trained model to be detoxified on the test sample set to obtain the fitness of each particle;
and updating the particle parameters of each particle according to the fitness.
3. The poisoning sample generation method of claim 2, wherein the step of updating the particle parameter of each particle according to the fitness comprises:
determining the current global optimal extreme value of the particle swarm according to the fitness, and determining the current individual optimal extreme value of each particle;
and updating the particle parameters of each particle according to the current global optimal extreme value and the current individual optimal extreme value.
4. The poisoning sample generation method of claim 3, wherein the particle parameters include particle position and particle velocity, and wherein updating the particle parameter for each particle based on the current global optimum extreme and the current individual optimum extreme comprises:
substituting the current global optimal extreme value, the current individual optimal extreme value and the initialized initial particle speed of the particle into a first preset calculation formula for calculation to obtain an updated particle speed;
and substituting the updated particle velocity and the initialized initial particle position of the particle into a second preset calculation formula for calculation to obtain an updated particle position.
5. The poisoning sample generation method of claim 4, wherein the first predetermined formula is:
vij＝αvi(j-1)+c1r1(pbestij-pi(j-1))+c2r2(gbestj-pi(j-1))
the second preset formula is as follows:
pij＝vij+pi(j-1)
wherein v isijAre particlesi is updated after j times of training, i is 1, 2, …, N is the group number of the preset particle group, α is the preset inertia weight, c is the updated particle speed1Is a first predetermined acceleration factor, c2Is a second predetermined acceleration factor, r1Is a first predetermined random number, 0 < r1＜1，r2Is a second predetermined random number, 0 < r2＜1，vi(j-1)Particle velocity, pbest, of particle i before jth trainingijIs the current individual optimum extreme value, gbest, of the particle i after j times of trainingjIs the current global optimum extreme value, p, of the particle swarm after j times of trainingi(j-1)Is the particle position of particle i before the jth training, pijAnd updating the obtained particle position after j times of training for the particle i.
6. The poisoning sample generation method according to any one of claims 1 to 5, wherein before the step of training the model to be poisoned based on the initialized particles and the sample set, and updating the particle parameters of each particle according to the training result, the method further includes:
detecting whether the initialized particle parameters of the particles exceed a preset attribute range or not to obtain a detection result;
processing the particles beyond the preset attribute range according to the detection result to obtain processed particles;
the step of training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of the particles according to the training result comprises the following steps:
and training the model to be poisoned based on the processed particles and the sample set, and updating the particle parameters of the particles according to the training result.
7. The toxicity sample generation method of any one of claims 1-5, wherein the step of obtaining the model to be administered and the sample set comprises:
when a poisoning sample generation instruction is received, acquiring model information to be poisoned and sample information carried in the poisoning sample generation instruction;
and acquiring the model to be poisoned according to the model information to be poisoned, and acquiring a sample set according to the sample information.
8. A poisoning sample generation apparatus, comprising:
the acquisition module is used for acquiring a model to be detoxified and a sample set and initializing particle parameters of particles in the particle swarm;
the updating module is used for training the model to be poisoned based on the initialized particles and the sample set and updating the particle parameters of the particles according to the training result;
the iteration module is used for carrying out iteration training on the model to be poisoned based on the updated particles so as to update the particle parameters of the particles in an iteration mode;
the output module is used for outputting a target global optimal extreme value of the particle swarm when a preset iteration stop condition is reached;
and the generating module is used for generating a poisoning sample based on the target global optimal extreme value and the sample set.
9. A poisoning sample generation apparatus, comprising: a memory, a processor and a poisoning sample generation program stored on the memory and executable on the processor, the poisoning sample generation program when executed by the processor implementing the steps of the poisoning sample generation method of any of claims 1 to 7.
10. A computer-readable storage medium, having a poisoning sample generation program stored thereon, which when executed by a processor implements the steps of the poisoning sample generation method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024362.XA CN111259404B (en) | 2020-01-09 | 2020-01-09 | Toxic sample generation method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024362.XA CN111259404B (en) | 2020-01-09 | 2020-01-09 | Toxic sample generation method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111259404A true CN111259404A (en) | 2020-06-09 |
| CN111259404B CN111259404B (en) | 2022-11-15 |
Family
ID=70950368
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010024362.XA Active CN111259404B (en) | 2020-01-09 | 2020-01-09 | Toxic sample generation method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111259404B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111898765A (en) * | 2020-07-29 | 2020-11-06 | 深圳前海微众银行股份有限公司 | Feature binning method, device, equipment and readable storage medium |
| US20210374247A1 (en) * | 2020-08-10 | 2021-12-02 | Intel Corporation | Utilizing data provenance to defend against data poisoning attacks |
| CN113792289A (en) * | 2021-11-16 | 2021-12-14 | 支付宝(杭州)信息技术有限公司 | Method and system for defending backdoor attack |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8756175B1 (en) * | 2012-02-22 | 2014-06-17 | Google Inc. | Robust and fast model fitting by adaptive sampling |
| CN109858368A (en) * | 2018-12-29 | 2019-06-07 | 浙江工业大学 | A kind of recognition of face attack defense method based on Rosenbrock-PSO |
| CN110175513A (en) * | 2019-04-15 | 2019-08-27 | 浙江工业大学 | A kind of guideboard identification attack defense method based on the optimization of multiple target road |
-
2020
- 2020-01-09 CN CN202010024362.XA patent/CN111259404B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8756175B1 (en) * | 2012-02-22 | 2014-06-17 | Google Inc. | Robust and fast model fitting by adaptive sampling |
| CN109858368A (en) * | 2018-12-29 | 2019-06-07 | 浙江工业大学 | A kind of recognition of face attack defense method based on Rosenbrock-PSO |
| CN110175513A (en) * | 2019-04-15 | 2019-08-27 | 浙江工业大学 | A kind of guideboard identification attack defense method based on the optimization of multiple target road |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111898765A (en) * | 2020-07-29 | 2020-11-06 | 深圳前海微众银行股份有限公司 | Feature binning method, device, equipment and readable storage medium |
| US20210374247A1 (en) * | 2020-08-10 | 2021-12-02 | Intel Corporation | Utilizing data provenance to defend against data poisoning attacks |
| CN113792289A (en) * | 2021-11-16 | 2021-12-14 | 支付宝(杭州)信息技术有限公司 | Method and system for defending backdoor attack |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111259404B (en) | 2022-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111259404B (en) | Toxic sample generation method, device, equipment and computer readable storage medium | |
| Naway et al. | A review on the use of deep learning in android malware detection | |
| EP3506139B1 (en) | Malware detection in event loops | |
| US11570211B1 (en) | Detection of phishing attacks using similarity analysis | |
| EP3647981B1 (en) | Security scanning method and apparatus for mini program, and electronic device | |
| CN107209818B (en) | Method and system for detecting false user interactions with a mobile device for improved malware protection | |
| US20200019821A1 (en) | Detecting and mitigating poison attacks using data provenance | |
| CN108268771A (en) | The malicious objects that Behavior-based control increment identifier is hidden | |
| WO2016122735A1 (en) | Methods and systems for identifying potential enterprise software threats based on visual and non-visual data | |
| US9934310B2 (en) | Determining repeat website users via browser uniqueness tracking | |
| WO2019222662A1 (en) | Methods and apparatuses to evaluate cyber security risk by establishing a probability of a cyber-attack being successful | |
| EP3789896A1 (en) | Method and system for managing security vulnerability in host system using artificial neural network | |
| US9779251B2 (en) | System, method, and computer program product for monitoring an execution flow of a function | |
| US20180129807A1 (en) | Shellcode Detection | |
| US9009819B1 (en) | Method and system for detecting rogue security software that displays frequent misleading warnings | |
| Yuan et al. | Mining software component interactions to detect security threats at the architectural level | |
| WO2020199163A1 (en) | Systems and methods for protecting remotely hosted application from malicious attacks | |
| Mao et al. | Toward exposing timing-based probing attacks in web applications | |
| US20230319099A1 (en) | Fuzz testing of machine learning models to detect malicious activity on a computer | |
| WO2016127037A1 (en) | Method and device for identifying computer virus variants | |
| CN112351008B (en) | Network attack analysis method and device, readable storage medium and computer equipment | |
| CN114021136A (en) | Back door attack defense system for artificial intelligence model | |
| US10552626B2 (en) | System and method for selecting a data entry mechanism for an application based on security requirements | |
| CN107908961B (en) | Malicious webpage detection method, equipment and storage medium based on virtualization | |
| Zhong et al. | Design for a cloud-based hybrid Android application security assessment framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |