CN109165671A - Confrontation sample testing method based on sample to decision boundary distance - Google Patents
Confrontation sample testing method based on sample to decision boundary distance Download PDFInfo
- Publication number
- CN109165671A CN109165671A CN201810768347.9A CN201810768347A CN109165671A CN 109165671 A CN109165671 A CN 109165671A CN 201810768347 A CN201810768347 A CN 201810768347A CN 109165671 A CN109165671 A CN 109165671A
- Authority
- CN
- China
- Prior art keywords
- sample
- resisting
- decision boundary
- classifier
- apart
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
A kind of confrontation sample testing method based on sample to decision boundary distance, it is generated according to common sample to resisting sample, and feature extraction is carried out to all samples, calculate each sample to decision boundary range estimation, then classifier is trained using range estimation as the feature of sample, classifier after training is detector, for detecting to resisting sample.The present invention can be widely applied for the machine learning model based on classifier, such as speech recognition, image classification multiple fields, improve confrontation pattern detection rate.For artificial intelligence API, input sample can be filtered, the safety of artificial intelligence is obviously improved.
Description
Technical field
It is specifically a kind of based on sample to decision boundary the present invention relates to a kind of technology in artificial intelligence confrontation field
The confrontation sample testing method of distance.
Background technique
Artificial intelligence develops rapidly in recent years, is also applicable in more and more fields.But research finds artificial intelligence
There are more serious security breaches in classifier, malicious attacker can by carrying out small sample perturbations to normal identification sample,
It becomes to resisting sample, classifier identification error can be made to resisting sample, can be supported to a certain extent using antagonistic training
It is anti-that resisting sample is attacked, but effect is always unsatisfactory, then, Many researchers are wished by some of resisting sample
Inherent characteristic is detected to resisting sample, to resist confrontation sexual assault.
Summary of the invention
The present invention, which is directed to, attacks resisting sample, proposes a kind of confrontation pattern detection side based on sample to decision boundary distance
Method, distance using sample to decision boundary as sample feature, whether to be to resisting sample as tag along sort, training one
Classifier, using this classifier as confrontation sample detector.The present invention, to attack resistance, can be widely applied for for artificial intelligence
Machine learning model based on classifier, such as speech recognition, image classification multiple fields improve confrontation pattern detection rate.With
In artificial intelligence API, input sample can be filtered, the safety of artificial intelligence is obviously improved.
The present invention is achieved by the following technical solutions:
The present invention is generated according to common sample to resisting sample, and to carry out feature extraction to all samples, that is, is calculated each
Then sample instructs a classifier using range estimation as the feature of sample to the range estimation of decision boundary
Practice, the classifier after training is detector, for detecting to resisting sample.
It is described to resisting sample, equal proportion is mixed to get after being generated by a variety of pairs of resisting sample generating modes, generation side
Method include the Fast Field descent algorithm (iter-FGSM) of iteration, based on optimization to resisting sample distance calculating method (C&W),
Confuse deep learning method (DeepFool), the greedy matching algorithm (JSMA) based on Jacobin matrix.
The feature extraction preferably first carries out invalid sample rejecting to all samples, and wherein invalid sample includes normal
Be classified in sample mistake sample, and can not deceive classifier (i.e. not across decision boundary) to resisting sample.
The classifier is specifically included by neural fusion: full articulamentum and Dropout layers.
The range estimation, by apart from upper bound distUWith apart from lower bound distLIts range is limited, by adjusting the distance
The distance of sample to decision boundary so that narrowed down to a more accurate range, i.e. [dist by the estimation of boundL,
distU]。
It is described apart from the upper bound, obtained using the distance calculating method based on attack；It is described apart from lower bound, use intersection
Lipschitz bounding method.
The distance calculating method based on attack specifically: use sample the Fast Field descent method of iteration
(iter-FGSM) it calculates to resisting sample, then using generation to needed for resisting sample, disturbance is as the estimation apart from the upper bound, specifically
Are as follows: Wherein:It is the sample that the i-th wheel FGSM is calculated,It is positive
Normal sample,ForLoss function, ε is the disturbance constant of each round FGSM,It produces when after k takes turns FGSM to resisting sample, is then exactly disturbing for k wheel apart from the upper bound
The sum of moving vector:
It is preferably 1 that constant ε is disturbed in the present invention.
The intersection Lipschitz bounding method specifically:Wherein: distLI.e.
Sample point is to decision boundary apart from lower bound, fj(x0) it is j-th component of the sample by output vector after model, fc(x0)
Subscript c is x0Affiliated classification,It is locally Lipschitz function constant, when with g (x0)=fc(x0)-fj(x0), then
Wherein: Bp(x0, R) and it is lpWith x under normal form0For the centre of sphere, radius is the sphere of R, p and q
Relationship be Specific calculation are as follows: in Bp(x0, R) and enough x are randomly selected in ball, by reversely passing
(back propagation) is passed to calculate at each x | | ▽ g (x) | |q, then it is maximized.
Radius is preferably 5 in the present invention, and sampling number is preferably 500.
Technical effect
The present invention is characterized by the distance of sample to decision boundary, and to identify to resisting sample, effect is obvious, discrimination compared with
It is high.In the calculating apart from the upper bound, using iter-FGSM as attack pattern, can find sample to decision boundary shortest path
Diameter, it is more accurate to measure.In the calculating apart from lower bound, using intersection Lipschitz bounding method, and part has been used
Lipschitz constant, rather than overall situation Lipschitz constant can make Lipschitz normal by sampling enough points
Number measurement result is accurate enough.The detector finally trained has reached preferable detection effect, and Detection accuracy is higher than existing
The intrinsic dimension method (LID) in part.
Detailed description of the invention
Fig. 1 is embodiment flow diagram；
Fig. 2 is the neural network structure figure of detector；
Fig. 3 is embodiment detection effect contrast schematic diagram.
Specific embodiment
As shown in Figure 1, the present embodiment uses BelgiumTS data set, method protects landmark identification through this embodiment
API is protected it from and is attacked resisting sample.
The present embodiment specifically includes:
Step 1: being generated to resisting sample: using the training sample set of API as normal sample, with a part of normal sample one's duty
Not Tong Guo iter-FGSM, C&W, DeepFool, tetra- kinds of attack patterns of JSMA generate to resisting sample (equal proportion mixing).
Step 2: invalid sample is rejected: rejecting invalid sample respectively to normal sample and to resisting sample: invalid sample packet
Include: 1. itself is normal sample, but identifies mistake by API, these samples are closer away from decision boundary, therefore is rejected；2. this is as right
Resisting sample, but identified correctly by API, this kind of attack resisting sample fails, and can not threaten API.
Step 3: being calculated apart from bound: calculating first apart from bound: the calculating apart from the upper bound: sampling this x0With
Iter-FGSM generate to resisting sample, then will to the perturbation vector of k iteration in resisting sample generating process superposition, obtain away from
From the upper bound, then obtained by following steps apart from lower bound:
3.1) this x is sampled0If it is correctly classified as c, to each classification j in addition to c, find outWhereinIt needs by with x0For the B of the centre of spherep(x0, R) and in ball, 500 points are randomly selected, it finds so that g (x)=fc(x)-fj(x)
Gradient lpNormal form | | ▽ g (x) | |qMaximum x；
3.2) find out | | ▽ g (x) | |q, asThus it finds outThen find so thatIt is minimum
J, then correspond toFor apart from lower bound.
Step 4: the training of detector: the network structure of detector is as shown in Fig. 2, specifically include: three full articulamentums,
Centre folder two layers Dropout layers；In training process, the size of each trained batch is 64.
As shown in figure 3, carrying out Contrast on effect with other existing confrontation sample testing methods: the intrinsic dimension in part after training
Counting method (LID): 93.5%
Cuclear density method (Kernel Density): 90.7%
K- average distance (k-mean distance): 86.0%
Confrontation sample testing method based on sample to decision boundary distance: 95.2%
Identical experiment is carried out on other data sets, comparative experiments effect: MNIST:
The intrinsic dimension methodology (LID) in part: 96.8%
Cuclear density method (Kernel Density): 95.7%
K- average distance (k-mean distance): 93.0%
Confrontation sample testing method based on sample to decision boundary distance: 98.4%
CIFAR
The intrinsic dimension methodology (LID) in part: 91.1%
Cuclear density method (Kernel Density): 83.5%
K- average distance (k-mean distance): 80.7%
Confrontation sample testing method based on sample to decision boundary distance: 94.3%
It can be seen that test effect of the invention is all higher than existing several detection methods on these sample sets, inspection
The histogram for surveying effect is as shown in Figure 3.
Above-mentioned specific implementation can by those skilled in the art under the premise of without departing substantially from the principle of the invention and objective with difference
Mode carry out local directed complete set to it, protection scope of the present invention is subject to claims and not by above-mentioned specific implementation institute
Limit, each implementation within its scope is by the constraint of the present invention.
Claims (7)
1. a kind of confrontation sample testing method based on sample to decision boundary distance, which is characterized in that raw according to common sample
Pairs of resisting sample, and will to all samples carry out feature extraction, that is, calculate each sample to decision boundary range estimation, so
Classifier is trained using range estimation as the feature of sample afterwards, the classifier after training is used to detect confrontation sample
This.
2. according to the method described in claim 1, it is characterized in that, it is described to resisting sample, pass through a variety of pairs of resisting sample generation sides
Equal proportion is mixed to get after formula generates, and generation method includes the Fast Field descent algorithm of iteration, the confrontation sample based on optimization
This distance calculating method, fascination deep learning method, the greedy matching algorithm based on Jacobin matrix.
3. according to the method described in claim 1, it is characterized in that, the feature extraction first carries out invalid sample to all samples
This rejecting, wherein invalid sample include be classified in normal sample mistake sample, and can not deceive classifier (i.e. not across
Decision boundary) to resisting sample.
4. according to the method described in claim 1, it is characterized in that, the classifier is specifically included by neural fusion:
Full articulamentum and Dropout layers.
5. according to the method described in claim 1, it is characterized in that, the range estimation, by apart from upper bound distUWith away from
From lower bound distLIts range is limited, the distance of sample to decision boundary is narrowed down to by the estimation for bound of adjusting the distance
One more accurate range, i.e. [distL, distU], wherein obtained apart from the upper bound using the distance calculating method based on attack,
It is calculated apart from lower bound using Lipschitz bounding method is intersected.
6. according to the method described in claim 5, it is characterized in that, the distance calculating method based on attack specifically: right
Sample is calculated using the Fast Field descent method (iter-FGSM) of iteration to resisting sample, then with generation to needed for resisting sample
It disturbs as the estimation apart from the upper bound, specifically:Wherein:It is
The sample that i-th wheel FGSM is calculated,For normal sample,ForLoss function, ε is the disturbance of each round FGSM
Constant,It produces when after k takes turns FGSM to resisting sample, is then exactly k wheel apart from the upper bound
The sum of perturbation vector:
7. according to the method described in claim 5, it is characterized in that, the intersection Lipschitz bounding method specifically:Wherein: distLI.e. sample point is to decision boundary apart from lower bound, fj(x0) it is that sample passes through
J-th of component of output vector, f after modelc(x0) subscript c be classification belonging to x0,It is locally Lipschitz function constant, when
With g (x0)=fc(x0)-fj(x0), thenWherein: Bp(x0, R) and it is lpWith x under normal form0For ball
The heart, radius are the sphere of R, and the relationship of p and q are Specific calculation are as follows: in Bp(x0, R) and it is taken out in ball at random
Enough x are taken fully, are calculated at each x by back transfer (back propagation) | | ▽ g (x) | |q, then take maximum
Value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810768347.9A CN109165671A (en) | 2018-07-13 | 2018-07-13 | Confrontation sample testing method based on sample to decision boundary distance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810768347.9A CN109165671A (en) | 2018-07-13 | 2018-07-13 | Confrontation sample testing method based on sample to decision boundary distance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109165671A true CN109165671A (en) | 2019-01-08 |
Family
ID=64897814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810768347.9A Pending CN109165671A (en) | 2018-07-13 | 2018-07-13 | Confrontation sample testing method based on sample to decision boundary distance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109165671A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784411A (en) * | 2019-01-23 | 2019-05-21 | 四川虹微技术有限公司 | To the defence method of resisting sample, device, system and storage medium |
| CN109978029A (en) * | 2019-03-13 | 2019-07-05 | 北京邮电大学 | A kind of invalid image pattern screening technique based on convolutional neural networks |
| CN110503974A (en) * | 2019-08-29 | 2019-11-26 | 泰康保险集团股份有限公司 | Fight audio recognition method, device, equipment and computer readable storage medium |
| CN111209370A (en) * | 2019-12-27 | 2020-05-29 | 同济大学 | Text classification method based on neural network interpretability |
| CN111881034A (en) * | 2020-07-23 | 2020-11-03 | 深圳慕智科技有限公司 | Confrontation sample generation method based on distance |
| CN112200257A (en) * | 2020-10-16 | 2021-01-08 | 支付宝(杭州)信息技术有限公司 | Method and device for generating confrontation sample |
| CN112329837A (en) * | 2020-11-02 | 2021-02-05 | 北京邮电大学 | Countermeasure sample detection method and device, electronic equipment and medium |
| CN112381150A (en) * | 2020-11-17 | 2021-02-19 | 上海科技大学 | Confrontation sample detection method based on sample robustness difference |
| CN112464776A (en) * | 2020-11-22 | 2021-03-09 | 德派(嘉兴)医疗器械有限公司 | Learning state monitoring method, system and device |
| WO2021056746A1 (en) * | 2019-09-23 | 2021-04-01 | 平安科技（深圳）有限公司 | Image model testing method and apparatus, electronic device and storage medium |
| WO2021109695A1 (en) * | 2019-12-06 | 2021-06-10 | 支付宝(杭州)信息技术有限公司 | Adversarial attack detection method and device |
| CN113052314A (en) * | 2021-05-27 | 2021-06-29 | 华中科技大学 | Authentication radius guide attack method, optimization training method and system |
| WO2021189364A1 (en) * | 2020-03-26 | 2021-09-30 | 深圳先进技术研究院 | Method and device for generating adversarial image, equipment, and readable storage medium |
-
2018
- 2018-07-13 CN CN201810768347.9A patent/CN109165671A/en active Pending
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784411A (en) * | 2019-01-23 | 2019-05-21 | 四川虹微技术有限公司 | To the defence method of resisting sample, device, system and storage medium |
| CN109784411B (en) * | 2019-01-23 | 2021-01-05 | 四川虹微技术有限公司 | Defense method, device and system for confrontation sample and storage medium |
| CN109978029A (en) * | 2019-03-13 | 2019-07-05 | 北京邮电大学 | A kind of invalid image pattern screening technique based on convolutional neural networks |
| CN110503974A (en) * | 2019-08-29 | 2019-11-26 | 泰康保险集团股份有限公司 | Fight audio recognition method, device, equipment and computer readable storage medium |
| CN110503974B (en) * | 2019-08-29 | 2022-02-22 | 泰康保险集团股份有限公司 | Confrontation voice recognition method, device, equipment and computer readable storage medium |
| WO2021056746A1 (en) * | 2019-09-23 | 2021-04-01 | 平安科技（深圳）有限公司 | Image model testing method and apparatus, electronic device and storage medium |
| WO2021109695A1 (en) * | 2019-12-06 | 2021-06-10 | 支付宝(杭州)信息技术有限公司 | Adversarial attack detection method and device |
| CN111209370A (en) * | 2019-12-27 | 2020-05-29 | 同济大学 | Text classification method based on neural network interpretability |
| US11995155B2 (en) | 2020-03-26 | 2024-05-28 | Shenzhen Institutes Of Advanced Technology | Adversarial image generation method, computer device, and computer-readable storage medium |
| WO2021189364A1 (en) * | 2020-03-26 | 2021-09-30 | 深圳先进技术研究院 | Method and device for generating adversarial image, equipment, and readable storage medium |
| CN111881034A (en) * | 2020-07-23 | 2020-11-03 | 深圳慕智科技有限公司 | Confrontation sample generation method based on distance |
| CN112200257A (en) * | 2020-10-16 | 2021-01-08 | 支付宝(杭州)信息技术有限公司 | Method and device for generating confrontation sample |
| CN112329837A (en) * | 2020-11-02 | 2021-02-05 | 北京邮电大学 | Countermeasure sample detection method and device, electronic equipment and medium |
| CN112381150A (en) * | 2020-11-17 | 2021-02-19 | 上海科技大学 | Confrontation sample detection method based on sample robustness difference |
| CN112381150B (en) * | 2020-11-17 | 2024-08-06 | 上海科技大学 | Sample robustness difference-based countersample detection method |
| CN112464776A (en) * | 2020-11-22 | 2021-03-09 | 德派(嘉兴)医疗器械有限公司 | Learning state monitoring method, system and device |
| CN113052314A (en) * | 2021-05-27 | 2021-06-29 | 华中科技大学 | Authentication radius guide attack method, optimization training method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109165671A (en) | Confrontation sample testing method based on sample to decision boundary distance | |
| CN108093406B (en) | Wireless sensor network intrusion detection method based on ensemble learning | |
| CN110674938B (en) | Anti-attack defense method based on cooperative multi-task training | |
| CN103716204B (en) | Abnormal intrusion detection ensemble learning method and apparatus based on Wiener process | |
| CN108932527A (en) | Using cross-training model inspection to the method for resisting sample | |
| CN102571486B (en) | Traffic identification method based on bag of word (BOW) model and statistic features | |
| CN108615048A (en) | It is evolved based on disturbance and fights the defence method of sexual assault to Image Classifier | |
| CN104464079B (en) | Multiple Currencies face amount recognition methods based on template characteristic point and topological structure thereof | |
| CN109729091A (en) | A kind of LDoS attack detection method based on multiple features fusion and CNN algorithm | |
| Wang et al. | Detecting adversarial samples for deep neural networks through mutation testing | |
| CN109977897A (en) | A kind of ship's particulars based on deep learning recognition methods, application method and system again | |
| CN110365647A (en) | A kind of false data detection method for injection attack based on PCA and BP neural network | |
| CN105096344B (en) | Group behavior recognition methods and system based on CD motion features | |
| CN107909027A (en) | It is a kind of that there is the quick human body target detection method for blocking processing | |
| CN102945374B (en) | Method for automatically detecting civil aircraft in high-resolution remote sensing image | |
| CN112333128B (en) | Web attack behavior detection system based on self-encoder | |
| Lu et al. | On the limitation of local intrinsic dimensionality for characterizing the subspaces of adversarial examples | |
| Lin | PCA/SVM‐based method for pattern detection in a multisensor system | |
| CN109522755A (en) | Hardware Trojan horse detection method based on probabilistic neural network | |
| CN108549866A (en) | Remote sensing aeroplane recognition methods based on intensive convolutional neural networks | |
| CN106250913B (en) | A kind of combining classifiers licence plate recognition method based on local canonical correlation analysis | |
| CN107145778A (en) | A kind of intrusion detection method and device | |
| CN106296697A (en) | A kind of distorted image method of inspection quickly calculated based on two dimension sliding window DFT | |
| CN108154089A (en) | A kind of people counting method of head detection and density map based on dimension self-adaption | |
| CN104899606B (en) | It is a kind of based on the Information Hiding ＆ Detecting method locally learnt |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190108 |
|
| RJ01 | Rejection of invention patent application after publication |