WO2022081553A4 - Systems and methods for providing a systemic error in artificial intelligence algorithms - Google Patents
Systems and methods for providing a systemic error in artificial intelligence algorithms Download PDFInfo
- Publication number
- WO2022081553A4 WO2022081553A4 PCT/US2021/054542 US2021054542W WO2022081553A4 WO 2022081553 A4 WO2022081553 A4 WO 2022081553A4 US 2021054542 W US2021054542 W US 2021054542W WO 2022081553 A4 WO2022081553 A4 WO 2022081553A4
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- model
- adversarial
- group
- source
- source model
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract 25
- 238000013473 artificial intelligence Methods 0.000 title 1
- 230000009885 systemic effect Effects 0.000 title 1
- 238000012360 testing method Methods 0.000 claims abstract 16
- 238000012795 verification Methods 0.000 claims 3
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/096—Transfer learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Abstract
Disclosed is a process for testing a suspect model to determine whether it was derived from a source model. An example method includes receiving, from a model owner node, a source model and a fingerprint associated with the source model, receiving a suspect model at a service node, based on a request to test the suspect model, applying the fingerprint to the suspect model to generate an output and, when the output has an accuracy that is equal to or greater than a threshold, determining that the suspect model is derived from the source model. Imperceptible noise can be used to generate the fingerprint which can cause predictable outputs from the source model and a potential derivative thereof.
Claims
1. (Original) A method comprising: generating, based on a training dataset, a reference model and a surrogate model; selecting datapoints from the training dataset that are predicted correctly by a source model as a group of adversarial candidates; selecting, from the group of adversarial candidates, a sub-group of candidates that each have a low confidence score according to a threshold to yield a sub-group of adversarial candidates; adding noise to each candidate of the sub-group of adversarial candidates to yield a noisy group of adversarial examples; testing the noisy group of adversarial examples against the source model to obtain a set of source model outputs that the source model predicts correctly; testing the set of source model successful adversarial examples against the reference model to yield a set of reference model outputs; testing the set of source model successful adversarial examples against the surrogate model to yield a set of surrogate model outputs; and identifying a set of fingerprints based on which ones from the set of source model outputs, the set of reference model outputs and the set of surrogate model outputs pass as adversarial examples against the source model and the surrogate model, but not the reference model.
2. (Original) The method of claim 1, wherein the training dataset is from a same distribution of a source model dataset.
3. (Original) The method of claim 1, wherein the training dataset comprises at least some data from a source model dataset.
38
AMENDED SHEET (ARTICLE 19)
Application/ Control Number: PCT/US2021/054542 Docket No.: 213-0106-PCT
4. (Original) The method of claim 1, wherein fingerprint candidates comprise ones of the noisy group of adversarial examples that lead to a fully successful adversarial attack accuracy against the source model.
5. (Original) The method of claim 1, further comprising: sharing a hashed version of the set of fingerprints with a trusted third party.
6. (Original) The method of claim 1, wherein generating the set of fingerprints further comprises constructing respective adversarial examples with the noise that causes a receiving model to misclassify an input.
7. (Original) The method of claim 6, wherein the noise is imperceptible noise.
8. (Original) The method of claim 1, further comprising: testing a suspect model by using the set of fingerprints against the suspect model to determine whether an overall accuracy operating on the set of fingerprints is equal to or greater than a testing threshold.
9. (Original) The method of claim 8, further comprising: determining, when the overall accuracy operating on the set of fingerprints is equal to or greater than the testing threshold, that the suspect model was derived from the source model.
10. (Original) A method comprising: receiving, from a model owner node, a source model and verification key at a service node; receiving a suspect model at the service node; transmitting a request to the model owner node for a proof of ownership relative to the suspect model; in response to the request, receiving a marking key at the service node from the model owner node; and
39
AMENDED SHEET (ARTICLE 19)
Application/ Control Number: PCT/US2021/054542 Docket No.: 213-0106-PCT based on the marking key and the verification key, determining whether the suspect model was derived from the source model.
11. (Original) The method of claim 10, wherein determining whether the suspect model was derived from the source model further comprises testing the suspect model to determine whether a fingerprint produces a same output from both the source model and the suspect model.
12. (Original) The method of claim 11, wherein the fingerprint passes against the source model and a surrogate model, but not a reference model.
13. (Original) The method of claim 10, wherein at least one of the marking key and the verification key comprises added noise which causes a predictable output from the source model and surrogate models derived therefrom.
14. (Currently Amended) A method comprising: receiving, from a model owner node, a source model and a fingerprint associated with the source model; receiving a suspect model at a service node; based on a request to test the suspect model, applying the fingerprint to the suspect model to generate an output; and when the output has an accuracy that is equal to or greater than a threshold, determining that the suspect model is derived from the source model, wherein the fingerprint is generated by a process comprising: generating, based on a training dataset, a reference model and a surrogate model; selecting datapoints from the training dataset that are predicted correctly by the source model as a group of adversarial candidates; selecting from the group of adversarial candidates a sub-group of candidates that each have a low confidence score according to a threshold to yield a sub-group of adversarial candidates;
40
AMENDED SHEET (ARTICLE 19)
Application/ Control Number: PCT/US2021/054542 Docket No.: 213-0106-PCT adding noise to each candidate of the sub-group of adversarial candidates to yield a noisy group of adversarial examples; testing the noisy group of adversarial examples against the source model to obtain a set of source model successful adversarial examples that the source model predicts correctly; testing the set of source model successful adversarial examples against the reference model to yield a set of reference model outputs; testing the set of source model successful adversarial examples against the surrogate model to yield a set of surrogate model outputs; and identifying the fingerprint based on which ones from the set of source model successful adversarial examples, the set of reference model outputs and the set of surrogate model outputs pass as adversarial examples against the source model and the surrogate model, but not the reference model.
15. (Cancelled) The method of claim 14, wherein the fingerprint is generated by a process comprising: generating, based on a training dataset, a reference model and a surrogate model; selecting datapoints from the training dataset that are predicted correctly by the source model as a group of adversarial candidates; selecting from the group of adversarial candidates a sub-group of candidates that each have a low confidence score according to a threshold to yield a sub-group of adversarial candidates; adding noise to each candidate of the sub-group of adversarial candidates to yield a noisy group of adversarial examples; testing the noisy group of adversarial examples against the source model to obtain a set of source model successful adversarial examples that the source model predicts correctly; testing the set of source model successful adversarial examples against the reference model to yield a set of reference model outputs;
41
AMENDED SHEET (ARTICLE 19)
Application/ Control Number: PCT/US2021/054542 Docket No.: 213-0106-PCT testing the set of source model successful adversarial examples against the surrogate model to yield a set of surrogate model outputs; and identifying the fingerprint based on which ones from the set of source model successful adversarial examples, the set of reference model outputs and the set of surrogate model outputs pass as adversarial examples against the source model and the surrogate model, but not the reference model.
16. (Currently Amended) The method of claim 14, wherein the noisy group of adversarial examples comprises ones of the group of adversarial candidates that lead to a fully successful adversarial attack accuracy against the source model.
17. (Currently Amended) The method of claim 14, further comprising: sharing a hashed version of the fingerprint with a trusted third party.
18. (Currently Amended) The method of claim 14, wherein generating the fingerprint further comprises constructing respective adversarial examples with the noise that causes a receiving model to misclassify an input.
19. (Currently Amended) The method of claim 18, wherein the noise is imperceptible noise.
20. (Currently Amended) The method of claim 14, wherein the threshold is approximately 0.60.
AMENDED SHEET (ARTICLE 19)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA3195434A CA3195434A1 (en) | 2020-10-13 | 2021-10-12 | Systems and methods for providing a systemic error in artificial intelligence algorithms |
| EP21880894.7A EP4229554A4 (en) | 2020-10-13 | 2021-10-12 | Systems and methods for providing a systemic error in artificial intelligence algorithms |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063090933P | 2020-10-13 | 2020-10-13 | |
| US63/090,933 | 2020-10-13 | ||
| US17/499,553 US20220039127A1 (en) | 2019-04-15 | 2021-10-12 | Uplink Transmission Method and Communication Apparatus |
| US17/499,553 | 2021-10-12 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2022081553A1 WO2022081553A1 (en) | 2022-04-21 |
| WO2022081553A4 true WO2022081553A4 (en) | 2022-06-16 |
Family
ID=81208609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2021/054542 WO2022081553A1 (en) | 2020-10-13 | 2021-10-12 | Systems and methods for providing a systemic error in artificial intelligence algorithms |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4229554A4 (en) |
| CA (1) | CA3195434A1 (en) |
| WO (1) | WO2022081553A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277065B (en) * | 2022-06-15 | 2024-01-23 | 北京信息科技大学 | Anti-attack method and device in abnormal traffic detection of Internet of things |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11163860B2 (en) * | 2018-06-04 | 2021-11-02 | International Business Machines Corporation | Protecting deep learning models using watermarking |
| US11042611B2 (en) * | 2018-12-10 | 2021-06-22 | XNOR.ai, Inc. | Digital watermarking of machine-learning models |
| US11409845B2 (en) * | 2019-01-17 | 2022-08-09 | Nxp B.V. | Method for determining if a machine learning model has been copied |
| US11836256B2 (en) * | 2019-01-24 | 2023-12-05 | International Business Machines Corporation | Testing adversarial robustness of systems with limited access |
| US20220108185A1 (en) * | 2019-03-22 | 2022-04-07 | Siemens Aktiengesellschaft | Inverse and forward modeling machine learning-based generative design |
-
2021
- 2021-10-12 CA CA3195434A patent/CA3195434A1/en active Pending
- 2021-10-12 EP EP21880894.7A patent/EP4229554A4/en active Pending
- 2021-10-12 WO PCT/US2021/054542 patent/WO2022081553A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| EP4229554A1 (en) | 2023-08-23 |
| EP4229554A4 (en) | 2024-04-03 |
| CA3195434A1 (en) | 2022-04-21 |
| WO2022081553A1 (en) | 2022-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11526745B2 (en) | Methods and apparatus for federated training of a neural network using trusted edge devices | |
| US11599750B2 (en) | Edge devices utilizing personalized machine learning and methods of operating the same | |
| CN113066484A (en) | System and method for distributed training of neural network models | |
| CN111160749B (en) | Information quality assessment and information fusion method and device | |
| CN106295338B (en) | SQL vulnerability detection method based on artificial neuron network | |
| CN111046425A (en) | Method and device for risk identification by combining multiple parties | |
| CN110728328B (en) | Training method and device for classification model | |
| WO2021120854A1 (en) | Model training method, and method and system for training member detection device | |
| CN111061740B (en) | Data synchronization method, device and storage medium | |
| US10916254B2 (en) | Systems, apparatuses, and methods for speaker verification using artificial neural networks | |
| Razmjooei et al. | A new approach to design a finite‐time extended state observer: uncertain robotic manipulators application | |
| WO2022081553A4 (en) | Systems and methods for providing a systemic error in artificial intelligence algorithms | |
| Alvim et al. | When not all bits are equal: Worth-based information flow | |
| CN114581966A (en) | Method, electronic device and computer program product for information processing | |
| US7756802B2 (en) | Combiner training and evaluation with random data partition | |
| US11528259B2 (en) | Systems and methods for providing a systemic error in artificial intelligence algorithms | |
| US11967314B2 (en) | Automatic generation of a contextual meeting summary | |
| US10418024B1 (en) | Systems and methods of speech generation for target user given limited data | |
| KR102066264B1 (en) | Speech recognition method and system using deep neural network | |
| CN111308423B (en) | Robust sound source positioning system and method thereof | |
| Zhou et al. | Simulation credibility evaluation based on multi-source data fusion | |
| CN100574218C (en) | A kind of method of setting up artificial distinct network | |
| US11973743B2 (en) | Systems and methods for providing a systemic error in artificial intelligence algorithms | |
| Atashbar et al. | Coherent l1‐SVD method for DOA estimation of wideband signals | |
| Singh et al. | A new trust model based on time series prediction and Markov model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21880894 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 3195434 Country of ref document: CA |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2021880894 Country of ref document: EP Effective date: 20230515 |