GB2603391B - Mitigating adversarial effects in machine learning systems - Google Patents

Mitigating adversarial effects in machine learning systems Download PDF

Info

Publication number
GB2603391B
GB2603391B GB2204966.2A GB202204966A GB2603391B GB 2603391 B GB2603391 B GB 2603391B GB 202204966 A GB202204966 A GB 202204966A GB 2603391 B GB2603391 B GB 2603391B
Authority
GB
United Kingdom
Prior art keywords
machine learning
learning systems
mitigating
effects
adversarial effects
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
GB2204966.2A
Other languages
English (en)
Other versions
GB202204966D0 (en
GB2603391A (en
Inventor
CHEN Pin-Yu
Das Payel
Natesan Ramamurthy Karthikeyan
Zhao Pu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of GB202204966D0 publication Critical patent/GB202204966D0/en
Publication of GB2603391A publication Critical patent/GB2603391A/en
Application granted granted Critical
Publication of GB2603391B publication Critical patent/GB2603391B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Cultivation Of Plants (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
GB2204966.2A 2019-09-24 2020-09-21 Mitigating adversarial effects in machine learning systems Active GB2603391B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962904869P 2019-09-24 2019-09-24
US16/702,817 US11568282B2 (en) 2019-09-24 2019-12-04 Mitigating adversarial effects in machine learning systems
PCT/IB2020/058763 WO2021059106A1 (en) 2019-09-24 2020-09-21 Mitigating adversarial effects in machine learning systems

Publications (3)

Publication Number Publication Date
GB202204966D0 GB202204966D0 (en) 2022-05-18
GB2603391A GB2603391A (en) 2022-08-03
GB2603391B true GB2603391B (en) 2024-05-22

Family

ID=74881033

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2204966.2A Active GB2603391B (en) 2019-09-24 2020-09-21 Mitigating adversarial effects in machine learning systems

Country Status (6)

Country Link
US (1) US11568282B2 (https=)
JP (1) JP7507851B2 (https=)
CN (1) CN114450695B (https=)
DE (1) DE112020003829T5 (https=)
GB (1) GB2603391B (https=)
WO (1) WO2021059106A1 (https=)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11609990B2 (en) * 2019-05-29 2023-03-21 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11514297B2 (en) * 2019-05-29 2022-11-29 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11601463B2 (en) * 2020-07-28 2023-03-07 The Boeing Company Cybersecurity threat modeling and analysis with text miner and data flow diagram editor
US12001553B2 (en) * 2020-08-20 2024-06-04 Red Bend Ltd. Detecting vehicle malfunctions and cyber attacks using machine learning
US20250240658A1 (en) * 2021-10-20 2025-07-24 Nokia Technologies Oy Criteria-based measurement reporting
US12182258B2 (en) * 2021-11-08 2024-12-31 Microsoft Technology Licensing, Llc Adversarial training to minimize data poisoning attacks
EP4430513A1 (en) * 2021-11-08 2024-09-18 Microsoft Technology Licensing, LLC Adversarial training to minimize data poisoning attacks
US12326940B2 (en) * 2021-11-28 2025-06-10 International Business Machines Corporation Graph exploration framework for adversarial example generation
US20230205872A1 (en) * 2021-12-23 2023-06-29 Advanced Micro Devices, Inc. Method and apparatus to address row hammer attacks at a host processor
CN114283341B (zh) * 2022-03-04 2022-05-17 西南石油大学 一种高转移性对抗样本生成方法、系统及终端
CN114610885B (zh) * 2022-03-09 2022-11-08 江南大学 一种文本分类后门攻击方法、系统及设备
JP2024047424A (ja) * 2022-09-26 2024-04-05 株式会社Ｓｃｒｅｅｎホールディングス 学習装置、学習方法および学習プログラムに関する。
US12475235B2 (en) 2023-01-19 2025-11-18 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US12282565B2 (en) 2023-01-19 2025-04-22 Citibank, N.A. Generative cybersecurity exploit synthesis and mitigation
US12596813B2 (en) 2023-01-19 2026-04-07 Citibank, N.A Autonomous agent observation and control
US12314406B1 (en) 2023-01-19 2025-05-27 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US11874934B1 (en) 2023-01-19 2024-01-16 Citibank, N.A. Providing user-induced variable identification of end-to-end computing system security impact information systems and methods
US12271491B2 (en) * 2023-01-19 2025-04-08 Citibank, N.A. Detection and mitigation of machine learning model adversarial attacks
US12608486B2 (en) 2023-01-19 2026-04-21 Citibank, N.A. Generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data systems and methods
US20240296219A1 (en) * 2023-03-05 2024-09-05 Microsoft Technology Licensing, Llc Adverse or malicious input mitigation for large language models
CN121532780A (zh) * 2023-07-05 2026-02-13 软银集团股份有限公司 信息提供装置、信息提供方法以及信息提供程序
US12489798B2 (en) 2023-08-31 2025-12-02 Dell Products L.P. Managing artificial intelligence models to identify goals of malicious attackers
US20250077656A1 (en) * 2023-08-31 2025-03-06 Dell Products L.P. Managing impact of poisoned inferences on inference consumers using digital twins
US12602624B2 (en) 2023-12-11 2026-04-14 Citibank, N.A. Anomaly detection method for model outputs
US12602418B2 (en) 2024-04-11 2026-04-14 Citibank, N.A. Intelligent query decomposition, specialized model routing, and hierarchical aggregation with conflict resolution
US12596738B2 (en) 2024-04-11 2026-04-07 Citibank, N.A. Explainable large language model routing with immutable audit trails
US12346820B1 (en) 2024-04-11 2025-07-01 Citibank, N. A. Identifying and remediating gaps in artificial intelligence use cases using a generative artificial intelligence model

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160048766A1 (en) * 2014-08-13 2016-02-18 Vitae Analytics, Inc. Method and system for generating and aggregating models based on disparate data from insurance, financial services, and public industries
US20170116182A1 (en) * 2013-12-06 2017-04-27 Microsoft Technology Licensing, Llc Discriminating Between Natural Language and Keyword Language Items
US20190197368A1 (en) * 2017-12-21 2019-06-27 International Business Machines Corporation Adapting a Generative Adversarial Network to New Data Sources for Image Classification
US20190220605A1 (en) * 2019-03-22 2019-07-18 Intel Corporation Adversarial training of neural networks using information about activation path differentials
US20190272375A1 (en) * 2019-03-28 2019-09-05 Intel Corporation Trust model for malware classification
CN110222762A (zh) * 2019-06-04 2019-09-10 恒安嘉新（北京）科技股份公司 对象预测方法、装置、设备、及介质

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010067033A (ja) * 2008-09-11 2010-03-25 Sony Corp データ処理装置、データ処理方法、及び、プログラム
KR102494139B1 (ko) * 2015-11-06 2023-01-31 삼성전자주식회사 뉴럴 네트워크 학습 장치 및 방법과, 음성 인식 장치 및 방법
US11080616B2 (en) * 2016-09-27 2021-08-03 Clarifai, Inc. Artificial intelligence model and data collection/development platform
CN106934462A (zh) 2017-02-09 2017-07-07 华南理工大学 基于迁移的对抗性环境下的防御毒化攻击的学习方法
CN108320026B (zh) * 2017-05-16 2022-02-11 腾讯科技（深圳）有限公司 机器学习模型训练方法和装置
CN107316083B (zh) * 2017-07-04 2021-05-25 北京百度网讯科技有限公司 用于更新深度学习模型的方法和装置
CA3033014A1 (en) * 2018-02-07 2019-08-07 Royal Bank Of Canada Robust pruned neural networks via adversarial training
US11195120B2 (en) * 2018-02-09 2021-12-07 Cisco Technology, Inc. Detecting dataset poisoning attacks independent of a learning algorithm
US10643602B2 (en) * 2018-03-16 2020-05-05 Microsoft Technology Licensing, Llc Adversarial teacher-student learning for unsupervised domain adaptation
CN108932527A (zh) 2018-06-06 2018-12-04 上海交通大学 使用交叉训练模型检测对抗样本的方法
CN108712448A (zh) 2018-07-09 2018-10-26 四川大学 一种基于动态污点分析的注入式攻击检测模型
CN109101999B (zh) 2018-07-16 2021-06-25 华东师范大学 基于支持向量机的协神经网络可信决策方法
US11568211B2 (en) 2018-12-27 2023-01-31 Intel Corporation Defending neural networks by randomizing model weights
CN109886210B (zh) * 2019-02-25 2022-07-19 百度在线网络技术（北京）有限公司 一种交通图像识别方法、装置、计算机设备和介质
CN109948663B (zh) 2019-02-27 2022-03-15 天津大学 一种基于模型抽取的步长自适应的对抗攻击方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170116182A1 (en) * 2013-12-06 2017-04-27 Microsoft Technology Licensing, Llc Discriminating Between Natural Language and Keyword Language Items
US20160048766A1 (en) * 2014-08-13 2016-02-18 Vitae Analytics, Inc. Method and system for generating and aggregating models based on disparate data from insurance, financial services, and public industries
US20190197368A1 (en) * 2017-12-21 2019-06-27 International Business Machines Corporation Adapting a Generative Adversarial Network to New Data Sources for Image Classification
US20190220605A1 (en) * 2019-03-22 2019-07-18 Intel Corporation Adversarial training of neural networks using information about activation path differentials
US20190272375A1 (en) * 2019-03-28 2019-09-05 Intel Corporation Trust model for malware classification
CN110222762A (zh) * 2019-06-04 2019-09-10 恒安嘉新（北京）科技股份公司 对象预测方法、装置、设备、及介质

Also Published As

Publication number Publication date
GB202204966D0 (en) 2022-05-18
DE112020003829T5 (de) 2022-05-19
CN114450695B (zh) 2025-08-12
US20210089941A1 (en) 2021-03-25
CN114450695A (zh) 2022-05-06
WO2021059106A1 (en) 2021-04-01
JP2022548770A (ja) 2022-11-21
JP7507851B2 (ja) 2024-06-28
GB2603391A (en) 2022-08-03
US11568282B2 (en) 2023-01-31

Similar Documents

Publication Publication Date Title
GB2603391B (en) Mitigating adversarial effects in machine learning systems
GB2576454B (en) Systems and methods for objective-based scoring using machine learning techniques
EP3568282A4 (en) MACHINE FOR LARGE-SCALE GENERATIVE PRODUCTION
GB201908530D0 (en) Robutness against manipulations n machine learning
GB201711880D0 (en) Machine learning for attack mitigation in virtual machine
GB202203407D0 (en) Improved machine learning for technical systems
GB201810944D0 (en) Machine learning
EP3117338A4 (en) Alternative training distribution data in machine learning
GB201913601D0 (en) Privacy enhanced machine learning
SG11202003446YA (en) Speed simulation effect
GB201700915D0 (en) Improvements in damper assemblies
HUE055578T2 (hu) Temporo-mandibuláris protézis
EP3216141A4 (en) Static delay compensation in a telecommunications system
EP3276067A4 (en) Washing machine damping mechanism and washing machine
GB201917292D0 (en) Machine learning
GB2562122B (en) Training machine
GB2558910B (en) Workstation
SG11202000031RA (en) Virtual office
GB201801231D0 (en) Improvements in dampers
HUE064291T2 (hu) Marási eljárás
GB201812835D0 (en) Improvements in dampers
PL3395200T3 (pl) Aranżacja stacji roboczej
GB201911719D0 (en) Simulator
GB201917680D0 (en) Wafr v1
GB201818834D0 (en) Improvements in dampers