CN114450695B - 减轻机器学习系统中的对抗效应 - Google Patents

减轻机器学习系统中的对抗效应

Info

Publication number
CN114450695B
CN114450695B CN202080067549.6A CN202080067549A CN114450695B CN 114450695 B CN114450695 B CN 114450695B CN 202080067549 A CN202080067549 A CN 202080067549A CN 114450695 B CN114450695 B CN 114450695B
Authority
CN
China
Prior art keywords
machine learning
learning model
model
cleansed
trained
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080067549.6A
Other languages
English (en)
Chinese (zh)
Other versions
CN114450695A (zh
Inventor
陈品谕
P·达斯
K·纳特桑拉马穆尔蒂
赵普
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CN114450695A publication Critical patent/CN114450695A/zh
Application granted granted Critical
Publication of CN114450695B publication Critical patent/CN114450695B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Cultivation Of Plants (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
CN202080067549.6A 2019-09-24 2020-09-21 减轻机器学习系统中的对抗效应 Active CN114450695B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201962904869P 2019-09-24 2019-09-24
US62/904,869 2019-09-24
US16/702,817 2019-12-04
US16/702,817 US11568282B2 (en) 2019-09-24 2019-12-04 Mitigating adversarial effects in machine learning systems
PCT/IB2020/058763 WO2021059106A1 (en) 2019-09-24 2020-09-21 Mitigating adversarial effects in machine learning systems

Publications (2)

Publication Number Publication Date
CN114450695A CN114450695A (zh) 2022-05-06
CN114450695B true CN114450695B (zh) 2025-08-12

Family

ID=74881033

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080067549.6A Active CN114450695B (zh) 2019-09-24 2020-09-21 减轻机器学习系统中的对抗效应

Country Status (6)

Country Link
US (1) US11568282B2 (https=)
JP (1) JP7507851B2 (https=)
CN (1) CN114450695B (https=)
DE (1) DE112020003829T5 (https=)
GB (1) GB2603391B (https=)
WO (1) WO2021059106A1 (https=)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11609990B2 (en) * 2019-05-29 2023-03-21 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11514297B2 (en) * 2019-05-29 2022-11-29 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11601463B2 (en) * 2020-07-28 2023-03-07 The Boeing Company Cybersecurity threat modeling and analysis with text miner and data flow diagram editor
US12001553B2 (en) * 2020-08-20 2024-06-04 Red Bend Ltd. Detecting vehicle malfunctions and cyber attacks using machine learning
US20250240658A1 (en) * 2021-10-20 2025-07-24 Nokia Technologies Oy Criteria-based measurement reporting
US12182258B2 (en) * 2021-11-08 2024-12-31 Microsoft Technology Licensing, Llc Adversarial training to minimize data poisoning attacks
EP4430513A1 (en) * 2021-11-08 2024-09-18 Microsoft Technology Licensing, LLC Adversarial training to minimize data poisoning attacks
US12326940B2 (en) * 2021-11-28 2025-06-10 International Business Machines Corporation Graph exploration framework for adversarial example generation
US20230205872A1 (en) * 2021-12-23 2023-06-29 Advanced Micro Devices, Inc. Method and apparatus to address row hammer attacks at a host processor
CN114283341B (zh) * 2022-03-04 2022-05-17 西南石油大学 一种高转移性对抗样本生成方法、系统及终端
CN114610885B (zh) * 2022-03-09 2022-11-08 江南大学 一种文本分类后门攻击方法、系统及设备
JP2024047424A (ja) * 2022-09-26 2024-04-05 株式会社Ｓｃｒｅｅｎホールディングス 学習装置、学習方法および学習プログラムに関する。
US12475235B2 (en) 2023-01-19 2025-11-18 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US12282565B2 (en) 2023-01-19 2025-04-22 Citibank, N.A. Generative cybersecurity exploit synthesis and mitigation
US12596813B2 (en) 2023-01-19 2026-04-07 Citibank, N.A Autonomous agent observation and control
US12314406B1 (en) 2023-01-19 2025-05-27 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US11874934B1 (en) 2023-01-19 2024-01-16 Citibank, N.A. Providing user-induced variable identification of end-to-end computing system security impact information systems and methods
US12271491B2 (en) * 2023-01-19 2025-04-08 Citibank, N.A. Detection and mitigation of machine learning model adversarial attacks
US12608486B2 (en) 2023-01-19 2026-04-21 Citibank, N.A. Generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data systems and methods
US20240296219A1 (en) * 2023-03-05 2024-09-05 Microsoft Technology Licensing, Llc Adverse or malicious input mitigation for large language models
CN121532780A (zh) * 2023-07-05 2026-02-13 软银集团股份有限公司 信息提供装置、信息提供方法以及信息提供程序
US12489798B2 (en) 2023-08-31 2025-12-02 Dell Products L.P. Managing artificial intelligence models to identify goals of malicious attackers
US20250077656A1 (en) * 2023-08-31 2025-03-06 Dell Products L.P. Managing impact of poisoned inferences on inference consumers using digital twins
US12602624B2 (en) 2023-12-11 2026-04-14 Citibank, N.A. Anomaly detection method for model outputs
US12602418B2 (en) 2024-04-11 2026-04-14 Citibank, N.A. Intelligent query decomposition, specialized model routing, and hierarchical aggregation with conflict resolution
US12596738B2 (en) 2024-04-11 2026-04-07 Citibank, N.A. Explainable large language model routing with immutable audit trails
US12346820B1 (en) 2024-04-11 2025-07-01 Citibank, N. A. Identifying and remediating gaps in artificial intelligence use cases using a generative artificial intelligence model

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106683663A (zh) * 2015-11-06 2017-05-17 三星电子株式会社 神经网络训练设备和方法以及语音识别设备和方法
CN108320026A (zh) * 2017-05-16 2018-07-24 腾讯科技（深圳）有限公司 机器学习模型训练方法和装置

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010067033A (ja) * 2008-09-11 2010-03-25 Sony Corp データ処理装置、データ処理方法、及び、プログラム
US9558176B2 (en) 2013-12-06 2017-01-31 Microsoft Technology Licensing, Llc Discriminating between natural language and keyword language items
US9697469B2 (en) 2014-08-13 2017-07-04 Andrew McMahon Method and system for generating and aggregating models based on disparate data from insurance, financial services, and public industries
US11080616B2 (en) * 2016-09-27 2021-08-03 Clarifai, Inc. Artificial intelligence model and data collection/development platform
CN106934462A (zh) 2017-02-09 2017-07-07 华南理工大学 基于迁移的对抗性环境下的防御毒化攻击的学习方法
CN107316083B (zh) * 2017-07-04 2021-05-25 北京百度网讯科技有限公司 用于更新深度学习模型的方法和装置
US10540578B2 (en) 2017-12-21 2020-01-21 International Business Machines Corporation Adapting a generative adversarial network to new data sources for image classification
CA3033014A1 (en) * 2018-02-07 2019-08-07 Royal Bank Of Canada Robust pruned neural networks via adversarial training
US11195120B2 (en) * 2018-02-09 2021-12-07 Cisco Technology, Inc. Detecting dataset poisoning attacks independent of a learning algorithm
US10643602B2 (en) * 2018-03-16 2020-05-05 Microsoft Technology Licensing, Llc Adversarial teacher-student learning for unsupervised domain adaptation
CN108932527A (zh) 2018-06-06 2018-12-04 上海交通大学 使用交叉训练模型检测对抗样本的方法
CN108712448A (zh) 2018-07-09 2018-10-26 四川大学 一种基于动态污点分析的注入式攻击检测模型
CN109101999B (zh) 2018-07-16 2021-06-25 华东师范大学 基于支持向量机的协神经网络可信决策方法
US11568211B2 (en) 2018-12-27 2023-01-31 Intel Corporation Defending neural networks by randomizing model weights
CN109886210B (zh) * 2019-02-25 2022-07-19 百度在线网络技术（北京）有限公司 一种交通图像识别方法、装置、计算机设备和介质
CN109948663B (zh) 2019-02-27 2022-03-15 天津大学 一种基于模型抽取的步长自适应的对抗攻击方法
US11657162B2 (en) 2019-03-22 2023-05-23 Intel Corporation Adversarial training of neural networks using information about activation path differentials
US20190272375A1 (en) 2019-03-28 2019-09-05 Intel Corporation Trust model for malware classification
CN110222762A (zh) 2019-06-04 2019-09-10 恒安嘉新（北京）科技股份公司 对象预测方法、装置、设备、及介质

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106683663A (zh) * 2015-11-06 2017-05-17 三星电子株式会社 神经网络训练设备和方法以及语音识别设备和方法
CN108320026A (zh) * 2017-05-16 2018-07-24 腾讯科技（深圳）有限公司 机器学习模型训练方法和装置

Also Published As

Publication number Publication date
GB202204966D0 (en) 2022-05-18
DE112020003829T5 (de) 2022-05-19
US20210089941A1 (en) 2021-03-25
CN114450695A (zh) 2022-05-06
GB2603391B (en) 2024-05-22
WO2021059106A1 (en) 2021-04-01
JP2022548770A (ja) 2022-11-21
JP7507851B2 (ja) 2024-06-28
GB2603391A (en) 2022-08-03
US11568282B2 (en) 2023-01-31

Similar Documents

Publication Publication Date Title
CN114450695B (zh) 减轻机器学习系统中的对抗效应
AU2020368222B2 (en) Adding adversarial robustness to trained machine learning models
US11875253B2 (en) Low-resource entity resolution with transfer learning
US20200412743A1 (en) Detection of an adversarial backdoor attack on a trained model at inference time
US12141704B2 (en) Neural flow attestation
US11176508B2 (en) Minimizing compliance risk using machine learning techniques
US11436129B2 (en) System, method and recording medium for generating mobile test sequences
WO2021047866A1 (en) Learning input preprocessing to harden machine learning models
US11250602B2 (en) Generating concept images of human poses using machine learning models
US12572657B2 (en) Generating high-quality threat intelligence from aggregated threat reports
David et al. Modelling social-technical attacks with timed automata
US20210243217A1 (en) Preventing vulnerable configurations in sensor-based devices
US12229509B2 (en) Contextual impact adjustment for machine learning models
US20200364104A1 (en) Identifying a problem based on log data analysis
US12326940B2 (en) Graph exploration framework for adversarial example generation
US11704542B2 (en) Convolutional dynamic Boltzmann Machine for temporal event sequence
WO2022194086A1 (en) A neuro-symbolic approach for entity linking
US11785038B2 (en) Transfer learning platform for improved mobile enterprise security
Madani et al. Study on the different types of neural networks to improve the classification of ransomwares
Sikos Defensive Generative AI
KR102955098B1 (ko) 신경 흐름 증명
Modi Towards Uncertainty Quantification of Common Weakness Enumerations
CN117152824A (zh) 跨场景人脸检测方法及装置

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant