JP7507851B2 - 機械学習システムにおける敵対的影響の軽減 - Google Patents

機械学習システムにおける敵対的影響の軽減 Download PDF

Info

Publication number
JP7507851B2
JP7507851B2 JP2022518169A JP2022518169A JP7507851B2 JP 7507851 B2 JP7507851 B2 JP 7507851B2 JP 2022518169 A JP2022518169 A JP 2022518169A JP 2022518169 A JP2022518169 A JP 2022518169A JP 7507851 B2 JP7507851 B2 JP 7507851B2
Authority
JP
Japan
Prior art keywords
model
sanitized
training data
trained
models
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022518169A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022548770A (ja
JP2022548770A5 (https=
Inventor
チェン、ピン－ユー
ダス、パイエル
ラマムシィ、カーシケヤン ネイトサン
ジャオ、プー
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2022548770A publication Critical patent/JP2022548770A/ja
Publication of JP2022548770A5 publication Critical patent/JP2022548770A5/ja
Application granted granted Critical
Publication of JP7507851B2 publication Critical patent/JP7507851B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Cultivation Of Plants (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
JP2022518169A 2019-09-24 2020-09-21 機械学習システムにおける敵対的影響の軽減 Active JP7507851B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201962904869P 2019-09-24 2019-09-24
US62/904,869 2019-09-24
US16/702,817 2019-12-04
US16/702,817 US11568282B2 (en) 2019-09-24 2019-12-04 Mitigating adversarial effects in machine learning systems
PCT/IB2020/058763 WO2021059106A1 (en) 2019-09-24 2020-09-21 Mitigating adversarial effects in machine learning systems

Publications (3)

Publication Number Publication Date
JP2022548770A JP2022548770A (ja) 2022-11-21
JP2022548770A5 JP2022548770A5 (https=) 2022-12-16
JP7507851B2 true JP7507851B2 (ja) 2024-06-28

Family

ID=74881033

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022518169A Active JP7507851B2 (ja) 2019-09-24 2020-09-21 機械学習システムにおける敵対的影響の軽減

Country Status (6)

Country Link
US (1) US11568282B2 (https=)
JP (1) JP7507851B2 (https=)
CN (1) CN114450695B (https=)
DE (1) DE112020003829T5 (https=)
GB (1) GB2603391B (https=)
WO (1) WO2021059106A1 (https=)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11609990B2 (en) * 2019-05-29 2023-03-21 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11514297B2 (en) * 2019-05-29 2022-11-29 Anomalee Inc. Post-training detection and identification of human-imperceptible backdoor-poisoning attacks
US11601463B2 (en) * 2020-07-28 2023-03-07 The Boeing Company Cybersecurity threat modeling and analysis with text miner and data flow diagram editor
US12001553B2 (en) * 2020-08-20 2024-06-04 Red Bend Ltd. Detecting vehicle malfunctions and cyber attacks using machine learning
US20250240658A1 (en) * 2021-10-20 2025-07-24 Nokia Technologies Oy Criteria-based measurement reporting
US12182258B2 (en) * 2021-11-08 2024-12-31 Microsoft Technology Licensing, Llc Adversarial training to minimize data poisoning attacks
EP4430513A1 (en) * 2021-11-08 2024-09-18 Microsoft Technology Licensing, LLC Adversarial training to minimize data poisoning attacks
US12326940B2 (en) * 2021-11-28 2025-06-10 International Business Machines Corporation Graph exploration framework for adversarial example generation
US20230205872A1 (en) * 2021-12-23 2023-06-29 Advanced Micro Devices, Inc. Method and apparatus to address row hammer attacks at a host processor
CN114283341B (zh) * 2022-03-04 2022-05-17 西南石油大学 一种高转移性对抗样本生成方法、系统及终端
CN114610885B (zh) * 2022-03-09 2022-11-08 江南大学 一种文本分类后门攻击方法、系统及设备
JP2024047424A (ja) * 2022-09-26 2024-04-05 株式会社Ｓｃｒｅｅｎホールディングス 学習装置、学習方法および学習プログラムに関する。
US12475235B2 (en) 2023-01-19 2025-11-18 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US12282565B2 (en) 2023-01-19 2025-04-22 Citibank, N.A. Generative cybersecurity exploit synthesis and mitigation
US12596813B2 (en) 2023-01-19 2026-04-07 Citibank, N.A Autonomous agent observation and control
US12314406B1 (en) 2023-01-19 2025-05-27 Citibank, N.A. Generative cybersecurity exploit discovery and evaluation
US11874934B1 (en) 2023-01-19 2024-01-16 Citibank, N.A. Providing user-induced variable identification of end-to-end computing system security impact information systems and methods
US12271491B2 (en) * 2023-01-19 2025-04-08 Citibank, N.A. Detection and mitigation of machine learning model adversarial attacks
US12608486B2 (en) 2023-01-19 2026-04-21 Citibank, N.A. Generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data systems and methods
US20240296219A1 (en) * 2023-03-05 2024-09-05 Microsoft Technology Licensing, Llc Adverse or malicious input mitigation for large language models
CN121532780A (zh) * 2023-07-05 2026-02-13 软银集团股份有限公司 信息提供装置、信息提供方法以及信息提供程序
US12489798B2 (en) 2023-08-31 2025-12-02 Dell Products L.P. Managing artificial intelligence models to identify goals of malicious attackers
US20250077656A1 (en) * 2023-08-31 2025-03-06 Dell Products L.P. Managing impact of poisoned inferences on inference consumers using digital twins
US12602624B2 (en) 2023-12-11 2026-04-14 Citibank, N.A. Anomaly detection method for model outputs
US12602418B2 (en) 2024-04-11 2026-04-14 Citibank, N.A. Intelligent query decomposition, specialized model routing, and hierarchical aggregation with conflict resolution
US12596738B2 (en) 2024-04-11 2026-04-07 Citibank, N.A. Explainable large language model routing with immutable audit trails
US12346820B1 (en) 2024-04-11 2025-07-01 Citibank, N. A. Identifying and remediating gaps in artificial intelligence use cases using a generative artificial intelligence model

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010067033A (ja) 2008-09-11 2010-03-25 Sony Corp データ処理装置、データ処理方法、及び、プログラム
US20190244103A1 (en) 2018-02-07 2019-08-08 Royal Bank Of Canada Robust pruned neural networks via adversarial training

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9558176B2 (en) 2013-12-06 2017-01-31 Microsoft Technology Licensing, Llc Discriminating between natural language and keyword language items
US9697469B2 (en) 2014-08-13 2017-07-04 Andrew McMahon Method and system for generating and aggregating models based on disparate data from insurance, financial services, and public industries
KR102494139B1 (ko) * 2015-11-06 2023-01-31 삼성전자주식회사 뉴럴 네트워크 학습 장치 및 방법과, 음성 인식 장치 및 방법
US11080616B2 (en) * 2016-09-27 2021-08-03 Clarifai, Inc. Artificial intelligence model and data collection/development platform
CN106934462A (zh) 2017-02-09 2017-07-07 华南理工大学 基于迁移的对抗性环境下的防御毒化攻击的学习方法
CN108320026B (zh) * 2017-05-16 2022-02-11 腾讯科技（深圳）有限公司 机器学习模型训练方法和装置
CN107316083B (zh) * 2017-07-04 2021-05-25 北京百度网讯科技有限公司 用于更新深度学习模型的方法和装置
US10540578B2 (en) 2017-12-21 2020-01-21 International Business Machines Corporation Adapting a generative adversarial network to new data sources for image classification
US11195120B2 (en) * 2018-02-09 2021-12-07 Cisco Technology, Inc. Detecting dataset poisoning attacks independent of a learning algorithm
US10643602B2 (en) * 2018-03-16 2020-05-05 Microsoft Technology Licensing, Llc Adversarial teacher-student learning for unsupervised domain adaptation
CN108932527A (zh) 2018-06-06 2018-12-04 上海交通大学 使用交叉训练模型检测对抗样本的方法
CN108712448A (zh) 2018-07-09 2018-10-26 四川大学 一种基于动态污点分析的注入式攻击检测模型
CN109101999B (zh) 2018-07-16 2021-06-25 华东师范大学 基于支持向量机的协神经网络可信决策方法
US11568211B2 (en) 2018-12-27 2023-01-31 Intel Corporation Defending neural networks by randomizing model weights
CN109886210B (zh) * 2019-02-25 2022-07-19 百度在线网络技术（北京）有限公司 一种交通图像识别方法、装置、计算机设备和介质
CN109948663B (zh) 2019-02-27 2022-03-15 天津大学 一种基于模型抽取的步长自适应的对抗攻击方法
US11657162B2 (en) 2019-03-22 2023-05-23 Intel Corporation Adversarial training of neural networks using information about activation path differentials
US20190272375A1 (en) 2019-03-28 2019-09-05 Intel Corporation Trust model for malware classification
CN110222762A (zh) 2019-06-04 2019-09-10 恒安嘉新（北京）科技股份公司 对象预测方法、装置、设备、及介质

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010067033A (ja) 2008-09-11 2010-03-25 Sony Corp データ処理装置、データ処理方法、及び、プログラム
US20190244103A1 (en) 2018-02-07 2019-08-08 Royal Bank Of Canada Robust pruned neural networks via adversarial training

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
古澤 一憲，"ディジタルエコノミー時代のサイバーセキュリティ －ディジタルトランスフォーメーション促進の基盤確立に向けて－ （６）ＡＩをセキュリティリスクから守るために －ＡＩへのサイバー攻撃とその対策－"，情報処理，2018年11月15日，第59巻第12号，pp.1102-1107

Also Published As

Publication number Publication date
GB202204966D0 (en) 2022-05-18
DE112020003829T5 (de) 2022-05-19
CN114450695B (zh) 2025-08-12
US20210089941A1 (en) 2021-03-25
CN114450695A (zh) 2022-05-06
GB2603391B (en) 2024-05-22
WO2021059106A1 (en) 2021-04-01
JP2022548770A (ja) 2022-11-21
GB2603391A (en) 2022-08-03
US11568282B2 (en) 2023-01-31

Similar Documents

Publication Publication Date Title
JP7507851B2 (ja) 機械学習システムにおける敵対的影響の軽減
AU2020368222B2 (en) Adding adversarial robustness to trained machine learning models
US11875253B2 (en) Low-resource entity resolution with transfer learning
US11601468B2 (en) Detection of an adversarial backdoor attack on a trained model at inference time
US11593642B2 (en) Combined data pre-process and architecture search for deep learning models
AU2020385264A1 (en) Fusing multimodal data using recurrent neural networks
JP7448562B2 (ja) 人工知能のための希な訓練データへの対処
US20190050465A1 (en) Methods and systems for feature engineering
US20220300822A1 (en) Forgetting data samples from pretrained neural network models
US11989626B2 (en) Generating performance predictions with uncertainty intervals
CN111612022A (zh) 用于分析数据的方法、设备和计算机存储介质
JP2025500854A (ja) グラフ・ニューラル・ネットワーク・アンサンブル学習
EP4476664A1 (en) Quantum computer performance enhancement
US11928849B2 (en) Action-object recognition in cluttered video scenes using text
US12141701B2 (en) Channel scaling: a scale-and-select approach for selective transfer learning
US12326940B2 (en) Graph exploration framework for adversarial example generation
US12596940B2 (en) Smart training and smart deployment of machine learning models
CN121753046A (zh) 使用集成来预测视频的下一帧
WO2022194086A1 (en) A neuro-symbolic approach for entity linking
JP7331938B2 (ja) 学習装置、推定装置、学習方法及び学習プログラム
CA3170297C (en) Generating performance predictions with uncertainty intervals

Legal Events

Date Code Title Description
RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20220518

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221208

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230224

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240131

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240205

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240416

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240528

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240618