JP7754599B2 - 機械学習モデルにおける敵対的サンプルに対する動的勾配策略 - Google Patents

機械学習モデルにおける敵対的サンプルに対する動的勾配策略

Info

Publication number
JP7754599B2
JP7754599B2 JP2023534141A JP2023534141A JP7754599B2 JP 7754599 B2 JP7754599 B2 JP 7754599B2 JP 2023534141 A JP2023534141 A JP 2023534141A JP 2023534141 A JP2023534141 A JP 2023534141A JP 7754599 B2 JP7754599 B2 JP 7754599B2
Authority
JP
Japan
Prior art keywords
variation
classification
subset
classification values
input data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023534141A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023551976A (ja
JP2023551976A5 (https=
Inventor
リー、タエスン
モロイ、イアン、マイケル
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2023551976A publication Critical patent/JP2023551976A/ja
Publication of JP2023551976A5 publication Critical patent/JP2023551976A5/ja
Application granted granted Critical
Publication of JP7754599B2 publication Critical patent/JP7754599B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)
  • Image Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing Of Engines (AREA)
  • Feedback Control In General (AREA)
  • Testing And Monitoring For Control Systems (AREA)
  • Complex Calculations (AREA)
  • Bioethics (AREA)
JP2023534141A 2020-12-08 2021-11-22 機械学習モデルにおける敵対的サンプルに対する動的勾配策略 Active JP7754599B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US17/114,819 US12050993B2 (en) 2020-12-08 2020-12-08 Dynamic gradient deception against adversarial examples in machine learning models
US17/114,819 2020-12-08
PCT/IB2021/060808 WO2022123372A1 (en) 2020-12-08 2021-11-22 Dynamic gradient deception against adversarial examples in machine learning models

Publications (3)

Publication Number Publication Date
JP2023551976A JP2023551976A (ja) 2023-12-13
JP2023551976A5 JP2023551976A5 (https=) 2023-12-21
JP7754599B2 true JP7754599B2 (ja) 2025-10-15

Family

ID=81849070

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023534141A Active JP7754599B2 (ja) 2020-12-08 2021-11-22 機械学習モデルにおける敵対的サンプルに対する動的勾配策略

Country Status (6)

Country Link
US (1) US12050993B2 (https=)
JP (1) JP7754599B2 (https=)
CN (1) CN116670693A (https=)
DE (1) DE112021005847T5 (https=)
GB (1) GB2617735A (https=)
WO (1) WO2022123372A1 (https=)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12493666B2 (en) * 2021-01-14 2025-12-09 Origin Research Wireless, Inc. Wireless sensing using classifier probing and refinement
US20220405531A1 (en) * 2021-06-15 2022-12-22 Etsy, Inc. Blackbox optimization via model ensembling
US20230071450A1 (en) * 2021-09-09 2023-03-09 Siemens Aktiengesellschaft System and method for controlling large scale power distribution systems using reinforcement learning
CN115278757B (zh) * 2022-07-25 2025-05-20 绿盟科技集团股份有限公司 一种检测异常数据的方法、装置及电子设备
CN114998707B (zh) * 2022-08-05 2022-11-04 深圳中集智能科技有限公司 评估目标检测模型鲁棒性的攻击方法和装置
US11947902B1 (en) * 2023-03-03 2024-04-02 Microsoft Technology Licensing, Llc Efficient multi-turn generative AI model suggested message generation
US11962546B1 (en) 2023-03-03 2024-04-16 Microsoft Technology Licensing, Llc Leveraging inferred context to improve suggested messages
US12282731B2 (en) 2023-03-03 2025-04-22 Microsoft Technology Licensing, Llc Guardrails for efficient processing and error prevention in generating suggested messages
US20240378726A1 (en) * 2023-05-12 2024-11-14 GE Precision Healthcare LLC Deep learning based medical imaging system and method
US12580929B2 (en) * 2023-07-25 2026-03-17 Crowdstrike, Inc. Techniques for assessing malware classification
CN116680727B (zh) * 2023-08-01 2023-11-03 北京航空航天大学 一种面向图像分类模型的功能窃取防御方法
US12587564B2 (en) * 2023-08-15 2026-03-24 Cisco Technology, Inc. Adversarial training of language models to prevent hijacking of conversational agents
US20250217255A1 (en) * 2024-01-03 2025-07-03 Samsung Electronics Co., Ltd. Method and apparatus with ai model performance measuring using perturbation
CN118747837B (zh) * 2024-08-12 2024-11-15 北京小蝇科技有限责任公司 基于机器学习的样本数据处理方法和装置
CN119150031B (zh) * 2024-11-13 2025-10-10 阿里云飞天(杭州)云计算技术有限公司 模型训练方法和数据处理方法
CN119202258B (zh) * 2024-11-25 2025-02-28 西安融军通用标准化研究院有限责任公司 一种基于机器学习的标准文本分类方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190095629A1 (en) 2017-09-25 2019-03-28 International Business Machines Corporation Protecting Cognitive Systems from Model Stealing Attacks
WO2019181099A1 (ja) 2018-03-20 2019-09-26 ソニー株式会社 情報処理装置および情報処理方法

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5359699A (en) 1991-12-02 1994-10-25 General Electric Company Method for using a feed forward neural network to perform classification with highly biased data
US5371809A (en) 1992-03-30 1994-12-06 Desieno; Duane D. Neural network for improved classification of patterns which adds a best performing trial branch node to the network
US7409372B2 (en) 2003-06-20 2008-08-05 Hewlett-Packard Development Company, L.P. Neural network trained with spatial errors
US8275803B2 (en) 2008-05-14 2012-09-25 International Business Machines Corporation System and method for providing answers to questions
US8280838B2 (en) 2009-09-17 2012-10-02 International Business Machines Corporation Evidence evaluation system and method based on question answering
US20110125734A1 (en) 2009-11-23 2011-05-26 International Business Machines Corporation Questions and answers generation
WO2012047557A1 (en) 2010-09-28 2012-04-12 International Business Machines Corporation Evidence diffusion among candidate answers during question answering
US8738617B2 (en) 2010-09-28 2014-05-27 International Business Machines Corporation Providing answers to questions using multiple models to score candidate answers
US8601030B2 (en) 2011-09-09 2013-12-03 International Business Machines Corporation Method for a natural language question-answering system to complement decision-support in a real-time command center
US9390370B2 (en) 2012-08-28 2016-07-12 International Business Machines Corporation Training deep neural network acoustic models using distributed hessian-free optimization
US20150170027A1 (en) 2013-12-13 2015-06-18 Qualcomm Incorporated Neuronal diversity in spiking neural networks and pattern classification
US10621487B2 (en) 2014-09-17 2020-04-14 Hewlett Packard Enterprise Development Lp Neural network verification
US9786270B2 (en) 2015-07-09 2017-10-10 Google Inc. Generating acoustic models
US10044751B2 (en) 2015-12-28 2018-08-07 Arbor Networks, Inc. Using recurrent neural networks to defeat DNS denial of service attacks
CN105718945B (zh) 2016-01-20 2020-07-31 江苏大学 基于分水岭和神经网络的苹果采摘机器人夜间图像识别方法
US9948666B2 (en) 2016-02-09 2018-04-17 International Business Machines Corporation Forecasting and classifying cyber-attacks using analytical data based neural embeddings
CN106127729A (zh) 2016-06-08 2016-11-16 浙江传媒学院 一种基于梯度的图像噪声水平估计方法
CN106296692A (zh) 2016-08-11 2017-01-04 深圳市未来媒体技术研究院 基于对抗网络的图像显著性检测方法
US10915817B2 (en) 2017-01-23 2021-02-09 Fotonation Limited Method of training a neural network
CN106845471A (zh) 2017-02-20 2017-06-13 深圳市唯特视科技有限公司 一种基于生成对抗网络的视觉显著性预测方法
EP3602316A4 (en) 2017-03-24 2020-12-30 D5A1 Llc LEARNING COACH FOR AUTOMATIC LEARNING SYSTEM
CN107025284B (zh) 2017-04-06 2020-10-27 中南大学 网络评论文本情感倾向的识别方法及卷积神经网络模型
CN107147603B (zh) 2017-05-05 2019-10-08 西安电子科技大学 基于多神经网络的dbpsk解调方法
CN107240085A (zh) 2017-05-08 2017-10-10 广州智慧城市发展研究院 一种基于卷积神经网络模型的图像融合方法及系统
WO2018231708A2 (en) 2017-06-12 2018-12-20 D5Ai Llc Robust anti-adversarial machine learning
US10642846B2 (en) 2017-10-13 2020-05-05 Microsoft Technology Licensing, Llc Using a generative adversarial network for query-keyword matching
US10657259B2 (en) 2017-11-01 2020-05-19 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients
US10733292B2 (en) 2018-07-10 2020-08-04 International Business Machines Corporation Defending against model inversion attacks on neural networks
US11227215B2 (en) 2019-03-08 2022-01-18 International Business Machines Corporation Quantifying vulnerabilities of deep learning computing systems to adversarial perturbations
US11017319B1 (en) * 2020-06-23 2021-05-25 Deeping Source Inc. Method for training obfuscation network which conceals original data to be used for machine learning and training surrogate network which uses obfuscated data generated by obfuscation network and method for testing trained obfuscation network and learning device and testing device using the same

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190095629A1 (en) 2017-09-25 2019-03-28 International Business Machines Corporation Protecting Cognitive Systems from Model Stealing Attacks
WO2019181099A1 (ja) 2018-03-20 2019-09-26 ソニー株式会社 情報処理装置および情報処理方法

Also Published As

Publication number Publication date
WO2022123372A1 (en) 2022-06-16
US12050993B2 (en) 2024-07-30
CN116670693A (zh) 2023-08-29
GB2617735A (en) 2023-10-18
DE112021005847T5 (de) 2023-08-24
GB202310212D0 (en) 2023-08-16
JP2023551976A (ja) 2023-12-13
US20220180242A1 (en) 2022-06-09

Similar Documents

Publication Publication Date Title
JP7754599B2 (ja) 機械学習モデルにおける敵対的サンプルに対する動的勾配策略
US11853436B2 (en) Protecting cognitive systems from model stealing attacks
US11783025B2 (en) Training diverse and robust ensembles of artificial intelligence computer models
US11681918B2 (en) Cohort based adversarial attack detection
US11373093B2 (en) Detecting and purifying adversarial inputs in deep learning computing systems
CN111295674B (zh) 通过使用欺骗梯度来保护认知系统免受基于梯度的攻击
JP7374545B2 (ja) 勾配を使用したニューラル・ネットワーク内のバックドアの検出
CN111667049B (zh) 量化深度学习计算系统对对抗性扰动的脆弱性
US12321492B2 (en) Cognitive iterative minimization of personally identifiable information in electronic documents
US11443178B2 (en) Deep neural network hardening framework
US10606893B2 (en) Expanding knowledge graphs based on candidate missing edges to optimize hypothesis set adjudication
US10586156B2 (en) Knowledge canvassing using a knowledge graph and a question and answer system
US10147051B2 (en) Candidate answer generation for explanatory questions directed to underlying reasoning regarding the existence of a fact
US9720981B1 (en) Multiple instance machine learning for question answering systems
US10147047B2 (en) Augmenting answer keys with key characteristics for training question and answer systems
US11663518B2 (en) Cognitive system virtual corpus training and utilization
US12014281B2 (en) Automatic processing of electronic files to identify genetic variants

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231212

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240516

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250311

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250318

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250606

RD12 Notification of acceptance of power of sub attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7432

Effective date: 20250606

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20250606

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250917

RD14 Notification of resignation of power of sub attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7434

Effective date: 20250919

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250930

R150 Certificate of patent or registration of utility model

Ref document number: 7754599

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150