CN116670693A - 机器学习模型中针对对抗样本的动态梯度欺骗 - Google Patents

机器学习模型中针对对抗样本的动态梯度欺骗 Download PDF

Info

Publication number
CN116670693A
CN116670693A CN202180082952.0A CN202180082952A CN116670693A CN 116670693 A CN116670693 A CN 116670693A CN 202180082952 A CN202180082952 A CN 202180082952A CN 116670693 A CN116670693 A CN 116670693A
Authority
CN
China
Prior art keywords
classification
disturbance
subset
machine learning
input data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202180082952.0A
Other languages
English (en)
Chinese (zh)
Inventor
李泰星
I·M·莫洛伊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CN116670693A publication Critical patent/CN116670693A/zh
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)
  • Image Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing Of Engines (AREA)
  • Feedback Control In General (AREA)
  • Testing And Monitoring For Control Systems (AREA)
  • Complex Calculations (AREA)
  • Bioethics (AREA)
CN202180082952.0A 2020-12-08 2021-11-22 机器学习模型中针对对抗样本的动态梯度欺骗 Pending CN116670693A (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US17/114,819 US12050993B2 (en) 2020-12-08 2020-12-08 Dynamic gradient deception against adversarial examples in machine learning models
US17/114,819 2020-12-08
PCT/IB2021/060808 WO2022123372A1 (en) 2020-12-08 2021-11-22 Dynamic gradient deception against adversarial examples in machine learning models

Publications (1)

Publication Number Publication Date
CN116670693A true CN116670693A (zh) 2023-08-29

Family

ID=81849070

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180082952.0A Pending CN116670693A (zh) 2020-12-08 2021-11-22 机器学习模型中针对对抗样本的动态梯度欺骗

Country Status (6)

Country Link
US (1) US12050993B2 (https=)
JP (1) JP7754599B2 (https=)
CN (1) CN116670693A (https=)
DE (1) DE112021005847T5 (https=)
GB (1) GB2617735A (https=)
WO (1) WO2022123372A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119202258A (zh) * 2024-11-25 2024-12-27 西安融军通用标准化研究院有限责任公司 一种基于机器学习的标准文本分类方法

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12493666B2 (en) * 2021-01-14 2025-12-09 Origin Research Wireless, Inc. Wireless sensing using classifier probing and refinement
US20220405531A1 (en) * 2021-06-15 2022-12-22 Etsy, Inc. Blackbox optimization via model ensembling
US20230071450A1 (en) * 2021-09-09 2023-03-09 Siemens Aktiengesellschaft System and method for controlling large scale power distribution systems using reinforcement learning
CN115278757B (zh) * 2022-07-25 2025-05-20 绿盟科技集团股份有限公司 一种检测异常数据的方法、装置及电子设备
CN114998707B (zh) * 2022-08-05 2022-11-04 深圳中集智能科技有限公司 评估目标检测模型鲁棒性的攻击方法和装置
US11947902B1 (en) * 2023-03-03 2024-04-02 Microsoft Technology Licensing, Llc Efficient multi-turn generative AI model suggested message generation
US11962546B1 (en) 2023-03-03 2024-04-16 Microsoft Technology Licensing, Llc Leveraging inferred context to improve suggested messages
US12282731B2 (en) 2023-03-03 2025-04-22 Microsoft Technology Licensing, Llc Guardrails for efficient processing and error prevention in generating suggested messages
US20240378726A1 (en) * 2023-05-12 2024-11-14 GE Precision Healthcare LLC Deep learning based medical imaging system and method
US12580929B2 (en) * 2023-07-25 2026-03-17 Crowdstrike, Inc. Techniques for assessing malware classification
CN116680727B (zh) * 2023-08-01 2023-11-03 北京航空航天大学 一种面向图像分类模型的功能窃取防御方法
US12587564B2 (en) * 2023-08-15 2026-03-24 Cisco Technology, Inc. Adversarial training of language models to prevent hijacking of conversational agents
US20250217255A1 (en) * 2024-01-03 2025-07-03 Samsung Electronics Co., Ltd. Method and apparatus with ai model performance measuring using perturbation
CN118747837B (zh) * 2024-08-12 2024-11-15 北京小蝇科技有限责任公司 基于机器学习的样本数据处理方法和装置
CN119150031B (zh) * 2024-11-13 2025-10-10 阿里云飞天(杭州)云计算技术有限公司 模型训练方法和数据处理方法

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5359699A (en) 1991-12-02 1994-10-25 General Electric Company Method for using a feed forward neural network to perform classification with highly biased data
US5371809A (en) 1992-03-30 1994-12-06 Desieno; Duane D. Neural network for improved classification of patterns which adds a best performing trial branch node to the network
US7409372B2 (en) 2003-06-20 2008-08-05 Hewlett-Packard Development Company, L.P. Neural network trained with spatial errors
US8275803B2 (en) 2008-05-14 2012-09-25 International Business Machines Corporation System and method for providing answers to questions
US8280838B2 (en) 2009-09-17 2012-10-02 International Business Machines Corporation Evidence evaluation system and method based on question answering
US20110125734A1 (en) 2009-11-23 2011-05-26 International Business Machines Corporation Questions and answers generation
WO2012047557A1 (en) 2010-09-28 2012-04-12 International Business Machines Corporation Evidence diffusion among candidate answers during question answering
US8738617B2 (en) 2010-09-28 2014-05-27 International Business Machines Corporation Providing answers to questions using multiple models to score candidate answers
US8601030B2 (en) 2011-09-09 2013-12-03 International Business Machines Corporation Method for a natural language question-answering system to complement decision-support in a real-time command center
US9390370B2 (en) 2012-08-28 2016-07-12 International Business Machines Corporation Training deep neural network acoustic models using distributed hessian-free optimization
US20150170027A1 (en) 2013-12-13 2015-06-18 Qualcomm Incorporated Neuronal diversity in spiking neural networks and pattern classification
US10621487B2 (en) 2014-09-17 2020-04-14 Hewlett Packard Enterprise Development Lp Neural network verification
US9786270B2 (en) 2015-07-09 2017-10-10 Google Inc. Generating acoustic models
US10044751B2 (en) 2015-12-28 2018-08-07 Arbor Networks, Inc. Using recurrent neural networks to defeat DNS denial of service attacks
CN105718945B (zh) 2016-01-20 2020-07-31 江苏大学 基于分水岭和神经网络的苹果采摘机器人夜间图像识别方法
US9948666B2 (en) 2016-02-09 2018-04-17 International Business Machines Corporation Forecasting and classifying cyber-attacks using analytical data based neural embeddings
CN106127729A (zh) 2016-06-08 2016-11-16 浙江传媒学院 一种基于梯度的图像噪声水平估计方法
CN106296692A (zh) 2016-08-11 2017-01-04 深圳市未来媒体技术研究院 基于对抗网络的图像显著性检测方法
US10915817B2 (en) 2017-01-23 2021-02-09 Fotonation Limited Method of training a neural network
CN106845471A (zh) 2017-02-20 2017-06-13 深圳市唯特视科技有限公司 一种基于生成对抗网络的视觉显著性预测方法
EP3602316A4 (en) 2017-03-24 2020-12-30 D5A1 Llc LEARNING COACH FOR AUTOMATIC LEARNING SYSTEM
CN107025284B (zh) 2017-04-06 2020-10-27 中南大学 网络评论文本情感倾向的识别方法及卷积神经网络模型
CN107147603B (zh) 2017-05-05 2019-10-08 西安电子科技大学 基于多神经网络的dbpsk解调方法
CN107240085A (zh) 2017-05-08 2017-10-10 广州智慧城市发展研究院 一种基于卷积神经网络模型的图像融合方法及系统
WO2018231708A2 (en) 2017-06-12 2018-12-20 D5Ai Llc Robust anti-adversarial machine learning
US11023593B2 (en) 2017-09-25 2021-06-01 International Business Machines Corporation Protecting cognitive systems from model stealing attacks
US10642846B2 (en) 2017-10-13 2020-05-05 Microsoft Technology Licensing, Llc Using a generative adversarial network for query-keyword matching
US10657259B2 (en) 2017-11-01 2020-05-19 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients
EP3770777A4 (en) 2018-03-20 2021-05-05 Sony Corporation INFORMATION PROCESSING DEVICE AND INFORMATION PROCESSING METHOD
US10733292B2 (en) 2018-07-10 2020-08-04 International Business Machines Corporation Defending against model inversion attacks on neural networks
US11227215B2 (en) 2019-03-08 2022-01-18 International Business Machines Corporation Quantifying vulnerabilities of deep learning computing systems to adversarial perturbations
US11017319B1 (en) * 2020-06-23 2021-05-25 Deeping Source Inc. Method for training obfuscation network which conceals original data to be used for machine learning and training surrogate network which uses obfuscated data generated by obfuscation network and method for testing trained obfuscation network and learning device and testing device using the same

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119202258A (zh) * 2024-11-25 2024-12-27 西安融军通用标准化研究院有限责任公司 一种基于机器学习的标准文本分类方法

Also Published As

Publication number Publication date
WO2022123372A1 (en) 2022-06-16
US12050993B2 (en) 2024-07-30
GB2617735A (en) 2023-10-18
DE112021005847T5 (de) 2023-08-24
JP7754599B2 (ja) 2025-10-15
GB202310212D0 (en) 2023-08-16
JP2023551976A (ja) 2023-12-13
US20220180242A1 (en) 2022-06-09

Similar Documents

Publication Publication Date Title
JP7754599B2 (ja) 機械学習モデルにおける敵対的サンプルに対する動的勾配策略
US11853436B2 (en) Protecting cognitive systems from model stealing attacks
US11783025B2 (en) Training diverse and robust ensembles of artificial intelligence computer models
US11681918B2 (en) Cohort based adversarial attack detection
CN111295674B (zh) 通过使用欺骗梯度来保护认知系统免受基于梯度的攻击
US11373093B2 (en) Detecting and purifying adversarial inputs in deep learning computing systems
CN111667049B (zh) 量化深度学习计算系统对对抗性扰动的脆弱性
JP7374545B2 (ja) 勾配を使用したニューラル・ネットワーク内のバックドアの検出
US11443178B2 (en) Deep neural network hardening framework
US11182562B2 (en) Deep embedding for natural language content based on semantic dependencies
US10606893B2 (en) Expanding knowledge graphs based on candidate missing edges to optimize hypothesis set adjudication
US12321492B2 (en) Cognitive iterative minimization of personally identifiable information in electronic documents
US10147051B2 (en) Candidate answer generation for explanatory questions directed to underlying reasoning regarding the existence of a fact
US10147047B2 (en) Augmenting answer keys with key characteristics for training question and answer systems
US10754969B2 (en) Method to allow for question and answer system to dynamically return different responses based on roles
US10382440B2 (en) Method to allow for question and answer system to dynamically return different responses based on roles
US11663518B2 (en) Cognitive system virtual corpus training and utilization
US11544460B1 (en) Adversarial anonymization and preservation of content
Kang et al. A hybrid style transfer with whale optimization algorithm model for textual adversarial attack

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination