GB2617735A - Dynamic gradient deception against adversarial examples in machine learning models - Google Patents

Dynamic gradient deception against adversarial examples in machine learning models Download PDF

Info

Publication number
GB2617735A
GB2617735A GB2310212.2A GB202310212A GB2617735A GB 2617735 A GB2617735 A GB 2617735A GB 202310212 A GB202310212 A GB 202310212A GB 2617735 A GB2617735 A GB 2617735A
Authority
GB
United Kingdom
Prior art keywords
classification values
subset
machine learning
perturbation
learning model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
GB2310212.2A
Other languages
English (en)
Other versions
GB202310212D0 (en
Inventor
Lee Taesung
Michael Molloy Ian
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of GB202310212D0 publication Critical patent/GB202310212D0/en
Publication of GB2617735A publication Critical patent/GB2617735A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)
  • Image Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Testing Of Engines (AREA)
  • Feedback Control In General (AREA)
  • Testing And Monitoring For Control Systems (AREA)
  • Complex Calculations (AREA)
  • Bioethics (AREA)
GB2310212.2A 2020-12-08 2021-11-22 Dynamic gradient deception against adversarial examples in machine learning models Pending GB2617735A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US17/114,819 US12050993B2 (en) 2020-12-08 2020-12-08 Dynamic gradient deception against adversarial examples in machine learning models
PCT/IB2021/060808 WO2022123372A1 (en) 2020-12-08 2021-11-22 Dynamic gradient deception against adversarial examples in machine learning models

Publications (2)

Publication Number Publication Date
GB202310212D0 GB202310212D0 (en) 2023-08-16
GB2617735A true GB2617735A (en) 2023-10-18

Family

ID=81849070

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2310212.2A Pending GB2617735A (en) 2020-12-08 2021-11-22 Dynamic gradient deception against adversarial examples in machine learning models

Country Status (6)

Country Link
US (1) US12050993B2 (https=)
JP (1) JP7754599B2 (https=)
CN (1) CN116670693A (https=)
DE (1) DE112021005847T5 (https=)
GB (1) GB2617735A (https=)
WO (1) WO2022123372A1 (https=)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12493666B2 (en) * 2021-01-14 2025-12-09 Origin Research Wireless, Inc. Wireless sensing using classifier probing and refinement
US20220405531A1 (en) * 2021-06-15 2022-12-22 Etsy, Inc. Blackbox optimization via model ensembling
US20230071450A1 (en) * 2021-09-09 2023-03-09 Siemens Aktiengesellschaft System and method for controlling large scale power distribution systems using reinforcement learning
CN115278757B (zh) * 2022-07-25 2025-05-20 绿盟科技集团股份有限公司 一种检测异常数据的方法、装置及电子设备
CN114998707B (zh) * 2022-08-05 2022-11-04 深圳中集智能科技有限公司 评估目标检测模型鲁棒性的攻击方法和装置
US11947902B1 (en) * 2023-03-03 2024-04-02 Microsoft Technology Licensing, Llc Efficient multi-turn generative AI model suggested message generation
US11962546B1 (en) 2023-03-03 2024-04-16 Microsoft Technology Licensing, Llc Leveraging inferred context to improve suggested messages
US12282731B2 (en) 2023-03-03 2025-04-22 Microsoft Technology Licensing, Llc Guardrails for efficient processing and error prevention in generating suggested messages
US20240378726A1 (en) * 2023-05-12 2024-11-14 GE Precision Healthcare LLC Deep learning based medical imaging system and method
US12580929B2 (en) * 2023-07-25 2026-03-17 Crowdstrike, Inc. Techniques for assessing malware classification
CN116680727B (zh) * 2023-08-01 2023-11-03 北京航空航天大学 一种面向图像分类模型的功能窃取防御方法
US12587564B2 (en) * 2023-08-15 2026-03-24 Cisco Technology, Inc. Adversarial training of language models to prevent hijacking of conversational agents
US20250217255A1 (en) * 2024-01-03 2025-07-03 Samsung Electronics Co., Ltd. Method and apparatus with ai model performance measuring using perturbation
CN118747837B (zh) * 2024-08-12 2024-11-15 北京小蝇科技有限责任公司 基于机器学习的样本数据处理方法和装置
CN119150031B (zh) * 2024-11-13 2025-10-10 阿里云飞天(杭州)云计算技术有限公司 模型训练方法和数据处理方法
CN119202258B (zh) * 2024-11-25 2025-02-28 西安融军通用标准化研究院有限责任公司 一种基于机器学习的标准文本分类方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7409372B2 (en) * 2003-06-20 2008-08-05 Hewlett-Packard Development Company, L.P. Neural network trained with spatial errors
US20190095629A1 (en) * 2017-09-25 2019-03-28 International Business Machines Corporation Protecting Cognitive Systems from Model Stealing Attacks
CN111295674A (zh) * 2017-11-01 2020-06-16 国际商业机器公司 通过使用欺骗梯度来保护认知系统免受基于梯度的攻击
CN111667049A (zh) * 2019-03-08 2020-09-15 国际商业机器公司 量化深度学习计算系统对对抗性扰动的脆弱性

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5359699A (en) 1991-12-02 1994-10-25 General Electric Company Method for using a feed forward neural network to perform classification with highly biased data
US5371809A (en) 1992-03-30 1994-12-06 Desieno; Duane D. Neural network for improved classification of patterns which adds a best performing trial branch node to the network
US8275803B2 (en) 2008-05-14 2012-09-25 International Business Machines Corporation System and method for providing answers to questions
US8280838B2 (en) 2009-09-17 2012-10-02 International Business Machines Corporation Evidence evaluation system and method based on question answering
US20110125734A1 (en) 2009-11-23 2011-05-26 International Business Machines Corporation Questions and answers generation
WO2012047557A1 (en) 2010-09-28 2012-04-12 International Business Machines Corporation Evidence diffusion among candidate answers during question answering
US8738617B2 (en) 2010-09-28 2014-05-27 International Business Machines Corporation Providing answers to questions using multiple models to score candidate answers
US8601030B2 (en) 2011-09-09 2013-12-03 International Business Machines Corporation Method for a natural language question-answering system to complement decision-support in a real-time command center
US9390370B2 (en) 2012-08-28 2016-07-12 International Business Machines Corporation Training deep neural network acoustic models using distributed hessian-free optimization
US20150170027A1 (en) 2013-12-13 2015-06-18 Qualcomm Incorporated Neuronal diversity in spiking neural networks and pattern classification
US10621487B2 (en) 2014-09-17 2020-04-14 Hewlett Packard Enterprise Development Lp Neural network verification
US9786270B2 (en) 2015-07-09 2017-10-10 Google Inc. Generating acoustic models
US10044751B2 (en) 2015-12-28 2018-08-07 Arbor Networks, Inc. Using recurrent neural networks to defeat DNS denial of service attacks
CN105718945B (zh) 2016-01-20 2020-07-31 江苏大学 基于分水岭和神经网络的苹果采摘机器人夜间图像识别方法
US9948666B2 (en) 2016-02-09 2018-04-17 International Business Machines Corporation Forecasting and classifying cyber-attacks using analytical data based neural embeddings
CN106127729A (zh) 2016-06-08 2016-11-16 浙江传媒学院 一种基于梯度的图像噪声水平估计方法
CN106296692A (zh) 2016-08-11 2017-01-04 深圳市未来媒体技术研究院 基于对抗网络的图像显著性检测方法
US10915817B2 (en) 2017-01-23 2021-02-09 Fotonation Limited Method of training a neural network
CN106845471A (zh) 2017-02-20 2017-06-13 深圳市唯特视科技有限公司 一种基于生成对抗网络的视觉显著性预测方法
EP3602316A4 (en) 2017-03-24 2020-12-30 D5A1 Llc LEARNING COACH FOR AUTOMATIC LEARNING SYSTEM
CN107025284B (zh) 2017-04-06 2020-10-27 中南大学 网络评论文本情感倾向的识别方法及卷积神经网络模型
CN107147603B (zh) 2017-05-05 2019-10-08 西安电子科技大学 基于多神经网络的dbpsk解调方法
CN107240085A (zh) 2017-05-08 2017-10-10 广州智慧城市发展研究院 一种基于卷积神经网络模型的图像融合方法及系统
WO2018231708A2 (en) 2017-06-12 2018-12-20 D5Ai Llc Robust anti-adversarial machine learning
US10642846B2 (en) 2017-10-13 2020-05-05 Microsoft Technology Licensing, Llc Using a generative adversarial network for query-keyword matching
EP3770777A4 (en) 2018-03-20 2021-05-05 Sony Corporation INFORMATION PROCESSING DEVICE AND INFORMATION PROCESSING METHOD
US10733292B2 (en) 2018-07-10 2020-08-04 International Business Machines Corporation Defending against model inversion attacks on neural networks
US11017319B1 (en) * 2020-06-23 2021-05-25 Deeping Source Inc. Method for training obfuscation network which conceals original data to be used for machine learning and training surrogate network which uses obfuscated data generated by obfuscation network and method for testing trained obfuscation network and learning device and testing device using the same

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7409372B2 (en) * 2003-06-20 2008-08-05 Hewlett-Packard Development Company, L.P. Neural network trained with spatial errors
US20190095629A1 (en) * 2017-09-25 2019-03-28 International Business Machines Corporation Protecting Cognitive Systems from Model Stealing Attacks
CN111295674A (zh) * 2017-11-01 2020-06-16 国际商业机器公司 通过使用欺骗梯度来保护认知系统免受基于梯度的攻击
CN111667049A (zh) * 2019-03-08 2020-09-15 国际商业机器公司 量化深度学习计算系统对对抗性扰动的脆弱性

Also Published As

Publication number Publication date
WO2022123372A1 (en) 2022-06-16
US12050993B2 (en) 2024-07-30
CN116670693A (zh) 2023-08-29
DE112021005847T5 (de) 2023-08-24
JP7754599B2 (ja) 2025-10-15
GB202310212D0 (en) 2023-08-16
JP2023551976A (ja) 2023-12-13
US20220180242A1 (en) 2022-06-09

Similar Documents

Publication Publication Date Title
GB2617735A (en) Dynamic gradient deception against adversarial examples in machine learning models
JP2023551976A5 (https=)
EP3839824A3 (en) Method and apparatus for generating image, device, medium and program
WO2019241570A8 (en) Quantum virtual machine for simulation of a quantum processing system
US20200065656A1 (en) Training neural networks using a clustering loss
CN104392174B (zh) 应用程序动态行为的特征向量的生成方法及装置
GB2597166A (en) Intent recognition model creation from randomized intent vector proximities
GB2624143A (en) Root cause analysis for deterministic machine learning model
GB2602767A (en) Processor and method of weighted feature importance estimation
CN114692889A (zh) 用于机器学习算法的元特征训练模型
GB2606881A (en) Region constrained regularized adversarial examples for model interpretability
GB2617722A (en) Generating data based on pre-trained models using generative adversarial models
WO2021114924A1 (zh) 一种模型盗用的检测、模型的训练方法和装置
CN109726554A (zh) 一种恶意程序的检测方法、装置及相关应用
CN112529114A (zh) 基于gan的目标信息识别方法、电子设备和介质
JPWO2022085150A5 (ja) 攻撃シナリオ生成装置、リスク分析装置、方法、及びプログラム
CN110598794A (zh) 一种分类对抗的网络攻击检测方法及系统
GB2581283A (en) Geological sediment provenance analysis and display system
Dong et al. Dynamic analysis of an HIV model with CTL immune response and logarithmic Ornstein–Uhlenbeck process
CN112532562B (zh) 一种对抗性网络的恶意数据流检测方法及系统
Tang et al. Swarm learning anomaly detection framework for cloud data center using multi-channel BiWGAN-GTN and CEEMDAN
CN107679567A (zh) 一种代码抄写行为识别方法、装置和系统
CN117574364B (zh) 一种基于PSEAM-MobileNet神经网络的安卓恶意软件检测方法及系统
Lakhno et al. Development of a system for the detection of cyber attacks based on the clustering and formation of reference deviations of attributes
CN114817893A (zh) 验证码图像加密方法、设备、存储介质和计算机程序产品