AU2020368222B2 - Adding adversarial robustness to trained machine learning models - Google Patents

Adding adversarial robustness to trained machine learning models Download PDF

Info

Publication number
AU2020368222B2
AU2020368222B2 AU2020368222A AU2020368222A AU2020368222B2 AU 2020368222 B2 AU2020368222 B2 AU 2020368222B2 AU 2020368222 A AU2020368222 A AU 2020368222A AU 2020368222 A AU2020368222 A AU 2020368222A AU 2020368222 B2 AU2020368222 B2 AU 2020368222B2
Authority
AU
Australia
Prior art keywords
machine learning
adversarial
learning models
trained machine
retraining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2020368222A
Other languages
English (en)
Other versions
AU2020368222A1 (en
Inventor
Beat Buesser
Maria-Irina Nicolae
Ambrish RAWAT
Mathieu Sinn
Ngoc Minh Tran
Martin Wistuba
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of AU2020368222A1 publication Critical patent/AU2020368222A1/en
Application granted granted Critical
Publication of AU2020368222B2 publication Critical patent/AU2020368222B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1471—Error detection or correction of the data by redundancy in operations involving logging of persistent data for recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Security & Cryptography (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)
AU2020368222A 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models Active AU2020368222B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/601,451 US11334671B2 (en) 2019-10-14 2019-10-14 Adding adversarial robustness to trained machine learning models
US16/601,451 2019-10-14
PCT/IB2020/059559 WO2021074770A1 (en) 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models

Publications (2)

Publication Number Publication Date
AU2020368222A1 AU2020368222A1 (en) 2022-03-31
AU2020368222B2 true AU2020368222B2 (en) 2023-11-23

Family

ID=75383118

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2020368222A Active AU2020368222B2 (en) 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models

Country Status (7)

Country Link
US (1) US11334671B2 (https=)
JP (1) JP7537709B2 (https=)
KR (1) KR102692100B1 (https=)
CN (1) CN114503108B (https=)
AU (1) AU2020368222B2 (https=)
GB (1) GB2604791B (https=)
WO (1) WO2021074770A1 (https=)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12346432B2 (en) * 2018-12-31 2025-07-01 Intel Corporation Securing systems employing artificial intelligence
JP7079502B2 (ja) * 2019-11-14 2022-06-02 株式会社アクセル 推論システム
WO2021176716A1 (ja) * 2020-03-06 2021-09-10 日本電気株式会社 嗜好推定装置、嗜好推定方法および嗜好推定プログラム
US11675896B2 (en) * 2020-04-09 2023-06-13 International Business Machines Corporation Using multimodal model consistency to detect adversarial attacks
EP4133346A1 (en) * 2020-06-30 2023-02-15 Siemens Aktiengesellschaft Providing an alarm relating to anomaly scores assigned to input data method and system
US12242613B2 (en) * 2020-09-30 2025-03-04 International Business Machines Corporation Automated evaluation of machine learning models
US12019747B2 (en) * 2020-10-13 2024-06-25 International Business Machines Corporation Adversarial interpolation backdoor detection
KR20220103247A (ko) * 2021-01-14 2022-07-22 성균관대학교산학협력단 학습 데이터 분류를 이용한 연합 학습 프레임워크의 로컬 모델 학습 방법
US11785024B2 (en) * 2021-03-22 2023-10-10 University Of South Florida Deploying neural-trojan-resistant convolutional neural networks
WO2022224246A1 (en) * 2021-04-19 2022-10-27 Deepkeep Ltd. Device, system, and method for protecting machine learning, artificial intelligence, and deep learning units
EP4348508B1 (en) * 2021-05-31 2025-06-25 Microsoft Technology Licensing, LLC Merging models on an edge server
US12536467B2 (en) 2021-05-31 2026-01-27 Microsoft Technology Licensing, Llc Merging models on an edge server
CN113935949B (zh) * 2021-09-10 2025-09-16 上海联影智能医疗科技有限公司 乳腺钼靶图像处理方法、装置及计算机可读存储介质
US12368739B2 (en) 2021-10-13 2025-07-22 Oracle International Corporation Adaptive network attack prediction system
US20230134546A1 (en) * 2021-10-29 2023-05-04 Oracle International Corporation Network threat analysis system
CN114355936A (zh) * 2021-12-31 2022-04-15 深兰人工智能(深圳)有限公司 智能体的控制方法、装置、智能体及计算机可读存储介质
CN114358282B (zh) * 2022-01-05 2024-10-29 深圳大学 深度网络对抗鲁棒性提升模型、构建方法、设备、介质
CN114694222B (zh) * 2022-03-28 2023-08-18 马上消费金融股份有限公司 图像处理方法、装置、计算机设备及存储介质
US12541683B2 (en) * 2022-04-06 2026-02-03 Nomura Research Institute, Ltd. Information processing apparatus for improving robustness of deep neural network by using adversarial training and formal method
WO2024013911A1 (ja) * 2022-07-13 2024-01-18 日本電信電話株式会社 学習装置、学習方法、学習プログラム、推論装置、推論方法、及び推論プログラム
GB2621838A (en) * 2022-08-23 2024-02-28 Mindgard Ltd Method and system
KR102753131B1 (ko) * 2022-09-19 2025-01-14 호서대학교 산학협력단 악성코드 변종 분석을 위한 ai 모델의 견고성 측정 시스템 및 어플리케이션
EP4425384A1 (en) * 2023-02-28 2024-09-04 Fujitsu Limited Training deep belief networks
US12609949B2 (en) * 2023-04-12 2026-04-21 Taif University System and method for DNN-based cyber-security using federated learning-based generative adversarial network
US20250156941A1 (en) * 2023-11-14 2025-05-15 The Pnc Financial Services Group, Inc. Technologies for Prediction of Recurring Transactions
US12518025B1 (en) * 2025-07-09 2026-01-06 The Florida International University Board Of Trustees Systems and methods for automatic vulnerability assessment of machine learning models

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING
US20190244103A1 (en) * 2018-02-07 2019-08-08 Royal Bank Of Canada Robust pruned neural networks via adversarial training

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08339360A (ja) * 1995-06-13 1996-12-24 Hitachi Ltd ニューラルネットを応用した学習システム
EP0974660A1 (en) 1998-06-19 2000-01-26 Stichting Instituut voor Dierhouderij en Diergezondheid (ID-DLO) Newcastle disease virus infectious clones, vaccines and diagnostic assays
JP2000181893A (ja) 1998-12-11 2000-06-30 Toshiba Mach Co Ltd ニューラルネットワークの構成方法
US20150134966A1 (en) 2013-11-10 2015-05-14 Sypris Electronics, Llc Authentication System
US9619749B2 (en) 2014-03-06 2017-04-11 Progress, Inc. Neural network and method of neural network training
US20160321523A1 (en) 2015-04-30 2016-11-03 The Regents Of The University Of California Using machine learning to filter monte carlo noise from images
EP3400419B1 (en) 2016-01-05 2025-08-27 Mobileye Vision Technologies Ltd. Trained navigational system with imposed constraints
US20180005136A1 (en) 2016-07-01 2018-01-04 Yi Gai Machine learning in adversarial environments
CN107273747A (zh) * 2017-05-22 2017-10-20 中国人民公安大学 勒索软件检测的方法
CN107463951A (zh) * 2017-07-19 2017-12-12 清华大学 一种提高深度学习模型鲁棒性的方法及装置
CN107390949B (zh) * 2017-09-13 2020-08-07 广州视源电子科技股份有限公司 获取触摸屏基准资料的方法和装置、存储介质及触摸显示系统
US10657259B2 (en) 2017-11-01 2020-05-19 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients
CN108304858B (zh) 2017-12-28 2022-01-04 中国银联股份有限公司 对抗样本识别模型生成方法、验证方法及其系统
US11315012B2 (en) 2018-01-12 2022-04-26 Intel Corporation Neural network training using generated random unit vector
CN108099598A (zh) 2018-01-29 2018-06-01 三汽车起重机械有限公司 用于起重机的驱动装置及起重机
CN108322349B (zh) 2018-02-11 2021-04-06 浙江工业大学 基于对抗式生成网络的深度学习对抗性攻击防御方法
US10347241B1 (en) * 2018-03-23 2019-07-09 Microsoft Technology Licensing, Llc Speaker-invariant training via adversarial learning
CN108537271B (zh) 2018-04-04 2021-02-05 重庆大学 一种基于卷积去噪自编码机防御对抗样本攻击的方法
CN108615048B (zh) 2018-04-04 2020-06-23 浙江工业大学 基于扰动进化对图像分类器对抗性攻击的防御方法
CN108734276B (zh) * 2018-04-28 2021-12-31 同济大学 一种基于对抗生成网络的模仿学习对话生成方法
CA3043809A1 (en) * 2018-05-17 2019-11-17 Royal Bank Of Canada System and method for machine learning architecture with adversarial attack defence
US10861439B2 (en) * 2018-10-22 2020-12-08 Ca, Inc. Machine learning model for identifying offensive, computer-generated natural-language text or speech
US20200125928A1 (en) * 2018-10-22 2020-04-23 Ca, Inc. Real-time supervised machine learning by models configured to classify offensiveness of computer-generated natural-language text
US11526746B2 (en) * 2018-11-20 2022-12-13 Bank Of America Corporation System and method for incremental learning through state-based real-time adaptations in neural networks
US11481617B2 (en) * 2019-01-22 2022-10-25 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN109885389B (zh) * 2019-02-19 2021-07-16 浪潮云信息技术股份公司 一种基于容器的并行深度学习调度训练方法及系统
CN110008680B (zh) * 2019-04-03 2020-11-13 华南师范大学 基于对抗样本的验证码生成系统及方法
CN110310206B (zh) * 2019-07-01 2023-09-29 创新先进技术有限公司 用于更新风险控制模型的方法和系统
EP3944159A1 (en) * 2020-07-17 2022-01-26 Tata Consultancy Services Limited Method and system for defending universal adversarial attacks on time-series data

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING
US20190244103A1 (en) * 2018-02-07 2019-08-08 Royal Bank Of Canada Robust pruned neural networks via adversarial training

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SPECHT, F. et al., "Generation of adversarial examples to prevent misclassification of deep neural network based condition monitoring systems for cyber-physical production systems", 2018. *

Also Published As

Publication number Publication date
US20210110045A1 (en) 2021-04-15
CN114503108A (zh) 2022-05-13
GB2604791B (en) 2024-03-13
GB2604791A (en) 2022-09-14
US11334671B2 (en) 2022-05-17
GB202207000D0 (en) 2022-06-29
KR102692100B1 (ko) 2024-08-05
JP7537709B2 (ja) 2024-08-21
CN114503108B (zh) 2025-03-14
WO2021074770A1 (en) 2021-04-22
KR20220054812A (ko) 2022-05-03
JP2022552243A (ja) 2022-12-15
AU2020368222A1 (en) 2022-03-31

Similar Documents

Publication Publication Date Title
AU2020368222B2 (en) Adding adversarial robustness to trained machine learning models
US11681796B2 (en) Learning input preprocessing to harden machine learning models
JP7640200B2 (ja) セキュリティ脅威に関する量子コンピューティング機械学習
US11036857B2 (en) Protecting a machine learning model
US11397891B2 (en) Interpretability-aware adversarial attack and defense method for deep learnings
US20200066255A1 (en) Unsupervised Learning of Interpretable Conversation Models from Conversation Logs
US11048718B2 (en) Methods and systems for feature engineering
US11847546B2 (en) Automatic data preprocessing
US11176508B2 (en) Minimizing compliance risk using machine learning techniques
US20220300822A1 (en) Forgetting data samples from pretrained neural network models
US11573785B2 (en) Predicting code vulnerabilities using machine learning classifier models trained on internal analysis states
US20220358358A1 (en) Accelerating inference of neural network models via dynamic early exits
US11526791B2 (en) Methods and systems for diverse instance generation in artificial intelligence planning
US12229509B2 (en) Contextual impact adjustment for machine learning models
US10839791B2 (en) Neural network-based acoustic model with softening target-layer
US12518153B2 (en) Training machine learning systems
US20200218990A1 (en) Target system optimization with domain knowledge
US12093814B2 (en) Hyper-parameter management
US11620435B2 (en) Domain specific model compression
US11275974B2 (en) Random feature transformation forests for automatic feature engineering
US20210216879A1 (en) Methods and systems for improving heuristic searches for artificial intelligence planning
US20230325469A1 (en) Determining analytical model accuracy with perturbation response
US12282848B2 (en) Estimated online hard negative mining via probabilistic selection and scores history consideration

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)