CN114503108B - 向经过训练的机器学习模型添加对抗鲁棒性 - Google Patents

向经过训练的机器学习模型添加对抗鲁棒性 Download PDF

Info

Publication number
CN114503108B
CN114503108B CN202080070524.1A CN202080070524A CN114503108B CN 114503108 B CN114503108 B CN 114503108B CN 202080070524 A CN202080070524 A CN 202080070524A CN 114503108 B CN114503108 B CN 114503108B
Authority
CN
China
Prior art keywords
machine learning
learning models
adversarial
trained machine
previously trained
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080070524.1A
Other languages
English (en)
Chinese (zh)
Other versions
CN114503108A (zh
Inventor
B.比塞尔
M-I.尼古拉
A.拉瓦特
M.辛恩
N.M.特兰
M.维斯图巴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CN114503108A publication Critical patent/CN114503108A/zh
Application granted granted Critical
Publication of CN114503108B publication Critical patent/CN114503108B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1471—Error detection or correction of the data by redundancy in operations involving logging of persistent data for recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Security & Cryptography (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)
CN202080070524.1A 2019-10-14 2020-10-12 向经过训练的机器学习模型添加对抗鲁棒性 Active CN114503108B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/601,451 US11334671B2 (en) 2019-10-14 2019-10-14 Adding adversarial robustness to trained machine learning models
US16/601,451 2019-10-14
PCT/IB2020/059559 WO2021074770A1 (en) 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models

Publications (2)

Publication Number Publication Date
CN114503108A CN114503108A (zh) 2022-05-13
CN114503108B true CN114503108B (zh) 2025-03-14

Family

ID=75383118

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080070524.1A Active CN114503108B (zh) 2019-10-14 2020-10-12 向经过训练的机器学习模型添加对抗鲁棒性

Country Status (7)

Country Link
US (1) US11334671B2 (https=)
JP (1) JP7537709B2 (https=)
KR (1) KR102692100B1 (https=)
CN (1) CN114503108B (https=)
AU (1) AU2020368222B2 (https=)
GB (1) GB2604791B (https=)
WO (1) WO2021074770A1 (https=)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12346432B2 (en) * 2018-12-31 2025-07-01 Intel Corporation Securing systems employing artificial intelligence
JP7079502B2 (ja) * 2019-11-14 2022-06-02 株式会社アクセル 推論システム
WO2021176716A1 (ja) * 2020-03-06 2021-09-10 日本電気株式会社 嗜好推定装置、嗜好推定方法および嗜好推定プログラム
US11675896B2 (en) * 2020-04-09 2023-06-13 International Business Machines Corporation Using multimodal model consistency to detect adversarial attacks
EP4133346A1 (en) * 2020-06-30 2023-02-15 Siemens Aktiengesellschaft Providing an alarm relating to anomaly scores assigned to input data method and system
US12242613B2 (en) * 2020-09-30 2025-03-04 International Business Machines Corporation Automated evaluation of machine learning models
US12019747B2 (en) * 2020-10-13 2024-06-25 International Business Machines Corporation Adversarial interpolation backdoor detection
KR20220103247A (ko) * 2021-01-14 2022-07-22 성균관대학교산학협력단 학습 데이터 분류를 이용한 연합 학습 프레임워크의 로컬 모델 학습 방법
US11785024B2 (en) * 2021-03-22 2023-10-10 University Of South Florida Deploying neural-trojan-resistant convolutional neural networks
WO2022224246A1 (en) * 2021-04-19 2022-10-27 Deepkeep Ltd. Device, system, and method for protecting machine learning, artificial intelligence, and deep learning units
EP4348508B1 (en) * 2021-05-31 2025-06-25 Microsoft Technology Licensing, LLC Merging models on an edge server
US12536467B2 (en) 2021-05-31 2026-01-27 Microsoft Technology Licensing, Llc Merging models on an edge server
CN113935949B (zh) * 2021-09-10 2025-09-16 上海联影智能医疗科技有限公司 乳腺钼靶图像处理方法、装置及计算机可读存储介质
US12368739B2 (en) 2021-10-13 2025-07-22 Oracle International Corporation Adaptive network attack prediction system
US20230134546A1 (en) * 2021-10-29 2023-05-04 Oracle International Corporation Network threat analysis system
CN114355936A (zh) * 2021-12-31 2022-04-15 深兰人工智能(深圳)有限公司 智能体的控制方法、装置、智能体及计算机可读存储介质
CN114358282B (zh) * 2022-01-05 2024-10-29 深圳大学 深度网络对抗鲁棒性提升模型、构建方法、设备、介质
CN114694222B (zh) * 2022-03-28 2023-08-18 马上消费金融股份有限公司 图像处理方法、装置、计算机设备及存储介质
US12541683B2 (en) * 2022-04-06 2026-02-03 Nomura Research Institute, Ltd. Information processing apparatus for improving robustness of deep neural network by using adversarial training and formal method
WO2024013911A1 (ja) * 2022-07-13 2024-01-18 日本電信電話株式会社 学習装置、学習方法、学習プログラム、推論装置、推論方法、及び推論プログラム
GB2621838A (en) * 2022-08-23 2024-02-28 Mindgard Ltd Method and system
KR102753131B1 (ko) * 2022-09-19 2025-01-14 호서대학교 산학협력단 악성코드 변종 분석을 위한 ai 모델의 견고성 측정 시스템 및 어플리케이션
EP4425384A1 (en) * 2023-02-28 2024-09-04 Fujitsu Limited Training deep belief networks
US12609949B2 (en) * 2023-04-12 2026-04-21 Taif University System and method for DNN-based cyber-security using federated learning-based generative adversarial network
US20250156941A1 (en) * 2023-11-14 2025-05-15 The Pnc Financial Services Group, Inc. Technologies for Prediction of Recurring Transactions
US12518025B1 (en) * 2025-07-09 2026-01-06 The Florida International University Board Of Trustees Systems and methods for automatic vulnerability assessment of machine learning models

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08339360A (ja) * 1995-06-13 1996-12-24 Hitachi Ltd ニューラルネットを応用した学習システム
EP0974660A1 (en) 1998-06-19 2000-01-26 Stichting Instituut voor Dierhouderij en Diergezondheid (ID-DLO) Newcastle disease virus infectious clones, vaccines and diagnostic assays
JP2000181893A (ja) 1998-12-11 2000-06-30 Toshiba Mach Co Ltd ニューラルネットワークの構成方法
US20150134966A1 (en) 2013-11-10 2015-05-14 Sypris Electronics, Llc Authentication System
US9619749B2 (en) 2014-03-06 2017-04-11 Progress, Inc. Neural network and method of neural network training
US20160321523A1 (en) 2015-04-30 2016-11-03 The Regents Of The University Of California Using machine learning to filter monte carlo noise from images
EP3400419B1 (en) 2016-01-05 2025-08-27 Mobileye Vision Technologies Ltd. Trained navigational system with imposed constraints
US20180005136A1 (en) 2016-07-01 2018-01-04 Yi Gai Machine learning in adversarial environments
CN107273747A (zh) * 2017-05-22 2017-10-20 中国人民公安大学 勒索软件检测的方法
CN107463951A (zh) * 2017-07-19 2017-12-12 清华大学 一种提高深度学习模型鲁棒性的方法及装置
CN107390949B (zh) * 2017-09-13 2020-08-07 广州视源电子科技股份有限公司 获取触摸屏基准资料的方法和装置、存储介质及触摸显示系统
US10657259B2 (en) 2017-11-01 2020-05-19 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients
CN108304858B (zh) 2017-12-28 2022-01-04 中国银联股份有限公司 对抗样本识别模型生成方法、验证方法及其系统
US11315012B2 (en) 2018-01-12 2022-04-26 Intel Corporation Neural network training using generated random unit vector
CN108099598A (zh) 2018-01-29 2018-06-01 三汽车起重机械有限公司 用于起重机的驱动装置及起重机
CA3033014A1 (en) * 2018-02-07 2019-08-07 Royal Bank Of Canada Robust pruned neural networks via adversarial training
CN108322349B (zh) 2018-02-11 2021-04-06 浙江工业大学 基于对抗式生成网络的深度学习对抗性攻击防御方法
US10347241B1 (en) * 2018-03-23 2019-07-09 Microsoft Technology Licensing, Llc Speaker-invariant training via adversarial learning
CN108537271B (zh) 2018-04-04 2021-02-05 重庆大学 一种基于卷积去噪自编码机防御对抗样本攻击的方法
CN108615048B (zh) 2018-04-04 2020-06-23 浙江工业大学 基于扰动进化对图像分类器对抗性攻击的防御方法
CN108734276B (zh) * 2018-04-28 2021-12-31 同济大学 一种基于对抗生成网络的模仿学习对话生成方法
CA3043809A1 (en) * 2018-05-17 2019-11-17 Royal Bank Of Canada System and method for machine learning architecture with adversarial attack defence
US10861439B2 (en) * 2018-10-22 2020-12-08 Ca, Inc. Machine learning model for identifying offensive, computer-generated natural-language text or speech
US20200125928A1 (en) * 2018-10-22 2020-04-23 Ca, Inc. Real-time supervised machine learning by models configured to classify offensiveness of computer-generated natural-language text
US11526746B2 (en) * 2018-11-20 2022-12-13 Bank Of America Corporation System and method for incremental learning through state-based real-time adaptations in neural networks
US11481617B2 (en) * 2019-01-22 2022-10-25 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN109885389B (zh) * 2019-02-19 2021-07-16 浪潮云信息技术股份公司 一种基于容器的并行深度学习调度训练方法及系统
CN110008680B (zh) * 2019-04-03 2020-11-13 华南师范大学 基于对抗样本的验证码生成系统及方法
CN110310206B (zh) * 2019-07-01 2023-09-29 创新先进技术有限公司 用于更新风险控制模型的方法和系统
EP3944159A1 (en) * 2020-07-17 2022-01-26 Tata Consultancy Services Limited Method and system for defending universal adversarial attacks on time-series data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于对抗攻击的人脸活体检测模型安全研究;毛骏杰;《中国优秀硕士学位论文全文数据库 信息科技辑》;20240615(第6期);I138-364 *

Also Published As

Publication number Publication date
US20210110045A1 (en) 2021-04-15
CN114503108A (zh) 2022-05-13
GB2604791B (en) 2024-03-13
GB2604791A (en) 2022-09-14
US11334671B2 (en) 2022-05-17
GB202207000D0 (en) 2022-06-29
KR102692100B1 (ko) 2024-08-05
JP7537709B2 (ja) 2024-08-21
AU2020368222B2 (en) 2023-11-23
WO2021074770A1 (en) 2021-04-22
KR20220054812A (ko) 2022-05-03
JP2022552243A (ja) 2022-12-15
AU2020368222A1 (en) 2022-03-31

Similar Documents

Publication Publication Date Title
CN114503108B (zh) 向经过训练的机器学习模型添加对抗鲁棒性
CN114270349B (zh) 用于加固机器学习模型的学习输入预处理
JP7640200B2 (ja) セキュリティ脅威に関する量子コンピューティング機械学習
US11411979B2 (en) Compliance process risk assessment
US11036857B2 (en) Protecting a machine learning model
US11397891B2 (en) Interpretability-aware adversarial attack and defense method for deep learnings
US11924239B2 (en) Vulnerability and attack technique association
US11048718B2 (en) Methods and systems for feature engineering
US11176508B2 (en) Minimizing compliance risk using machine learning techniques
CN115427967A (zh) 确定多变量时间序列数据依赖性
CA3033014A1 (en) Robust pruned neural networks via adversarial training
US20200177634A1 (en) Hybrid Network Infrastructure Management
US11847546B2 (en) Automatic data preprocessing
US11573785B2 (en) Predicting code vulnerabilities using machine learning classifier models trained on internal analysis states
US11836220B2 (en) Updating of statistical sets for decentralized distributed training of a machine learning model
US11526791B2 (en) Methods and systems for diverse instance generation in artificial intelligence planning
US11741377B2 (en) Target system optimization with domain knowledge
US20230325469A1 (en) Determining analytical model accuracy with perturbation response
TWI792923B (zh) 使用基於用戶互動歷史之模型來增強移動設備中之用戶驗證之電腦實施方法、電腦系統及電腦程式產品
US12468977B2 (en) Uncertainty aware parameter provision for a variational quantum algorithm
US11288046B2 (en) Methods and systems for program optimization utilizing intelligent space exploration
US12555009B2 (en) Updating of a statistical set for decentralized distributed training of a machine learning model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant