JP7537709B2 - トレーニングされた機械学習モデルへの敵対的ロバスト性の追加 - Google Patents

トレーニングされた機械学習モデルへの敵対的ロバスト性の追加 Download PDF

Info

Publication number
JP7537709B2
JP7537709B2 JP2022521116A JP2022521116A JP7537709B2 JP 7537709 B2 JP7537709 B2 JP 7537709B2 JP 2022521116 A JP2022521116 A JP 2022521116A JP 2022521116 A JP2022521116 A JP 2022521116A JP 7537709 B2 JP7537709 B2 JP 7537709B2
Authority
JP
Japan
Prior art keywords
machine learning
adversarial
learning models
trained machine
retraining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022521116A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022552243A5 (https=
JP2022552243A (ja
Inventor
ベッサー、ビート
ニコラエ、マリア－イリナ
ラワット、アンブリッシュ
シン、マテュー
トラン、ゴック、ミン
ウィツバ、マーティン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2022552243A publication Critical patent/JP2022552243A/ja
Publication of JP2022552243A5 publication Critical patent/JP2022552243A5/ja
Application granted granted Critical
Publication of JP7537709B2 publication Critical patent/JP7537709B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1471—Error detection or correction of the data by redundancy in operations involving logging of persistent data for recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Security & Cryptography (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)
JP2022521116A 2019-10-14 2020-10-12 トレーニングされた機械学習モデルへの敵対的ロバスト性の追加 Active JP7537709B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/601,451 US11334671B2 (en) 2019-10-14 2019-10-14 Adding adversarial robustness to trained machine learning models
US16/601,451 2019-10-14
PCT/IB2020/059559 WO2021074770A1 (en) 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models

Publications (3)

Publication Number Publication Date
JP2022552243A JP2022552243A (ja) 2022-12-15
JP2022552243A5 JP2022552243A5 (https=) 2022-12-22
JP7537709B2 true JP7537709B2 (ja) 2024-08-21

Family

ID=75383118

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022521116A Active JP7537709B2 (ja) 2019-10-14 2020-10-12 トレーニングされた機械学習モデルへの敵対的ロバスト性の追加

Country Status (7)

Country Link
US (1) US11334671B2 (https=)
JP (1) JP7537709B2 (https=)
KR (1) KR102692100B1 (https=)
CN (1) CN114503108B (https=)
AU (1) AU2020368222B2 (https=)
GB (1) GB2604791B (https=)
WO (1) WO2021074770A1 (https=)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12346432B2 (en) * 2018-12-31 2025-07-01 Intel Corporation Securing systems employing artificial intelligence
JP7079502B2 (ja) * 2019-11-14 2022-06-02 株式会社アクセル 推論システム
WO2021176716A1 (ja) * 2020-03-06 2021-09-10 日本電気株式会社 嗜好推定装置、嗜好推定方法および嗜好推定プログラム
US11675896B2 (en) * 2020-04-09 2023-06-13 International Business Machines Corporation Using multimodal model consistency to detect adversarial attacks
EP4133346A1 (en) * 2020-06-30 2023-02-15 Siemens Aktiengesellschaft Providing an alarm relating to anomaly scores assigned to input data method and system
US12242613B2 (en) * 2020-09-30 2025-03-04 International Business Machines Corporation Automated evaluation of machine learning models
US12019747B2 (en) * 2020-10-13 2024-06-25 International Business Machines Corporation Adversarial interpolation backdoor detection
KR20220103247A (ko) * 2021-01-14 2022-07-22 성균관대학교산학협력단 학습 데이터 분류를 이용한 연합 학습 프레임워크의 로컬 모델 학습 방법
US11785024B2 (en) * 2021-03-22 2023-10-10 University Of South Florida Deploying neural-trojan-resistant convolutional neural networks
WO2022224246A1 (en) * 2021-04-19 2022-10-27 Deepkeep Ltd. Device, system, and method for protecting machine learning, artificial intelligence, and deep learning units
EP4348508B1 (en) * 2021-05-31 2025-06-25 Microsoft Technology Licensing, LLC Merging models on an edge server
US12536467B2 (en) 2021-05-31 2026-01-27 Microsoft Technology Licensing, Llc Merging models on an edge server
CN113935949B (zh) * 2021-09-10 2025-09-16 上海联影智能医疗科技有限公司 乳腺钼靶图像处理方法、装置及计算机可读存储介质
US12368739B2 (en) 2021-10-13 2025-07-22 Oracle International Corporation Adaptive network attack prediction system
US20230134546A1 (en) * 2021-10-29 2023-05-04 Oracle International Corporation Network threat analysis system
CN114355936A (zh) * 2021-12-31 2022-04-15 深兰人工智能(深圳)有限公司 智能体的控制方法、装置、智能体及计算机可读存储介质
CN114358282B (zh) * 2022-01-05 2024-10-29 深圳大学 深度网络对抗鲁棒性提升模型、构建方法、设备、介质
CN114694222B (zh) * 2022-03-28 2023-08-18 马上消费金融股份有限公司 图像处理方法、装置、计算机设备及存储介质
US12541683B2 (en) * 2022-04-06 2026-02-03 Nomura Research Institute, Ltd. Information processing apparatus for improving robustness of deep neural network by using adversarial training and formal method
WO2024013911A1 (ja) * 2022-07-13 2024-01-18 日本電信電話株式会社 学習装置、学習方法、学習プログラム、推論装置、推論方法、及び推論プログラム
GB2621838A (en) * 2022-08-23 2024-02-28 Mindgard Ltd Method and system
KR102753131B1 (ko) * 2022-09-19 2025-01-14 호서대학교 산학협력단 악성코드 변종 분석을 위한 ai 모델의 견고성 측정 시스템 및 어플리케이션
EP4425384A1 (en) * 2023-02-28 2024-09-04 Fujitsu Limited Training deep belief networks
US12609949B2 (en) * 2023-04-12 2026-04-21 Taif University System and method for DNN-based cyber-security using federated learning-based generative adversarial network
US20250156941A1 (en) * 2023-11-14 2025-05-15 The Pnc Financial Services Group, Inc. Technologies for Prediction of Recurring Transactions
US12518025B1 (en) * 2025-07-09 2026-01-06 The Florida International University Board Of Trustees Systems and methods for automatic vulnerability assessment of machine learning models

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000181893A (ja) 1998-12-11 2000-06-30 Toshiba Mach Co Ltd ニューラルネットワークの構成方法
JP2006141398A (ja) 1998-06-19 2006-06-08 Id-Lelystad Inst Voor Dierhouderij En Diergezondheid Bv ニューカッスル病ウイルス感染性クローン、ワクチンおよび診断アッセイ
WO2019087033A1 (en) 2017-11-01 2019-05-09 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08339360A (ja) * 1995-06-13 1996-12-24 Hitachi Ltd ニューラルネットを応用した学習システム
US20150134966A1 (en) 2013-11-10 2015-05-14 Sypris Electronics, Llc Authentication System
US9619749B2 (en) 2014-03-06 2017-04-11 Progress, Inc. Neural network and method of neural network training
US20160321523A1 (en) 2015-04-30 2016-11-03 The Regents Of The University Of California Using machine learning to filter monte carlo noise from images
EP3400419B1 (en) 2016-01-05 2025-08-27 Mobileye Vision Technologies Ltd. Trained navigational system with imposed constraints
US20180005136A1 (en) 2016-07-01 2018-01-04 Yi Gai Machine learning in adversarial environments
CN107273747A (zh) * 2017-05-22 2017-10-20 中国人民公安大学 勒索软件检测的方法
US11526601B2 (en) 2017-07-12 2022-12-13 The Regents Of The University Of California Detection and prevention of adversarial deep learning
CN107463951A (zh) * 2017-07-19 2017-12-12 清华大学 一种提高深度学习模型鲁棒性的方法及装置
CN107390949B (zh) * 2017-09-13 2020-08-07 广州视源电子科技股份有限公司 获取触摸屏基准资料的方法和装置、存储介质及触摸显示系统
CN108304858B (zh) 2017-12-28 2022-01-04 中国银联股份有限公司 对抗样本识别模型生成方法、验证方法及其系统
US11315012B2 (en) 2018-01-12 2022-04-26 Intel Corporation Neural network training using generated random unit vector
CN108099598A (zh) 2018-01-29 2018-06-01 三汽车起重机械有限公司 用于起重机的驱动装置及起重机
CA3033014A1 (en) * 2018-02-07 2019-08-07 Royal Bank Of Canada Robust pruned neural networks via adversarial training
CN108322349B (zh) 2018-02-11 2021-04-06 浙江工业大学 基于对抗式生成网络的深度学习对抗性攻击防御方法
US10347241B1 (en) * 2018-03-23 2019-07-09 Microsoft Technology Licensing, Llc Speaker-invariant training via adversarial learning
CN108537271B (zh) 2018-04-04 2021-02-05 重庆大学 一种基于卷积去噪自编码机防御对抗样本攻击的方法
CN108615048B (zh) 2018-04-04 2020-06-23 浙江工业大学 基于扰动进化对图像分类器对抗性攻击的防御方法
CN108734276B (zh) * 2018-04-28 2021-12-31 同济大学 一种基于对抗生成网络的模仿学习对话生成方法
CA3043809A1 (en) * 2018-05-17 2019-11-17 Royal Bank Of Canada System and method for machine learning architecture with adversarial attack defence
US10861439B2 (en) * 2018-10-22 2020-12-08 Ca, Inc. Machine learning model for identifying offensive, computer-generated natural-language text or speech
US20200125928A1 (en) * 2018-10-22 2020-04-23 Ca, Inc. Real-time supervised machine learning by models configured to classify offensiveness of computer-generated natural-language text
US11526746B2 (en) * 2018-11-20 2022-12-13 Bank Of America Corporation System and method for incremental learning through state-based real-time adaptations in neural networks
US11481617B2 (en) * 2019-01-22 2022-10-25 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN109885389B (zh) * 2019-02-19 2021-07-16 浪潮云信息技术股份公司 一种基于容器的并行深度学习调度训练方法及系统
CN110008680B (zh) * 2019-04-03 2020-11-13 华南师范大学 基于对抗样本的验证码生成系统及方法
CN110310206B (zh) * 2019-07-01 2023-09-29 创新先进技术有限公司 用于更新风险控制模型的方法和系统
EP3944159A1 (en) * 2020-07-17 2022-01-26 Tata Consultancy Services Limited Method and system for defending universal adversarial attacks on time-series data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006141398A (ja) 1998-06-19 2006-06-08 Id-Lelystad Inst Voor Dierhouderij En Diergezondheid Bv ニューカッスル病ウイルス感染性クローン、ワクチンおよび診断アッセイ
JP2000181893A (ja) 1998-12-11 2000-06-30 Toshiba Mach Co Ltd ニューラルネットワークの構成方法
WO2019087033A1 (en) 2017-11-01 2019-05-09 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients

Also Published As

Publication number Publication date
US20210110045A1 (en) 2021-04-15
CN114503108A (zh) 2022-05-13
GB2604791B (en) 2024-03-13
GB2604791A (en) 2022-09-14
US11334671B2 (en) 2022-05-17
GB202207000D0 (en) 2022-06-29
KR102692100B1 (ko) 2024-08-05
CN114503108B (zh) 2025-03-14
AU2020368222B2 (en) 2023-11-23
WO2021074770A1 (en) 2021-04-22
KR20220054812A (ko) 2022-05-03
JP2022552243A (ja) 2022-12-15
AU2020368222A1 (en) 2022-03-31

Similar Documents

Publication Publication Date Title
JP7537709B2 (ja) トレーニングされた機械学習モデルへの敵対的ロバスト性の追加
US12101341B2 (en) Quantum computing machine learning for security threats
CN114270349B (zh) 用于加固机器学习模型的学习输入预处理
US11748648B2 (en) Quantum pulse optimization using machine learning
US10909327B2 (en) Unsupervised learning of interpretable conversation models from conversation logs
US11048718B2 (en) Methods and systems for feature engineering
US11847546B2 (en) Automatic data preprocessing
US11836220B2 (en) Updating of statistical sets for decentralized distributed training of a machine learning model
US20220358358A1 (en) Accelerating inference of neural network models via dynamic early exits
US11526791B2 (en) Methods and systems for diverse instance generation in artificial intelligence planning
US10839791B2 (en) Neural network-based acoustic model with softening target-layer
US20240037439A1 (en) Quantum system selection via coupling map comparison
US20210216858A1 (en) Training machine learning systems
US11741377B2 (en) Target system optimization with domain knowledge
US12353966B2 (en) Spectral clustering of high-dimensional data
US20230325469A1 (en) Determining analytical model accuracy with perturbation response
US12468977B2 (en) Uncertainty aware parameter provision for a variational quantum algorithm
US12282848B2 (en) Estimated online hard negative mining via probabilistic selection and scores history consideration

Legal Events

Date Code Title Description
RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20220518

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221209

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230324

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240327

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240402

RD12 Notification of acceptance of power of sub attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7432

Effective date: 20240520

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240625

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240716

RD14 Notification of resignation of power of sub attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7434

Effective date: 20240718

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240730

R150 Certificate of patent or registration of utility model

Ref document number: 7537709

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150