KR102692100B1 - 훈련된 머신 러닝 모델에 적대적 견고성 추가 - Google Patents

훈련된 머신 러닝 모델에 적대적 견고성 추가 Download PDF

Info

Publication number
KR102692100B1
KR102692100B1 KR1020227008142A KR20227008142A KR102692100B1 KR 102692100 B1 KR102692100 B1 KR 102692100B1 KR 1020227008142 A KR1020227008142 A KR 1020227008142A KR 20227008142 A KR20227008142 A KR 20227008142A KR 102692100 B1 KR102692100 B1 KR 102692100B1
Authority
KR
South Korea
Prior art keywords
machine learning
learning models
adversarial
trained machine
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020227008142A
Other languages
English (en)
Korean (ko)
Other versions
KR20220054812A (ko
Inventor
비트 뷰서
마리아-이리나 니콜라에
앰브리시 라와트
마티에 신
응옥 민 트랜
마틴 위스투바
Original Assignee
인터내셔널 비지네스 머신즈 코포레이션
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 인터내셔널 비지네스 머신즈 코포레이션 filed Critical 인터내셔널 비지네스 머신즈 코포레이션
Publication of KR20220054812A publication Critical patent/KR20220054812A/ko
Application granted granted Critical
Publication of KR102692100B1 publication Critical patent/KR102692100B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/552—Detecting local intrusion or implementing counter-measures involving long-term monitoring or reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operations
    • G06F11/1471—Error detection or correction of the data by redundancy in operations involving logging of persistent data for recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computer Security & Cryptography (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Hardware Design (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)
KR1020227008142A 2019-10-14 2020-10-12 훈련된 머신 러닝 모델에 적대적 견고성 추가 Active KR102692100B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/601,451 US11334671B2 (en) 2019-10-14 2019-10-14 Adding adversarial robustness to trained machine learning models
US16/601,451 2019-10-14
PCT/IB2020/059559 WO2021074770A1 (en) 2019-10-14 2020-10-12 Adding adversarial robustness to trained machine learning models

Publications (2)

Publication Number Publication Date
KR20220054812A KR20220054812A (ko) 2022-05-03
KR102692100B1 true KR102692100B1 (ko) 2024-08-05

Family

ID=75383118

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020227008142A Active KR102692100B1 (ko) 2019-10-14 2020-10-12 훈련된 머신 러닝 모델에 적대적 견고성 추가

Country Status (7)

Country Link
US (1) US11334671B2 (https=)
JP (1) JP7537709B2 (https=)
KR (1) KR102692100B1 (https=)
CN (1) CN114503108B (https=)
AU (1) AU2020368222B2 (https=)
GB (1) GB2604791B (https=)
WO (1) WO2021074770A1 (https=)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12346432B2 (en) * 2018-12-31 2025-07-01 Intel Corporation Securing systems employing artificial intelligence
JP7079502B2 (ja) * 2019-11-14 2022-06-02 株式会社アクセル 推論システム
WO2021176716A1 (ja) * 2020-03-06 2021-09-10 日本電気株式会社 嗜好推定装置、嗜好推定方法および嗜好推定プログラム
US11675896B2 (en) * 2020-04-09 2023-06-13 International Business Machines Corporation Using multimodal model consistency to detect adversarial attacks
EP4133346A1 (en) * 2020-06-30 2023-02-15 Siemens Aktiengesellschaft Providing an alarm relating to anomaly scores assigned to input data method and system
US12242613B2 (en) * 2020-09-30 2025-03-04 International Business Machines Corporation Automated evaluation of machine learning models
US12019747B2 (en) * 2020-10-13 2024-06-25 International Business Machines Corporation Adversarial interpolation backdoor detection
KR20220103247A (ko) * 2021-01-14 2022-07-22 성균관대학교산학협력단 학습 데이터 분류를 이용한 연합 학습 프레임워크의 로컬 모델 학습 방법
US11785024B2 (en) * 2021-03-22 2023-10-10 University Of South Florida Deploying neural-trojan-resistant convolutional neural networks
WO2022224246A1 (en) * 2021-04-19 2022-10-27 Deepkeep Ltd. Device, system, and method for protecting machine learning, artificial intelligence, and deep learning units
EP4348508B1 (en) * 2021-05-31 2025-06-25 Microsoft Technology Licensing, LLC Merging models on an edge server
US12536467B2 (en) 2021-05-31 2026-01-27 Microsoft Technology Licensing, Llc Merging models on an edge server
CN113935949B (zh) * 2021-09-10 2025-09-16 上海联影智能医疗科技有限公司 乳腺钼靶图像处理方法、装置及计算机可读存储介质
US12368739B2 (en) 2021-10-13 2025-07-22 Oracle International Corporation Adaptive network attack prediction system
US20230134546A1 (en) * 2021-10-29 2023-05-04 Oracle International Corporation Network threat analysis system
CN114355936A (zh) * 2021-12-31 2022-04-15 深兰人工智能(深圳)有限公司 智能体的控制方法、装置、智能体及计算机可读存储介质
CN114358282B (zh) * 2022-01-05 2024-10-29 深圳大学 深度网络对抗鲁棒性提升模型、构建方法、设备、介质
CN114694222B (zh) * 2022-03-28 2023-08-18 马上消费金融股份有限公司 图像处理方法、装置、计算机设备及存储介质
US12541683B2 (en) * 2022-04-06 2026-02-03 Nomura Research Institute, Ltd. Information processing apparatus for improving robustness of deep neural network by using adversarial training and formal method
WO2024013911A1 (ja) * 2022-07-13 2024-01-18 日本電信電話株式会社 学習装置、学習方法、学習プログラム、推論装置、推論方法、及び推論プログラム
GB2621838A (en) * 2022-08-23 2024-02-28 Mindgard Ltd Method and system
KR102753131B1 (ko) * 2022-09-19 2025-01-14 호서대학교 산학협력단 악성코드 변종 분석을 위한 ai 모델의 견고성 측정 시스템 및 어플리케이션
EP4425384A1 (en) * 2023-02-28 2024-09-04 Fujitsu Limited Training deep belief networks
US12609949B2 (en) * 2023-04-12 2026-04-21 Taif University System and method for DNN-based cyber-security using federated learning-based generative adversarial network
US20250156941A1 (en) * 2023-11-14 2025-05-15 The Pnc Financial Services Group, Inc. Technologies for Prediction of Recurring Transactions
US12518025B1 (en) * 2025-07-09 2026-01-06 The Florida International University Board Of Trustees Systems and methods for automatic vulnerability assessment of machine learning models

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08339360A (ja) * 1995-06-13 1996-12-24 Hitachi Ltd ニューラルネットを応用した学習システム
EP0974660A1 (en) 1998-06-19 2000-01-26 Stichting Instituut voor Dierhouderij en Diergezondheid (ID-DLO) Newcastle disease virus infectious clones, vaccines and diagnostic assays
JP2000181893A (ja) 1998-12-11 2000-06-30 Toshiba Mach Co Ltd ニューラルネットワークの構成方法
US20150134966A1 (en) 2013-11-10 2015-05-14 Sypris Electronics, Llc Authentication System
US9619749B2 (en) 2014-03-06 2017-04-11 Progress, Inc. Neural network and method of neural network training
US20160321523A1 (en) 2015-04-30 2016-11-03 The Regents Of The University Of California Using machine learning to filter monte carlo noise from images
EP3400419B1 (en) 2016-01-05 2025-08-27 Mobileye Vision Technologies Ltd. Trained navigational system with imposed constraints
US20180005136A1 (en) 2016-07-01 2018-01-04 Yi Gai Machine learning in adversarial environments
CN107273747A (zh) * 2017-05-22 2017-10-20 中国人民公安大学 勒索软件检测的方法
CN107463951A (zh) * 2017-07-19 2017-12-12 清华大学 一种提高深度学习模型鲁棒性的方法及装置
CN107390949B (zh) * 2017-09-13 2020-08-07 广州视源电子科技股份有限公司 获取触摸屏基准资料的方法和装置、存储介质及触摸显示系统
US10657259B2 (en) 2017-11-01 2020-05-19 International Business Machines Corporation Protecting cognitive systems from gradient based attacks through the use of deceiving gradients
CN108304858B (zh) 2017-12-28 2022-01-04 中国银联股份有限公司 对抗样本识别模型生成方法、验证方法及其系统
US11315012B2 (en) 2018-01-12 2022-04-26 Intel Corporation Neural network training using generated random unit vector
CN108099598A (zh) 2018-01-29 2018-06-01 三汽车起重机械有限公司 用于起重机的驱动装置及起重机
CA3033014A1 (en) * 2018-02-07 2019-08-07 Royal Bank Of Canada Robust pruned neural networks via adversarial training
CN108322349B (zh) 2018-02-11 2021-04-06 浙江工业大学 基于对抗式生成网络的深度学习对抗性攻击防御方法
US10347241B1 (en) * 2018-03-23 2019-07-09 Microsoft Technology Licensing, Llc Speaker-invariant training via adversarial learning
CN108537271B (zh) 2018-04-04 2021-02-05 重庆大学 一种基于卷积去噪自编码机防御对抗样本攻击的方法
CN108615048B (zh) 2018-04-04 2020-06-23 浙江工业大学 基于扰动进化对图像分类器对抗性攻击的防御方法
CN108734276B (zh) * 2018-04-28 2021-12-31 同济大学 一种基于对抗生成网络的模仿学习对话生成方法
CA3043809A1 (en) * 2018-05-17 2019-11-17 Royal Bank Of Canada System and method for machine learning architecture with adversarial attack defence
US10861439B2 (en) * 2018-10-22 2020-12-08 Ca, Inc. Machine learning model for identifying offensive, computer-generated natural-language text or speech
US20200125928A1 (en) * 2018-10-22 2020-04-23 Ca, Inc. Real-time supervised machine learning by models configured to classify offensiveness of computer-generated natural-language text
US11526746B2 (en) * 2018-11-20 2022-12-13 Bank Of America Corporation System and method for incremental learning through state-based real-time adaptations in neural networks
US11481617B2 (en) * 2019-01-22 2022-10-25 Adobe Inc. Generating trained neural networks with increased robustness against adversarial attacks
CN109885389B (zh) * 2019-02-19 2021-07-16 浪潮云信息技术股份公司 一种基于容器的并行深度学习调度训练方法及系统
CN110008680B (zh) * 2019-04-03 2020-11-13 华南师范大学 基于对抗样本的验证码生成系统及方法
CN110310206B (zh) * 2019-07-01 2023-09-29 创新先进技术有限公司 用于更新风险控制模型的方法和系统
EP3944159A1 (en) * 2020-07-17 2022-01-26 Tata Consultancy Services Limited Method and system for defending universal adversarial attacks on time-series data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019014487A1 (en) * 2017-07-12 2019-01-17 The Regents Of The University Of California DETECTION AND PREVENTION OF DEEP ANTAGONIST LEARNING

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Jonathan Hui, "GAN - Unrolled GAN (How to reduce mode collapse)"(2018.06.)*

Also Published As

Publication number Publication date
US20210110045A1 (en) 2021-04-15
CN114503108A (zh) 2022-05-13
GB2604791B (en) 2024-03-13
GB2604791A (en) 2022-09-14
US11334671B2 (en) 2022-05-17
GB202207000D0 (en) 2022-06-29
JP7537709B2 (ja) 2024-08-21
CN114503108B (zh) 2025-03-14
AU2020368222B2 (en) 2023-11-23
WO2021074770A1 (en) 2021-04-22
KR20220054812A (ko) 2022-05-03
JP2022552243A (ja) 2022-12-15
AU2020368222A1 (en) 2022-03-31

Similar Documents

Publication Publication Date Title
KR102692100B1 (ko) 훈련된 머신 러닝 모델에 적대적 견고성 추가
US12101341B2 (en) Quantum computing machine learning for security threats
CN111476264B (zh) 访问受限的系统的对抗鲁棒性的测试
US11681914B2 (en) Determining multivariate time series data dependencies
US11036857B2 (en) Protecting a machine learning model
JP7513358B2 (ja) 機械学習モデルを堅牢化するための学習入力のプリプロセッシング
US11397891B2 (en) Interpretability-aware adversarial attack and defense method for deep learnings
US11847546B2 (en) Automatic data preprocessing
US11573785B2 (en) Predicting code vulnerabilities using machine learning classifier models trained on internal analysis states
US20230206029A1 (en) Graph Neural Network Ensemble Learning
US20210279621A1 (en) Methods and systems for graph computing with hybrid reasoning
US11526791B2 (en) Methods and systems for diverse instance generation in artificial intelligence planning
US20230108135A1 (en) Neuro-symbolic reinforcement learning with first-order logic
US11275974B2 (en) Random feature transformation forests for automatic feature engineering
CN114424216A (zh) 域特定模型压缩
US20210056457A1 (en) Hyper-parameter management
US20230325469A1 (en) Determining analytical model accuracy with perturbation response

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20220311

Patent event code: PA01051R01D

Comment text: International Patent Application

A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20220331

Comment text: Request for Examination of Application

PG1501 Laying open of application
E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

Comment text: Notification of reason for refusal

Patent event date: 20240229

Patent event code: PE09021S01D

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20240625

GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20240731

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20240801

End annual number: 3

Start annual number: 1

PG1601 Publication of registration