JP7232599B2 - 畳み込みニューラルネットワークのための刈り込みと再学習法 - Google Patents

畳み込みニューラルネットワークのための刈り込みと再学習法 Download PDF

Info

Publication number
JP7232599B2
JP7232599B2 JP2018165782A JP2018165782A JP7232599B2 JP 7232599 B2 JP7232599 B2 JP 7232599B2 JP 2018165782 A JP2018165782 A JP 2018165782A JP 2018165782 A JP2018165782 A JP 2018165782A JP 7232599 B2 JP7232599 B2 JP 7232599B2
Authority
JP
Japan
Prior art keywords
neural network
pruning
rate
connections
accuracy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2018165782A
Other languages
English (en)
Japanese (ja)
Other versions
JP2019049977A5 (https=
JP2019049977A (ja
Inventor
ウォン シン
リン シャン‐ハン
Original Assignee
ビバンテ コーポレーション
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ビバンテ コーポレーション filed Critical ビバンテ コーポレーション
Publication of JP2019049977A publication Critical patent/JP2019049977A/ja
Publication of JP2019049977A5 publication Critical patent/JP2019049977A5/ja
Application granted granted Critical
Publication of JP7232599B2 publication Critical patent/JP7232599B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
JP2018165782A 2017-09-08 2018-09-05 畳み込みニューラルネットワークのための刈り込みと再学習法 Active JP7232599B2 (ja)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/699,438 2017-09-08
US15/699,438 US11200495B2 (en) 2017-09-08 2017-09-08 Pruning and retraining method for a convolution neural network

Publications (3)

Publication Number Publication Date
JP2019049977A JP2019049977A (ja) 2019-03-28
JP2019049977A5 JP2019049977A5 (https=) 2021-10-07
JP7232599B2 true JP7232599B2 (ja) 2023-03-03

Family

ID=63490311

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018165782A Active JP7232599B2 (ja) 2017-09-08 2018-09-05 畳み込みニューラルネットワークのための刈り込みと再学習法

Country Status (5)

Country Link
US (1) US11200495B2 (https=)
EP (1) EP3454262A1 (https=)
JP (1) JP7232599B2 (https=)
KR (1) KR102793134B1 (https=)
CN (1) CN109472357A (https=)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11734568B2 (en) * 2018-02-14 2023-08-22 Google Llc Systems and methods for modification of neural networks based on estimated edge utility
KR20200023238A (ko) * 2018-08-23 2020-03-04 삼성전자주식회사 딥러닝 모델을 생성하는 방법 및 시스템
CN111738401A (zh) * 2019-03-25 2020-10-02 北京三星通信技术研究有限公司 模型优化方法、分组压缩方法、相应的装置、设备
US12373696B2 (en) * 2019-06-21 2025-07-29 Samsung Electronics Co., Ltd. Neural network hardware accelerator system with zero-skipping and hierarchical structured pruning methods
US20210089921A1 (en) * 2019-09-25 2021-03-25 Nvidia Corporation Transfer learning for neural networks
US11704571B2 (en) * 2019-10-11 2023-07-18 Qualcomm Incorporated Learned threshold pruning for deep neural networks
CN112699990B (zh) * 2019-10-22 2024-06-07 杭州海康威视数字技术股份有限公司 神经网络模型训练方法、装置及电子设备
CN111061909B (zh) * 2019-11-22 2023-11-28 腾讯音乐娱乐科技（深圳）有限公司 一种伴奏分类方法和装置
US11935271B2 (en) * 2020-01-10 2024-03-19 Tencent America LLC Neural network model compression with selective structured weight unification
US11562235B2 (en) 2020-02-21 2023-01-24 International Business Machines Corporation Activation function computation for neural networks
CN111582456B (zh) * 2020-05-11 2023-12-15 抖音视界有限公司 用于生成网络模型信息的方法、装置、设备和介质
WO2021248409A1 (en) * 2020-06-11 2021-12-16 Alibaba Group Holding Limited Pruning hardware unit for training neural network
CN111553169B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
CN111539224B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
US12488250B2 (en) 2020-11-02 2025-12-02 International Business Machines Corporation Weight repetition on RPU crossbar arrays
US20220156574A1 (en) * 2020-11-19 2022-05-19 Kabushiki Kaisha Toshiba Methods and systems for remote training of a machine learning model
US20220318633A1 (en) * 2021-03-26 2022-10-06 Qualcomm Incorporated Model compression using pruning quantization and knowledge distillation
US20220405571A1 (en) * 2021-06-16 2022-12-22 Microsoft Technology Licensing, Llc Sparsifying narrow data formats for neural networks
CN113469326B (zh) * 2021-06-24 2024-04-02 上海寒武纪信息科技有限公司 在神经网络模型中执行剪枝优化的集成电路装置及板卡
US12524673B2 (en) * 2021-07-16 2026-01-13 Industry-Academic Cooperation Foundation, Yonsei University Multitask distributed learning system and method based on lottery ticket neural network
JP7666289B2 (ja) 2021-10-25 2025-04-22 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
US12591778B2 (en) 2021-11-17 2026-03-31 Samsung Electronics Co., Ltd. System and method for torque-based structured pruning for deep neural networks
CN115115045B (zh) * 2021-12-24 2026-02-06 杭州海康威视数字技术股份有限公司 一种模型剪枝方法、装置及电子设备
CN114462582A (zh) * 2022-02-25 2022-05-10 腾讯科技（深圳）有限公司 基于卷积神经网络模型的数据处理方法及装置、设备
JP7782319B2 (ja) 2022-03-04 2025-12-09 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
CN120562503A (zh) * 2022-03-31 2025-08-29 支付宝(杭州)信息技术有限公司 模型剪枝方法、装置和计算机设备
CN114863243B (zh) * 2022-04-28 2024-12-17 国家电网有限公司大数据中心 一种模型的数据遗忘方法、装置、设备及存储介质
US20240013050A1 (en) * 2022-07-05 2024-01-11 International Business Machines Corporation Packing machine learning models using pruning and permutation
US12524999B2 (en) * 2022-09-30 2026-01-13 Samsung Electronics Co., Ltd. Generating images with small objects for training a pruned super-resolution network

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011054200A (ja) 2010-11-11 2011-03-17 Fuji Electric Systems Co Ltd ニューラルネットワークの学習方法
US20160307098A1 (en) 2015-04-19 2016-10-20 International Business Machines Corporation Annealed dropout training of neural networks
JP2018152000A (ja) 2017-03-15 2018-09-27 株式会社島津製作所 分析データ解析装置及び分析データ解析方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5288645A (en) * 1992-09-04 1994-02-22 Mtm Engineering, Inc. Hydrogen evolution analyzer
US11423311B2 (en) * 2015-06-04 2022-08-23 Samsung Electronics Co., Ltd. Automatic tuning of artificial neural networks
US10832136B2 (en) * 2016-05-18 2020-11-10 Nec Corporation Passive pruning of filters in a convolutional neural network
US10832123B2 (en) * 2016-08-12 2020-11-10 Xilinx Technology Beijing Limited Compression of deep neural networks with proper use of mask
US10984308B2 (en) * 2016-08-12 2021-04-20 Xilinx Technology Beijing Limited Compression method for deep neural networks with load balance
US10762426B2 (en) * 2016-08-12 2020-09-01 Beijing Deephi Intelligent Technology Co., Ltd. Multi-iteration compression for deep neural networks
CN107688850B (zh) * 2017-08-08 2021-04-13 赛灵思公司 一种深度神经网络压缩方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011054200A (ja) 2010-11-11 2011-03-17 Fuji Electric Systems Co Ltd ニューラルネットワークの学習方法
US20160307098A1 (en) 2015-04-19 2016-10-20 International Business Machines Corporation Annealed dropout training of neural networks
JP2018152000A (ja) 2017-03-15 2018-09-27 株式会社島津製作所 分析データ解析装置及び分析データ解析方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HU Hengyuan et al.，Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures，arXiv.org [online]，2016年，pp.1-9，[2022年7月28日検索], インターネット<URL: https://arxiv.org/abs/1607.03250>

Also Published As

Publication number Publication date
US20190080238A1 (en) 2019-03-14
KR20190028320A (ko) 2019-03-18
EP3454262A1 (en) 2019-03-13
KR102793134B1 (ko) 2025-04-07
US11200495B2 (en) 2021-12-14
CN109472357A (zh) 2019-03-15
JP2019049977A (ja) 2019-03-28

Similar Documents

Publication Publication Date Title
JP7232599B2 (ja) 畳み込みニューラルネットワークのための刈り込みと再学習法
US11741342B2 (en) Resource-efficient neural architects
CN113168559B (zh) 机器学习模型的自动化生成
CN110546656B (zh) 前馈生成式神经网络
CN107729996B (zh) 零系数跳过卷积神经网络引擎
JP6876814B2 (ja) バッチ再正規化層
CN111178258B (zh) 一种图像识别的方法、系统、设备及可读存储介质
WO2023098544A1 (zh) 基于局部稀疏约束的结构化剪枝方法和装置
US12198032B2 (en) Electronic device and control method therefor
CN114267366B (zh) 通过离散表示学习进行语音降噪
CN113795880B (zh) 电子设备及其控制方法
CN111587441B (zh) 使用以比特值为条件的回归神经网络生成输出示例
WO2020118608A1 (zh) 一种反卷积神经网络的硬件加速方法、装置和电子设备
US11113578B1 (en) Learned model-based image rendering
JP2019087072A (ja) 処理装置、推論装置、学習装置、処理システム、処理方法、及び処理プログラム
CN108960425B (zh) 一种渲染模型训练方法、系统、设备、介质及渲染方法
CN117351299B (zh) 图像生成及模型训练方法、装置、设备和存储介质
US20190205757A1 (en) Model-free control for reinforcement learning agents
CN114186097A (zh) 用于训练模型的方法和装置
CN113139466B (zh) 基于单隐层神经网络的图像识别方法及相关设备
CN120047332B (zh) 一种结构扰动指导和一致性蒸馏融合的扩散模型图像生成方法及装置
KR102813612B1 (ko) 신경망 구조 설계를 위한 프루닝 방법 및 이를 위한 컴퓨팅 장치
CN119127189A (zh) 网页代码处理方法、装置、设备、存储介质和程序产品
CN113778375B (zh) 用于神经网络的增强型乘法累加设备
CN116822407A (zh) 流场模拟方法、装置、计算机设备、存储介质

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20210825

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20210825

RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20210825

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20220729

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20220802

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221021

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20230207

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20230220

R150 Certificate of patent or registration of utility model

Ref document number: 7232599

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250