CN109472357A - 用于卷积神经网络的修剪和再训练方法 - Google Patents

用于卷积神经网络的修剪和再训练方法 Download PDF

Info

Publication number
CN109472357A
CN109472357A CN201811051626.XA CN201811051626A CN109472357A CN 109472357 A CN109472357 A CN 109472357A CN 201811051626 A CN201811051626 A CN 201811051626A CN 109472357 A CN109472357 A CN 109472357A
Authority
CN
China
Prior art keywords
trimming
neural network
ratio
processing units
accuracy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811051626.XA
Other languages
English (en)
Chinese (zh)
Inventor
王新
林尚宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivante Corp
Original Assignee
Vivante Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivante Corp filed Critical Vivante Corp
Publication of CN109472357A publication Critical patent/CN109472357A/zh
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
CN201811051626.XA 2017-09-08 2018-09-10 用于卷积神经网络的修剪和再训练方法 Pending CN109472357A (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/699,438 2017-09-08
US15/699,438 US11200495B2 (en) 2017-09-08 2017-09-08 Pruning and retraining method for a convolution neural network

Publications (1)

Publication Number Publication Date
CN109472357A true CN109472357A (zh) 2019-03-15

Family

ID=63490311

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811051626.XA Pending CN109472357A (zh) 2017-09-08 2018-09-10 用于卷积神经网络的修剪和再训练方法

Country Status (5)

Country Link
US (1) US11200495B2 (https=)
EP (1) EP3454262A1 (https=)
JP (1) JP7232599B2 (https=)
KR (1) KR102793134B1 (https=)
CN (1) CN109472357A (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112699990A (zh) * 2019-10-22 2021-04-23 杭州海康威视数字技术股份有限公司 神经网络模型训练方法、装置及电子设备
WO2021248409A1 (en) * 2020-06-11 2021-12-16 Alibaba Group Holding Limited Pruning hardware unit for training neural network
CN114467098A (zh) * 2019-10-11 2022-05-10 高通股份有限公司 用于深度神经网络的经学习阈值修剪

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11734568B2 (en) * 2018-02-14 2023-08-22 Google Llc Systems and methods for modification of neural networks based on estimated edge utility
KR20200023238A (ko) * 2018-08-23 2020-03-04 삼성전자주식회사 딥러닝 모델을 생성하는 방법 및 시스템
CN111738401A (zh) * 2019-03-25 2020-10-02 北京三星通信技术研究有限公司 模型优化方法、分组压缩方法、相应的装置、设备
US12373696B2 (en) * 2019-06-21 2025-07-29 Samsung Electronics Co., Ltd. Neural network hardware accelerator system with zero-skipping and hierarchical structured pruning methods
US20210089921A1 (en) * 2019-09-25 2021-03-25 Nvidia Corporation Transfer learning for neural networks
CN111061909B (zh) * 2019-11-22 2023-11-28 腾讯音乐娱乐科技（深圳）有限公司 一种伴奏分类方法和装置
US11935271B2 (en) * 2020-01-10 2024-03-19 Tencent America LLC Neural network model compression with selective structured weight unification
US11562235B2 (en) 2020-02-21 2023-01-24 International Business Machines Corporation Activation function computation for neural networks
CN111582456B (zh) * 2020-05-11 2023-12-15 抖音视界有限公司 用于生成网络模型信息的方法、装置、设备和介质
CN111553169B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
CN111539224B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
US12488250B2 (en) 2020-11-02 2025-12-02 International Business Machines Corporation Weight repetition on RPU crossbar arrays
US20220156574A1 (en) * 2020-11-19 2022-05-19 Kabushiki Kaisha Toshiba Methods and systems for remote training of a machine learning model
US20220318633A1 (en) * 2021-03-26 2022-10-06 Qualcomm Incorporated Model compression using pruning quantization and knowledge distillation
US20220405571A1 (en) * 2021-06-16 2022-12-22 Microsoft Technology Licensing, Llc Sparsifying narrow data formats for neural networks
CN113469326B (zh) * 2021-06-24 2024-04-02 上海寒武纪信息科技有限公司 在神经网络模型中执行剪枝优化的集成电路装置及板卡
US12524673B2 (en) * 2021-07-16 2026-01-13 Industry-Academic Cooperation Foundation, Yonsei University Multitask distributed learning system and method based on lottery ticket neural network
JP7666289B2 (ja) 2021-10-25 2025-04-22 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
US12591778B2 (en) 2021-11-17 2026-03-31 Samsung Electronics Co., Ltd. System and method for torque-based structured pruning for deep neural networks
CN115115045B (zh) * 2021-12-24 2026-02-06 杭州海康威视数字技术股份有限公司 一种模型剪枝方法、装置及电子设备
CN114462582A (zh) * 2022-02-25 2022-05-10 腾讯科技（深圳）有限公司 基于卷积神经网络模型的数据处理方法及装置、设备
JP7782319B2 (ja) 2022-03-04 2025-12-09 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
CN120562503A (zh) * 2022-03-31 2025-08-29 支付宝(杭州)信息技术有限公司 模型剪枝方法、装置和计算机设备
CN114863243B (zh) * 2022-04-28 2024-12-17 国家电网有限公司大数据中心 一种模型的数据遗忘方法、装置、设备及存储介质
US20240013050A1 (en) * 2022-07-05 2024-01-11 International Business Machines Corporation Packing machine learning models using pruning and permutation
US12524999B2 (en) * 2022-09-30 2026-01-13 Samsung Electronics Co., Ltd. Generating images with small objects for training a pruned super-resolution network

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5288645A (en) * 1992-09-04 1994-02-22 Mtm Engineering, Inc. Hydrogen evolution analyzer
JP5234085B2 (ja) 2010-11-11 2013-07-10 富士電機株式会社 ニューラルネットワークの学習方法
US10373054B2 (en) 2015-04-19 2019-08-06 International Business Machines Corporation Annealed dropout training of neural networks
US11423311B2 (en) * 2015-06-04 2022-08-23 Samsung Electronics Co., Ltd. Automatic tuning of artificial neural networks
US10832136B2 (en) * 2016-05-18 2020-11-10 Nec Corporation Passive pruning of filters in a convolutional neural network
US10832123B2 (en) * 2016-08-12 2020-11-10 Xilinx Technology Beijing Limited Compression of deep neural networks with proper use of mask
US10984308B2 (en) * 2016-08-12 2021-04-20 Xilinx Technology Beijing Limited Compression method for deep neural networks with load balance
US10762426B2 (en) * 2016-08-12 2020-09-01 Beijing Deephi Intelligent Technology Co., Ltd. Multi-iteration compression for deep neural networks
JP6729455B2 (ja) 2017-03-15 2020-07-22 株式会社島津製作所 分析データ解析装置及び分析データ解析方法
CN107688850B (zh) * 2017-08-08 2021-04-13 赛灵思公司 一种深度神经网络压缩方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114467098A (zh) * 2019-10-11 2022-05-10 高通股份有限公司 用于深度神经网络的经学习阈值修剪
CN112699990A (zh) * 2019-10-22 2021-04-23 杭州海康威视数字技术股份有限公司 神经网络模型训练方法、装置及电子设备
CN112699990B (zh) * 2019-10-22 2024-06-07 杭州海康威视数字技术股份有限公司 神经网络模型训练方法、装置及电子设备
WO2021248409A1 (en) * 2020-06-11 2021-12-16 Alibaba Group Holding Limited Pruning hardware unit for training neural network

Also Published As

Publication number Publication date
US20190080238A1 (en) 2019-03-14
KR20190028320A (ko) 2019-03-18
JP7232599B2 (ja) 2023-03-03
EP3454262A1 (en) 2019-03-13
KR102793134B1 (ko) 2025-04-07
US11200495B2 (en) 2021-12-14
JP2019049977A (ja) 2019-03-28

Similar Documents

Publication Publication Date Title
CN109472357A (zh) 用于卷积神经网络的修剪和再训练方法
US11069345B2 (en) Speech recognition using convolutional neural networks
US12361305B2 (en) Neural architecture search for convolutional neural networks
US20240135955A1 (en) Generating audio using neural networks
CN110546656B (zh) 前馈生成式神经网络
US20210342670A1 (en) Processing sequences using convolutional neural networks
JP2019533257A (ja) ニューラルアーキテクチャ検索
CN110291540A (zh) 批再归一化层
KR102199066B1 (ko) 미세먼지 측정기를 통해 산출된 미세먼지 측정 데이터의 오류를 기계학습을 기반으로 보정할 수 있는 미세먼지 측정 데이터 보정 장치
CN110751941B (zh) 语音合成模型的生成方法、装置、设备及存储介质
CN117114074A (zh) 神经网络模型的训练、数据处理方法、装置以及介质
HK40029473B (zh) 对话生成方法、装置、计算机设备和存储介质

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination