KR102793134B1 - 컨볼루션 신경망의 가지-치기 및 재훈련 방법 - Google Patents

컨볼루션 신경망의 가지-치기 및 재훈련 방법 Download PDF

Info

Publication number
KR102793134B1
KR102793134B1 KR1020180105880A KR20180105880A KR102793134B1 KR 102793134 B1 KR102793134 B1 KR 102793134B1 KR 1020180105880 A KR1020180105880 A KR 1020180105880A KR 20180105880 A KR20180105880 A KR 20180105880A KR 102793134 B1 KR102793134 B1 KR 102793134B1
Authority
KR
South Korea
Prior art keywords
neural network
pruning
ratio
connections
accuracy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020180105880A
Other languages
English (en)
Korean (ko)
Other versions
KR20190028320A (ko
Inventor
왕 신
린 상-헝
Original Assignee
비반테 코포레이션
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 비반테 코포레이션 filed Critical 비반테 코포레이션
Publication of KR20190028320A publication Critical patent/KR20190028320A/ko
Application granted granted Critical
Publication of KR102793134B1 publication Critical patent/KR102793134B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
KR1020180105880A 2017-09-08 2018-09-05 컨볼루션 신경망의 가지-치기 및 재훈련 방법 Active KR102793134B1 (ko)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/699,438 2017-09-08
US15/699,438 US11200495B2 (en) 2017-09-08 2017-09-08 Pruning and retraining method for a convolution neural network

Publications (2)

Publication Number Publication Date
KR20190028320A KR20190028320A (ko) 2019-03-18
KR102793134B1 true KR102793134B1 (ko) 2025-04-07

Family

ID=63490311

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020180105880A Active KR102793134B1 (ko) 2017-09-08 2018-09-05 컨볼루션 신경망의 가지-치기 및 재훈련 방법

Country Status (5)

Country Link
US (1) US11200495B2 (https=)
EP (1) EP3454262A1 (https=)
JP (1) JP7232599B2 (https=)
KR (1) KR102793134B1 (https=)
CN (1) CN109472357A (https=)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11734568B2 (en) * 2018-02-14 2023-08-22 Google Llc Systems and methods for modification of neural networks based on estimated edge utility
KR20200023238A (ko) * 2018-08-23 2020-03-04 삼성전자주식회사 딥러닝 모델을 생성하는 방법 및 시스템
CN111738401A (zh) * 2019-03-25 2020-10-02 北京三星通信技术研究有限公司 模型优化方法、分组压缩方法、相应的装置、设备
US12373696B2 (en) * 2019-06-21 2025-07-29 Samsung Electronics Co., Ltd. Neural network hardware accelerator system with zero-skipping and hierarchical structured pruning methods
US20210089921A1 (en) * 2019-09-25 2021-03-25 Nvidia Corporation Transfer learning for neural networks
US11704571B2 (en) * 2019-10-11 2023-07-18 Qualcomm Incorporated Learned threshold pruning for deep neural networks
CN112699990B (zh) * 2019-10-22 2024-06-07 杭州海康威视数字技术股份有限公司 神经网络模型训练方法、装置及电子设备
CN111061909B (zh) * 2019-11-22 2023-11-28 腾讯音乐娱乐科技（深圳）有限公司 一种伴奏分类方法和装置
US11935271B2 (en) * 2020-01-10 2024-03-19 Tencent America LLC Neural network model compression with selective structured weight unification
US11562235B2 (en) 2020-02-21 2023-01-24 International Business Machines Corporation Activation function computation for neural networks
CN111582456B (zh) * 2020-05-11 2023-12-15 抖音视界有限公司 用于生成网络模型信息的方法、装置、设备和介质
WO2021248409A1 (en) * 2020-06-11 2021-12-16 Alibaba Group Holding Limited Pruning hardware unit for training neural network
CN111553169B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
CN111539224B (zh) * 2020-06-25 2023-08-25 北京百度网讯科技有限公司 语义理解模型的剪枝方法、装置、电子设备和存储介质
US12488250B2 (en) 2020-11-02 2025-12-02 International Business Machines Corporation Weight repetition on RPU crossbar arrays
US20220156574A1 (en) * 2020-11-19 2022-05-19 Kabushiki Kaisha Toshiba Methods and systems for remote training of a machine learning model
US20220318633A1 (en) * 2021-03-26 2022-10-06 Qualcomm Incorporated Model compression using pruning quantization and knowledge distillation
US20220405571A1 (en) * 2021-06-16 2022-12-22 Microsoft Technology Licensing, Llc Sparsifying narrow data formats for neural networks
CN113469326B (zh) * 2021-06-24 2024-04-02 上海寒武纪信息科技有限公司 在神经网络模型中执行剪枝优化的集成电路装置及板卡
US12524673B2 (en) * 2021-07-16 2026-01-13 Industry-Academic Cooperation Foundation, Yonsei University Multitask distributed learning system and method based on lottery ticket neural network
JP7666289B2 (ja) 2021-10-25 2025-04-22 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
US12591778B2 (en) 2021-11-17 2026-03-31 Samsung Electronics Co., Ltd. System and method for torque-based structured pruning for deep neural networks
CN115115045B (zh) * 2021-12-24 2026-02-06 杭州海康威视数字技术股份有限公司 一种模型剪枝方法、装置及电子设备
CN114462582A (zh) * 2022-02-25 2022-05-10 腾讯科技（深圳）有限公司 基于卷积神经网络模型的数据处理方法及装置、设备
JP7782319B2 (ja) 2022-03-04 2025-12-09 富士通株式会社 機械学習プログラム、機械学習方法、及び、情報処理装置
CN120562503A (zh) * 2022-03-31 2025-08-29 支付宝(杭州)信息技术有限公司 模型剪枝方法、装置和计算机设备
CN114863243B (zh) * 2022-04-28 2024-12-17 国家电网有限公司大数据中心 一种模型的数据遗忘方法、装置、设备及存储介质
US20240013050A1 (en) * 2022-07-05 2024-01-11 International Business Machines Corporation Packing machine learning models using pruning and permutation
US12524999B2 (en) * 2022-09-30 2026-01-13 Samsung Electronics Co., Ltd. Generating images with small objects for training a pruned super-resolution network

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5288645A (en) * 1992-09-04 1994-02-22 Mtm Engineering, Inc. Hydrogen evolution analyzer
JP5234085B2 (ja) 2010-11-11 2013-07-10 富士電機株式会社 ニューラルネットワークの学習方法
US10373054B2 (en) 2015-04-19 2019-08-06 International Business Machines Corporation Annealed dropout training of neural networks
US11423311B2 (en) * 2015-06-04 2022-08-23 Samsung Electronics Co., Ltd. Automatic tuning of artificial neural networks
US10832136B2 (en) * 2016-05-18 2020-11-10 Nec Corporation Passive pruning of filters in a convolutional neural network
US10832123B2 (en) * 2016-08-12 2020-11-10 Xilinx Technology Beijing Limited Compression of deep neural networks with proper use of mask
US10984308B2 (en) * 2016-08-12 2021-04-20 Xilinx Technology Beijing Limited Compression method for deep neural networks with load balance
US10762426B2 (en) * 2016-08-12 2020-09-01 Beijing Deephi Intelligent Technology Co., Ltd. Multi-iteration compression for deep neural networks
JP6729455B2 (ja) 2017-03-15 2020-07-22 株式会社島津製作所 分析データ解析装置及び分析データ解析方法
CN107688850B (zh) * 2017-08-08 2021-04-13 赛灵思公司 一种深度神经网络压缩方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Song Han et al., "Learning bothWeights and Connections for Efficient Neural Networks," arXiv:1506.02626v3 [cs.NE] 30 Oct 2015 (2015.10.30.)*

Also Published As

Publication number Publication date
US20190080238A1 (en) 2019-03-14
KR20190028320A (ko) 2019-03-18
JP7232599B2 (ja) 2023-03-03
EP3454262A1 (en) 2019-03-13
US11200495B2 (en) 2021-12-14
CN109472357A (zh) 2019-03-15
JP2019049977A (ja) 2019-03-28

Similar Documents

Publication Publication Date Title
KR102793134B1 (ko) 컨볼루션 신경망의 가지-치기 및 재훈련 방법
US20240242125A1 (en) Learning data augmentation policies
CN116363261B (zh) 图像编辑模型的训练方法、图像编辑方法和装置
US10521729B2 (en) Neural architecture search for convolutional neural networks
CN110546656B (zh) 前馈生成式神经网络
CN111105375B (zh) 图像生成方法及其模型训练方法、装置及电子设备
US20180046898A1 (en) Zero Coefficient Skipping Convolution Neural Network Engine
CN111178258B (zh) 一种图像识别的方法、系统、设备及可读存储介质
KR102898294B1 (ko) 전자 장치 및 이의 제어 방법
JP2020502625A (ja) ニューラルネットワークを使用したテキストシーケンスの処理
CN111587441B (zh) 使用以比特值为条件的回归神经网络生成输出示例
US12393840B2 (en) Granular neural network architecture search over low-level primitives
WO2020118608A1 (zh) 一种反卷积神经网络的硬件加速方法、装置和电子设备
CN108960425B (zh) 一种渲染模型训练方法、系统、设备、介质及渲染方法
CN117351299B (zh) 图像生成及模型训练方法、装置、设备和存储介质
US20220101145A1 (en) Training energy-based variational autoencoders
KR102813612B1 (ko) 신경망 구조 설계를 위한 프루닝 방법 및 이를 위한 컴퓨팅 장치
CN119127189A (zh) 网页代码处理方法、装置、设备、存储介质和程序产品
CN119047523A (zh) 模型训练方法、装置、计算机设备、存储介质和程序产品
CN117151193A (zh) 模型压缩方法、装置、电子设备和可读存储介质
CN116822407A (zh) 流场模拟方法、装置、计算机设备、存储介质
Lee et al. Fast on-device learning framework for single-image super-resolution
KR20210152957A (ko) 신경망을 위한 향상된 곱셈 누산 디바이스
CN113298248B (zh) 一种针对神经网络模型的处理方法、装置以及电子设备
KR102865872B1 (ko) 곱셈/누산 연산을 수행하기 위한 디바이스

Legal Events

Date Code Title Description
PA0109 Patent application

Patent event code: PA01091R01D

Comment text: Patent Application

Patent event date: 20180905

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20210820

Comment text: Request for Examination of Application

Patent event code: PA02011R01I

Patent event date: 20180905

Comment text: Patent Application

PE0902 Notice of grounds for rejection

Comment text: Notification of reason for refusal

Patent event date: 20240624

Patent event code: PE09021S01D

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20250206

GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20250403

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20250403

End annual number: 3

Start annual number: 1

PG1601 Publication of registration