JP2018506785A - モデル圧縮および微調整 - Google Patents

モデル圧縮および微調整 Download PDF

Info

Publication number
JP2018506785A
JP2018506785A JP2017538296A JP2017538296A JP2018506785A JP 2018506785 A JP2018506785 A JP 2018506785A JP 2017538296 A JP2017538296 A JP 2017538296A JP 2017538296 A JP2017538296 A JP 2017538296A JP 2018506785 A JP2018506785 A JP 2018506785A
Authority
JP
Japan
Prior art keywords
compressed
layer
neural network
layers
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2017538296A
Other languages
English (en)
Japanese (ja)
Other versions
JP2018506785A5 (enExample
Inventor
アンナプレディー、ベンカタ・スリーカンタ・レッディ
ダイクマン、ダニエル・ヘンドリカス・フランシスカス
ジュリアン、デイビッド・ジョナサン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of JP2018506785A publication Critical patent/JP2018506785A/ja
Publication of JP2018506785A5 publication Critical patent/JP2018506785A5/ja
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Complex Calculations (AREA)
  • Feedback Control In General (AREA)
  • Aiming, Guidance, Guns With A Light Source, Armor, Camouflage, And Targets (AREA)
JP2017538296A 2015-01-22 2015-12-15 モデル圧縮および微調整 Pending JP2018506785A (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201562106608P 2015-01-22 2015-01-22
US62/106,608 2015-01-22
US14/846,579 2015-09-04
US14/846,579 US10223635B2 (en) 2015-01-22 2015-09-04 Model compression and fine-tuning
PCT/US2015/065783 WO2016118257A1 (en) 2015-01-22 2015-12-15 Model compression and fine-tuning

Publications (2)

Publication Number Publication Date
JP2018506785A true JP2018506785A (ja) 2018-03-08
JP2018506785A5 JP2018506785A5 (enExample) 2019-01-10

Family

ID=55085908

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2017538296A Pending JP2018506785A (ja) 2015-01-22 2015-12-15 モデル圧縮および微調整

Country Status (8)

Country Link
US (1) US10223635B2 (enExample)
EP (1) EP3248148A1 (enExample)
JP (1) JP2018506785A (enExample)
KR (1) KR20170106338A (enExample)
CN (1) CN107004157A (enExample)
BR (1) BR112017015560A2 (enExample)
TW (1) TW201627923A (enExample)
WO (1) WO2016118257A1 (enExample)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020107042A (ja) * 2018-12-27 2020-07-09 Kddi株式会社 学習モデル生成装置、学習モデル生成方法、及びプログラム
WO2021234967A1 (ja) * 2020-05-22 2021-11-25 日本電信電話株式会社 音声波形生成モデル学習装置、音声合成装置、それらの方法、およびプログラム
JP2022008571A (ja) * 2016-07-13 2022-01-13 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
JP2022533307A (ja) * 2019-11-22 2022-07-22 テンセント・アメリカ・エルエルシー ニューラルネットワークモデル圧縮のための量子化、適応ブロック分割、及びコードブック符号化の方法及び装置、並びにコンピュータープログラム
JP2022109807A (ja) * 2021-01-15 2022-07-28 富士通株式会社 情報処理装置、情報処理方法、および情報処理プログラム
JP2024509435A (ja) * 2021-03-03 2024-03-01 ベイジン ダジア インターネット インフォメーション テクノロジー カンパニー リミテッド 映像符号化のためにニューラルネットワークをハイブリッド訓練するための方法および装置

Families Citing this family (150)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9408551B2 (en) 2013-11-14 2016-08-09 Bardy Diagnostics, Inc. System and method for facilitating diagnosis of cardiac rhythm disorders with the aid of a digital computer
US10624551B2 (en) 2013-09-25 2020-04-21 Bardy Diagnostics, Inc. Insertable cardiac monitor for use in performing long term electrocardiographic monitoring
US10799137B2 (en) 2013-09-25 2020-10-13 Bardy Diagnostics, Inc. System and method for facilitating a cardiac rhythm disorder diagnosis with the aid of a digital computer
US10463269B2 (en) * 2013-09-25 2019-11-05 Bardy Diagnostics, Inc. System and method for machine-learning-based atrial fibrillation detection
US10806360B2 (en) 2013-09-25 2020-10-20 Bardy Diagnostics, Inc. Extended wear ambulatory electrocardiography and physiological sensor monitor
US10433748B2 (en) 2013-09-25 2019-10-08 Bardy Diagnostics, Inc. Extended wear electrocardiography and physiological sensor monitor
US9619660B1 (en) 2013-09-25 2017-04-11 Bardy Diagnostics, Inc. Computer-implemented system for secure physiological data collection and processing
US10820801B2 (en) 2013-09-25 2020-11-03 Bardy Diagnostics, Inc. Electrocardiography monitor configured for self-optimizing ECG data compression
US10736531B2 (en) 2013-09-25 2020-08-11 Bardy Diagnostics, Inc. Subcutaneous insertable cardiac monitor optimized for long term, low amplitude electrocardiographic data collection
US20190167139A1 (en) 2017-12-05 2019-06-06 Gust H. Bardy Subcutaneous P-Wave Centric Insertable Cardiac Monitor For Long Term Electrocardiographic Monitoring
US10433751B2 (en) 2013-09-25 2019-10-08 Bardy Diagnostics, Inc. System and method for facilitating a cardiac rhythm disorder diagnosis based on subcutaneous cardiac monitoring data
US9345414B1 (en) 2013-09-25 2016-05-24 Bardy Diagnostics, Inc. Method for providing dynamic gain over electrocardiographic data with the aid of a digital computer
US9953425B2 (en) 2014-07-30 2018-04-24 Adobe Systems Incorporated Learning image categorization using related attributes
US9536293B2 (en) * 2014-07-30 2017-01-03 Adobe Systems Incorporated Image assessment using deep convolutional neural networks
US10515301B2 (en) * 2015-04-17 2019-12-24 Microsoft Technology Licensing, Llc Small-footprint deep neural network
US11250335B2 (en) * 2015-10-26 2022-02-15 NetraDyne, Inc. Joint processing for embedded data inference
US20170132511A1 (en) * 2015-11-10 2017-05-11 Facebook, Inc. Systems and methods for utilizing compressed convolutional neural networks to perform media content processing
KR102100977B1 (ko) * 2016-02-03 2020-04-14 구글 엘엘씨 압축된 순환 신경망 모델
EP3427192A4 (en) * 2016-03-11 2019-03-27 Magic Leap, Inc. STRUCTURAL LEARNING IN NEURAL FOLDING NETWORKS
KR102805829B1 (ko) * 2016-04-15 2025-05-12 삼성전자주식회사 인터페이스 뉴럴 네트워크
US10290197B2 (en) * 2016-08-15 2019-05-14 Nec Corporation Mass transit surveillance camera system
EP3293682A1 (en) * 2016-09-13 2018-03-14 Alcatel Lucent Method and device for analyzing sensor data
US10748057B1 (en) * 2016-09-21 2020-08-18 X Development Llc Neural network modules
US10175980B2 (en) * 2016-10-27 2019-01-08 Google Llc Neural network compute tile
EP4220630A1 (en) 2016-11-03 2023-08-02 Samsung Electronics Co., Ltd. Electronic device and controlling method thereof
KR102631381B1 (ko) * 2016-11-07 2024-01-31 삼성전자주식회사 컨볼루션 신경망 처리 방법 및 장치
TWI634490B (zh) 2016-11-14 2018-09-01 美商耐能股份有限公司 卷積運算裝置及卷積運算方法
CN108073548B (zh) * 2016-11-14 2021-09-10 耐能股份有限公司 卷积运算装置及卷积运算方法
US11157814B2 (en) * 2016-11-15 2021-10-26 Google Llc Efficient convolutional neural networks and techniques to reduce associated computational costs
US10032256B1 (en) * 2016-11-18 2018-07-24 The Florida State University Research Foundation, Inc. System and method for image processing using automatically estimated tuning parameters
US10685285B2 (en) * 2016-11-23 2020-06-16 Microsoft Technology Licensing, Llc Mirror deep neural networks that regularize to linear networks
KR102879261B1 (ko) 2016-12-22 2025-10-31 삼성전자주식회사 컨볼루션 신경망 처리 방법 및 장치
CN108243216B (zh) * 2016-12-26 2020-02-14 华为技术有限公司 数据处理的方法、端侧设备、云侧设备与端云协同系统
CN108242046B (zh) * 2016-12-27 2022-02-18 阿里巴巴集团控股有限公司 图片处理方法及相关设备
CN108229673B (zh) * 2016-12-27 2021-02-26 北京市商汤科技开发有限公司 卷积神经网络的处理方法、装置和电子设备
WO2018120019A1 (zh) * 2016-12-30 2018-07-05 上海寒武纪信息科技有限公司 用于神经网络数据的压缩/解压缩的装置和系统
US10387751B2 (en) * 2017-01-12 2019-08-20 Arizona Board Of Regents On Behalf Of Arizona State University Methods, apparatuses, and systems for reconstruction-free image recognition from compressive sensors
US11195094B2 (en) * 2017-01-17 2021-12-07 Fujitsu Limited Neural network connection reduction
JP6820764B2 (ja) * 2017-02-28 2021-01-27 日本放送協会 音響モデル学習装置および音響モデル学習プログラム
US20180260695A1 (en) * 2017-03-07 2018-09-13 Qualcomm Incorporated Neural network compression via weak supervision
US10691886B2 (en) 2017-03-09 2020-06-23 Samsung Electronics Co., Ltd. Electronic apparatus for compressing language model, electronic apparatus for providing recommendation word and operation methods thereof
US10803378B2 (en) * 2017-03-15 2020-10-13 Samsung Electronics Co., Ltd System and method for designing efficient super resolution deep convolutional neural networks by cascade network training, cascade network trimming, and dilated convolutions
KR102415508B1 (ko) 2017-03-28 2022-07-01 삼성전자주식회사 컨볼루션 신경망 처리 방법 및 장치
US10902312B2 (en) * 2017-03-28 2021-01-26 Qualcomm Incorporated Tracking axes during model conversion
US11037330B2 (en) 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US10795836B2 (en) 2017-04-17 2020-10-06 Microsoft Technology Licensing, Llc Data processing performance enhancement for neural networks using a virtualized data iterator
US11164071B2 (en) * 2017-04-18 2021-11-02 Samsung Electronics Co., Ltd. Method and apparatus for reducing computational complexity of convolutional neural networks
US10497084B2 (en) 2017-04-24 2019-12-03 Intel Corporation Efficient sharing and compression expansion of data across processing systems
US20180314945A1 (en) * 2017-04-27 2018-11-01 Advanced Micro Devices, Inc. Graph matching for optimized deep network processing
CN109102074B (zh) * 2017-06-21 2021-06-01 上海寒武纪信息科技有限公司 一种训练装置
DE102017213247A1 (de) * 2017-06-30 2019-01-03 Conti Temic Microelectronic Gmbh Wissenstransfer zwischen verschiedenen Deep-Learning Architekturen
KR102153786B1 (ko) * 2017-07-20 2020-09-08 한국과학기술원 선택 유닛을 이용한 이미지 처리 방법 및 장치
CN107992329B (zh) * 2017-07-20 2021-05-11 上海寒武纪信息科技有限公司 一种计算方法及相关产品
US11676004B2 (en) * 2017-08-15 2023-06-13 Xilinx, Inc. Architecture optimized training of neural networks
WO2019033380A1 (en) * 2017-08-18 2019-02-21 Intel Corporation SLURRY OF NEURAL NETWORKS IN MACHINE LEARNING ENVIRONMENTS
JP2019036899A (ja) 2017-08-21 2019-03-07 株式会社東芝 情報処理装置、情報処理方法およびプログラム
EP3679524A4 (en) * 2017-09-05 2020-10-28 Panasonic Intellectual Property Corporation of America EXECUTION PROCESS, EXECUTION DEVICE, LEARNING PROCESS, LEARNING DEVICE AND DEEP NEURONAL NETWORK PROGRAM
US12210958B2 (en) 2017-09-21 2025-01-28 Qualcomm Incorporated Compression of sparse deep convolutional network weights
US11093832B2 (en) 2017-10-19 2021-08-17 International Business Machines Corporation Pruning redundant neurons and kernels of deep convolutional neural networks
KR102727052B1 (ko) * 2017-10-23 2024-11-06 삼성전자주식회사 뉴럴 네트워크에서 파라미터를 처리하는 방법 및 장치
US10726335B2 (en) 2017-10-26 2020-07-28 Uber Technologies, Inc. Generating compressed representation neural networks having high degree of accuracy
WO2019086104A1 (en) * 2017-10-30 2019-05-09 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Neural network representation
US11164078B2 (en) 2017-11-08 2021-11-02 International Business Machines Corporation Model matching and learning rate selection for fine tuning
US11663476B2 (en) 2017-12-15 2023-05-30 Electronics And Telecommunications Research Institute Method and device for providing compression and transmission of training parameters in distributed processing environment
CN109978150A (zh) * 2017-12-27 2019-07-05 北京中科寒武纪科技有限公司 神经网络处理器板卡及相关产品
KR101864412B1 (ko) * 2017-12-28 2018-06-04 (주)휴톰 학습용 데이터 관리방법, 장치 및 프로그램
CN109993298B (zh) * 2017-12-29 2023-08-08 百度在线网络技术（北京）有限公司 用于压缩神经网络的方法和装置
CN108062780B (zh) * 2017-12-29 2019-08-09 百度在线网络技术（北京）有限公司 图像压缩方法和装置
CN109993292B (zh) 2017-12-30 2020-08-04 中科寒武纪科技股份有限公司 集成电路芯片装置及相关产品
WO2019129302A1 (zh) 2017-12-30 2019-07-04 北京中科寒武纪科技有限公司 集成电路芯片装置及相关产品
CN109993291B (zh) * 2017-12-30 2020-07-07 中科寒武纪科技股份有限公司 集成电路芯片装置及相关产品
US10546393B2 (en) * 2017-12-30 2020-01-28 Intel Corporation Compression in machine learning and deep learning processing
CN109993290B (zh) 2017-12-30 2021-08-06 中科寒武纪科技股份有限公司 集成电路芯片装置及相关产品
CN109993289B (zh) 2017-12-30 2021-09-21 中科寒武纪科技股份有限公司 集成电路芯片装置及相关产品
US11961000B2 (en) * 2018-01-22 2024-04-16 Qualcomm Incorporated Lossy layer compression for dynamic scaling of deep neural network processing
US11586924B2 (en) * 2018-01-23 2023-02-21 Qualcomm Incorporated Determining layer ranks for compression of deep networks
US10841577B2 (en) 2018-02-08 2020-11-17 Electronics And Telecommunications Research Institute Method and apparatus for video encoding and video decoding based on neural network
US10516415B2 (en) * 2018-02-09 2019-12-24 Kneron, Inc. Method of compressing convolution parameters, convolution operation chip and system
CN108415888A (zh) * 2018-02-12 2018-08-17 苏州思必驰信息科技有限公司 用于神经网络语言模型的压缩方法和系统
JP6811736B2 (ja) * 2018-03-12 2021-01-13 Kddi株式会社 情報処理装置、情報処理方法、及びプログラム
US11468302B2 (en) * 2018-03-13 2022-10-11 Recogni Inc. Efficient convolutional engine
US11468316B2 (en) * 2018-03-13 2022-10-11 Recogni Inc. Cluster compression for compressing weights in neural networks
US11461869B2 (en) 2018-03-14 2022-10-04 Samsung Electronics Co., Ltd. Slab based memory management for machine learning training
JP7228961B2 (ja) * 2018-04-02 2023-02-27 キヤノン株式会社 ニューラルネットワークの学習装置およびその制御方法
US11019355B2 (en) 2018-04-03 2021-05-25 Electronics And Telecommunications Research Institute Inter-prediction method and apparatus using reference frame generated based on deep learning
US11238346B2 (en) 2018-04-25 2022-02-01 Qualcomm Incorproated Learning a truncation rank of singular value decomposed matrices representing weight tensors in neural networks
WO2019216514A1 (en) 2018-05-09 2019-11-14 Samsung Electronics Co., Ltd. Electronic apparatus for compression and decompression of data and compression method thereof
US11562208B2 (en) 2018-05-17 2023-01-24 Qualcomm Incorporated Continuous relaxation of quantization for discretized deep neural networks
CN108764487B (zh) * 2018-05-29 2022-07-08 北京百度网讯科技有限公司 用于生成模型的方法和装置、用于识别信息的方法和装置
KR102199484B1 (ko) * 2018-06-01 2021-01-06 아주대학교산학협력단 대용량 네트워크를 압축하기 위한 방법 및 장치
KR102096388B1 (ko) * 2018-06-05 2020-04-06 네이버 주식회사 모바일 환경에서 실시간 추론이 가능한 ｄｎｎ 구성을 위한 최적화 기법
US20190378013A1 (en) * 2018-06-06 2019-12-12 Kneron Inc. Self-tuning model compression methodology for reconfiguring deep neural network and electronic device
KR102695519B1 (ko) * 2018-07-02 2024-08-14 삼성전자주식회사 영상 모델 구축 장치 및 방법
CN112437930A (zh) * 2018-07-12 2021-03-02 华为技术有限公司 以熟练的推理速度和功耗，生成神经网络的压缩表示
CN109101999B (zh) * 2018-07-16 2021-06-25 华东师范大学 基于支持向量机的协神经网络可信决策方法
KR102728476B1 (ko) * 2018-07-19 2024-11-12 삼성전자주식회사 전자 장치 및 그의 제어 방법
CN110826706B (zh) 2018-08-10 2023-10-03 北京百度网讯科技有限公司 用于神经网络的数据处理方法和装置
KR102159953B1 (ko) * 2018-08-13 2020-09-25 인천대학교 산학협력단 딥러닝 모델을 통한 추론 서비스를 제공할 때, 적어도 하나의 프로세서의 성능을 제어하는 전자 장치 및 그의 동작 방법
CN110874636B (zh) * 2018-09-04 2023-06-30 杭州海康威视数字技术股份有限公司 一种神经网络模型压缩方法、装置和计算机设备
CN109344731B (zh) * 2018-09-10 2022-05-03 电子科技大学 基于神经网络的轻量级的人脸识别方法
US11588499B2 (en) * 2018-11-05 2023-02-21 Samsung Electronics Co., Ltd. Lossless compression of neural network weights
CN111291882A (zh) * 2018-12-06 2020-06-16 北京百度网讯科技有限公司 一种模型转换的方法、装置、设备和计算机存储介质
KR102796861B1 (ko) * 2018-12-10 2025-04-17 삼성전자주식회사 인공 신경망을 압축하기 위한 장치 및 방법
US12353971B1 (en) * 2018-12-13 2025-07-08 Amazon Technologies, Inc. Machine learning model adaptation via segment replacement and student-teacher training
CN109766993B (zh) * 2018-12-13 2020-12-18 浙江大学 一种适合硬件的卷积神经网络压缩方法
US11263323B2 (en) 2018-12-19 2022-03-01 Google Llc Systems and methods for increasing robustness of machine-learned models and other software systems against adversarial attacks
CN111353591B (zh) * 2018-12-20 2024-08-20 中科寒武纪科技股份有限公司 一种计算装置及相关产品
CN111382848B (zh) * 2018-12-27 2024-08-23 中科寒武纪科技股份有限公司 一种计算装置及相关产品
US12333428B2 (en) * 2019-02-27 2025-06-17 Huawei Technologies Co., Ltd. Neural network model processing method and apparatus
CN109886394B (zh) * 2019-03-05 2021-06-18 北京时代拓灵科技有限公司 嵌入式设备中三值神经网络权值处理方法及装置
US11444845B1 (en) * 2019-03-05 2022-09-13 Amazon Technologies, Inc. Processing requests using compressed and complete machine learning models
WO2020199056A1 (zh) * 2019-03-30 2020-10-08 华为技术有限公司 一种数据处理方法、服务器和可读介质
CN110111234B (zh) * 2019-04-11 2023-12-15 上海集成电路研发中心有限公司 一种基于神经网络的图像处理系统架构
KR102774162B1 (ko) * 2019-05-16 2025-03-04 삼성전자주식회사 전자 장치 및 이의 제어 방법
US20220237454A1 (en) * 2019-05-21 2022-07-28 Interdigital Vc Holding, Inc. Linear neural reconstruction for deep neural network compression
US10716089B1 (en) * 2019-06-03 2020-07-14 Mapsted Corp. Deployment of trained neural network based RSS fingerprint dataset
US11696681B2 (en) 2019-07-03 2023-07-11 Bardy Diagnostics Inc. Configurable hardware platform for physiological monitoring of a living body
US11116451B2 (en) 2019-07-03 2021-09-14 Bardy Diagnostics, Inc. Subcutaneous P-wave centric insertable cardiac monitor with energy harvesting capabilities
US11096579B2 (en) 2019-07-03 2021-08-24 Bardy Diagnostics, Inc. System and method for remote ECG data streaming in real-time
CN112308197B (zh) * 2019-07-26 2024-04-09 杭州海康威视数字技术股份有限公司 一种卷积神经网络的压缩方法、装置及电子设备
KR102147912B1 (ko) 2019-08-13 2020-08-25 삼성전자주식회사 프로세서 칩 및 그 제어 방법들
US11551054B2 (en) * 2019-08-27 2023-01-10 International Business Machines Corporation System-aware selective quantization for performance optimized distributed deep learning
US12175359B2 (en) 2019-09-03 2024-12-24 International Business Machines Corporation Machine learning hardware having reduced precision parameter components for efficient parameter update
US12217158B2 (en) 2019-09-03 2025-02-04 International Business Machines Corporation Neural network circuitry having floating point format with asymmetric range
US11604647B2 (en) 2019-09-03 2023-03-14 International Business Machines Corporation Mixed precision capable hardware for tuning a machine learning model
WO2021064292A1 (en) * 2019-10-02 2021-04-08 Nokia Technologies Oy High-level syntax for priority signaling in neural network compression
US11620435B2 (en) 2019-10-10 2023-04-04 International Business Machines Corporation Domain specific model compression
KR102660728B1 (ko) 2019-11-22 2024-04-26 텐센트 아메리카 엘엘씨 신경망 모델 압축을 위한 3차원(3d)-트리 코딩을 위한 방법 및 장치
US11234024B2 (en) 2019-11-26 2022-01-25 Tencent America LLC Method and apparatus for three-dimensional (3D)-tree coding for neural network model compression
US11245903B2 (en) 2019-11-22 2022-02-08 Tencent America LLC Method and apparatus for quantization, adaptive block partitioning and codebook coding for neural network model compression
RU2734579C1 (ru) * 2019-12-30 2020-10-20 Автономная некоммерческая образовательная организация высшего образования "Сколковский институт науки и технологий" Система сжатия искусственных нейронных сетей на основе итеративного применения тензорных аппроксимаций
US12443830B2 (en) 2020-01-03 2025-10-14 International Business Machines Corporation Compressed weight distribution in networks of neural processors
US12072806B2 (en) 2020-01-22 2024-08-27 Alibaba Group Holding Limited Compression and decompression module in a cache controller for reducing off-chip data traffic
CN113537485B (zh) * 2020-04-15 2024-09-06 北京金山数字娱乐科技有限公司 一种神经网络模型的压缩方法及装置
KR20210136706A (ko) 2020-05-08 2021-11-17 삼성전자주식회사 전자 장치 및 이의 제어 방법
TWI737300B (zh) * 2020-05-15 2021-08-21 國立陽明交通大學 深度神經網路壓縮的方法
US20210397963A1 (en) * 2020-06-17 2021-12-23 Tencent America LLC Method and apparatus for neural network model compression with micro-structured weight pruning and weight unification
KR20220032861A (ko) * 2020-09-08 2022-03-15 삼성전자주식회사 하드웨어에서의 성능을 고려한 뉴럴 아키텍처 서치 방법 빛 장치
US20220094713A1 (en) * 2020-09-21 2022-03-24 Sophos Limited Malicious message detection
CN112132278A (zh) * 2020-09-23 2020-12-25 平安科技（深圳）有限公司 模型压缩方法、装置、计算机设备及存储介质
US11462033B2 (en) 2020-09-30 2022-10-04 Wipro Limited Method and system for performing classification of real-time input sample using compressed classification model
US11335056B1 (en) * 2020-11-30 2022-05-17 Nvidia Corporation Real-time rendering with implicit shapes
EP4377894A4 (en) * 2021-07-29 2025-06-18 Beijing Dajia Internet Information Technology Co., Ltd. Network-based image filtering for video encoding
CN114114363B (zh) * 2021-11-08 2025-06-10 北京邮电大学 基于时频和卷积神经网络的机会信号感知方法、系统及机会信号定位方法
US11972108B2 (en) 2021-11-15 2024-04-30 International Business Machines Corporation Parameter redundancy reduction method
WO2023101472A1 (ko) * 2021-12-01 2023-06-08 주식회사 딥엑스 프로그램된 활성화 함수 실행 유닛을 포함하는 신경 프로세싱 유닛
KR102651560B1 (ko) * 2021-12-01 2024-03-26 주식회사 딥엑스 프로그램된 활성화 함수 실행 유닛을 포함하는 신경 프로세싱 유닛
EP4540711A1 (en) 2022-07-11 2025-04-23 Huawei Cloud Computing Technologies Co., Ltd. Performant collaborative transfer learning between cloud storage and cloud compute
US20240160889A1 (en) * 2022-11-14 2024-05-16 Arm Limited Neural network processing
US20250077887A1 (en) * 2022-12-12 2025-03-06 Rakuten Mobile, Inc. Collaborative training with compressed transmissions

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH064504A (ja) * 1992-06-18 1994-01-14 Matsushita Electric Ind Co Ltd ニューラルネットワーク回路
JPH07146852A (ja) * 1993-11-24 1995-06-06 Ricoh Co Ltd ニューラルネットワークの構造簡略化方法
JP2006163808A (ja) * 2004-12-07 2006-06-22 Fuji Electric Holdings Co Ltd ニューラルネットワークの構造
US20140019388A1 (en) * 2012-07-13 2014-01-16 International Business Machines Corporation System and method for low-rank matrix factorization for deep belief network training with high-dimensional output targets

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5058179A (en) 1990-01-31 1991-10-15 At&T Bell Laboratories Hierarchical constrained automatic learning network for character recognition
US5376962A (en) 1993-03-31 1994-12-27 Panasonic Technologies, Inc. Neural network video image processor
US6269351B1 (en) * 1999-03-31 2001-07-31 Dryken Technologies, Inc. Method and system for training an artificial neural network
WO2003015026A1 (en) 2001-08-10 2003-02-20 Saffron Technology, Inc. Artificial neurons including weights that define maximal projections
EP1444600A1 (en) * 2001-11-16 2004-08-11 Yuan Yan Chen Pausible neural network with supervised and unsupervised cluster analysis
US20040199482A1 (en) * 2002-04-15 2004-10-07 Wilson Scott B. Systems and methods for automatic and incremental learning of patient states from biomedical signals
CN101183873B (zh) * 2007-12-11 2011-09-28 广州中珩电子科技有限公司 一种基于bp神经网络的嵌入式系统数据压缩解压缩方法
US9565439B2 (en) 2009-10-15 2017-02-07 Nbcuniversal Media, Llc System and method for enhancing data compression using dynamic learning and control
CN101795344B (zh) * 2010-03-02 2013-03-27 北京大学 数字全息图像压缩、解码方法及系统、传输方法及系统
KR20120040015A (ko) 2010-10-18 2012-04-26 한국전자통신연구원 벡터 분류기 및 그것의 벡터 분류 방법
US10068170B2 (en) * 2013-09-23 2018-09-04 Oracle International Corporation Minimizing global error in an artificial neural network
US9400955B2 (en) * 2013-12-13 2016-07-26 Amazon Technologies, Inc. Reducing dynamic range of low-rank decomposition matrices

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH064504A (ja) * 1992-06-18 1994-01-14 Matsushita Electric Ind Co Ltd ニューラルネットワーク回路
JPH07146852A (ja) * 1993-11-24 1995-06-06 Ricoh Co Ltd ニューラルネットワークの構造簡略化方法
JP2006163808A (ja) * 2004-12-07 2006-06-22 Fuji Electric Holdings Co Ltd ニューラルネットワークの構造
US20140019388A1 (en) * 2012-07-13 2014-01-16 International Business Machines Corporation System and method for low-rank matrix factorization for deep belief network training with high-dimensional output targets

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7244598B2 (ja) 2016-07-13 2023-03-22 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
US12450466B2 (en) 2016-07-13 2025-10-21 Google Llc Superpixel methods for convolutional neural networks
JP2022008571A (ja) * 2016-07-13 2022-01-13 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
JP2025016553A (ja) * 2016-07-13 2025-02-04 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
JP7578742B2 (ja) 2016-07-13 2024-11-06 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
JP2023078247A (ja) * 2016-07-13 2023-06-06 グーグル エルエルシー 畳み込みニューラルネットワークのためのスーパーピクセル法
JP7042210B2 (ja) 2018-12-27 2022-03-25 Kddi株式会社 学習モデル生成装置、学習モデル生成方法、及びプログラム
JP2020107042A (ja) * 2018-12-27 2020-07-09 Kddi株式会社 学習モデル生成装置、学習モデル生成方法、及びプログラム
JP7337950B2 (ja) 2019-11-22 2023-09-04 テンセント・アメリカ・エルエルシー ニューラルネットワークモデル圧縮のための量子化、適応ブロック分割、及びコードブック符号化の方法及び装置、並びにコンピュータープログラム
JP2022533307A (ja) * 2019-11-22 2022-07-22 テンセント・アメリカ・エルエルシー ニューラルネットワークモデル圧縮のための量子化、適応ブロック分割、及びコードブック符号化の方法及び装置、並びにコンピュータープログラム
WO2021234967A1 (ja) * 2020-05-22 2021-11-25 日本電信電話株式会社 音声波形生成モデル学習装置、音声合成装置、それらの方法、およびプログラム
JP2022109807A (ja) * 2021-01-15 2022-07-28 富士通株式会社 情報処理装置、情報処理方法、および情報処理プログラム
JP7673412B2 (ja) 2021-01-15 2025-05-09 富士通株式会社 情報処理装置、情報処理方法、および情報処理プログラム
JP2024509435A (ja) * 2021-03-03 2024-03-01 ベイジン ダジア インターネット インフォメーション テクノロジー カンパニー リミテッド 映像符号化のためにニューラルネットワークをハイブリッド訓練するための方法および装置
JP7783289B2 (ja) 2021-03-03 2025-12-09 ベイジン ダジア インターネット インフォメーション テクノロジー カンパニー リミテッド 映像符号化のためにニューラルネットワークをハイブリッド訓練するための方法および装置

Also Published As

Publication number Publication date
WO2016118257A1 (en) 2016-07-28
US10223635B2 (en) 2019-03-05
BR112017015560A2 (pt) 2018-03-13
CN107004157A (zh) 2017-08-01
KR20170106338A (ko) 2017-09-20
US20160217369A1 (en) 2016-07-28
TW201627923A (zh) 2016-08-01
EP3248148A1 (en) 2017-11-29

Similar Documents

Publication Publication Date Title
JP2018506785A (ja) モデル圧縮および微調整
JP6732795B2 (ja) 深層畳み込みネットワークにおいて画像解像度を低減すること
JP6862426B2 (ja) トレーニングされた機械学習モデルのパフォーマンスを改善するための方法
JP6869948B2 (ja) ニューラルネットワークにおける転移学習
JP7037478B2 (ja) 分類のための強制的なスパース性
US20160283864A1 (en) Sequential image sampling and storage of fine-tuned features
US20170228646A1 (en) Spiking multi-layer perceptron
CN107533669A (zh) 滤波器特异性作为用于神经网络的训练准则
CN107209873A (zh) 用于深度卷积网络的超参数选择
CN108140142A (zh) 选择性反向传播
US20190228311A1 (en) Determining layer ranks for compression of deep networks
EP4162405A1 (en) Federated mixture models
US10902312B2 (en) Tracking axes during model conversion
JP2024509862A (ja) ビデオ処理における改善された時間的一貫性のための効率的なテスト時間適応
US20240086699A1 (en) Hardware-aware federated learning
US20240232585A1 (en) Channel-guided nested loop transformation and scalar replacement
TW202512021A (zh) 人工智慧（ａｉ）加速裝置中基於轉接器的高效上下文切換

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171005

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171004

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20181120

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20181120

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20191129

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20200107

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20200811