CN113574887B - 基于低位移秩的深度神经网络压缩 - Google Patents

基于低位移秩的深度神经网络压缩 Download PDF

Info

Publication number
CN113574887B
CN113574887B CN202080021701.7A CN202080021701A CN113574887B CN 113574887 B CN113574887 B CN 113574887B CN 202080021701 A CN202080021701 A CN 202080021701A CN 113574887 B CN113574887 B CN 113574887B
Authority
CN
China
Prior art keywords
layer
neural network
deep neural
output
weights
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080021701.7A
Other languages
English (en)
Chinese (zh)
Other versions
CN113574887A (zh
Inventor
F·拉卡佩
S·杰恩
S·哈米迪拉德
D·帕帕季米特里乌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
InterDigital VC Holdings Inc
Original Assignee
InterDigital VC Holdings Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by InterDigital VC Holdings Inc filed Critical InterDigital VC Holdings Inc
Publication of CN113574887A publication Critical patent/CN113574887A/zh
Application granted granted Critical
Publication of CN113574887B publication Critical patent/CN113574887B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Image Processing (AREA)
CN202080021701.7A 2019-03-15 2020-03-13 基于低位移秩的深度神经网络压缩 Active CN113574887B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962818914P 2019-03-15 2019-03-15
US62/818,914 2019-03-15
PCT/US2020/022585 WO2020190696A1 (en) 2019-03-15 2020-03-13 Low displacement rank based deep neural network compression

Publications (2)

Publication Number Publication Date
CN113574887A CN113574887A (zh) 2021-10-29
CN113574887B true CN113574887B (zh) 2024-09-27

Family

ID=70228824

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080021701.7A Active CN113574887B (zh) 2019-03-15 2020-03-13 基于低位移秩的深度神经网络压缩

Country Status (6)

Country Link
US (1) US20220188633A1 (enExample)
EP (1) EP3939301A1 (enExample)
JP (1) JP7575388B2 (enExample)
CN (1) CN113574887B (enExample)
MX (1) MX2021011131A (enExample)
WO (1) WO2020190696A1 (enExample)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US11700518B2 (en) * 2019-05-31 2023-07-11 Huawei Technologies Co., Ltd. Methods and systems for relaying feature-driven communications
US20210326710A1 (en) * 2020-04-16 2021-10-21 Tencent America LLC Neural network model compression
CN114698394A (zh) * 2020-10-29 2022-07-01 华为技术有限公司 一种基于神经网络模型的量化方法及其相关设备
US11818399B2 (en) * 2021-01-04 2023-11-14 Tencent America LLC Techniques for signaling neural network topology and parameters in the coded video stream
CN112836801A (zh) * 2021-02-03 2021-05-25 上海商汤智能科技有限公司 深度学习网络确定方法、装置、电子设备及存储介质

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145940A (zh) * 2016-03-01 2017-09-08 谷歌公司 压缩的递归神经网络模型

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100850729B1 (ko) * 2000-07-06 2008-08-06 더 트러스티스 오브 콜롬비아 유니버시티 인 더 시티 오브 뉴욕 데이터 해상도를 향상시키는 방법 및 장치
US7133568B2 (en) * 2000-08-04 2006-11-07 Nikitin Alexei V Method and apparatus for analysis of variables
CN107736027B (zh) * 2015-06-12 2021-06-01 松下知识产权经营株式会社 图像编码方法、图像解码方法、图像编码装置及图像解码装置
US11321609B2 (en) * 2016-10-19 2022-05-03 Samsung Electronics Co., Ltd Method and apparatus for neural network quantization
US12079700B2 (en) * 2016-10-26 2024-09-03 Google Llc Structured orthogonal random features for kernel-based machine learning
US10599935B2 (en) 2017-02-22 2020-03-24 Arm Limited Processing artificial neural network weights
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
BR112019027664B1 (pt) 2017-07-07 2023-12-19 Mitsubishi Electric Corporation Dispositivo e método de processamento de dados, e, meio de armazenamento
JP6789894B2 (ja) * 2017-07-31 2020-11-25 株式会社東芝 ネットワーク係数圧縮装置、ネットワーク係数圧縮方法およびプログラム
EP3451293A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with multi-branch deep learning
CN107396124B (zh) * 2017-08-29 2019-09-20 南京大学 基于深度神经网络的视频压缩方法
EP3704638A1 (en) * 2017-10-30 2020-09-09 Fraunhofer Gesellschaft zur Förderung der Angewand Neural network representation
US11423259B1 (en) * 2017-12-12 2022-08-23 Amazon Technologies, Inc. Trained model approximation
WO2019115865A1 (en) * 2017-12-13 2019-06-20 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
US11429849B2 (en) * 2018-05-11 2022-08-30 Intel Corporation Deep compressed network
KR20200115239A (ko) * 2019-03-26 2020-10-07 (주)인시그널 훈련된 심층 신경망의 압축 장치 및 방법

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107145940A (zh) * 2016-03-01 2017-09-08 谷歌公司 压缩的递归神经网络模型

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank;Liang Zhao ET AL;《arXiv》;20170331;第1-13页 *

Also Published As

Publication number Publication date
CN113574887A (zh) 2021-10-29
US20220188633A1 (en) 2022-06-16
JP7575388B2 (ja) 2024-10-29
MX2021011131A (es) 2021-10-14
JP2022525392A (ja) 2022-05-13
WO2020190696A1 (en) 2020-09-24
EP3939301A1 (en) 2022-01-19

Similar Documents

Publication Publication Date Title
CN113574887B (zh) 基于低位移秩的深度神经网络压缩
JP7794765B2 (ja) ディープニューラルネットワークを符号化／復号するためのシステム及び方法
CN113950834B (zh) 用于隐式多变换选择的变换选择
CN113728637B (zh) 用于对深度神经网络的基于低秩和位移秩的层进行编码和解码的框架
CN117256142A (zh) 用于使用基于人工神经网络的工具对图像和视频进行编码/解码的方法和装置
JP7813731B2 (ja) ディープニューラルネットワークを符号化／復号するためのシステム及び方法
CN113994348A (zh) 对于深度神经网络压缩的线性神经重建
TWI908728B (zh) 編解碼深度神經網路權重張量之裝置及方法
EP4602557A1 (en) Image and video compression using learned dictionary of implicit neural representations
CN114731396A (zh) 图像块的深度帧内预测
JP2024513873A (ja) 切り替え可能な補間フィルタを用いる幾何学的分割
US12581080B2 (en) System and method for compression of data stream
EP3994623A1 (en) Systems and methods for encoding a deep neural network
EP4675498A1 (en) Video specific dictionary learning for implicit neural compression
CN120153659A (zh) 使用灵活网络的深度特征压缩的熵适应

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant