JP7575388B2 - 低変位ランクベースのディープニューラルネットワーク圧縮 - Google Patents

低変位ランクベースのディープニューラルネットワーク圧縮 Download PDF

Info

Publication number
JP7575388B2
JP7575388B2 JP2021548231A JP2021548231A JP7575388B2 JP 7575388 B2 JP7575388 B2 JP 7575388B2 JP 2021548231 A JP2021548231 A JP 2021548231A JP 2021548231 A JP2021548231 A JP 2021548231A JP 7575388 B2 JP7575388 B2 JP 7575388B2
Authority
JP
Japan
Prior art keywords
layer
output matrix
weights
outputs
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2021548231A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022525392A5 (enExample
JP2022525392A (ja
Inventor
ラケイプ，ファビアン
ジェイン，スワヤンブー
アミディ－ラッド，シャハブ
パパディミトリウ，ディミトリス
Original Assignee
インターデジタル ヴイシー ホールディングス， インコーポレイテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by インターデジタル ヴイシー ホールディングス， インコーポレイテッド filed Critical インターデジタル ヴイシー ホールディングス， インコーポレイテッド
Publication of JP2022525392A publication Critical patent/JP2022525392A/ja
Publication of JP2022525392A5 publication Critical patent/JP2022525392A5/ja
Application granted granted Critical
Publication of JP7575388B2 publication Critical patent/JP7575388B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Image Processing (AREA)
JP2021548231A 2019-03-15 2020-03-13 低変位ランクベースのディープニューラルネットワーク圧縮 Active JP7575388B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962818914P 2019-03-15 2019-03-15
US62/818,914 2019-03-15
PCT/US2020/022585 WO2020190696A1 (en) 2019-03-15 2020-03-13 Low displacement rank based deep neural network compression

Publications (3)

Publication Number Publication Date
JP2022525392A JP2022525392A (ja) 2022-05-13
JP2022525392A5 JP2022525392A5 (enExample) 2023-03-13
JP7575388B2 true JP7575388B2 (ja) 2024-10-29

Family

ID=70228824

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021548231A Active JP7575388B2 (ja) 2019-03-15 2020-03-13 低変位ランクベースのディープニューラルネットワーク圧縮

Country Status (6)

Country Link
US (1) US20220188633A1 (enExample)
EP (1) EP3939301A1 (enExample)
JP (1) JP7575388B2 (enExample)
CN (1) CN113574887B (enExample)
MX (1) MX2021011131A (enExample)
WO (1) WO2020190696A1 (enExample)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
US11700518B2 (en) * 2019-05-31 2023-07-11 Huawei Technologies Co., Ltd. Methods and systems for relaying feature-driven communications
US20210326710A1 (en) * 2020-04-16 2021-10-21 Tencent America LLC Neural network model compression
CN114698394A (zh) * 2020-10-29 2022-07-01 华为技术有限公司 一种基于神经网络模型的量化方法及其相关设备
US11818399B2 (en) * 2021-01-04 2023-11-14 Tencent America LLC Techniques for signaling neural network topology and parameters in the coded video stream
CN112836801A (zh) * 2021-02-03 2021-05-25 上海商汤智能科技有限公司 深度学习网络确定方法、装置、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016199330A1 (ja) 2015-06-12 2016-12-15 パナソニックＩｐマネジメント株式会社 画像符号化方法、画像復号方法、画像符号化装置および画像復号装置
US20180239992A1 (en) 2017-02-22 2018-08-23 Arm Limited Processing artificial neural network weights
WO2019008752A1 (ja) 2017-07-07 2019-01-10 三菱電機株式会社 データ処理装置、データ処理方法および圧縮データ

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100850729B1 (ko) * 2000-07-06 2008-08-06 더 트러스티스 오브 콜롬비아 유니버시티 인 더 시티 오브 뉴욕 데이터 해상도를 향상시키는 방법 및 장치
US7133568B2 (en) * 2000-08-04 2006-11-07 Nikitin Alexei V Method and apparatus for analysis of variables
US10515307B2 (en) * 2015-06-05 2019-12-24 Google Llc Compressed recurrent neural network models
US11321609B2 (en) * 2016-10-19 2022-05-03 Samsung Electronics Co., Ltd Method and apparatus for neural network quantization
US12079700B2 (en) * 2016-10-26 2024-09-03 Google Llc Structured orthogonal random features for kernel-based machine learning
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
JP6789894B2 (ja) * 2017-07-31 2020-11-25 株式会社東芝 ネットワーク係数圧縮装置、ネットワーク係数圧縮方法およびプログラム
EP3451293A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with multi-branch deep learning
CN107396124B (zh) * 2017-08-29 2019-09-20 南京大学 基于深度神经网络的视频压缩方法
EP3704638A1 (en) * 2017-10-30 2020-09-09 Fraunhofer Gesellschaft zur Förderung der Angewand Neural network representation
US11423259B1 (en) * 2017-12-12 2022-08-23 Amazon Technologies, Inc. Trained model approximation
WO2019115865A1 (en) * 2017-12-13 2019-06-20 Nokia Technologies Oy An apparatus, a method and a computer program for video coding and decoding
US11429849B2 (en) * 2018-05-11 2022-08-30 Intel Corporation Deep compressed network
KR20200115239A (ko) * 2019-03-26 2020-10-07 (주)인시그널 훈련된 심층 신경망의 압축 장치 및 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016199330A1 (ja) 2015-06-12 2016-12-15 パナソニックＩｐマネジメント株式会社 画像符号化方法、画像復号方法、画像符号化装置および画像復号装置
US20180239992A1 (en) 2017-02-22 2018-08-23 Arm Limited Processing artificial neural network weights
WO2019008752A1 (ja) 2017-07-07 2019-01-10 三菱電機株式会社 データ処理装置、データ処理方法および圧縮データ

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
"Use cases and requirements for compressed representation of neural networks"，N17740，[online], ISO/IEC JTC1/SC29/WG11，2018年07月20日，Pages 1-4，[令和6年1月15日検索], インターネット, <URL: https://www.mpeg.org/wp-content/uploads/mpeg_meetings/123_Ljubljana/w17740.zip> and <URL: https://www.mpeg.org/standards/Explorations/29/>.，(See document file "w17740.docx" in the zip file "w17740.zip".)
Anna T. Thomas, et al.，"Learning Compressed Transforms with Low Displacement Rank"，arXiv:1810.02309v3，version v3，[online], arXiv (Cornell University)，2019年01月01日，Pages 1-33，[令和6年6月7日検索], インターネット, <URL: https://arxiv.org/abs/1810.02309v3> and <URL: https://arxiv.org/pdf/1810.02309v3>.
Liang Zhao, et al.，"Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank"，arXiv:1703.00144v4，version v4，[online], arXiv (Cornell University)，2017年09月22日，Pages 1-13，[令和6年1月14日検索], インターネット, <URL: https://arxiv.org/abs/1703.00144v4> and <URL: https://arxiv.org/pdf/1703.00144v4.pdf>.
VICTOR Y. PAN, et al.，"INVERSION OF DISPLACEMENT OPERATORS"，SIAM J. MATRIX ANAL. APPL.，Vol.24, No.3，[online], Society for Industrial and Applied Mathematics，2003年01月23日，Pages 660-677，[令和6年1月14日検索], インターネット, <URL: http://comet.lehman.cuny.edu/vpan/pdf/38627.pdf>.
Zhiyun Lu, et al.，"LEARNING COMPACT RECURRENT NEURAL NETWORKS"，Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2016)，IEEE，2016年03月25日，Pages 5960-5964，ISBN: 978-1-4799-9988-0, <DOI: 10.1109/ICASSP.2016.7472821>.
武井 宏将，「初めてのディープラーニング オープンソース"Caffe"による演習付き」，第１版，日本，株式会社リックテレコム，2016年03月04日，第１１１～１１５，１２３頁，ISBN: 978-4-86594-022-0.
長尾 真（外７名）編，「岩波情報科学辞典」，第１刷，日本，株式会社岩波書店，1990年05月25日，第５９０頁，ISBN: 4-00-080074-4.

Also Published As

Publication number Publication date
CN113574887A (zh) 2021-10-29
US20220188633A1 (en) 2022-06-16
CN113574887B (zh) 2024-09-27
MX2021011131A (es) 2021-10-14
JP2022525392A (ja) 2022-05-13
WO2020190696A1 (en) 2020-09-24
EP3939301A1 (en) 2022-01-19

Similar Documents

Publication Publication Date Title
JP7575388B2 (ja) 低変位ランクベースのディープニューラルネットワーク圧縮
JP7794765B2 (ja) ディープニューラルネットワークを符号化／復号するためのシステム及び方法
JP7761735B2 (ja) 暗黙的多重変換選択の変換選択
JP2023543985A (ja) 多用途ビデオコーディングのためのテンプレートマッチング予測
TWI878283B (zh) 用於編碼及解碼深度神經網路之低等級及基於位移等級層之架構
KR20230025879A (ko) 신경 네트워크 기반 인트라 예측 모드에 대한 변환 프로세스의 적응
TWI908728B (zh) 編解碼深度神經網路權重張量之裝置及方法
JP2025535086A (ja) 暗黙的ニューラル表現の学習された辞書を使用する画像及びビデオ圧縮
JP7654407B2 (ja) 可変重みを使用する複数参照イントラ予測
KR20240072180A (ko) Isp 모드를 사용한 템플릿 기반 인트라 모드 도출(timd)의 확장
JP7578584B2 (ja) イントラ変換コード化及び広角イントラ予測の調和
EP4055824A1 (en) Deep intra prediction of an image block
JP2024513873A (ja) 切り替え可能な補間フィルタを用いる幾何学的分割
WO2020260110A1 (en) Hmvc for affine and sbtmvp motion vector prediciton modes
EP4675498A1 (en) Video specific dictionary learning for implicit neural compression
TW202420823A (zh) 使用彈性網路之深度特徵壓縮的熵調適

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230302

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230302

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240123

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240327

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240612

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240801

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20241007

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20241017

R150 Certificate of patent or registration of utility model

Ref document number: 7575388

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150