JP7640552B2 - ニューラルネットワークのパラメータを符号化するための概念 - Google Patents

ニューラルネットワークのパラメータを符号化するための概念 Download PDF

Info

Publication number
JP7640552B2
JP7640552B2 JP2022538077A JP2022538077A JP7640552B2 JP 7640552 B2 JP7640552 B2 JP 7640552B2 JP 2022538077 A JP2022538077 A JP 2022538077A JP 2022538077 A JP2022538077 A JP 2022538077A JP 7640552 B2 JP7640552 B2 JP 7640552B2
Authority
JP
Japan
Prior art keywords
neural network
network parameters
quantization
current
reconstruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022538077A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023507502A (ja
Inventor
ポール ハーセ
ハイナー キルヒホッファー
ハイコ シュヴァルツ
デトレフ マルペ
トーマス ウィーガント
Original Assignee
フラウンホッファー－ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by フラウンホッファー－ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ filed Critical フラウンホッファー－ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ
Publication of JP2023507502A publication Critical patent/JP2023507502A/ja
Priority to JP2024179366A priority Critical patent/JP7783376B2/ja
Application granted granted Critical
Publication of JP7640552B2 publication Critical patent/JP7640552B2/ja
Priority to JP2025205537A priority patent/JP2026067854A/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2483—Traffic characterised by specific attributes, e.g. priority or QoS involving identification of individual flows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Neurology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Executing Machine-Instructions (AREA)
JP2022538077A 2019-12-20 2020-12-21 ニューラルネットワークのパラメータを符号化するための概念 Active JP7640552B2 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024179366A JP7783376B2 (ja) 2019-12-20 2024-10-11 ニューラルネットワークのパラメータを符号化するための概念
JP2025205537A JP2026067854A (ja) 2019-12-20 2025-11-27 ニューラルネットワークのパラメータを符号化するための概念

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP19218862.1 2019-12-20
EP19218862 2019-12-20
PCT/EP2020/087489 WO2021123438A1 (en) 2019-12-20 2020-12-21 Concepts for coding neural networks parameters

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2024179366A Division JP7783376B2 (ja) 2019-12-20 2024-10-11 ニューラルネットワークのパラメータを符号化するための概念

Publications (2)

Publication Number Publication Date
JP2023507502A JP2023507502A (ja) 2023-02-22
JP7640552B2 true JP7640552B2 (ja) 2025-03-05

Family

ID=69104239

Family Applications (3)

Application Number Title Priority Date Filing Date
JP2022538077A Active JP7640552B2 (ja) 2019-12-20 2020-12-21 ニューラルネットワークのパラメータを符号化するための概念
JP2024179366A Active JP7783376B2 (ja) 2019-12-20 2024-10-11 ニューラルネットワークのパラメータを符号化するための概念
JP2025205537A Pending JP2026067854A (ja) 2019-12-20 2025-11-27 ニューラルネットワークのパラメータを符号化するための概念

Family Applications After (2)

Application Number Title Priority Date Filing Date
JP2024179366A Active JP7783376B2 (ja) 2019-12-20 2024-10-11 ニューラルネットワークのパラメータを符号化するための概念
JP2025205537A Pending JP2026067854A (ja) 2019-12-20 2025-11-27 ニューラルネットワークのパラメータを符号化するための概念

Country Status (6)

Country Link
US (2) US20220393986A1 (https=)
EP (1) EP4078454A1 (https=)
JP (3) JP7640552B2 (https=)
KR (1) KR20220127261A (https=)
CN (1) CN115087988A (https=)
WO (1) WO2021123438A1 (https=)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
CN115699020A (zh) * 2020-06-05 2023-02-03 华为技术有限公司 用于神经网络的量化
WO2021255567A1 (en) * 2020-06-16 2021-12-23 Nokia Technologies Oy Guided probability model for compressed representation of neural networks
CN115720666A (zh) * 2020-06-25 2023-02-28 英迪股份有限公司 用于神经网络的压缩和训练的方法及装置
US20240056575A1 (en) * 2020-12-22 2024-02-15 Intellectual Discovery Co., Ltd. Deep learning-based image coding method and device
JP7325015B2 (ja) * 2021-03-24 2023-08-14 パナソニックＩｐマネジメント株式会社 量子化方法、量子化装置、及び、プログラム
US11909975B2 (en) * 2021-06-18 2024-02-20 Tencent America LLC Dependent scalar quantization with substitution in neural image compression
CN118140458A (zh) * 2021-10-13 2024-06-04 谷歌有限责任公司 量化的机器学习配置信息
WO2023131641A1 (en) * 2022-01-09 2023-07-13 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Concepts for encoding and decoding neural network parameters
WO2024013109A1 (en) * 2022-07-11 2024-01-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoder, decoder and methods for coding a data structure
KR20250047001A (ko) * 2023-09-27 2025-04-03 삼성전자주식회사 심층 신경망 모델을 포함하는 전자 장치 및 그 동작 방법

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019185769A1 (en) 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dependent quantization

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2006201490B2 (en) * 2005-04-19 2008-05-22 Samsung Electronics Co., Ltd. Method and apparatus for adaptively selecting context model for entropy coding
KR101158439B1 (ko) * 2005-07-08 2012-07-13 엘지전자 주식회사 영상 신호의 코딩정보를 압축/해제하기 위해 모델링하는 방법
US9584802B2 (en) * 2012-04-13 2017-02-28 Texas Instruments Incorporated Reducing context coded and bypass coded bins to improve context adaptive binary arithmetic coding (CABAC) throughput
US20140003488A1 (en) * 2012-06-30 2014-01-02 Research In Motion Limited Position-based context selection for greater-than-one flag decoding and encoding
US11451840B2 (en) * 2018-06-18 2022-09-20 Qualcomm Incorporated Trellis coded quantization coefficient coding
CN114761970A (zh) 2019-10-01 2022-07-15 弗劳恩霍夫应用研究促进协会 神经网络表示格式

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019185769A1 (en) 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dependent quantization

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Marta Karczewicz et al.，CE8-related: Sign context modelling and level mapping for TS residual coding，Joint Video Experts Team (JVET)，2019年03月21日，[JVET-N0455] (version 3)
Simon Wiedemann et al.，DeepCABAC: A universal compression algorithm for deep neural networks，arXiv，2019年07月27日，pp.1-18

Also Published As

Publication number Publication date
JP2025016517A (ja) 2025-02-04
JP2023507502A (ja) 2023-02-22
KR20220127261A (ko) 2022-09-19
EP4078454A1 (en) 2022-10-26
US20250343764A1 (en) 2025-11-06
CN115087988A (zh) 2022-09-20
JP2026067854A (ja) 2026-04-21
WO2021123438A1 (en) 2021-06-24
US20220393986A1 (en) 2022-12-08
JP7783376B2 (ja) 2025-12-09

Similar Documents

Publication Publication Date Title
JP7640552B2 (ja) ニューラルネットワークのパラメータを符号化するための概念
Kirchhoffer et al. Overview of the neural network compression and representation (NNR) standard
US20250278604A1 (en) Methods and apparatuses for compressing parameters of neural networks
JP2025513886A (ja) ニューラルネットワークの復号化パラメータを提供するためのデコーダ、エンコーダ、方法、及び並べ替えを使用するコンピュータプログラム
JP2025186542A (ja) 更新モデルを使用して、ニューラルネットワークパラメーターを復号化する装置、方法及びコンピュータープログラム、並びにニューラルネットワークパラメーターを符号化する装置、方法及びコンピュータープログラム
US20240364362A1 (en) Concepts for encoding and decoding neural network parameters
Farkash et al. Transform trellis coding of images at low bit rates

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221019

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20221019

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20231130

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20231212

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240312

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20240611

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20241011

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20241011

A911 Transfer to examiner for re-examination before appeal (zenchi)

Free format text: JAPANESE INTERMEDIATE CODE: A911

Effective date: 20241031

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250121

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250220

R150 Certificate of patent or registration of utility model

Ref document number: 7640552

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150