KR102964609B1 - 신경망 매개변수 코딩에 대한 개념 - Google Patents

신경망 매개변수 코딩에 대한 개념

Info

Publication number
KR102964609B1
KR102964609B1 KR1020227025245A KR20227025245A KR102964609B1 KR 102964609 B1 KR102964609 B1 KR 102964609B1 KR 1020227025245 A KR1020227025245 A KR 1020227025245A KR 20227025245 A KR20227025245 A KR 20227025245A KR 102964609 B1 KR102964609 B1 KR 102964609B1
Authority
KR
South Korea
Prior art keywords
neural network
delete delete
network parameter
quantization
restoration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020227025245A
Other languages
English (en)
Korean (ko)
Other versions
KR20220127261A (ko
Inventor
파울 하제
하이너 키르흐호퍼
하이코 슈바르츠
데틀레프 마르페
토마스 비간트
Original Assignee
프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. filed Critical 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베.
Priority to KR1020267014703A priority Critical patent/KR20260070586A/ko
Publication of KR20220127261A publication Critical patent/KR20220127261A/ko
Application granted granted Critical
Publication of KR102964609B1 publication Critical patent/KR102964609B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2483—Traffic characterised by specific attributes, e.g. priority or QoS involving identification of individual flows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Neurology (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Executing Machine-Instructions (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
KR1020227025245A 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념 Active KR102964609B1 (ko)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020267014703A KR20260070586A (ko) 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP19218862.1 2019-12-20
EP19218862 2019-12-20
PCT/EP2020/087489 WO2021123438A1 (en) 2019-12-20 2020-12-21 Concepts for coding neural networks parameters

Related Child Applications (1)

Application Number Title Priority Date Filing Date
KR1020267014703A Division KR20260070586A (ko) 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념

Publications (2)

Publication Number Publication Date
KR20220127261A KR20220127261A (ko) 2022-09-19
KR102964609B1 true KR102964609B1 (ko) 2026-05-14

Family

ID=69104239

Family Applications (2)

Application Number Title Priority Date Filing Date
KR1020227025245A Active KR102964609B1 (ko) 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념
KR1020267014703A Pending KR20260070586A (ko) 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념

Family Applications After (1)

Application Number Title Priority Date Filing Date
KR1020267014703A Pending KR20260070586A (ko) 2019-12-20 2020-12-21 신경망 매개변수 코딩에 대한 개념

Country Status (6)

Country Link
US (2) US20220393986A1 (cg-RX-API-DMAC7.html)
EP (1) EP4078454A1 (cg-RX-API-DMAC7.html)
JP (3) JP7640552B2 (cg-RX-API-DMAC7.html)
KR (2) KR102964609B1 (cg-RX-API-DMAC7.html)
CN (1) CN115087988A (cg-RX-API-DMAC7.html)
WO (1) WO2021123438A1 (cg-RX-API-DMAC7.html)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11037330B2 (en) * 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
WO2021246913A1 (en) * 2020-06-05 2021-12-09 Huawei Technologies Co., Ltd. Quantization for neural networks
US12363310B2 (en) * 2020-06-16 2025-07-15 Nokia Technologies Oy Guided probability model for compressed representation of neural networks
WO2021261950A1 (ko) * 2020-06-25 2021-12-30 인텔렉추얼디스커버리 주식회사 뉴럴 네트워크의 압축 및 학습을 위한 방법 및 장치
WO2022074007A1 (en) * 2020-10-06 2022-04-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Arithmetic encoder for arithmetically encoding and arithmetic decoder for arithmetically decoding a sequence of information values, methods for arithmetically encoding and decoding a sequence of information values and computer program for implementing these methods
WO2022139438A1 (ko) * 2020-12-22 2022-06-30 인텔렉추얼디스커버리 주식회사 딥러닝 기반 이미지 코딩 방법 및 장치
JP7325015B2 (ja) * 2021-03-24 2023-08-14 パナソニックIpマネジメント株式会社 量子化方法、量子化装置、及び、プログラム
US11909975B2 (en) * 2021-06-18 2024-02-20 Tencent America LLC Dependent scalar quantization with substitution in neural image compression
WO2023064419A1 (en) * 2021-10-13 2023-04-20 Google Inc. Quantized machine-learning configuration information
KR20240132484A (ko) * 2022-01-09 2024-09-03 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 신경 네트워크 파라미터를 인코딩 및 디코딩하는 컨셉
WO2024013109A1 (en) * 2022-07-11 2024-01-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoder, decoder and methods for coding a data structure
KR20250047001A (ko) * 2023-09-27 2025-04-03 삼성전자주식회사 심층 신경망 모델을 포함하는 전자 장치 및 그 동작 방법

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019185769A1 (en) * 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dependent quantization
US20190373264A1 (en) 2018-05-29 2019-12-05 Qualcomm Incorporated Bandwidth compression for neural network systems

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2006201490B2 (en) * 2005-04-19 2008-05-22 Samsung Electronics Co., Ltd. Method and apparatus for adaptively selecting context model for entropy coding
US7894523B2 (en) * 2005-09-05 2011-02-22 Lg Electronics Inc. Method for modeling coding information of a video signal for compressing/decompressing coding information
US9584802B2 (en) * 2012-04-13 2017-02-28 Texas Instruments Incorporated Reducing context coded and bypass coded bins to improve context adaptive binary arithmetic coding (CABAC) throughput
US20140003488A1 (en) * 2012-06-30 2014-01-02 Research In Motion Limited Position-based context selection for greater-than-one flag decoding and encoding
US11451840B2 (en) * 2018-06-18 2022-09-20 Qualcomm Incorporated Trellis coded quantization coefficient coding
TW202601463A (zh) 2019-10-01 2026-01-01 弗勞恩霍夫爾協會 用於編/解碼神經網路參數之設備與方法、及相關資料串流與電腦程式

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019185769A1 (en) * 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dependent quantization
US20190373264A1 (en) 2018-05-29 2019-12-05 Qualcomm Incorporated Bandwidth compression for neural network systems

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
S. Wiedemann 등. "DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks". arXiv:1907.11900v1*

Also Published As

Publication number Publication date
US20250343764A1 (en) 2025-11-06
JP2026067854A (ja) 2026-04-21
KR20220127261A (ko) 2022-09-19
JP2025016517A (ja) 2025-02-04
EP4078454A1 (en) 2022-10-26
WO2021123438A1 (en) 2021-06-24
KR20260070586A (ko) 2026-05-19
CN115087988A (zh) 2022-09-20
JP7783376B2 (ja) 2025-12-09
US20220393986A1 (en) 2022-12-08
JP2023507502A (ja) 2023-02-22
JP7640552B2 (ja) 2025-03-05

Similar Documents

Publication Publication Date Title
US20250343764A1 (en) Concepts for Coding Neural Networks Parameters
Kirchhoffer et al. Overview of the neural network compression and representation (NNR) standard
TWI748201B (zh) 適於相依純量量化之變換係數熵寫碼技術
TWI813126B (zh) 相依量化技術
US20220004844A1 (en) Methods and apparatuses for compressing parameters of neural networks
KR20210065155A (ko) 종속 스칼라 양자화를 사용하거나 이와 조합하는 데에 적합한 변환 계수의 효율적인 코딩
US20250045973A1 (en) Decoder for providing decoded Parameters of a Neural Network, Encoder, Methods and Computer Programs using a Reordering
US12532032B2 (en) Coding concept for a sequence of information values
JP2025186542A (ja) 更新モデルを使用して、ニューラルネットワークパラメーターを復号化する装置、方法及びコンピュータープログラム、並びにニューラルネットワークパラメーターを符号化する装置、方法及びコンピュータープログラム
US20240364362A1 (en) Concepts for encoding and decoding neural network parameters

Legal Events

Date Code Title Description
E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

P22-X000 Classification modified

St.27 status event code: A-2-2-P10-P22-nap-X000

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E601 Decision to refuse application
PE0601 Decision on rejection of patent

St.27 status event code: N-2-6-B10-B15-exm-PE0601

T11-X000 Administrative time limit extension requested

St.27 status event code: U-3-3-T10-T11-oth-X000

T13-X000 Administrative time limit extension granted

St.27 status event code: U-3-3-T10-T13-oth-X000

T13-X000 Administrative time limit extension granted

St.27 status event code: U-3-3-T10-T13-oth-X000

E13 Pre-grant limitation requested

Free format text: ST27 STATUS EVENT CODE: A-2-3-E10-E13-LIM-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-NAP-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

F13 Ip right granted in full following pre-grant review

Free format text: ST27 STATUS EVENT CODE: A-3-4-F10-F13-REX-PX0701 (AS PROVIDED BY THE NATIONAL OFFICE)

PX0701 Decision of registration after re-examination

St.27 status event code: A-3-4-F10-F13-rex-PX0701

A16 Divisional, continuation or continuation in part application filed

Free format text: ST27 STATUS EVENT CODE: A-0-1-A10-A16-DIV-PA0104 (AS PROVIDED BY THE NATIONAL OFFICE)

PA0104 Divisional application for international application

St.27 status event code: A-0-1-A10-A16-div-PA0104

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U12-oth-PR1002

Fee payment year number: 1

U12 Designation fee paid

Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U12-OTH-PR1002 (AS PROVIDED BY THE NATIONAL OFFICE)

Year of fee payment: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

Q13 Ip right document published

Free format text: ST27 STATUS EVENT CODE: A-4-4-Q10-Q13-NAP-PG1601 (AS PROVIDED BY THE NATIONAL OFFICE)