KR102964609B1 - 신경망 매개변수 코딩에 대한 개념 - Google Patents
신경망 매개변수 코딩에 대한 개념Info
- Publication number
- KR102964609B1 KR102964609B1 KR1020227025245A KR20227025245A KR102964609B1 KR 102964609 B1 KR102964609 B1 KR 102964609B1 KR 1020227025245 A KR1020227025245 A KR 1020227025245A KR 20227025245 A KR20227025245 A KR 20227025245A KR 102964609 B1 KR102964609 B1 KR 102964609B1
- Authority
- KR
- South Korea
- Prior art keywords
- neural network
- delete delete
- network parameter
- quantization
- restoration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2483—Traffic characterised by specific attributes, e.g. priority or QoS involving identification of individual flows
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Neurology (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Executing Machine-Instructions (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020267014703A KR20260070586A (ko) | 2019-12-20 | 2020-12-21 | 신경망 매개변수 코딩에 대한 개념 |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP19218862.1 | 2019-12-20 | ||
| EP19218862 | 2019-12-20 | ||
| PCT/EP2020/087489 WO2021123438A1 (en) | 2019-12-20 | 2020-12-21 | Concepts for coding neural networks parameters |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020267014703A Division KR20260070586A (ko) | 2019-12-20 | 2020-12-21 | 신경망 매개변수 코딩에 대한 개념 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20220127261A KR20220127261A (ko) | 2022-09-19 |
| KR102964609B1 true KR102964609B1 (ko) | 2026-05-14 |
Family
ID=69104239
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020227025245A Active KR102964609B1 (ko) | 2019-12-20 | 2020-12-21 | 신경망 매개변수 코딩에 대한 개념 |
| KR1020267014703A Pending KR20260070586A (ko) | 2019-12-20 | 2020-12-21 | 신경망 매개변수 코딩에 대한 개념 |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020267014703A Pending KR20260070586A (ko) | 2019-12-20 | 2020-12-21 | 신경망 매개변수 코딩에 대한 개념 |
Country Status (6)
| Country | Link |
|---|---|
| US (2) | US20220393986A1 (cg-RX-API-DMAC7.html) |
| EP (1) | EP4078454A1 (cg-RX-API-DMAC7.html) |
| JP (3) | JP7640552B2 (cg-RX-API-DMAC7.html) |
| KR (2) | KR102964609B1 (cg-RX-API-DMAC7.html) |
| CN (1) | CN115087988A (cg-RX-API-DMAC7.html) |
| WO (1) | WO2021123438A1 (cg-RX-API-DMAC7.html) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11037330B2 (en) * | 2017-04-08 | 2021-06-15 | Intel Corporation | Low rank matrix compression |
| WO2021246913A1 (en) * | 2020-06-05 | 2021-12-09 | Huawei Technologies Co., Ltd. | Quantization for neural networks |
| US12363310B2 (en) * | 2020-06-16 | 2025-07-15 | Nokia Technologies Oy | Guided probability model for compressed representation of neural networks |
| WO2021261950A1 (ko) * | 2020-06-25 | 2021-12-30 | 인텔렉추얼디스커버리 주식회사 | 뉴럴 네트워크의 압축 및 학습을 위한 방법 및 장치 |
| WO2022074007A1 (en) * | 2020-10-06 | 2022-04-14 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Arithmetic encoder for arithmetically encoding and arithmetic decoder for arithmetically decoding a sequence of information values, methods for arithmetically encoding and decoding a sequence of information values and computer program for implementing these methods |
| WO2022139438A1 (ko) * | 2020-12-22 | 2022-06-30 | 인텔렉추얼디스커버리 주식회사 | 딥러닝 기반 이미지 코딩 방법 및 장치 |
| JP7325015B2 (ja) * | 2021-03-24 | 2023-08-14 | パナソニックIpマネジメント株式会社 | 量子化方法、量子化装置、及び、プログラム |
| US11909975B2 (en) * | 2021-06-18 | 2024-02-20 | Tencent America LLC | Dependent scalar quantization with substitution in neural image compression |
| WO2023064419A1 (en) * | 2021-10-13 | 2023-04-20 | Google Inc. | Quantized machine-learning configuration information |
| KR20240132484A (ko) * | 2022-01-09 | 2024-09-03 | 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. | 신경 네트워크 파라미터를 인코딩 및 디코딩하는 컨셉 |
| WO2024013109A1 (en) * | 2022-07-11 | 2024-01-18 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Encoder, decoder and methods for coding a data structure |
| KR20250047001A (ko) * | 2023-09-27 | 2025-04-03 | 삼성전자주식회사 | 심층 신경망 모델을 포함하는 전자 장치 및 그 동작 방법 |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019185769A1 (en) * | 2018-03-29 | 2019-10-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Dependent quantization |
| US20190373264A1 (en) | 2018-05-29 | 2019-12-05 | Qualcomm Incorporated | Bandwidth compression for neural network systems |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2006201490B2 (en) * | 2005-04-19 | 2008-05-22 | Samsung Electronics Co., Ltd. | Method and apparatus for adaptively selecting context model for entropy coding |
| US7894523B2 (en) * | 2005-09-05 | 2011-02-22 | Lg Electronics Inc. | Method for modeling coding information of a video signal for compressing/decompressing coding information |
| US9584802B2 (en) * | 2012-04-13 | 2017-02-28 | Texas Instruments Incorporated | Reducing context coded and bypass coded bins to improve context adaptive binary arithmetic coding (CABAC) throughput |
| US20140003488A1 (en) * | 2012-06-30 | 2014-01-02 | Research In Motion Limited | Position-based context selection for greater-than-one flag decoding and encoding |
| US11451840B2 (en) * | 2018-06-18 | 2022-09-20 | Qualcomm Incorporated | Trellis coded quantization coefficient coding |
| TW202601463A (zh) | 2019-10-01 | 2026-01-01 | 弗勞恩霍夫爾協會 | 用於編/解碼神經網路參數之設備與方法、及相關資料串流與電腦程式 |
-
2020
- 2020-12-21 EP EP20830246.3A patent/EP4078454A1/en active Pending
- 2020-12-21 KR KR1020227025245A patent/KR102964609B1/ko active Active
- 2020-12-21 JP JP2022538077A patent/JP7640552B2/ja active Active
- 2020-12-21 KR KR1020267014703A patent/KR20260070586A/ko active Pending
- 2020-12-21 WO PCT/EP2020/087489 patent/WO2021123438A1/en not_active Ceased
- 2020-12-21 CN CN202080094840.2A patent/CN115087988A/zh active Pending
-
2022
- 2022-06-17 US US17/843,772 patent/US20220393986A1/en active Pending
-
2024
- 2024-10-11 JP JP2024179366A patent/JP7783376B2/ja active Active
-
2025
- 2025-07-11 US US19/267,146 patent/US20250343764A1/en active Pending
- 2025-11-27 JP JP2025205537A patent/JP2026067854A/ja active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019185769A1 (en) * | 2018-03-29 | 2019-10-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Dependent quantization |
| US20190373264A1 (en) | 2018-05-29 | 2019-12-05 | Qualcomm Incorporated | Bandwidth compression for neural network systems |
Non-Patent Citations (1)
| Title |
|---|
| S. Wiedemann 등. "DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks". arXiv:1907.11900v1* |
Also Published As
| Publication number | Publication date |
|---|---|
| US20250343764A1 (en) | 2025-11-06 |
| JP2026067854A (ja) | 2026-04-21 |
| KR20220127261A (ko) | 2022-09-19 |
| JP2025016517A (ja) | 2025-02-04 |
| EP4078454A1 (en) | 2022-10-26 |
| WO2021123438A1 (en) | 2021-06-24 |
| KR20260070586A (ko) | 2026-05-19 |
| CN115087988A (zh) | 2022-09-20 |
| JP7783376B2 (ja) | 2025-12-09 |
| US20220393986A1 (en) | 2022-12-08 |
| JP2023507502A (ja) | 2023-02-22 |
| JP7640552B2 (ja) | 2025-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20250343764A1 (en) | Concepts for Coding Neural Networks Parameters | |
| Kirchhoffer et al. | Overview of the neural network compression and representation (NNR) standard | |
| TWI748201B (zh) | 適於相依純量量化之變換係數熵寫碼技術 | |
| TWI813126B (zh) | 相依量化技術 | |
| US20220004844A1 (en) | Methods and apparatuses for compressing parameters of neural networks | |
| KR20210065155A (ko) | 종속 스칼라 양자화를 사용하거나 이와 조합하는 데에 적합한 변환 계수의 효율적인 코딩 | |
| US20250045973A1 (en) | Decoder for providing decoded Parameters of a Neural Network, Encoder, Methods and Computer Programs using a Reordering | |
| US12532032B2 (en) | Coding concept for a sequence of information values | |
| JP2025186542A (ja) | 更新モデルを使用して、ニューラルネットワークパラメーターを復号化する装置、方法及びコンピュータープログラム、並びにニューラルネットワークパラメーターを符号化する装置、方法及びコンピュータープログラム | |
| US20240364362A1 (en) | Concepts for encoding and decoding neural network parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| E13-X000 | Pre-grant limitation requested |
St.27 status event code: A-2-3-E10-E13-lim-X000 |
|
| P11-X000 | Amendment of application requested |
St.27 status event code: A-2-2-P10-P11-nap-X000 |
|
| P13-X000 | Application amended |
St.27 status event code: A-2-2-P10-P13-nap-X000 |
|
| PA0105 | International application |
St.27 status event code: A-0-1-A10-A15-nap-PA0105 |
|
| PA0201 | Request for examination |
St.27 status event code: A-1-2-D10-D11-exm-PA0201 |
|
| P11-X000 | Amendment of application requested |
St.27 status event code: A-2-2-P10-P11-nap-X000 |
|
| P13-X000 | Application amended |
St.27 status event code: A-2-2-P10-P13-nap-X000 |
|
| PG1501 | Laying open of application |
St.27 status event code: A-1-1-Q10-Q12-nap-PG1501 |
|
| E902 | Notification of reason for refusal | ||
| PE0902 | Notice of grounds for rejection |
St.27 status event code: A-1-2-D10-D21-exm-PE0902 |
|
| P22-X000 | Classification modified |
St.27 status event code: A-2-2-P10-P22-nap-X000 |
|
| E13-X000 | Pre-grant limitation requested |
St.27 status event code: A-2-3-E10-E13-lim-X000 |
|
| P11-X000 | Amendment of application requested |
St.27 status event code: A-2-2-P10-P11-nap-X000 |
|
| P13-X000 | Application amended |
St.27 status event code: A-2-2-P10-P13-nap-X000 |
|
| E601 | Decision to refuse application | ||
| PE0601 | Decision on rejection of patent |
St.27 status event code: N-2-6-B10-B15-exm-PE0601 |
|
| T11-X000 | Administrative time limit extension requested |
St.27 status event code: U-3-3-T10-T11-oth-X000 |
|
| T13-X000 | Administrative time limit extension granted |
St.27 status event code: U-3-3-T10-T13-oth-X000 |
|
| T13-X000 | Administrative time limit extension granted |
St.27 status event code: U-3-3-T10-T13-oth-X000 |
|
| E13 | Pre-grant limitation requested |
Free format text: ST27 STATUS EVENT CODE: A-2-3-E10-E13-LIM-X000 (AS PROVIDED BY THE NATIONAL OFFICE) |
|
| E13-X000 | Pre-grant limitation requested |
St.27 status event code: A-2-3-E10-E13-lim-X000 |
|
| P11 | Amendment of application requested |
Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-NAP-X000 (AS PROVIDED BY THE NATIONAL OFFICE) |
|
| P11-X000 | Amendment of application requested |
St.27 status event code: A-2-2-P10-P11-nap-X000 |
|
| F13 | Ip right granted in full following pre-grant review |
Free format text: ST27 STATUS EVENT CODE: A-3-4-F10-F13-REX-PX0701 (AS PROVIDED BY THE NATIONAL OFFICE) |
|
| PX0701 | Decision of registration after re-examination |
St.27 status event code: A-3-4-F10-F13-rex-PX0701 |
|
| A16 | Divisional, continuation or continuation in part application filed |
Free format text: ST27 STATUS EVENT CODE: A-0-1-A10-A16-DIV-PA0104 (AS PROVIDED BY THE NATIONAL OFFICE) |
|
| PA0104 | Divisional application for international application |
St.27 status event code: A-0-1-A10-A16-div-PA0104 |
|
| PR1002 | Payment of registration fee |
St.27 status event code: A-2-2-U10-U12-oth-PR1002 Fee payment year number: 1 |
|
| U12 | Designation fee paid |
Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U12-OTH-PR1002 (AS PROVIDED BY THE NATIONAL OFFICE) Year of fee payment: 1 |
|
| PG1601 | Publication of registration |
St.27 status event code: A-4-4-Q10-Q13-nap-PG1601 |
|
| Q13 | Ip right document published |
Free format text: ST27 STATUS EVENT CODE: A-4-4-Q10-Q13-NAP-PG1601 (AS PROVIDED BY THE NATIONAL OFFICE) |