JP7815247B2 - ニューラルネットワークベースのビデオコーディングのためのフロントエンドアーキテクチャ - Google Patents
ニューラルネットワークベースのビデオコーディングのためのフロントエンドアーキテクチャInfo
- Publication number
- JP7815247B2 JP7815247B2 JP2023532549A JP2023532549A JP7815247B2 JP 7815247 B2 JP7815247 B2 JP 7815247B2 JP 2023532549 A JP2023532549 A JP 2023532549A JP 2023532549 A JP2023532549 A JP 2023532549A JP 7815247 B2 JP7815247 B2 JP 7815247B2
- Authority
- JP
- Japan
- Prior art keywords
- frame
- channel
- layer
- convolutional layer
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/439—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using cascaded computational arrangements for performing a single operation, e.g. filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/88—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Applications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063124016P | 2020-12-10 | 2020-12-10 | |
| US63/124,016 | 2020-12-10 | ||
| US202063131802P | 2020-12-30 | 2020-12-30 | |
| US63/131,802 | 2020-12-30 | ||
| US17/643,383 US12231666B2 (en) | 2020-12-10 | 2021-12-08 | Front-end architecture for neural network based video coding |
| US17/643,383 | 2021-12-08 | ||
| PCT/US2021/072824 WO2022126120A1 (en) | 2020-12-10 | 2021-12-09 | A front-end architecture for neural network based video coding |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| JP2023553369A JP2023553369A (ja) | 2023-12-21 |
| JP2023553369A5 JP2023553369A5 (cg-RX-API-DMAC7.html) | 2024-11-20 |
| JP7815247B2 true JP7815247B2 (ja) | 2026-02-17 |
Family
ID=79283114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2023532549A Active JP7815247B2 (ja) | 2020-12-10 | 2021-12-09 | ニューラルネットワークベースのビデオコーディングのためのフロントエンドアーキテクチャ |
Country Status (6)
| Country | Link |
|---|---|
| EP (1) | EP4260561A1 (cg-RX-API-DMAC7.html) |
| JP (1) | JP7815247B2 (cg-RX-API-DMAC7.html) |
| KR (1) | KR20230117346A (cg-RX-API-DMAC7.html) |
| CN (1) | CN116547965B (cg-RX-API-DMAC7.html) |
| TW (1) | TWI883294B (cg-RX-API-DMAC7.html) |
| WO (1) | WO2022126120A1 (cg-RX-API-DMAC7.html) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4494341A1 (en) * | 2022-07-01 | 2025-01-22 | Huawei Technologies Co., Ltd. | Parallel processing of image regions with neural networks ? decoding, post filtering, and rdoq |
| WO2024208149A1 (en) * | 2023-04-01 | 2024-10-10 | Douyin Vision Co., Ltd. | Method, apparatus, and medium for visual data processing |
| US20240357118A1 (en) * | 2023-04-11 | 2024-10-24 | Alibaba Innovation Private Limited | Methods and non-transitory computer readable storage medium for spatial resampling towards machine vision |
| CN121190582A (zh) * | 2023-04-18 | 2025-12-23 | 华为技术有限公司 | 图像解压缩方法和装置 |
| CN119450037A (zh) * | 2023-07-28 | 2025-02-14 | 腾讯科技(深圳)有限公司 | 滤波及编解码方法、装置及电子设备 |
| CN121646789A (zh) * | 2024-06-29 | 2026-03-10 | 京东方科技集团股份有限公司 | 图像处理装置、图像处理方法和显示装置 |
| WO2026086522A1 (en) * | 2024-10-25 | 2026-04-30 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Video coding using colorization |
| CN119743644B (zh) * | 2024-12-18 | 2026-01-13 | 北京潞晨科技有限公司 | 视频生成方法、装置、电子设备、存储介质及产品 |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019525544A (ja) | 2016-06-24 | 2019-09-05 | 韓国科学技術院Korea Advanced Institute Of Science And Technology | Cnn基盤インループフィルタを含む符号化方法と装置及び復号化方法と装置 |
| WO2020101257A1 (en) | 2018-11-12 | 2020-05-22 | Samsung Electronics Co., Ltd. | Display apparatus and method of controlling the same |
| JP2020191630A (ja) | 2019-05-22 | 2020-11-26 | 富士通株式会社 | 画像コーディング装置、確率モデル生成装置及び画像デコーディング装置 |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015134198A1 (en) * | 2014-02-21 | 2015-09-11 | Silicon Image, Inc. | Adaptive processing of video streams with reduced color resolution |
| CN110622169A (zh) * | 2017-05-15 | 2019-12-27 | 渊慧科技有限公司 | 用于视频中的动作识别的神经网络系统 |
| CN109426858B (zh) * | 2017-08-29 | 2021-04-06 | 京东方科技集团股份有限公司 | 神经网络、训练方法、图像处理方法及图像处理装置 |
| CN109842799B (zh) * | 2017-11-29 | 2021-02-09 | 杭州海康威视数字技术股份有限公司 | 颜色分量的帧内预测方法、装置及计算机设备 |
| CN108184129B (zh) * | 2017-12-11 | 2020-01-10 | 北京大学 | 一种视频编解码方法、装置及用于图像滤波的神经网络 |
| US20200053388A1 (en) * | 2018-08-10 | 2020-02-13 | Disney Enterprises, Inc. | Machine learning based video compression |
| US10999606B2 (en) * | 2019-01-08 | 2021-05-04 | Intel Corporation | Method and system of neural network loop filtering for video coding |
| CN111861877A (zh) * | 2019-04-25 | 2020-10-30 | 华为技术有限公司 | 视频超分变率的方法和装置 |
| CN111294604B (zh) * | 2020-02-13 | 2022-03-22 | 山东新一代信息产业技术研究院有限公司 | 一种基于深度学习的视频压缩方法 |
-
2021
- 2021-12-09 CN CN202180071392.9A patent/CN116547965B/zh active Active
- 2021-12-09 KR KR1020237018485A patent/KR20230117346A/ko active Pending
- 2021-12-09 TW TW110146050A patent/TWI883294B/zh active
- 2021-12-09 EP EP21839804.8A patent/EP4260561A1/en active Pending
- 2021-12-09 JP JP2023532549A patent/JP7815247B2/ja active Active
- 2021-12-09 WO PCT/US2021/072824 patent/WO2022126120A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019525544A (ja) | 2016-06-24 | 2019-09-05 | 韓国科学技術院Korea Advanced Institute Of Science And Technology | Cnn基盤インループフィルタを含む符号化方法と装置及び復号化方法と装置 |
| WO2020101257A1 (en) | 2018-11-12 | 2020-05-22 | Samsung Electronics Co., Ltd. | Display apparatus and method of controlling the same |
| JP2020191630A (ja) | 2019-05-22 | 2020-11-26 | 富士通株式会社 | 画像コーディング装置、確率モデル生成装置及び画像デコーディング装置 |
Non-Patent Citations (1)
| Title |
|---|
| Ankitesh K. Singh. Singh, et al.,DNNVC: A study of handling YUV420 input format for DNN-based video coding [online],Joint Video Exploration Team (JVET) of ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29 20th Meeting, by teleconference, 7-16 Oct. 2020,[JVET-T0123],2020年10月14日,pp.1-8,[retrieved on 2025-09-17],Retrieved from <https://jvet-experts.org/doc_end_user/documents/20_Teleconference/wg11/JVET-T0123-v3.zip><JVET-T0123.docx> |
Also Published As
| Publication number | Publication date |
|---|---|
| EP4260561A1 (en) | 2023-10-18 |
| KR20230117346A (ko) | 2023-08-08 |
| TWI883294B (zh) | 2025-05-11 |
| CN116547965A (zh) | 2023-08-04 |
| JP2023553369A (ja) | 2023-12-21 |
| WO2022126120A1 (en) | 2022-06-16 |
| CN116547965B (zh) | 2026-07-17 |
| TW202243476A (zh) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116114247B (zh) | 用于处理视频数据的装置和方法 | |
| JP7628550B2 (ja) | 再帰ベースの機械学習システムを使用したビデオ圧縮 | |
| US12231666B2 (en) | Front-end architecture for neural network based video coding | |
| US12003734B2 (en) | Machine learning based flow determination for video coding | |
| TWI883294B (zh) | 用於基於神經網路的視訊譯碼的前端架構 | |
| CN117980916A (zh) | 用于媒体的变换译码的基于变换器的架构 | |
| KR102831175B1 (ko) | 머신 러닝 향상을 갖는 비디오 코딩을 위한 비트-레이트 추정 | |
| US11399198B1 (en) | Learned B-frame compression | |
| US12394100B2 (en) | Video coding using camera motion compensation and object motion compensation | |
| US12177473B2 (en) | Video coding using optical flow and residual predictors | |
| US20240214578A1 (en) | Regularizing neural networks with data quantization using exponential family priors | |
| JP7840973B2 (ja) | ビデオコーディングのための機械学習ベースのフロー決定 | |
| CN116965032A (zh) | 用于视频译码的基于机器学习的流确定 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20241111 |
|
| A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20241111 |
|
| A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20250918 |
|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20250930 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20251217 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20260113 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20260204 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 7815247 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |