JP7863094B2 - ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 - Google Patents
ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理Info
- Publication number
- JP7863094B2 JP7863094B2 JP2023519515A JP2023519515A JP7863094B2 JP 7863094 B2 JP7863094 B2 JP 7863094B2 JP 2023519515 A JP2023519515 A JP 2023519515A JP 2023519515 A JP2023519515 A JP 2023519515A JP 7863094 B2 JP7863094 B2 JP 7863094B2
- Authority
- JP
- Japan
- Prior art keywords
- color component
- size
- downsampled
- neural network
- filtered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
- H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/96—Tree coding, e.g. quad-tree coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Processing (AREA)
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063087784P | 2020-10-05 | 2020-10-05 | |
| US63/087,784 | 2020-10-05 | ||
| US17/493,543 | 2021-10-04 | ||
| US17/493,543 US11825101B2 (en) | 2020-10-05 | 2021-10-04 | Joint-component neural network based filtering during video coding |
| PCT/US2021/053490 WO2022076355A1 (en) | 2020-10-05 | 2021-10-05 | Joint-component neural network based filtering during video coding |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| JP2023544705A JP2023544705A (ja) | 2023-10-25 |
| JP2023544705A5 JP2023544705A5 (cg-RX-API-DMAC7.html) | 2024-09-17 |
| JP7863094B2 true JP7863094B2 (ja) | 2026-05-20 |
Family
ID=80931851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2023519515A Active JP7863094B2 (ja) | 2020-10-05 | 2021-10-05 | ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US11825101B2 (cg-RX-API-DMAC7.html) |
| EP (1) | EP4226632A1 (cg-RX-API-DMAC7.html) |
| JP (1) | JP7863094B2 (cg-RX-API-DMAC7.html) |
| KR (1) | KR20230081701A (cg-RX-API-DMAC7.html) |
| CN (1) | CN116508321B (cg-RX-API-DMAC7.html) |
| BR (1) | BR112023005436A2 (cg-RX-API-DMAC7.html) |
| WO (1) | WO2022076355A1 (cg-RX-API-DMAC7.html) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020262408A1 (ja) * | 2019-06-25 | 2020-12-30 | 日本放送協会 | イントラ予測装置、画像復号装置、及びプログラム |
| CN116569548A (zh) * | 2020-12-04 | 2023-08-08 | 北京达佳互联信息技术有限公司 | 用于视频编解码的基于网络的图像滤波 |
| US12212776B2 (en) * | 2021-09-24 | 2025-01-28 | Apple Inc. | Systems and methods for low resolution motion estimation searches |
| US12555201B2 (en) * | 2021-12-14 | 2026-02-17 | Netflix, Inc. | Techniques for component-based image preprocessing |
| US12689736B2 (en) * | 2022-04-12 | 2026-07-21 | Electronics And Telecommunications Research Institute | Method and device for video encoding/decoding using intra-block copy |
| WO2023198057A1 (en) * | 2022-04-12 | 2023-10-19 | Beijing Bytedance Network Technology Co., Ltd. | Method, apparatus, and medium for video processing |
| WO2023241690A1 (en) * | 2022-06-16 | 2023-12-21 | Douyin Vision (Beijing) Co., Ltd. | Variable-rate neural network based compression |
| JP2025524438A (ja) * | 2022-07-04 | 2025-07-30 | ドルビー ラボラトリーズ ライセンシング コーポレイション | ニューラルネットワークを使用するループ・フィルタリング |
| KR20240019638A (ko) * | 2022-08-04 | 2024-02-14 | 삼성전자주식회사 | 크로마 성분 예측을 수행하는 ai에 기반한 비디오 복호화 장치 및 방법, 및 비디오 부호화 장치 및 방법 |
| US20240080462A1 (en) * | 2022-09-06 | 2024-03-07 | Apple Inc. | Systems and Methods for Low-Resolution Motion Estimation Searches |
| WO2024146446A1 (en) * | 2023-01-04 | 2024-07-11 | Douyin Vision Co., Ltd. | Method, apparatus, and medium for video processing |
| WO2024149081A1 (en) * | 2023-01-11 | 2024-07-18 | Douyin Vision Co., Ltd. | Method, apparatus, and medium for video processing |
| WO2024164963A1 (en) * | 2023-02-08 | 2024-08-15 | Mediatek Inc. | Method and apparatus for improving performance of neural network filter based video coding |
| WO2024193710A1 (en) * | 2023-03-22 | 2024-09-26 | Douyin Vision Co., Ltd. | Method, apparatus, and medium for visual data processing |
| CN121190582A (zh) * | 2023-04-18 | 2025-12-23 | 华为技术有限公司 | 图像解压缩方法和装置 |
| CN121002885A (zh) * | 2023-04-25 | 2025-11-21 | 抖音视界有限公司 | 用于视频处理的方法、装置和介质 |
| US12457368B2 (en) * | 2023-06-12 | 2025-10-28 | Qualcomm Incorporated | NN-based in loop filter architectures with separable convolution and switching order of decomposition |
| US20240422361A1 (en) * | 2023-06-14 | 2024-12-19 | Qualcomm Incorporated | Neural network based in loop filter architecture with unified supplementary data processing for video coding |
| CN121753335A (zh) * | 2023-08-26 | 2026-03-27 | 抖音视界有限公司 | 用于可视数据处理的方法、装置和介质 |
| CN117635424B (zh) * | 2023-11-28 | 2026-02-10 | 抖音视界有限公司 | 一种图像处理方法、装置、设备和存储介质 |
| US12581085B2 (en) * | 2024-05-13 | 2026-03-17 | Tencent America LLC | CCSO with filter shapes |
| WO2026075920A1 (en) * | 2024-10-02 | 2026-04-09 | Bytedance Inc. | Method, apparatus, and medium for video processing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019201239A1 (en) | 2018-04-17 | 2019-10-24 | Mediatek Inc. | Method and apparatus of neural network for video coding |
| JP2019201256A (ja) | 2018-05-14 | 2019-11-21 | シャープ株式会社 | 画像フィルタ装置 |
| US20200145661A1 (en) | 2017-07-06 | 2020-05-07 | Samsung Electronics Co., Ltd. | Method for encoding/decoding image, and device therefor |
| WO2020192020A1 (zh) | 2019-03-24 | 2020-10-01 | Oppo广东移动通信有限公司 | 滤波方法、装置、编码器以及计算机存储介质 |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180041778A1 (en) * | 2016-08-02 | 2018-02-08 | Qualcomm Incorporated | Geometry transformation-based adaptive loop filtering |
| US10419757B2 (en) * | 2016-08-31 | 2019-09-17 | Qualcomm Incorporated | Cross-component filter |
| EP3451670A1 (en) * | 2017-08-28 | 2019-03-06 | Thomson Licensing | Method and apparatus for filtering with mode-aware deep learning |
| US11363302B2 (en) * | 2017-10-12 | 2022-06-14 | Mediatek Inc. | Method and apparatus of neural network for video coding |
| CN108184129B (zh) * | 2017-12-11 | 2020-01-10 | 北京大学 | 一种视频编解码方法、装置及用于图像滤波的神经网络 |
| US20190246122A1 (en) * | 2018-02-08 | 2019-08-08 | Qualcomm Incorporated | Palette coding for video coding |
| US11615873B2 (en) * | 2018-08-31 | 2023-03-28 | The Board Of Regents Of The University Of Texas System | Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy |
| US11284075B2 (en) * | 2018-09-12 | 2022-03-22 | Qualcomm Incorporated | Prediction of adaptive loop filter parameters with reduced memory consumption for video coding |
| EP3706046A1 (en) | 2019-03-04 | 2020-09-09 | InterDigital VC Holdings, Inc. | Method and device for picture encoding and decoding |
-
2021
- 2021-10-04 US US17/493,543 patent/US11825101B2/en active Active
- 2021-10-05 EP EP21801732.5A patent/EP4226632A1/en active Pending
- 2021-10-05 WO PCT/US2021/053490 patent/WO2022076355A1/en not_active Ceased
- 2021-10-05 BR BR112023005436A patent/BR112023005436A2/pt unknown
- 2021-10-05 CN CN202180066933.9A patent/CN116508321B/zh active Active
- 2021-10-05 JP JP2023519515A patent/JP7863094B2/ja active Active
- 2021-10-05 KR KR1020237010420A patent/KR20230081701A/ko active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200145661A1 (en) | 2017-07-06 | 2020-05-07 | Samsung Electronics Co., Ltd. | Method for encoding/decoding image, and device therefor |
| WO2019201239A1 (en) | 2018-04-17 | 2019-10-24 | Mediatek Inc. | Method and apparatus of neural network for video coding |
| JP2019201256A (ja) | 2018-05-14 | 2019-11-21 | シャープ株式会社 | 画像フィルタ装置 |
| WO2020192020A1 (zh) | 2019-03-24 | 2020-10-01 | Oppo广东移动通信有限公司 | 滤波方法、装置、编码器以及计算机存储介质 |
Non-Patent Citations (5)
| Title |
|---|
| Hongtao Wang, et al.,AHG11: Neural Network-based In-Loop Filter,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29,JVET-T0079,20th Meeting, by teleconference,2020年10月01日,pp.1-5 |
| Hujun Yin, et al.,AHG9: Adaptive convolutional neural network loop filter,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-M0566,13th Meeting: Marrakesh,2019年01月,pp.1-9 |
| Shuai Wan, et al.,CE13-related: Integrated in-loop filter based on CNN,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-N0133-v2,14th Meeting: Geneva, CH,2019年03月,pp.1-7 |
| Yu-Ling Hsiao, et al.,AHG9: Convolutional neural network loop filter,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-M0159-v1,13th Meeting: Marrakech, MA,2019年01月,pp.1-6 |
| Yu-Ling Hsiao, et al.,CE10-1.2: Convolutional neural network loop filter,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-O0056-v1,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,2019年06月,pp.1-5 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116508321B (zh) | 2026-07-10 |
| BR112023005436A2 (pt) | 2023-05-09 |
| CN116508321A (zh) | 2023-07-28 |
| KR20230081701A (ko) | 2023-06-07 |
| US11825101B2 (en) | 2023-11-21 |
| US20220109860A1 (en) | 2022-04-07 |
| JP2023544705A (ja) | 2023-10-25 |
| EP4226632A1 (en) | 2023-08-16 |
| WO2022076355A1 (en) | 2022-04-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7610661B2 (ja) | ビデオコーディングにおける改善されたイントラ予測 | |
| JP7629413B2 (ja) | ビデオエンコードのための0化パターンに基づく低周波数非分離変換シグナリング | |
| CN113940069B (zh) | 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令 | |
| TWI862578B (zh) | 適應性迴路濾波器組之索引發信 | |
| TWI853918B (zh) | 用於視訊寫碼之區塊內複製合併資料語法 | |
| CN110463202B (zh) | 一种用于解码视频数据的方法、装置和设备 | |
| CN114128286B (zh) | 视频编解码中的环绕运动补偿 | |
| JP2023544705A (ja) | ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 | |
| JP7504925B2 (ja) | ビデオコーディングのための組み合わされたインターおよびイントラ予測モード | |
| JP7787883B2 (ja) | ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計 | |
| JP2023542841A (ja) | ビデオコーディング中にフィルタ処理するための複数のニューラルネットワークモデル | |
| TWI860326B (zh) | 用於視訊資料之框間-框內預測模式 | |
| TWI877192B (zh) | 用於視訊寫碼之色度內預測單元 | |
| JP7637675B2 (ja) | ビデオコーディングのための変換スキップにおける残差値のためのコーディング方式をシグナリングすること | |
| JP7423647B2 (ja) | 異なるクロマフォーマットを使用した三角予測ユニットモードでのビデオコーディング | |
| US20220337811A1 (en) | Method and apparatus for coding transform coefficient in video/image coding system | |
| CN114342389A (zh) | 视频编解码和变换单元的以高层语法信令通知的最大和最小块大小 | |
| CN114223202A (zh) | 低频不可分离变换(lfnst)信令 | |
| JP7637693B2 (ja) | ビデオコーディングにおけるコード化ビデオシーケンス開始アクセスユニット | |
| CN114830667A (zh) | 视频编解码中用于参考图片重采样的参考图片缩放比例 | |
| CN114846801A (zh) | 基于色度变换跳过的用于色度的lfnst信令 | |
| TWI873150B (zh) | 增強內部寫碼區塊之解碼產出量 | |
| JP2024501331A (ja) | ビデオコーディング中にフィルタ処理するための複数のニューラルネットワークモデル | |
| TWI898055B (zh) | 用於視頻譯碼中的跨分量線性模型(cclm)模式的固定位元深度處理 | |
| TWI876066B (zh) | 解塊濾波器參數訊號傳遞 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20240906 |
|
| A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20240906 |
|
| A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20250822 |
|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20250902 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20251127 |
|
| A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20251223 |
|
| A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20260323 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20260428 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20260508 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 7863094 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |