JP2023544705A - ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 - Google Patents

ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 Download PDF

Info

Publication number
JP2023544705A
JP2023544705A JP2023519515A JP2023519515A JP2023544705A JP 2023544705 A JP2023544705 A JP 2023544705A JP 2023519515 A JP2023519515 A JP 2023519515A JP 2023519515 A JP2023519515 A JP 2023519515A JP 2023544705 A JP2023544705 A JP 2023544705A
Authority
JP
Japan
Prior art keywords
color component
size
component
downsampled
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2023519515A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023544705A5 (https=
JPWO2022076355A5 (https=
Inventor
チェン、ジャンレー
ワン、ホンタオ
コトラ、ベンカタ・メヘル・サトチット・アナンド
カルチェビチ、マルタ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of JP2023544705A publication Critical patent/JP2023544705A/ja
Publication of JP2023544705A5 publication Critical patent/JP2023544705A5/ja
Publication of JPWO2022076355A5 publication Critical patent/JPWO2022076355A5/ja
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
JP2023519515A 2020-10-05 2021-10-05 ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理 Pending JP2023544705A (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202063087784P 2020-10-05 2020-10-05
US63/087,784 2020-10-05
US17/493,543 2021-10-04
US17/493,543 US11825101B2 (en) 2020-10-05 2021-10-04 Joint-component neural network based filtering during video coding
PCT/US2021/053490 WO2022076355A1 (en) 2020-10-05 2021-10-05 Joint-component neural network based filtering during video coding

Publications (3)

Publication Number Publication Date
JP2023544705A true JP2023544705A (ja) 2023-10-25
JP2023544705A5 JP2023544705A5 (https=) 2024-09-17
JPWO2022076355A5 JPWO2022076355A5 (https=) 2024-09-17

Family

ID=80931851

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023519515A Pending JP2023544705A (ja) 2020-10-05 2021-10-05 ビデオコーディング中の、ジョイント成分ニューラルネットワークベースのフィルタ処理

Country Status (7)

Country Link
US (1) US11825101B2 (https=)
EP (1) EP4226632A1 (https=)
JP (1) JP2023544705A (https=)
KR (1) KR20230081701A (https=)
CN (1) CN116508321A (https=)
BR (1) BR112023005436A2 (https=)
WO (1) WO2022076355A1 (https=)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7030246B2 (ja) * 2019-06-25 2022-03-04 日本放送協会 イントラ予測装置、画像復号装置、及びプログラム
WO2022120285A1 (en) * 2020-12-04 2022-06-09 Beijing Dajia Internet Information Technology Co., Ltd. Network based image filtering for video coding
US12212776B2 (en) * 2021-09-24 2025-01-28 Apple Inc. Systems and methods for low resolution motion estimation searches
US12555201B2 (en) * 2021-12-14 2026-02-17 Netflix, Inc. Techniques for component-based image preprocessing
WO2023198057A1 (en) * 2022-04-12 2023-10-19 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for video processing
CN119404507A (zh) * 2022-04-12 2025-02-07 韩国电子通信研究院 用于使用帧内块复制的视频编码/解码的方法和设备
CN119422374A (zh) * 2022-06-16 2025-02-11 抖音视界有限公司 基于可变速率神经网络的压缩
WO2024010710A1 (en) * 2022-07-04 2024-01-11 Dolby Laboratories Licensing Corporation Loop filtering using neural networks
KR20240019638A (ko) * 2022-08-04 2024-02-14 삼성전자주식회사 크로마 성분 예측을 수행하는 ai에 기반한 비디오 복호화 장치 및 방법, 및 비디오 부호화 장치 및 방법
US20240080462A1 (en) * 2022-09-06 2024-03-07 Apple Inc. Systems and Methods for Low-Resolution Motion Estimation Searches
WO2024146446A1 (en) * 2023-01-04 2024-07-11 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
CN120569959A (zh) * 2023-01-11 2025-08-29 抖音视界有限公司 用于视频处理的方法、装置和介质
US12593040B2 (en) * 2023-02-08 2026-03-31 Mediatek Inc. Method and apparatus for improving performance of neural network filter based video coding
CN120898426A (zh) * 2023-03-22 2025-11-04 抖音视界有限公司 用于可视数据处理的方法、装置和介质
EP4702759A1 (en) * 2023-04-25 2026-03-04 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
US12457368B2 (en) * 2023-06-12 2025-10-28 Qualcomm Incorporated NN-based in loop filter architectures with separable convolution and switching order of decomposition
US20240422361A1 (en) * 2023-06-14 2024-12-19 Qualcomm Incorporated Neural network based in loop filter architecture with unified supplementary data processing for video coding
CN121753335A (zh) * 2023-08-26 2026-03-27 抖音视界有限公司 用于可视数据处理的方法、装置和介质
US12581085B2 (en) * 2024-05-13 2026-03-17 Tencent America LLC CCSO with filter shapes

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019201256A (ja) * 2018-05-14 2019-11-21 シャープ株式会社 画像フィルタ装置
US20200145661A1 (en) * 2017-07-06 2020-05-07 Samsung Electronics Co., Ltd. Method for encoding/decoding image, and device therefor
WO2020192020A1 (zh) * 2019-03-24 2020-10-01 Oppo广东移动通信有限公司 滤波方法、装置、编码器以及计算机存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11095922B2 (en) * 2016-08-02 2021-08-17 Qualcomm Incorporated Geometry transformation-based adaptive loop filtering
US10419757B2 (en) * 2016-08-31 2019-09-17 Qualcomm Incorporated Cross-component filter
EP3451670A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with mode-aware deep learning
EP3685577A4 (en) * 2017-10-12 2021-07-28 MediaTek Inc. METHOD AND DEVICE OF A NEURAL NETWORK FOR VIDEO ENCODING
CN108184129B (zh) * 2017-12-11 2020-01-10 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20190246122A1 (en) * 2018-02-08 2019-08-08 Qualcomm Incorporated Palette coding for video coding
WO2020047536A1 (en) * 2018-08-31 2020-03-05 Board Of Regents, University Of Texas System Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy
US11284075B2 (en) * 2018-09-12 2022-03-22 Qualcomm Incorporated Prediction of adaptive loop filter parameters with reduced memory consumption for video coding
EP3706046A1 (en) 2019-03-04 2020-09-09 InterDigital VC Holdings, Inc. Method and device for picture encoding and decoding

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200145661A1 (en) * 2017-07-06 2020-05-07 Samsung Electronics Co., Ltd. Method for encoding/decoding image, and device therefor
JP2019201256A (ja) * 2018-05-14 2019-11-21 シャープ株式会社 画像フィルタ装置
WO2020192020A1 (zh) * 2019-03-24 2020-10-01 Oppo广东移动通信有限公司 滤波方法、装置、编码器以及计算机存储介质

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
HONGTAO WANG, ET AL.: "AHG11: Neural Network-based In-Loop Filter", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29, vol. JVET-T0079, JPN6025051639, 1 October 2020 (2020-10-01), pages 1 - 5, ISSN: 0005760132 *
HUJUN YIN, ET AL.: "AHG9: Adaptive convolutional neural network loop filter", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, vol. JVET-M0566, JPN6025051634, January 2019 (2019-01-01), pages 1 - 9, ISSN: 0005760129 *
SHUAI WAN, ET AL.: "CE13-related: Integrated in-loop filter based on CNN", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, vol. JVET-N0133-v2, JPN6025051637, March 2019 (2019-03-01), pages 1 - 7, ISSN: 0005760130 *
YU-LING HSIAO, ET AL.: "AHG9: Convolutional neural network loop filter", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, vol. JVET-M0159-v1, JPN6025051631, January 2019 (2019-01-01), pages 1 - 6, ISSN: 0005760128 *
YU-LING HSIAO, ET AL.: "CE10-1.2: Convolutional neural network loop filter", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, vol. JVET-O0056-v1, JPN6025051638, June 2019 (2019-06-01), pages 1 - 5, ISSN: 0005760131 *

Also Published As

Publication number Publication date
EP4226632A1 (en) 2023-08-16
KR20230081701A (ko) 2023-06-07
WO2022076355A1 (en) 2022-04-14
US11825101B2 (en) 2023-11-21
BR112023005436A2 (pt) 2023-05-09
US20220109860A1 (en) 2022-04-07
CN116508321A (zh) 2023-07-28

Similar Documents

Publication Publication Date Title
US11825101B2 (en) Joint-component neural network based filtering during video coding
JP2023542841A (ja) ビデオコーディング中にフィルタ処理するための複数のニューラルネットワークモデル
US12327384B2 (en) Multiple neural network models for filtering during video coding
CN113940069A (zh) 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令
JP7423647B2 (ja) 異なるクロマフォーマットを使用した三角予測ユニットモードでのビデオコーディング
CN114128286A (zh) 视频编解码中的环绕运动补偿
JP7637675B2 (ja) ビデオコーディングのための変換スキップにおける残差値のためのコーディング方式をシグナリングすること
US11706425B2 (en) Multiple transform set signaling for video coding
US20250218052A1 (en) Multiple neural network models for filtering during video coding
US12439038B2 (en) Reduced complexity multi-mode neural network filtering of video data
JP2023543762A (ja) ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計
CN114846801A (zh) 基于色度变换跳过的用于色度的lfnst信令
JP7579279B2 (ja) ビデオ符号化および復号における空間スケーラビリティのサポート
JP2023517892A (ja) ビデオコーディングにおけるコード化ビデオシーケンス開始アクセスユニット
TWI898055B (zh) 用於視頻譯碼中的跨分量線性模型（ｃｃｌｍ）模式的固定位元深度處理
JP2023507099A (ja) ビデオコーディングにおけるマルチプル変換選択シグナリングに対する係数グループベースの制限
CN114731403A (zh) 基于量化参数的残差编解码选择和低层级信令
JP7662540B2 (ja) ビデオコーディングにおけるｄｃイントラモード予測
US20210314567A1 (en) Block partitioning for image and video coding
CN114930821A (zh) 视频编解码中的自适应色彩变换的qp偏移的灵活信令通知
JP2025522740A (ja) ビデオコーディングにおける複数の色成分のためのニューラルネットワークベースのフィルタ処理プロセス
US11863787B2 (en) Maximum allowed block size for BDPCM mode
US12598314B2 (en) Neural network based filtering process for multiple color components in video coding
KR20250034038A (ko) 비디오 데이터의 감소된 복잡도 다중 모드 뉴럴 네트워크 필터링
CN116746146A (zh) 在视频编解码期间用于滤波的多个神经网络模型

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240906

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240906

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250822

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250902

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20251127

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20251223

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20260323