CN116508321B - 视频译码期间基于联合分量神经网络的滤波 - Google Patents

视频译码期间基于联合分量神经网络的滤波

Info

Publication number
CN116508321B
CN116508321B CN202180066933.9A CN202180066933A CN116508321B CN 116508321 B CN116508321 B CN 116508321B CN 202180066933 A CN202180066933 A CN 202180066933A CN 116508321 B CN116508321 B CN 116508321B
Authority
CN
China
Prior art keywords
color component
downsampled
size
neural network
filtered
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202180066933.9A
Other languages
English (en)
Chinese (zh)
Other versions
CN116508321A (zh
Inventor
J·陈
王洪涛
V·M·S·A·科特拉
M·卡尔切维茨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of CN116508321A publication Critical patent/CN116508321A/zh
Application granted granted Critical
Publication of CN116508321B publication Critical patent/CN116508321B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
CN202180066933.9A 2020-10-05 2021-10-05 视频译码期间基于联合分量神经网络的滤波 Active CN116508321B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202063087784P 2020-10-05 2020-10-05
US63/087,784 2020-10-05
US17/493,543 2021-10-04
US17/493,543 US11825101B2 (en) 2020-10-05 2021-10-04 Joint-component neural network based filtering during video coding
PCT/US2021/053490 WO2022076355A1 (en) 2020-10-05 2021-10-05 Joint-component neural network based filtering during video coding

Publications (2)

Publication Number Publication Date
CN116508321A CN116508321A (zh) 2023-07-28
CN116508321B true CN116508321B (zh) 2026-07-10

Family

ID=80931851

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202180066933.9A Active CN116508321B (zh) 2020-10-05 2021-10-05 视频译码期间基于联合分量神经网络的滤波

Country Status (7)

Country Link
US (1) US11825101B2 (cg-RX-API-DMAC7.html)
EP (1) EP4226632A1 (cg-RX-API-DMAC7.html)
JP (1) JP7863094B2 (cg-RX-API-DMAC7.html)
KR (1) KR20230081701A (cg-RX-API-DMAC7.html)
CN (1) CN116508321B (cg-RX-API-DMAC7.html)
BR (1) BR112023005436A2 (cg-RX-API-DMAC7.html)
WO (1) WO2022076355A1 (cg-RX-API-DMAC7.html)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020262408A1 (ja) * 2019-06-25 2020-12-30 日本放送協会 イントラ予測装置、画像復号装置、及びプログラム
CN116569548A (zh) * 2020-12-04 2023-08-08 北京达佳互联信息技术有限公司 用于视频编解码的基于网络的图像滤波
US12212776B2 (en) * 2021-09-24 2025-01-28 Apple Inc. Systems and methods for low resolution motion estimation searches
US12555201B2 (en) * 2021-12-14 2026-02-17 Netflix, Inc. Techniques for component-based image preprocessing
US12689736B2 (en) * 2022-04-12 2026-07-21 Electronics And Telecommunications Research Institute Method and device for video encoding/decoding using intra-block copy
WO2023198057A1 (en) * 2022-04-12 2023-10-19 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for video processing
WO2023241690A1 (en) * 2022-06-16 2023-12-21 Douyin Vision (Beijing) Co., Ltd. Variable-rate neural network based compression
JP2025524438A (ja) * 2022-07-04 2025-07-30 ドルビー ラボラトリーズ ライセンシング コーポレイション ニューラルネットワークを使用するループ・フィルタリング
KR20240019638A (ko) * 2022-08-04 2024-02-14 삼성전자주식회사 크로마 성분 예측을 수행하는 ai에 기반한 비디오 복호화 장치 및 방법, 및 비디오 부호화 장치 및 방법
US20240080462A1 (en) * 2022-09-06 2024-03-07 Apple Inc. Systems and Methods for Low-Resolution Motion Estimation Searches
WO2024146446A1 (en) * 2023-01-04 2024-07-11 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
WO2024149081A1 (en) * 2023-01-11 2024-07-18 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
WO2024164963A1 (en) * 2023-02-08 2024-08-15 Mediatek Inc. Method and apparatus for improving performance of neural network filter based video coding
WO2024193710A1 (en) * 2023-03-22 2024-09-26 Douyin Vision Co., Ltd. Method, apparatus, and medium for visual data processing
CN121190582A (zh) * 2023-04-18 2025-12-23 华为技术有限公司 图像解压缩方法和装置
CN121002885A (zh) * 2023-04-25 2025-11-21 抖音视界有限公司 用于视频处理的方法、装置和介质
US12457368B2 (en) * 2023-06-12 2025-10-28 Qualcomm Incorporated NN-based in loop filter architectures with separable convolution and switching order of decomposition
US20240422361A1 (en) * 2023-06-14 2024-12-19 Qualcomm Incorporated Neural network based in loop filter architecture with unified supplementary data processing for video coding
CN121753335A (zh) * 2023-08-26 2026-03-27 抖音视界有限公司 用于可视数据处理的方法、装置和介质
CN117635424B (zh) * 2023-11-28 2026-02-10 抖音视界有限公司 一种图像处理方法、装置、设备和存储介质
US12581085B2 (en) * 2024-05-13 2026-03-17 Tencent America LLC CCSO with filter shapes
WO2026075920A1 (en) * 2024-10-02 2026-04-09 Bytedance Inc. Method, apparatus, and medium for video processing

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180041778A1 (en) * 2016-08-02 2018-02-08 Qualcomm Incorporated Geometry transformation-based adaptive loop filtering
US10419757B2 (en) * 2016-08-31 2019-09-17 Qualcomm Incorporated Cross-component filter
WO2019009449A1 (ko) * 2017-07-06 2019-01-10 삼성전자 주식회사 영상을 부호화/복호화 하는 방법 및 그 장치
EP3451670A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with mode-aware deep learning
US11363302B2 (en) * 2017-10-12 2022-06-14 Mediatek Inc. Method and apparatus of neural network for video coding
CN108184129B (zh) * 2017-12-11 2020-01-10 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20190246122A1 (en) * 2018-02-08 2019-08-08 Qualcomm Incorporated Palette coding for video coding
US11470356B2 (en) * 2018-04-17 2022-10-11 Mediatek Inc. Method and apparatus of neural network for video coding
JP7073186B2 (ja) * 2018-05-14 2022-05-23 シャープ株式会社 画像フィルタ装置
US11615873B2 (en) * 2018-08-31 2023-03-28 The Board Of Regents Of The University Of Texas System Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy
US11284075B2 (en) * 2018-09-12 2022-03-22 Qualcomm Incorporated Prediction of adaptive loop filter parameters with reduced memory consumption for video coding
EP3706046A1 (en) 2019-03-04 2020-09-09 InterDigital VC Holdings, Inc. Method and device for picture encoding and decoding
WO2020192020A1 (zh) * 2019-03-24 2020-10-01 Oppo广东移动通信有限公司 滤波方法、装置、编码器以及计算机存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CE10-1.2: Convolutional neural network loop filter;Yu-Ling Hsiao et al;JVET;20190618;第1-2页以及附图1-3 *

Also Published As

Publication number Publication date
BR112023005436A2 (pt) 2023-05-09
CN116508321A (zh) 2023-07-28
KR20230081701A (ko) 2023-06-07
US11825101B2 (en) 2023-11-21
US20220109860A1 (en) 2022-04-07
JP7863094B2 (ja) 2026-05-20
JP2023544705A (ja) 2023-10-25
EP4226632A1 (en) 2023-08-16
WO2022076355A1 (en) 2022-04-14

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant