KR20230081701A - 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 - Google Patents

비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 Download PDF

Info

Publication number
KR20230081701A
KR20230081701A KR1020237010420A KR20237010420A KR20230081701A KR 20230081701 A KR20230081701 A KR 20230081701A KR 1020237010420 A KR1020237010420 A KR 1020237010420A KR 20237010420 A KR20237010420 A KR 20237010420A KR 20230081701 A KR20230081701 A KR 20230081701A
Authority
KR
South Korea
Prior art keywords
color component
size
neural network
filtered
convolutional neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237010420A
Other languages
English (en)
Korean (ko)
Inventor
지안레 천
홍타오 왕
벤카타 메헤르 사칫 아난드 코트라
마르타 카르체비츠
Original Assignee
퀄컴 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 퀄컴 인코포레이티드 filed Critical 퀄컴 인코포레이티드
Publication of KR20230081701A publication Critical patent/KR20230081701A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
KR1020237010420A 2020-10-05 2021-10-05 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 Pending KR20230081701A (ko)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202063087784P 2020-10-05 2020-10-05
US63/087,784 2020-10-05
US17/493,543 2021-10-04
US17/493,543 US11825101B2 (en) 2020-10-05 2021-10-04 Joint-component neural network based filtering during video coding
PCT/US2021/053490 WO2022076355A1 (en) 2020-10-05 2021-10-05 Joint-component neural network based filtering during video coding

Publications (1)

Publication Number Publication Date
KR20230081701A true KR20230081701A (ko) 2023-06-07

Family

ID=80931851

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237010420A Pending KR20230081701A (ko) 2020-10-05 2021-10-05 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링

Country Status (7)

Country Link
US (1) US11825101B2 (cg-RX-API-DMAC7.html)
EP (1) EP4226632A1 (cg-RX-API-DMAC7.html)
JP (1) JP7863094B2 (cg-RX-API-DMAC7.html)
KR (1) KR20230081701A (cg-RX-API-DMAC7.html)
CN (1) CN116508321B (cg-RX-API-DMAC7.html)
BR (1) BR112023005436A2 (cg-RX-API-DMAC7.html)
WO (1) WO2022076355A1 (cg-RX-API-DMAC7.html)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020262408A1 (ja) * 2019-06-25 2020-12-30 日本放送協会 イントラ予測装置、画像復号装置、及びプログラム
CN116569548A (zh) * 2020-12-04 2023-08-08 北京达佳互联信息技术有限公司 用于视频编解码的基于网络的图像滤波
US12212776B2 (en) * 2021-09-24 2025-01-28 Apple Inc. Systems and methods for low resolution motion estimation searches
US12555201B2 (en) * 2021-12-14 2026-02-17 Netflix, Inc. Techniques for component-based image preprocessing
US12689736B2 (en) * 2022-04-12 2026-07-21 Electronics And Telecommunications Research Institute Method and device for video encoding/decoding using intra-block copy
WO2023198057A1 (en) * 2022-04-12 2023-10-19 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for video processing
WO2023241690A1 (en) * 2022-06-16 2023-12-21 Douyin Vision (Beijing) Co., Ltd. Variable-rate neural network based compression
JP2025524438A (ja) * 2022-07-04 2025-07-30 ドルビー ラボラトリーズ ライセンシング コーポレイション ニューラルネットワークを使用するループ・フィルタリング
KR20240019638A (ko) * 2022-08-04 2024-02-14 삼성전자주식회사 크로마 성분 예측을 수행하는 ai에 기반한 비디오 복호화 장치 및 방법, 및 비디오 부호화 장치 및 방법
US20240080462A1 (en) * 2022-09-06 2024-03-07 Apple Inc. Systems and Methods for Low-Resolution Motion Estimation Searches
WO2024146446A1 (en) * 2023-01-04 2024-07-11 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
WO2024149081A1 (en) * 2023-01-11 2024-07-18 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
WO2024164963A1 (en) * 2023-02-08 2024-08-15 Mediatek Inc. Method and apparatus for improving performance of neural network filter based video coding
WO2024193710A1 (en) * 2023-03-22 2024-09-26 Douyin Vision Co., Ltd. Method, apparatus, and medium for visual data processing
CN121190582A (zh) * 2023-04-18 2025-12-23 华为技术有限公司 图像解压缩方法和装置
CN121002885A (zh) * 2023-04-25 2025-11-21 抖音视界有限公司 用于视频处理的方法、装置和介质
US12457368B2 (en) * 2023-06-12 2025-10-28 Qualcomm Incorporated NN-based in loop filter architectures with separable convolution and switching order of decomposition
US20240422361A1 (en) * 2023-06-14 2024-12-19 Qualcomm Incorporated Neural network based in loop filter architecture with unified supplementary data processing for video coding
CN121753335A (zh) * 2023-08-26 2026-03-27 抖音视界有限公司 用于可视数据处理的方法、装置和介质
CN117635424B (zh) * 2023-11-28 2026-02-10 抖音视界有限公司 一种图像处理方法、装置、设备和存储介质
US12581085B2 (en) * 2024-05-13 2026-03-17 Tencent America LLC CCSO with filter shapes
WO2026075920A1 (en) * 2024-10-02 2026-04-09 Bytedance Inc. Method, apparatus, and medium for video processing

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180041778A1 (en) * 2016-08-02 2018-02-08 Qualcomm Incorporated Geometry transformation-based adaptive loop filtering
US10419757B2 (en) * 2016-08-31 2019-09-17 Qualcomm Incorporated Cross-component filter
WO2019009449A1 (ko) * 2017-07-06 2019-01-10 삼성전자 주식회사 영상을 부호화/복호화 하는 방법 및 그 장치
EP3451670A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with mode-aware deep learning
US11363302B2 (en) * 2017-10-12 2022-06-14 Mediatek Inc. Method and apparatus of neural network for video coding
CN108184129B (zh) * 2017-12-11 2020-01-10 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20190246122A1 (en) * 2018-02-08 2019-08-08 Qualcomm Incorporated Palette coding for video coding
US11470356B2 (en) * 2018-04-17 2022-10-11 Mediatek Inc. Method and apparatus of neural network for video coding
JP7073186B2 (ja) * 2018-05-14 2022-05-23 シャープ株式会社 画像フィルタ装置
US11615873B2 (en) * 2018-08-31 2023-03-28 The Board Of Regents Of The University Of Texas System Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy
US11284075B2 (en) * 2018-09-12 2022-03-22 Qualcomm Incorporated Prediction of adaptive loop filter parameters with reduced memory consumption for video coding
EP3706046A1 (en) 2019-03-04 2020-09-09 InterDigital VC Holdings, Inc. Method and device for picture encoding and decoding
WO2020192020A1 (zh) * 2019-03-24 2020-10-01 Oppo广东移动通信有限公司 滤波方法、装置、编码器以及计算机存储介质

Also Published As

Publication number Publication date
CN116508321B (zh) 2026-07-10
BR112023005436A2 (pt) 2023-05-09
CN116508321A (zh) 2023-07-28
US11825101B2 (en) 2023-11-21
US20220109860A1 (en) 2022-04-07
JP7863094B2 (ja) 2026-05-20
JP2023544705A (ja) 2023-10-25
EP4226632A1 (en) 2023-08-16
WO2022076355A1 (en) 2022-04-14

Similar Documents

Publication Publication Date Title
US11825101B2 (en) Joint-component neural network based filtering during video coding
KR20230079360A (ko) 비디오 코딩 동안 필터링을 위한 다중 뉴럴 네트워크 모델들
CN113940069A (zh) 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令
KR20220008265A (ko) 비디오 코딩을 위한 제로-아웃 패턴들에 기초한 저 주파수 비 분리가능 변환 시그널링
CN113812148A (zh) 用于视频译码的参考图片重采样和帧间译码工具
KR20230038709A (ko) 다중 적응형 루프 필터 세트들
EP3868108B1 (en) Scans and last coefficient position coding for zero-out transforms
EP3959891A1 (en) Adaptive loop filter set index signaling
EP4035390A1 (en) Low-frequency non-separable transform (lfnst) simplifications
SG11202109009PA (en) Coefficient domain block differential pulse-code modulation in video coding
KR20230078658A (ko) 비디오 코딩을 위한 뉴럴 네트워크-기반 필터링 프로세스에서의 활성화 함수 설계
EP3935840A2 (en) Simplification of sub-block transforms in video coding
EP4082211A1 (en) Lfnst signaling for chroma based on chroma transform skip
KR20230129015A (ko) 비디오 코딩 동안의 필터링을 위한 다수의 신경망 모델들
KR20220073755A (ko) 비디오 코딩을 위한 변환 스킵에서 잔차 값들을 위한 코딩 스킴 시그널링
WO2020257566A1 (en) Increasing decoding throughput of intra-coded blocks
KR20230079049A (ko) 비디오 코딩에서의 크로스-컴포넌트 리니어 모델 (cclm) 모드에 대한 고정된 비트 심도 프로세싱
KR20230012489A (ko) 비디오 코딩에서의 하이 레벨 디블록킹 필터 (dbf), 적응적 루프 필터 (alf) 및 샘플 적응적 오프셋 (sao) 제어, 및 적응 파라미터 세트 (aps) 수 제약
WO2021061799A1 (en) Arithmetic coder byte stuffing signaling for video coding
KR20230075443A (ko) 상이한 비트 심도에서 비디오 데이터의 코딩을 위한 적응적 루프 필터링의 동작 비트 심도의 제한
KR20230011303A (ko) 슬라이스 헤더들에서 비디오 데이터의 픽처들의 픽처 헤더 데이터의 코딩 여부 결정
EP4133729A1 (en) Signaling number of subblock merge candidates in video coding
KR20220163376A (ko) 비디오 코딩에서의 변환 스킵 블록들에 대한 하이-레벨 제약들
EP4035393A1 (en) Signaling number of sub-pictures in high-level syntax for video coding
KR20230002323A (ko) 비디오 코딩을 위한 적응적 스케일링 리스트 제어

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20230327

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20240919

Comment text: Request for Examination of Application