KR20230081701A - 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 - Google Patents

비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 Download PDF

Info

Publication number
KR20230081701A
KR20230081701A KR1020237010420A KR20237010420A KR20230081701A KR 20230081701 A KR20230081701 A KR 20230081701A KR 1020237010420 A KR1020237010420 A KR 1020237010420A KR 20237010420 A KR20237010420 A KR 20237010420A KR 20230081701 A KR20230081701 A KR 20230081701A
Authority
KR
South Korea
Prior art keywords
color component
size
neural network
filtered
convolutional neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237010420A
Other languages
English (en)
Korean (ko)
Inventor
지안레 천
홍타오 왕
벤카타 메헤르 사칫 아난드 코트라
마르타 카르체비츠
Original Assignee
퀄컴 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 퀄컴 인코포레이티드 filed Critical 퀄컴 인코포레이티드
Publication of KR20230081701A publication Critical patent/KR20230081701A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Processing (AREA)
KR1020237010420A 2020-10-05 2021-10-05 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링 Pending KR20230081701A (ko)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202063087784P 2020-10-05 2020-10-05
US63/087,784 2020-10-05
US17/493,543 2021-10-04
US17/493,543 US11825101B2 (en) 2020-10-05 2021-10-04 Joint-component neural network based filtering during video coding
PCT/US2021/053490 WO2022076355A1 (en) 2020-10-05 2021-10-05 Joint-component neural network based filtering during video coding

Publications (1)

Publication Number Publication Date
KR20230081701A true KR20230081701A (ko) 2023-06-07

Family

ID=80931851

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237010420A Pending KR20230081701A (ko) 2020-10-05 2021-10-05 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링

Country Status (7)

Country Link
US (1) US11825101B2 (https=)
EP (1) EP4226632A1 (https=)
JP (1) JP2023544705A (https=)
KR (1) KR20230081701A (https=)
CN (1) CN116508321A (https=)
BR (1) BR112023005436A2 (https=)
WO (1) WO2022076355A1 (https=)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7030246B2 (ja) * 2019-06-25 2022-03-04 日本放送協会 イントラ予測装置、画像復号装置、及びプログラム
WO2022120285A1 (en) * 2020-12-04 2022-06-09 Beijing Dajia Internet Information Technology Co., Ltd. Network based image filtering for video coding
US12212776B2 (en) * 2021-09-24 2025-01-28 Apple Inc. Systems and methods for low resolution motion estimation searches
US12555201B2 (en) * 2021-12-14 2026-02-17 Netflix, Inc. Techniques for component-based image preprocessing
WO2023198057A1 (en) * 2022-04-12 2023-10-19 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for video processing
CN119404507A (zh) * 2022-04-12 2025-02-07 韩国电子通信研究院 用于使用帧内块复制的视频编码/解码的方法和设备
CN119422374A (zh) * 2022-06-16 2025-02-11 抖音视界有限公司 基于可变速率神经网络的压缩
WO2024010710A1 (en) * 2022-07-04 2024-01-11 Dolby Laboratories Licensing Corporation Loop filtering using neural networks
KR20240019638A (ko) * 2022-08-04 2024-02-14 삼성전자주식회사 크로마 성분 예측을 수행하는 ai에 기반한 비디오 복호화 장치 및 방법, 및 비디오 부호화 장치 및 방법
US20240080462A1 (en) * 2022-09-06 2024-03-07 Apple Inc. Systems and Methods for Low-Resolution Motion Estimation Searches
WO2024146446A1 (en) * 2023-01-04 2024-07-11 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
CN120569959A (zh) * 2023-01-11 2025-08-29 抖音视界有限公司 用于视频处理的方法、装置和介质
US12593040B2 (en) * 2023-02-08 2026-03-31 Mediatek Inc. Method and apparatus for improving performance of neural network filter based video coding
CN120898426A (zh) * 2023-03-22 2025-11-04 抖音视界有限公司 用于可视数据处理的方法、装置和介质
EP4702759A1 (en) * 2023-04-25 2026-03-04 Douyin Vision Co., Ltd. Method, apparatus, and medium for video processing
US12457368B2 (en) * 2023-06-12 2025-10-28 Qualcomm Incorporated NN-based in loop filter architectures with separable convolution and switching order of decomposition
US20240422361A1 (en) * 2023-06-14 2024-12-19 Qualcomm Incorporated Neural network based in loop filter architecture with unified supplementary data processing for video coding
CN121753335A (zh) * 2023-08-26 2026-03-27 抖音视界有限公司 用于可视数据处理的方法、装置和介质
US12581085B2 (en) * 2024-05-13 2026-03-17 Tencent America LLC CCSO with filter shapes

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11095922B2 (en) * 2016-08-02 2021-08-17 Qualcomm Incorporated Geometry transformation-based adaptive loop filtering
US10419757B2 (en) * 2016-08-31 2019-09-17 Qualcomm Incorporated Cross-component filter
WO2019009449A1 (ko) * 2017-07-06 2019-01-10 삼성전자 주식회사 영상을 부호화/복호화 하는 방법 및 그 장치
EP3451670A1 (en) * 2017-08-28 2019-03-06 Thomson Licensing Method and apparatus for filtering with mode-aware deep learning
EP3685577A4 (en) * 2017-10-12 2021-07-28 MediaTek Inc. METHOD AND DEVICE OF A NEURAL NETWORK FOR VIDEO ENCODING
CN108184129B (zh) * 2017-12-11 2020-01-10 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20190246122A1 (en) * 2018-02-08 2019-08-08 Qualcomm Incorporated Palette coding for video coding
JP7073186B2 (ja) * 2018-05-14 2022-05-23 シャープ株式会社 画像フィルタ装置
WO2020047536A1 (en) * 2018-08-31 2020-03-05 Board Of Regents, University Of Texas System Deep learning based dosed prediction for treatment planning and quality assurance in radiation therapy
US11284075B2 (en) * 2018-09-12 2022-03-22 Qualcomm Incorporated Prediction of adaptive loop filter parameters with reduced memory consumption for video coding
EP3706046A1 (en) 2019-03-04 2020-09-09 InterDigital VC Holdings, Inc. Method and device for picture encoding and decoding
WO2020192020A1 (zh) * 2019-03-24 2020-10-01 Oppo广东移动通信有限公司 滤波方法、装置、编码器以及计算机存储介质

Also Published As

Publication number Publication date
EP4226632A1 (en) 2023-08-16
WO2022076355A1 (en) 2022-04-14
JP2023544705A (ja) 2023-10-25
US11825101B2 (en) 2023-11-21
BR112023005436A2 (pt) 2023-05-09
US20220109860A1 (en) 2022-04-07
CN116508321A (zh) 2023-07-28

Similar Documents

Publication Publication Date Title
US11825101B2 (en) Joint-component neural network based filtering during video coding
KR20230079360A (ko) 비디오 코딩 동안 필터링을 위한 다중 뉴럴 네트워크 모델들
CN113940069A (zh) 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令
KR20220008265A (ko) 비디오 코딩을 위한 제로-아웃 패턴들에 기초한 저 주파수 비 분리가능 변환 시그널링
CN113812148A (zh) 用于视频译码的参考图片重采样和帧间译码工具
KR20230038709A (ko) 다중 적응형 루프 필터 세트들
EP4035390A1 (en) Low-frequency non-separable transform (lfnst) simplifications
EP3868108B1 (en) Scans and last coefficient position coding for zero-out transforms
AU2020235622A1 (en) Coefficient domain block differential pulse-code modulation in video coding
KR20230078658A (ko) 비디오 코딩을 위한 뉴럴 네트워크-기반 필터링 프로세스에서의 활성화 함수 설계
EP4082211A1 (en) Lfnst signaling for chroma based on chroma transform skip
EP3935840A2 (en) Simplification of sub-block transforms in video coding
KR20230129015A (ko) 비디오 코딩 동안의 필터링을 위한 다수의 신경망 모델들
KR20220073755A (ko) 비디오 코딩을 위한 변환 스킵에서 잔차 값들을 위한 코딩 스킴 시그널링
WO2020257566A1 (en) Increasing decoding throughput of intra-coded blocks
EP4035371A1 (en) Arithmetic coder byte stuffing signaling for video coding
KR20230079049A (ko) 비디오 코딩에서의 크로스-컴포넌트 리니어 모델 (cclm) 모드에 대한 고정된 비트 심도 프로세싱
KR20230075443A (ko) 상이한 비트 심도에서 비디오 데이터의 코딩을 위한 적응적 루프 필터링의 동작 비트 심도의 제한
KR20230012489A (ko) 비디오 코딩에서의 하이 레벨 디블록킹 필터 (dbf), 적응적 루프 필터 (alf) 및 샘플 적응적 오프셋 (sao) 제어, 및 적응 파라미터 세트 (aps) 수 제약
KR20230011303A (ko) 슬라이스 헤더들에서 비디오 데이터의 픽처들의 픽처 헤더 데이터의 코딩 여부 결정
KR20230002323A (ko) 비디오 코딩을 위한 적응적 스케일링 리스트 제어
KR20220163376A (ko) 비디오 코딩에서의 변환 스킵 블록들에 대한 하이-레벨 제약들
EP4035393A1 (en) Signaling number of sub-pictures in high-level syntax for video coding
KR20220163374A (ko) 이미지 및 비디오 코딩을 위한 블록 파티셔닝
EP4133729A1 (en) Signaling number of subblock merge candidates in video coding

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20230327

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20240919

Comment text: Request for Examination of Application