JP7787883B2 - ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計 - Google Patents

ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計

Info

Publication number
JP7787883B2
JP7787883B2 JP2023518813A JP2023518813A JP7787883B2 JP 7787883 B2 JP7787883 B2 JP 7787883B2 JP 2023518813 A JP2023518813 A JP 2023518813A JP 2023518813 A JP2023518813 A JP 2023518813A JP 7787883 B2 JP7787883 B2 JP 7787883B2
Authority
JP
Japan
Prior art keywords
video
cnn
block
video data
alpha
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023518813A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023543762A (ja
JP2023543762A5 (https=
JPWO2022072684A5 (https=
Inventor
ワン、ホンタオ
チェン、ジャンレー
カルチェビチ、マルタ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of JP2023543762A publication Critical patent/JP2023543762A/ja
Publication of JP2023543762A5 publication Critical patent/JP2023543762A5/ja
Publication of JPWO2022072684A5 publication Critical patent/JPWO2022072684A5/ja
Application granted granted Critical
Publication of JP7787883B2 publication Critical patent/JP7787883B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Picture Signal Circuits (AREA)
JP2023518813A 2020-09-30 2021-09-30 ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計 Active JP7787883B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US202063085936P 2020-09-30 2020-09-30
US63/085,936 2020-09-30
US17/489,459 2021-09-29
US17/489,459 US11647212B2 (en) 2020-09-30 2021-09-29 Activation function design in neural network-based filtering process for video coding
PCT/US2021/052950 WO2022072684A1 (en) 2020-09-30 2021-09-30 Activation function design in neural network-based filtering process for video coding

Publications (4)

Publication Number Publication Date
JP2023543762A JP2023543762A (ja) 2023-10-18
JP2023543762A5 JP2023543762A5 (https=) 2024-09-11
JPWO2022072684A5 JPWO2022072684A5 (https=) 2024-09-11
JP7787883B2 true JP7787883B2 (ja) 2025-12-17

Family

ID=80821961

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023518813A Active JP7787883B2 (ja) 2020-09-30 2021-09-30 ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計

Country Status (7)

Country Link
US (2) US11647212B2 (https=)
EP (1) EP4222954A1 (https=)
JP (1) JP7787883B2 (https=)
KR (1) KR20230078658A (https=)
CN (1) CN116325729B (https=)
PH (1) PH12023550209A1 (https=)
WO (1) WO2022072684A1 (https=)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11647212B2 (en) 2020-09-30 2023-05-09 Qualcomm Incorporated Activation function design in neural network-based filtering process for video coding
US20220321919A1 (en) * 2021-03-23 2022-10-06 Sharp Kabushiki Kaisha Systems and methods for signaling neural network-based in-loop filter parameter information in video coding
US12167047B2 (en) * 2022-01-13 2024-12-10 Tencent America LLC Neural network-based deblocking filters
US12556718B2 (en) * 2022-12-29 2026-02-17 Samsung Electronics Co., Ltd. Electronic device and method with image encoding and decoding
CN116805971B (zh) * 2023-04-11 2024-07-12 腾讯科技（深圳）有限公司 图像编解码方法、装置、设备
US12542932B2 (en) * 2023-04-12 2026-02-03 Qualcomm Incorporated Neural network-based in loop filter architectures with separable convolution and multi-scale enhancement for video coding
CN119211544A (zh) * 2023-06-26 2024-12-27 腾讯科技（深圳）有限公司 基于神经网络的图像滤波及编解码方法、装置、设备、存储介质
CN120345242A (zh) * 2023-07-05 2025-07-18 Lg电子株式会社 图像编码/解码方法、存储比特流的记录介质和发送比特流的方法
WO2025217290A1 (en) * 2024-04-10 2025-10-16 Qualcomm Incorporated Improvements of resnet based in-loop filter architecture for video coding

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190273948A1 (en) 2019-01-08 2019-09-05 Intel Corporation Method and system of neural network loop filtering for video coding
JP2020010331A (ja) 2018-07-03 2020-01-16 株式会社ユビタス 画質を向上させる方法
US20200244997A1 (en) 2017-08-28 2020-07-30 Interdigital Vc Holdings, Inc. Method and apparatus for filtering with multi-branch deep learning
WO2020180449A1 (en) 2019-03-04 2020-09-10 Interdigital Vc Holdings, Inc. Method and device for picture encoding and decoding
WO2020187587A1 (en) 2019-03-15 2020-09-24 Dolby International Ab Method and apparatus for updating a neural network

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11113800B2 (en) * 2017-01-18 2021-09-07 Nvidia Corporation Filtering image data using a neural network
US11025907B2 (en) * 2019-02-28 2021-06-01 Google Llc Receptive-field-conforming convolution models for video coding
US12282840B2 (en) 2019-01-11 2025-04-22 Samsung Electronics Co., Ltd. Method and apparatus with neural network layer contraction
KR102646695B1 (ko) 2019-01-15 2024-03-12 포틀랜드 스테이트 유니버시티 비디오 프레임 보간을 위한 특징 피라미드 워핑
GB2620499B (en) 2019-03-20 2024-04-03 V Nova Int Ltd Low complexity enhancement video coding
KR20200114436A (ko) 2019-03-28 2020-10-07 국방과학연구소 스케일러블 영상 부호화를 수행하는 장치 및 방법
US10909728B1 (en) * 2019-05-01 2021-02-02 Amazon Technologies, Inc. Learned lossy image compression codec
US11216917B2 (en) 2019-05-03 2022-01-04 Amazon Technologies, Inc. Video enhancement using a neural network
US10944996B2 (en) 2019-08-19 2021-03-09 Intel Corporation Visual quality optimized video compression
US11647212B2 (en) 2020-09-30 2023-05-09 Qualcomm Incorporated Activation function design in neural network-based filtering process for video coding

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200244997A1 (en) 2017-08-28 2020-07-30 Interdigital Vc Holdings, Inc. Method and apparatus for filtering with multi-branch deep learning
JP2020010331A (ja) 2018-07-03 2020-01-16 株式会社ユビタス 画質を向上させる方法
US20190273948A1 (en) 2019-01-08 2019-09-05 Intel Corporation Method and system of neural network loop filtering for video coding
WO2020180449A1 (en) 2019-03-04 2020-09-10 Interdigital Vc Holdings, Inc. Method and device for picture encoding and decoding
WO2020187587A1 (en) 2019-03-15 2020-09-24 Dolby International Ab Method and apparatus for updating a neural network

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Aurelien Geron，ｓｃｉｋｉｔ－ｌｅａｒｎとＴｅｎｓｏｒＦｌｏｗによる実践機械学習 ，初版，株式会社オライリー・ジャパン，2019年06月13日，pp.279-281
Hongtao Wang, et al.，AHG11: Neural Network-based In-Loop Filter，Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29，JVET-T0079-v3，20th Meeting, by teleconference，2020年10月，pp.1-10
Hujun Yin, et al.，CE10-1.7: Adaptive convolutional neural network loop filter，Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11，JVET-O0063，15th Meeting: Gothenburg，2019年06月，pp.1-4
Yue Li, et al.，AHG11: Convolutional Neural Networks-based In-Loop Filter，Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29，JVET-T0088-v2，20th Meeting, by teleconference，2020年10月，pp.1-4
Yu-Ling Hsiao, et al.，CE10-1.2: Convolutional neural network loop filter，Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11，JVET-O0056-v1，15th Meeting: Gothenburg, SE，2019年06月，pp.1-5

Also Published As

Publication number Publication date
EP4222954A1 (en) 2023-08-09
KR20230078658A (ko) 2023-06-02
JP2023543762A (ja) 2023-10-18
US11778213B2 (en) 2023-10-03
CN116325729B (zh) 2026-03-27
US20230012661A1 (en) 2023-01-19
PH12023550209A1 (en) 2024-06-24
WO2022072684A1 (en) 2022-04-07
US20220103845A1 (en) 2022-03-31
CN116325729A (zh) 2023-06-23
US11647212B2 (en) 2023-05-09

Similar Documents

Publication Publication Date Title
JP7795528B2 (ja) ビデオコーディング中にフィルタ処理するための複数のニューラルネットワークモデル
CN113940069B (zh) 用于视频译码中的低频不可分离变换的变换和最后有效系数位置信令
JP7787883B2 (ja) ビデオコーディングのためのニューラルネットワークベースフィルタ処理プロセスにおける活性化関数設計
TWI862578B (zh) 適應性迴路濾波器組之索引發信
CN113853784B (zh) 用于视频译码的多个自适应环路滤波器集合的方法和装置
TWI877183B (zh) 視訊寫碼中用於變換略過模式及調色板模式之最小允許量化參數
CN114731394B (zh) 用于视频编解码的角度帧内预测模式的位置相关帧内预测组合
CN114223202B (zh) 低频不可分离变换（lfnst）信令
KR20230081701A (ko) 비디오 코딩 동안 조인트-컴포넌트 뉴럴 네트워크 기반 필터링
AU2020278519A1 (en) Low-frequency non-separable transform signaling based on zero-out patterns for video coding
CN112385233B (zh) 合并的依赖模式的帧内平滑（mdis）与具有依赖位置的帧内预测组合（pdpc）的内插值滤波器切换
TWI877192B (zh) 用於視訊寫碼之色度內預測單元
CN113812148A (zh) 用于视频译码的参考图片重采样和帧间译码工具
TWI840427B (zh) 用於置零轉換之掃描及最後係數位置寫碼
CN118921465A (zh) 视频译码中的系数域块差分脉冲译码调制
CN113170162B (zh) 用于视频译码的共享候选列表和并行候选列表推导
JP7637675B2 (ja) ビデオコーディングのための変換スキップにおける残差値のためのコーディング方式をシグナリングすること
CN114080805A (zh) 用于视频译码的自适应环路滤波的非线性扩展
EP3935840A2 (en) Simplification of sub-block transforms in video coding
KR20230043101A (ko) 디블록킹 필터 파라미터 시그널링
JP7824958B2 (ja) 固定されたフィルタを用いる適応ループフィルタ
CN114175643A (zh) 调色板和预测模式信令
CN114731403B (zh) 基于量化参数的残差编解码选择和低层级信令
CN116235495A (zh) 用于视频译码中的跨分量线性模型（cclm）模式的固定比特深度处理
TW202304201A (zh) 使用重疊區塊運動補償、組合訊框間－訊框內預測及／或亮度映射和色度縮放的視訊譯碼

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240902

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240902

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250818

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250826

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20251014

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20251125

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20251205

R150 Certificate of patent or registration of utility model

Ref document number: 7787883

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150