TWI883294B - 用於基於神經網路的視訊譯碼的前端架構 - Google Patents

用於基於神經網路的視訊譯碼的前端架構 Download PDF

Info

Publication number
TWI883294B
TWI883294B TW110146050A TW110146050A TWI883294B TW I883294 B TWI883294 B TW I883294B TW 110146050 A TW110146050 A TW 110146050A TW 110146050 A TW110146050 A TW 110146050A TW I883294 B TWI883294 B TW I883294B
Authority
TW
Taiwan
Prior art keywords
frame
channel
layer
output
subnetwork
Prior art date
Application number
TW110146050A
Other languages
English (en)
Chinese (zh)
Other versions
TW202243476A (zh
Inventor
希勒米艾內斯 艾吉爾梅茲
安基泰許庫瑪 辛格
莫哈美德塞伊德 克班
瑪塔 卡克基維克茲
Original Assignee
美商高通公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/643,383 external-priority patent/US12231666B2/en
Application filed by 美商高通公司 filed Critical 美商高通公司
Publication of TW202243476A publication Critical patent/TW202243476A/zh
Application granted granted Critical
Publication of TWI883294B publication Critical patent/TWI883294B/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/439—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using cascaded computational arrangements for performing a single operation, e.g. filtering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/88—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
TW110146050A 2020-12-10 2021-12-09 用於基於神經網路的視訊譯碼的前端架構 TWI883294B (zh)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US202063124016P 2020-12-10 2020-12-10
US63/124,016 2020-12-10
US202063131802P 2020-12-30 2020-12-30
US63/131,802 2020-12-30
US17/643,383 2021-12-08
US17/643,383 US12231666B2 (en) 2020-12-10 2021-12-08 Front-end architecture for neural network based video coding

Publications (2)

Publication Number Publication Date
TW202243476A TW202243476A (zh) 2022-11-01
TWI883294B true TWI883294B (zh) 2025-05-11

Family

ID=79283114

Family Applications (1)

Application Number Title Priority Date Filing Date
TW110146050A TWI883294B (zh) 2020-12-10 2021-12-09 用於基於神經網路的視訊譯碼的前端架構

Country Status (5)

Country Link
EP (1) EP4260561A1 (https=)
JP (1) JP7815247B2 (https=)
KR (1) KR20230117346A (https=)
TW (1) TWI883294B (https=)
WO (1) WO2022126120A1 (https=)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4494341A1 (en) * 2022-07-01 2025-01-22 Huawei Technologies Co., Ltd. Parallel processing of image regions with neural networks ? decoding, post filtering, and rdoq
WO2024208149A1 (en) * 2023-04-01 2024-10-10 Douyin Vision Co., Ltd. Method, apparatus, and medium for visual data processing
US20240357118A1 (en) * 2023-04-11 2024-10-24 Alibaba Innovation Private Limited Methods and non-transitory computer readable storage medium for spatial resampling towards machine vision
CN121190582A (zh) * 2023-04-18 2025-12-23 华为技术有限公司 图像解压缩方法和装置
CN119450037A (zh) * 2023-07-28 2025-02-14 腾讯科技（深圳）有限公司 滤波及编解码方法、装置及电子设备
CN121646789A (zh) * 2024-06-29 2026-03-10 京东方科技集团股份有限公司 图像处理装置、图像处理方法和显示装置
CN119743644B (zh) * 2024-12-18 2026-01-13 北京潞晨科技有限公司 视频生成方法、装置、电子设备、存储介质及产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108184129A (zh) * 2017-12-11 2018-06-19 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20190220746A1 (en) * 2017-08-29 2019-07-18 Boe Technology Group Co., Ltd. Image processing method, image processing device, and training method of neural network
US20200053388A1 (en) * 2018-08-10 2020-02-13 Disney Enterprises, Inc. Machine learning based video compression
CN111861877A (zh) * 2019-04-25 2020-10-30 华为技术有限公司 视频超分变率的方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101974261B1 (ko) * 2016-06-24 2019-04-30 한국과학기술원 Cnn 기반 인루프 필터를 포함하는 부호화 방법과 장치 및 복호화 방법과 장치
KR102622950B1 (ko) * 2018-11-12 2024-01-10 삼성전자주식회사 디스플레이장치, 그 제어방법 및 기록매체
CN111988609B (zh) * 2019-05-22 2024-07-16 富士通株式会社 图像编码装置、概率模型生成装置和图像解码装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190220746A1 (en) * 2017-08-29 2019-07-18 Boe Technology Group Co., Ltd. Image processing method, image processing device, and training method of neural network
CN108184129A (zh) * 2017-12-11 2018-06-19 北京大学 一种视频编解码方法、装置及用于图像滤波的神经网络
US20200053388A1 (en) * 2018-08-10 2020-02-13 Disney Enterprises, Inc. Machine learning based video compression
CN111861877A (zh) * 2019-04-25 2020-10-30 华为技术有限公司 视频超分变率的方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
網路文獻 Y. Ye et al., "[DNNVC] AhG on deep neural networks based video coding," Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 20th Meeting, by teleconference, 7 – 16 Oct. 2020. Document: JVET-T0121 (version 1 - date 2020-10-13 11:01:46) https://jvet-experts.org/doc_end_user/current_document.php?id=10518 *

Also Published As

Publication number Publication date
WO2022126120A1 (en) 2022-06-16
KR20230117346A (ko) 2023-08-08
EP4260561A1 (en) 2023-10-18
JP2023553369A (ja) 2023-12-21
TW202243476A (zh) 2022-11-01
JP7815247B2 (ja) 2026-02-17

Similar Documents

Publication Publication Date Title
CN116114247B (zh) 用于处理视频数据的装置和方法
US11405626B2 (en) Video compression using recurrent-based machine learning systems
TWI883294B (zh) 用於基於神經網路的視訊譯碼的前端架構
US12231666B2 (en) Front-end architecture for neural network based video coding
US12003734B2 (en) Machine learning based flow determination for video coding
TW202318878A (zh) 用於媒體變換譯碼的基於變換器的架構
EP4305839A1 (en) Learned b-frame coding using p-frame coding system
KR102831175B1 (ko) 머신 러닝 향상을 갖는 비디오 코딩을 위한 비트-레이트 추정
US12177473B2 (en) Video coding using optical flow and residual predictors
CN119923667A (zh) 基于扩散的数据压缩
US20240214578A1 (en) Regularizing neural networks with data quantization using exponential family priors
TWI914390B (zh) 基於端到端神經網路的視訊譯碼
JP7840973B2 (ja) ビデオコーディングのための機械学習ベースのフロー決定
CN116965032A (zh) 用于视频译码的基于机器学习的流确定
CN116547965A (zh) 用于基于神经网络的视频译码的前端架构