KR20230154022A - P-프레임 코딩 시스템을 이용한 학습된 b-프레임 코딩 - Google Patents

P-프레임 코딩 시스템을 이용한 학습된 b-프레임 코딩 Download PDF

Info

Publication number
KR20230154022A
KR20230154022A KR1020237030157A KR20237030157A KR20230154022A KR 20230154022 A KR20230154022 A KR 20230154022A KR 1020237030157 A KR1020237030157 A KR 1020237030157A KR 20237030157 A KR20237030157 A KR 20237030157A KR 20230154022 A KR20230154022 A KR 20230154022A
Authority
KR
South Korea
Prior art keywords
frame
reference frame
motion
warping
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
KR1020237030157A
Other languages
English (en)
Korean (ko)
Inventor
레자 푸레자
타코 세바스티안 코헌
Original Assignee
퀄컴 인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 퀄컴 인코포레이티드 filed Critical 퀄컴 인코포레이티드
Publication of KR20230154022A publication Critical patent/KR20230154022A/ko
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
KR1020237030157A 2021-03-11 2022-01-27 P-프레임 코딩 시스템을 이용한 학습된 b-프레임 코딩 Pending KR20230154022A (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US17/198,813 2021-03-11
US17/198,813 US11831909B2 (en) 2021-03-11 2021-03-11 Learned B-frame coding using P-frame coding system
PCT/US2022/014143 WO2022191933A1 (en) 2021-03-11 2022-01-27 Learned b-frame coding using p-frame coding system

Publications (1)

Publication Number Publication Date
KR20230154022A true KR20230154022A (ko) 2023-11-07

Family

ID=80787118

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020237030157A Pending KR20230154022A (ko) 2021-03-11 2022-01-27 P-프레임 코딩 시스템을 이용한 학습된 b-프레임 코딩

Country Status (8)

Country Link
US (2) US11831909B2 (https=)
EP (1) EP4305839A1 (https=)
JP (1) JP2024509881A (https=)
KR (1) KR20230154022A (https=)
CN (1) CN117015968A (https=)
BR (1) BR112023017637A2 (https=)
TW (1) TW202236849A (https=)
WO (1) WO2022191933A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025127479A1 (ko) * 2023-12-14 2025-06-19 현대자동차주식회사 동적 3차원 공간 정보를 압축 및 전달을 위한 방법

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI804181B (zh) * 2021-02-02 2023-06-01 聯詠科技股份有限公司 影像編碼方法及其影像編碼器
US11831909B2 (en) 2021-03-11 2023-11-28 Qualcomm Incorporated Learned B-frame coding using P-frame coding system
US12548204B2 (en) * 2021-06-03 2026-02-10 Intel Corporation Neural frame extrapolation rendering mechanism
WO2023050072A1 (en) * 2021-09-28 2023-04-06 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Methods and systems for video compression
US20230117247A1 (en) * 2021-10-18 2023-04-20 Adp, Inc. Multi-Modal Deep Learning of Structured and Non-Structured Data
DE112022006625T5 (de) * 2022-02-08 2024-12-05 Nvidia Corporation Bilderzeugung unter verwendung eines neuronalen netzes
WO2024008814A1 (en) * 2022-07-05 2024-01-11 Telefonaktiebolaget Lm Ericsson (Publ) Filtering for video encoding and decoding
CN117974814A (zh) * 2022-10-26 2024-05-03 荣耀终端有限公司 用于图像处理的方法、装置及存储介质
CN116233462A (zh) * 2023-03-06 2023-06-06 格兰菲智能科技有限公司 视频编码方法、视频编码系统及视频编码器
KR20240173786A (ko) * 2023-06-07 2024-12-16 삼성전자주식회사 영상 처리 장치 및 영상의 움직임 추정 방법
CN119854502B (zh) * 2024-12-26 2025-10-10 西安电子科技大学 一种长时参考和运动时空关联端到端监控视频编解码方法

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0865682A (ja) * 1994-08-25 1996-03-08 Sanyo Electric Co Ltd 動画像の動き補償予測方式
US8934552B2 (en) 2011-03-31 2015-01-13 Qualcomm Incorporated Combined reference picture list construction and mapping
WO2013009716A2 (en) * 2011-07-08 2013-01-17 Dolby Laboratories Licensing Corporation Hybrid encoding and decoding methods for single and multiple layered video coding systems
US9426463B2 (en) 2012-02-08 2016-08-23 Qualcomm Incorporated Restriction of prediction units in B slices to uni-directional inter prediction
US9451277B2 (en) 2012-02-08 2016-09-20 Qualcomm Incorporated Restriction of prediction units in B slices to uni-directional inter prediction
US9258562B2 (en) * 2012-06-13 2016-02-09 Qualcomm Incorporated Derivation of depth map estimate
JP2014082540A (ja) * 2012-10-12 2014-05-08 National Institute Of Information & Communication Technology 互いに類似した情報を含む複数画像のデータサイズを低減する方法、プログラム、および装置、ならびに、互いに類似した情報を含む複数画像を表現するデータ構造
CN104704827B (zh) * 2012-11-13 2019-04-12 英特尔公司 用于下一代视频的内容自适应变换译码
US10136119B2 (en) * 2013-01-10 2018-11-20 Qualcomm Incoporated View synthesis in 3D video
US10404992B2 (en) * 2015-07-27 2019-09-03 Qualcomm Incorporated Methods and systems of restricting bi-prediction in video coding
CN110741640B (zh) 2017-08-22 2024-03-29 谷歌有限责任公司 用于视频代码化中的运动补偿预测的光流估计
CN115695790B (zh) * 2018-01-15 2025-10-28 三星电子株式会社 编码方法及其设备以及解码方法及其设备
US11019355B2 (en) 2018-04-03 2021-05-25 Electronics And Telecommunications Research Institute Inter-prediction method and apparatus using reference frame generated based on deep learning
WO2021130357A1 (en) * 2019-12-27 2021-07-01 Koninklijke Kpn N.V. Motion vector prediction for video coding
US11405626B2 (en) * 2020-03-03 2022-08-02 Qualcomm Incorporated Video compression using recurrent-based machine learning systems
US11430138B2 (en) * 2020-03-05 2022-08-30 Huawei Technologies Co., Ltd. Systems and methods for multi-frame video frame interpolation
US11831909B2 (en) 2021-03-11 2023-11-28 Qualcomm Incorporated Learned B-frame coding using P-frame coding system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025127479A1 (ko) * 2023-12-14 2025-06-19 현대자동차주식회사 동적 3차원 공간 정보를 압축 및 전달을 위한 방법

Also Published As

Publication number Publication date
WO2022191933A1 (en) 2022-09-15
EP4305839A1 (en) 2024-01-17
US20240022761A1 (en) 2024-01-18
US11831909B2 (en) 2023-11-28
TW202236849A (zh) 2022-09-16
JP2024509881A (ja) 2024-03-05
US12184893B2 (en) 2024-12-31
CN117015968A (zh) 2023-11-07
BR112023017637A2 (pt) 2024-01-23
US20220295095A1 (en) 2022-09-15

Similar Documents

Publication Publication Date Title
US12184893B2 (en) Learned B-frame coding using P-frame coding system
US11405626B2 (en) Video compression using recurrent-based machine learning systems
US12003734B2 (en) Machine learning based flow determination for video coding
CN116965029A (zh) 使用卷积神经网络对图像进行译码的装置和方法
US11399198B1 (en) Learned B-frame compression
TWI883294B (zh) 用於基於神經網路的視訊譯碼的前端架構
JP7780059B2 (ja) 機械学習強化を用いたビデオコーディングのためのビットレート推定
US12177473B2 (en) Video coding using optical flow and residual predictors
US12394100B2 (en) Video coding using camera motion compensation and object motion compensation
WO2024015665A1 (en) Bit-rate estimation for video coding with machine learning enhancement
WO2024137094A1 (en) Regularizing neural networks with data quantization using exponential family priors
JP7840973B2 (ja) ビデオコーディングのための機械学習ベースのフロー決定
CN116965032A (zh) 用于视频译码的基于机器学习的流确定
KR20250071263A (ko) 가변 채널 수를 갖는 신경망 및 이를 작동시키는 방법

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20230904

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20250110

Comment text: Request for Examination of Application