JP2024509881A - Ｐフレームコーディングシステムを使用する学習型ｂフレームコーディング - Google Patents

Ｐフレームコーディングシステムを使用する学習型ｂフレームコーディング Download PDF

Info

Publication number
JP2024509881A
JP2024509881A JP2023554362A JP2023554362A JP2024509881A JP 2024509881 A JP2024509881 A JP 2024509881A JP 2023554362 A JP2023554362 A JP 2023554362A JP 2023554362 A JP2023554362 A JP 2023554362A JP 2024509881 A JP2024509881 A JP 2024509881A
Authority
JP
Japan
Prior art keywords
frame
reference frame
motion
prediction
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2023554362A
Other languages
English (en)
Japanese (ja)
Other versions
JP2024509881A5 (https=
Inventor
レザ・プレッザ
タコ・セバスティアーン・コーヘン
Original Assignee
クアルコム，インコーポレイテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by クアルコム，インコーポレイテッド filed Critical クアルコム，インコーポレイテッド
Publication of JP2024509881A publication Critical patent/JP2024509881A/ja
Publication of JP2024509881A5 publication Critical patent/JP2024509881A5/ja
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
JP2023554362A 2021-03-11 2022-01-27 Ｐフレームコーディングシステムを使用する学習型ｂフレームコーディング Pending JP2024509881A (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US17/198,813 2021-03-11
US17/198,813 US11831909B2 (en) 2021-03-11 2021-03-11 Learned B-frame coding using P-frame coding system
PCT/US2022/014143 WO2022191933A1 (en) 2021-03-11 2022-01-27 Learned b-frame coding using p-frame coding system

Publications (2)

Publication Number Publication Date
JP2024509881A true JP2024509881A (ja) 2024-03-05
JP2024509881A5 JP2024509881A5 (https=) 2025-01-16

Family

ID=80787118

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023554362A Pending JP2024509881A (ja) 2021-03-11 2022-01-27 Ｐフレームコーディングシステムを使用する学習型ｂフレームコーディング

Country Status (8)

Country Link
US (2) US11831909B2 (https=)
EP (1) EP4305839A1 (https=)
JP (1) JP2024509881A (https=)
KR (1) KR20230154022A (https=)
CN (1) CN117015968A (https=)
BR (1) BR112023017637A2 (https=)
TW (1) TW202236849A (https=)
WO (1) WO2022191933A1 (https=)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI804181B (zh) * 2021-02-02 2023-06-01 聯詠科技股份有限公司 影像編碼方法及其影像編碼器
US11831909B2 (en) 2021-03-11 2023-11-28 Qualcomm Incorporated Learned B-frame coding using P-frame coding system
US12548204B2 (en) * 2021-06-03 2026-02-10 Intel Corporation Neural frame extrapolation rendering mechanism
WO2023050072A1 (en) * 2021-09-28 2023-04-06 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Methods and systems for video compression
US20230117247A1 (en) * 2021-10-18 2023-04-20 Adp, Inc. Multi-Modal Deep Learning of Structured and Non-Structured Data
DE112022006625T5 (de) * 2022-02-08 2024-12-05 Nvidia Corporation Bilderzeugung unter verwendung eines neuronalen netzes
WO2024008814A1 (en) * 2022-07-05 2024-01-11 Telefonaktiebolaget Lm Ericsson (Publ) Filtering for video encoding and decoding
CN117974814A (zh) * 2022-10-26 2024-05-03 荣耀终端有限公司 用于图像处理的方法、装置及存储介质
CN116233462A (zh) * 2023-03-06 2023-06-06 格兰菲智能科技有限公司 视频编码方法、视频编码系统及视频编码器
KR20240173786A (ko) * 2023-06-07 2024-12-16 삼성전자주식회사 영상 처리 장치 및 영상의 움직임 추정 방법
WO2025127479A1 (ko) * 2023-12-14 2025-06-19 현대자동차주식회사 동적 3차원 공간 정보를 압축 및 전달을 위한 방법
CN119854502B (zh) * 2024-12-26 2025-10-10 西安电子科技大学 一种长时参考和运动时空关联端到端监控视频编解码方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0865682A (ja) * 1994-08-25 1996-03-08 Sanyo Electric Co Ltd 動画像の動き補償予測方式
US20190306526A1 (en) * 2018-04-03 2019-10-03 Electronics And Telecommunications Research Institute Inter-prediction method and apparatus using reference frame generated based on deep learning
JP2020522200A (ja) * 2017-08-22 2020-07-27 グーグル エルエルシー 映像コーディングにおける動き補償予測のオプティカルフロー推定

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8934552B2 (en) 2011-03-31 2015-01-13 Qualcomm Incorporated Combined reference picture list construction and mapping
WO2013009716A2 (en) * 2011-07-08 2013-01-17 Dolby Laboratories Licensing Corporation Hybrid encoding and decoding methods for single and multiple layered video coding systems
US9426463B2 (en) 2012-02-08 2016-08-23 Qualcomm Incorporated Restriction of prediction units in B slices to uni-directional inter prediction
US9451277B2 (en) 2012-02-08 2016-09-20 Qualcomm Incorporated Restriction of prediction units in B slices to uni-directional inter prediction
US9258562B2 (en) * 2012-06-13 2016-02-09 Qualcomm Incorporated Derivation of depth map estimate
JP2014082540A (ja) * 2012-10-12 2014-05-08 National Institute Of Information & Communication Technology 互いに類似した情報を含む複数画像のデータサイズを低減する方法、プログラム、および装置、ならびに、互いに類似した情報を含む複数画像を表現するデータ構造
CN104704827B (zh) * 2012-11-13 2019-04-12 英特尔公司 用于下一代视频的内容自适应变换译码
US10136119B2 (en) * 2013-01-10 2018-11-20 Qualcomm Incoporated View synthesis in 3D video
US10404992B2 (en) * 2015-07-27 2019-09-03 Qualcomm Incorporated Methods and systems of restricting bi-prediction in video coding
CN115695790B (zh) * 2018-01-15 2025-10-28 三星电子株式会社 编码方法及其设备以及解码方法及其设备
WO2021130357A1 (en) * 2019-12-27 2021-07-01 Koninklijke Kpn N.V. Motion vector prediction for video coding
US11405626B2 (en) * 2020-03-03 2022-08-02 Qualcomm Incorporated Video compression using recurrent-based machine learning systems
US11430138B2 (en) * 2020-03-05 2022-08-30 Huawei Technologies Co., Ltd. Systems and methods for multi-frame video frame interpolation
US11831909B2 (en) 2021-03-11 2023-11-28 Qualcomm Incorporated Learned B-frame coding using P-frame coding system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0865682A (ja) * 1994-08-25 1996-03-08 Sanyo Electric Co Ltd 動画像の動き補償予測方式
JP2020522200A (ja) * 2017-08-22 2020-07-27 グーグル エルエルシー 映像コーディングにおける動き補償予測のオプティカルフロー推定
US20190306526A1 (en) * 2018-04-03 2019-10-03 Electronics And Telecommunications Research Institute Inter-prediction method and apparatus using reference frame generated based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
NISHCHAL K. VERMA, ET.AL.: "High Accuracy Optical Flow Based Future Image Frame Predictor Model", 2015 IEEE APPLIEDIMAGERY PATTERNRECOGNITIONWORKSHOP (AIPR), JPN6025041180, 15 October 2015 (2015-10-15), ISSN: 0005703424 *

Also Published As

Publication number Publication date
WO2022191933A1 (en) 2022-09-15
EP4305839A1 (en) 2024-01-17
US20240022761A1 (en) 2024-01-18
US11831909B2 (en) 2023-11-28
TW202236849A (zh) 2022-09-16
US12184893B2 (en) 2024-12-31
KR20230154022A (ko) 2023-11-07
CN117015968A (zh) 2023-11-07
BR112023017637A2 (pt) 2024-01-23
US20220295095A1 (en) 2022-09-15

Similar Documents

Publication Publication Date Title
US12184893B2 (en) Learned B-frame coding using P-frame coding system
US11405626B2 (en) Video compression using recurrent-based machine learning systems
US12542919B2 (en) Apparatus and method for coding pictures using a convolutional neural network
US12003734B2 (en) Machine learning based flow determination for video coding
US11399198B1 (en) Learned B-frame compression
JP7780059B2 (ja) 機械学習強化を用いたビデオコーディングのためのビットレート推定
US12177473B2 (en) Video coding using optical flow and residual predictors
KR20230117346A (ko) 뉴럴 네트워크 기반 비디오 코딩을 위한 프론트-엔드 아키텍처
US12394100B2 (en) Video coding using camera motion compensation and object motion compensation
WO2024015665A1 (en) Bit-rate estimation for video coding with machine learning enhancement
JP7840973B2 (ja) ビデオコーディングのための機械学習ベースのフロー決定
US20260129220A1 (en) Apparatus and method for coding pictures using convolutional neural network
CN116965032A (zh) 用于视频译码的基于机器学习的流确定

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250106

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20250106

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20250929

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20251007

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20260107

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20260428