CN114073071A - 视频插帧方法及装置、计算机可读存储介质 - Google Patents

视频插帧方法及装置、计算机可读存储介质 Download PDF

Info

Publication number
CN114073071A
CN114073071A CN202080000871.7A CN202080000871A CN114073071A CN 114073071 A CN114073071 A CN 114073071A CN 202080000871 A CN202080000871 A CN 202080000871A CN 114073071 A CN114073071 A CN 114073071A
Authority
CN
China
Prior art keywords
frame
input frames
depth
input
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202080000871.7A
Other languages
English (en)
Other versions
CN114073071B (zh
Inventor
卢运华
段然
陈冠男
张丽杰
刘瀚文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Publication of CN114073071A publication Critical patent/CN114073071A/zh
Application granted granted Critical
Publication of CN114073071B publication Critical patent/CN114073071B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H04N7/0137—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes dependent on presence/absence of motion, e.g. of motion zones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及信息展示技术领域，具体涉及一种视频插帧方法及装置、计算机可读存储介质及电子设备，方法包括：根据两个输入帧得到与两个输入帧对应的两个初始光流图；优化初始光流图得到目标光流图；根据两个输入帧得到插帧核、两个深度图和两个上下文特征图；根据目标光流图、深度图、上下文特征图和插帧核利用帧合成方法得到输出帧；上述步骤至少满足以下条件之一：对两个输入帧进行迭代残差光流估计得到两个初始光流图；根据两个输入帧利用像素自适应卷积联合上采样优化初始光流图得到目标光流图；根据两个输入帧利用目标深度估计模型得到两个深度图；根据目标光流图、深度图、上下文特征图以及插帧核利用像素自适应卷积帧合成方法得到输出帧。

Description

PCT国内申请，说明书已公开。

Claims (15)

  1. PCT国内申请，权利要求书已公开。
CN202080000871.7A 2020-05-29 2020-05-29 视频插帧方法及装置、计算机可读存储介质 Active CN114073071B (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/093530 WO2021237743A1 (zh) 2020-05-29 2020-05-29 视频插帧方法及装置、计算机可读存储介质

Publications (2)

Publication Number Publication Date
CN114073071A true CN114073071A (zh) 2022-02-18
CN114073071B CN114073071B (zh) 2023-12-05

Family

ID=78745417

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080000871.7A Active CN114073071B (zh) 2020-05-29 2020-05-29 视频插帧方法及装置、计算机可读存储介质

Country Status (3)

Country Link
US (1) US11800053B2 (zh)
CN (1) CN114073071B (zh)
WO (1) WO2021237743A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11871145B2 (en) * 2021-04-06 2024-01-09 Adobe Inc. Optimization of adaptive convolutions for video frame interpolation
US11640668B2 (en) 2021-06-10 2023-05-02 Qualcomm Incorporated Volumetric sampling with correlative characterization for dense estimation
CN114745545A (zh) * 2022-04-11 2022-07-12 北京字节跳动网络技术有限公司 一种视频插帧方法、装置、设备和介质
CN115661304B (zh) * 2022-10-11 2024-05-03 北京汉仪创新科技股份有限公司 基于帧插值的字库生成方法、电子设备、存储介质和系统
CN116546183B (zh) * 2023-04-06 2024-03-22 华中科技大学 基于单帧图像的具有视差效果的动态图像生成方法及系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109151474A (zh) * 2018-08-23 2019-01-04 复旦大学 一种生成新视频帧的方法
CN109379550A (zh) * 2018-09-12 2019-02-22 上海交通大学 基于卷积神经网络的视频帧率上变换方法及系统
US20190138889A1 (en) * 2017-11-06 2019-05-09 Nvidia Corporation Multi-frame video interpolation using optical flow
WO2019168765A1 (en) * 2018-02-27 2019-09-06 Portland State University Context-aware synthesis for video frame interpolation
CN110351511A (zh) * 2019-06-28 2019-10-18 上海交通大学 基于场景深度估计的视频帧率上变换系统及方法
CN110392282A (zh) * 2018-04-18 2019-10-29 优酷网络技术（北京）有限公司 一种视频插帧的方法、计算机存储介质及服务器
CN110738697A (zh) * 2019-10-10 2020-01-31 福州大学 基于深度学习的单目深度估计方法
CN110913230A (zh) * 2019-11-29 2020-03-24 合肥图鸭信息科技有限公司 一种视频帧预测方法、装置及终端设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8570426B2 (en) * 2008-11-25 2013-10-29 Lytro, Inc. System of and method for video refocusing
EP3433816A1 (en) * 2016-03-22 2019-01-30 URU, Inc. Apparatus, systems, and methods for integrating digital media content into other digital media content
US10091435B2 (en) * 2016-06-07 2018-10-02 Disney Enterprises, Inc. Video segmentation from an uncalibrated camera array
CN109145922B (zh) 2018-09-10 2022-03-29 成都品果科技有限公司 一种自动抠图系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190138889A1 (en) * 2017-11-06 2019-05-09 Nvidia Corporation Multi-frame video interpolation using optical flow
WO2019168765A1 (en) * 2018-02-27 2019-09-06 Portland State University Context-aware synthesis for video frame interpolation
CN110392282A (zh) * 2018-04-18 2019-10-29 优酷网络技术（北京）有限公司 一种视频插帧的方法、计算机存储介质及服务器
CN109151474A (zh) * 2018-08-23 2019-01-04 复旦大学 一种生成新视频帧的方法
CN109379550A (zh) * 2018-09-12 2019-02-22 上海交通大学 基于卷积神经网络的视频帧率上变换方法及系统
CN110351511A (zh) * 2019-06-28 2019-10-18 上海交通大学 基于场景深度估计的视频帧率上变换系统及方法
CN110738697A (zh) * 2019-10-10 2020-01-31 福州大学 基于深度学习的单目深度估计方法
CN110913230A (zh) * 2019-11-29 2020-03-24 合肥图鸭信息科技有限公司 一种视频帧预测方法、装置及终端设备

Also Published As

Publication number Publication date
WO2021237743A1 (zh) 2021-12-02
US11800053B2 (en) 2023-10-24
CN114073071B (zh) 2023-12-05
US20220201242A1 (en) 2022-06-23

Similar Documents

Publication Publication Date Title
CN114073071A (zh) 视频插帧方法及装置、计算机可读存储介质
US9430817B2 (en) Blind image deblurring with cascade architecture
US10404962B2 (en) Drift correction for camera tracking
CN111327926B (zh) 视频插帧方法、装置、电子设备及存储介质
CN111372087B (zh) 全景视频插帧方法、装置及对应的存储介质
EP3872760A2 (en) Method and apparatus of training depth estimation network, and method and apparatus of estimating depth of image
EP2984622A1 (en) Image deblurring
CN113365110B (zh) 模型训练、视频插帧方法，装置，设备以及存储介质
US11641446B2 (en) Method for video frame interpolation, and electronic device
CN110706262A (zh) 图像处理方法、装置、设备及存储介质
JP2023525462A (ja) 特徴を抽出するための方法、装置、電子機器、記憶媒体およびコンピュータプログラム
CN113129352A (zh) 一种稀疏光场重建方法及装置
CN113780326A (zh) 一种图像处理方法、装置、存储介质及电子设备
CN110958363B (zh) 图像处理方法及装置、计算机可读介质和电子设备
CN110827341A (zh) 一种图片深度估计方法、装置和存储介质
CN113870399B (zh) 表情驱动方法、装置、电子设备及存储介质
WO2020092051A1 (en) Rolling shutter rectification in images/videos using convolutional neural networks with applications to sfm/slam with rolling shutter images/videos
CN113379877A (zh) 人脸视频生成方法、装置、电子设备及存储介质
CN111062883B (zh) 图像处理方法及装置、计算机可读介质和电子设备
CN111866492A (zh) 基于头戴显示设备的图像处理方法、装置及设备
CN113450276B (zh) 视频图像增强方法及其模型训练方法、相关设备
CN115576470A (zh) 图像处理方法和装置、增强现实系统及介质
CN109543557B (zh) 视频帧的处理方法、装置、设备以及存储介质
EP4248365A1 (en) Gating of contextual attention and convolutional features
US11481871B2 (en) Image-guided depth propagation for space-warping images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant