WO2022155990A1 - Procédé et système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé - Google Patents

Procédé et système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé Download PDF

Info

Publication number
WO2022155990A1
WO2022155990A1 PCT/CN2021/074281 CN2021074281W WO2022155990A1 WO 2022155990 A1 WO2022155990 A1 WO 2022155990A1 CN 2021074281 W CN2021074281 W CN 2021074281W WO 2022155990 A1 WO2022155990 A1 WO 2022155990A1
Authority
WO
WIPO (PCT)
Prior art keywords
resolution
video
network
reconstruction
video frame
Prior art date
Application number
PCT/CN2021/074281
Other languages
English (en)
Chinese (zh)
Inventor
潘金山
白浩然
唐金辉
Original Assignee
南京理工大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京理工大学 filed Critical 南京理工大学
Publication of WO2022155990A1 publication Critical patent/WO2022155990A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks

Definitions

  • the present invention relates to the technical field of video resolution reconstruction, in particular to a method and system for blind video super-resolution reconstruction based on self-supervised learning.
  • the goal of video super-resolution technology is to reconstruct a high-resolution video from a given low-resolution video.
  • the degradation process of the video super-resolution problem is usually defined as:
  • y j , x i , and n represent the jth video frame low-resolution image, the i-th video frame high-resolution image and noise, respectively;
  • S and K j represent the downsampling matrix and the blur matrix, respectively;
  • F i ⁇ j represents the deformation Matrix (related to optical flow u i ⁇ j , used to warp xi to the jth video frame to achieve alignment).
  • Video super-resolution is a highly ill-conditioned problem since the underlying high-resolution intermediate video frames x i , the blur matrix K j and the deformation matrix F i ⁇ j are all unknown.
  • the blur kernel in the actual scene is more complex, so the dataset constructed with the hypothetical blur kernel, and then the deep model trained with the dataset has poor generalization ability on the real video, but the image is degraded in the actual application scene
  • the process is more complicated, so the deep model trained by the above method will have false artifacts and wrong structural information when reconstructing the real video in high resolution.
  • the video rate is used for downstream tasks, it will cause a decrease in accuracy.
  • the purpose of the present invention is to provide a video blind super-resolution reconstruction method and system based on self-supervised learning, so as to improve the phenomenon of false artifacts and erroneous structural information when reconstructing high-resolution videos.
  • the present invention provides a video blind super-resolution reconstruction method based on self-supervised learning, the method comprising:
  • S1 Determine a first resolution video sequence based on the first resolution video
  • S2 Adopt self-supervised learning method to determine blur kernel estimation network, optical flow estimation network, feature extraction network and potential high-resolution intermediate frame reconstruction network;
  • S5 Extract the feature of each video frame in the first resolution video sequence by using the feature extraction network, align the feature of each video frame according to the deformation matrix, and obtain the feature of each video frame after alignment;
  • S6 utilize the potential high-resolution intermediate frame reconstruction network and the features of the aligned video frames to construct a second-resolution intermediate video frame;
  • S7 Determine a second resolution video based on the second resolution intermediate video frame; the resolution of the second resolution video is greater than the resolution of the first resolution video.
  • a self-supervised learning method to determine a blur kernel estimation network, an optical flow estimation network, a feature extraction network and a potential high-resolution intermediate frame reconstruction network specifically includes:
  • S25 Determine a total loss function according to the cycle consistency loss function, the fuzzy kernel regularization loss function, and the auxiliary reconstruction loss function;
  • S26 Determine the blur kernel estimation network, optical flow estimation network, feature extraction network and potential high-resolution intermediate frame reconstruction network when the total loss function is the smallest.
  • the determining a deformation matrix based on the optical flow estimation network and the first resolution video sequence specifically includes:
  • S42 Calculate a deformation matrix according to the optical flow using a bilinear interpolation method; determine a deformation operation according to the deformation matrix.
  • the feature extraction network to extract the features of each video frame in the first resolution video sequence, aligning the features of each video frame according to the deformation matrix, and obtaining the features of each video frame after alignment, specifically including:
  • S52 Use the deformation operation to align the features of each video frame with the features of the intermediate video frames, and obtain the features of each video frame after alignment.
  • N I ( ) is the potential high-resolution intermediate frame reconstruction network
  • C[ ] is the connection operation
  • x i is the second resolution intermediate video frame.
  • the present invention also provides a video blind super-resolution reconstruction system based on self-supervised learning, the system comprising:
  • a first resolution video sequence determination module configured to determine a first resolution video sequence based on the first resolution video
  • a multi-network determination module for determining a blur kernel estimation network, an optical flow estimation network, a feature extraction network, and a potentially high-resolution intermediate frame reconstruction network using a self-supervised learning method
  • a blur kernel determination module for estimating a blur kernel based on the blur kernel estimation network using the first resolution video sequence
  • a deformation matrix determination module configured to determine a deformation matrix based on the optical flow estimation network and the first resolution video sequence
  • a feature alignment module configured to extract the features of each video frame in the first resolution video sequence by using the feature extraction network, align the features of each video frame according to the deformation matrix, and obtain the features of each video frame after alignment;
  • a second-resolution intermediate video frame determining module configured to construct a second-resolution intermediate video frame by utilizing the potential high-resolution intermediate frame reconstruction network and the features of the aligned video frames;
  • a second resolution video determination module configured to determine a second resolution video based on the second resolution intermediate video frame; the resolution of the second resolution video is greater than the resolution of the first resolution video.
  • the multi-network determination module specifically includes:
  • a fuzzy matrix determination unit configured to determine a fuzzy matrix according to the fuzzy kernel through a convolution operation
  • a cycle-consistent loss function construction unit configured to construct a cycle-consistency loss function according to the fuzzy matrix
  • the fuzzy kernel regularized loss function building unit is used to construct the fuzzy kernel regularized loss function
  • Auxiliary reconstruction loss function construction unit used to construct auxiliary reconstruction loss function
  • a total loss function construction unit configured to determine a total loss function according to the cycle consistency loss function, the fuzzy kernel regularization loss function, and the auxiliary reconstruction loss function;
  • a multi-network determination unit for determining a blur kernel estimation network, an optical flow estimation network, a feature extraction network, and a potentially high-resolution intermediate frame reconstruction network when the total loss function is minimized.
  • the deformation matrix determination module specifically includes:
  • an optical flow determination unit configured to calculate the optical flow of each video frame and an intermediate video frame in the first resolution video sequence based on the optical flow estimation network
  • a deformation matrix determination unit configured to calculate a deformation matrix according to the optical flow using a bilinear interpolation method; and determine a deformation operation according to the deformation matrix.
  • the feature alignment module specifically includes:
  • a feature determination unit used for extracting the feature of each video frame in the first resolution video sequence by using the feature extraction network
  • the feature alignment unit is used for aligning the features of each video frame to the features of the intermediate video frames by using the deformation operation, and obtaining the features of each video frame after alignment.
  • N I ( ) is the potential high-resolution intermediate frame reconstruction network
  • C[ ] is the connection operation
  • x i is the second resolution intermediate video frame.
  • the present invention discloses the following technical effects:
  • the present invention provides a video blind super-resolution reconstruction method and system based on self-supervised learning.
  • the method includes: firstly, using a self-supervised learning method to determine a fuzzy kernel estimation network, an optical flow estimation network, a feature extraction network and a potential high-resolution intermediate Frame reconstruction network; based on the fuzzy kernel estimation network, use the first resolution video sequence to estimate the blur kernel; secondly, determine the deformation matrix based on the optical flow estimation network and the first resolution video sequence; then use the feature extraction network to extract the first resolution video sequence
  • the features of each video frame in , align the features of each video frame according to the deformation matrix; again use the potential high-resolution intermediate frame reconstruction network and the features of each video frame after alignment to construct the second resolution intermediate video frame; finally, based on the second resolution
  • the rate of the intermediate video frame determines the second resolution video.
  • Embodiment 1 is a flowchart of a method for blind video super-resolution reconstruction based on self-supervised learning according to Embodiment 1 of the present invention
  • FIG. 2 is a network structure diagram of a fuzzy kernel estimation network in Embodiment 1 of the present invention.
  • FIG. 3 is a network structure diagram of a feature extraction network in Embodiment 1 of the present invention.
  • FIG. 4 is a network structure diagram of a potential high-resolution intermediate frame reconstruction network in Embodiment 1 of the present invention.
  • FIG. 5 is a structural diagram of a video blind super-resolution reconstruction system based on self-supervised learning according to Embodiment 2 of the present invention.
  • the purpose of the present invention is to provide a video blind super-resolution reconstruction method and system based on self-supervised learning, so as to improve the phenomenon of false artifacts and wrong structural information when reconstructing high-resolution video.
  • a video blind super-resolution reconstruction method based on self-supervised learning includes:
  • S1 Determine a first resolution video sequence based on the first resolution video.
  • a self-supervised learning method is used to determine a blur kernel estimation network, an optical flow estimation network, a feature extraction network, and a potentially high-resolution intermediate frame reconstruction network.
  • S4 Determine a deformation matrix based on the optical flow estimation network and the first resolution video sequence.
  • S5 Extract the features of each video frame in the first resolution video sequence by using the feature extraction network, align the features of each video frame according to the deformation matrix, and obtain the features of each video frame after alignment.
  • S6 Use the potential high-resolution intermediate frame reconstruction network and the features of the aligned video frames to construct a second-resolution intermediate video frame.
  • S7 Determine a second resolution video based on the second resolution intermediate video frame; the resolution of the second resolution video is greater than the resolution of the first resolution video.
  • S1 Determine a first resolution video sequence based on the first resolution video; the first resolution video sequence includes a total of 2N+1 video frames; the first resolution is low resolution, that is, the first resolution video is low resolution high-resolution video, the first-resolution video sequence is a low-resolution video sequence.
  • the first resolution video is obtained directly.
  • Adopt self-supervised learning method to determine blur kernel estimation network N k ( ), optical flow estimation network N f ( ), feature extraction network Ne ( ) and potential high-resolution intermediate frame reconstruction network N I ( ) , including:
  • L self represents the cycle consistency loss function
  • ⁇ ( ) represents a robust function, usually using the L 1 norm or the L 2 norm
  • S represents the downsampling matrix
  • K i represents the fuzzy matrix
  • x i represents the The second resolution intermediate video frame
  • y i represents the ith video frame in the first resolution video sequence.
  • Equation (2) uses only Equation (2) to constrain the training of the above network tends to lead to trivial solutions.
  • the present invention further excavates the properties of the fuzzy kernel and formula (1) to constrain the training of the above network.
  • the elements in the fuzzy kernel are often sparse.
  • the present invention introduces a hyper-Laplace prior to describe the fuzzy kernel.
  • the sparsity of that is, the training of the fuzzy kernel estimation network N k ( ) is constrained by minimizing the fuzzy kernel regularization loss function.
  • L k represents the fuzzy kernel regular loss function
  • K i represents the fuzzy kernel
  • ⁇ represents the hyperparameter, which is usually 0.5.
  • set M represents the accurate blur matrix, deformation matrix, and low-resolution video mapping function, respectively, which are:
  • x i M(y iN ,...,y i-1 ,y i ,y i+1 ,...,y i+N ) (4);
  • H i M(L iN ,...,L i-1 ,L i ,L i+1 ,...,L i+N ) (5);
  • the present invention can use the auxiliary data pair ⁇ L i , H i ⁇ to train the deep network, and the present invention uses the input first resolution video ⁇ y i ⁇ as the ⁇ L i , H i ⁇ here.
  • H i ⁇ uses an auxiliary reconstruction loss function to constrain the optical flow estimation network N f ( ), the feature extraction network Ne ( ) and the potential high-resolution intermediate frame reconstruction network N I ( ), so there are the following steps:
  • L I represents the auxiliary reconstruction loss function
  • ⁇ ( ) represents a robust function, usually using the L 1 norm or the L 2 norm
  • C[ ] represents the connection operation
  • N I ( ) represents the potential high Resolution intermediate frame reconstruction network
  • the optical flow used in the deformation is N f (L j ,L i ), and
  • L total represents the total loss function
  • L self represents the cycle consistency loss function
  • L k represents the fuzzy kernel regularization loss function
  • L I represents the auxiliary reconstruction loss function
  • ⁇ and ⁇ represent the hyperparameters.
  • K i N k (C[y iN ,...,y i-1 ,y i ,y i+1 ,...,y i+N ]) (8);
  • K i is the blur kernel
  • N k ( ⁇ ) is the fuzzy kernel estimation network
  • C[ ⁇ ] is the connection operation
  • y iN is the iNth video frame in the first resolution video sequence.
  • S4 Determine a deformation matrix based on the optical flow estimation network and the first resolution video sequence, specifically including:
  • u j ⁇ i is the optical flow of the jth video frame and the intermediate video frame in the first resolution video sequence
  • N f ( ) is the optical flow estimation network
  • y i is the intermediate video frame in the first resolution video sequence
  • y j is the jth video frame in the first resolution video sequence
  • j iN,...j-1,j+1,...,i+N.
  • the existing optical flow estimation algorithm PWC-Net is used as the optical flow estimation network.
  • the present invention provides a second technical solution, which is as follows:
  • S5 Use the feature extraction network to extract the features of each video frame in the first resolution video sequence, align the features of each video frame according to the deformation matrix, and obtain the features of each video frame after alignment, specifically including:
  • Ne ( ⁇ ) represents the feature extraction network.
  • each video frame after the alignment is the feature corresponding to the jth video frame
  • the features of each video frame after the alignment include:
  • N I ( ) is the potential high-resolution intermediate frame reconstruction network
  • C[ ] is the connection operation
  • x i is the second resolution intermediate video frame.
  • S7 Determine a second resolution video based on the second resolution intermediate video frame; the resolution of the second resolution video is greater than the resolution of the first resolution video.
  • the present invention also provides a video blind super-resolution reconstruction system based on self-supervised learning, the system includes:
  • the first resolution video sequence determination module 501 is configured to determine a first resolution video sequence based on the first resolution video.
  • the multi-network determination module 502 is used for determining a blur kernel estimation network, an optical flow estimation network, a feature extraction network and a potential high-resolution intermediate frame reconstruction network using a self-supervised learning method.
  • a blur kernel determination module 503, configured to estimate a blur kernel by using the first resolution video sequence based on the blur kernel estimation network.
  • a deformation matrix determination module 504 configured to determine a deformation matrix based on the optical flow estimation network and the first resolution video sequence.
  • the feature alignment module 505 is configured to extract the features of each video frame in the first resolution video sequence by using the feature extraction network, align the features of each video frame according to the deformation matrix, and obtain the features of each video frame after alignment.
  • a second-resolution intermediate video frame determining module 506 is configured to construct a second-resolution intermediate video frame by utilizing the potential high-resolution intermediate frame reconstruction network and the features of the aligned video frames.
  • a second resolution video determination module 507 configured to determine a second resolution video based on the second resolution intermediate video frame; the resolution of the second resolution video is greater than the resolution of the first resolution video.
  • the multi-network determination module 502 of the present invention specifically includes:
  • a blur matrix determination unit configured to determine a blur matrix according to the blur kernel through a convolution operation.
  • a cycle-consistent loss function construction unit configured to construct a cycle-consistency loss function according to the fuzzy matrix.
  • the fuzzy kernel canonical loss function building unit is used to construct the fuzzy kernel canonical loss function.
  • the auxiliary reconstruction loss function building unit is used to construct the auxiliary reconstruction loss function.
  • a total loss function construction unit configured to determine a total loss function according to the cycle consistency loss function, the fuzzy kernel regularization loss function and the auxiliary reconstruction loss function.
  • a multi-network determination unit for determining a blur kernel estimation network, an optical flow estimation network, a feature extraction network, and a potentially high-resolution intermediate frame reconstruction network when the total loss function is minimized.
  • the deformation matrix determination module 504 of the present invention specifically includes:
  • An optical flow determination unit configured to calculate the optical flow of each video frame and an intermediate video frame in the first resolution video sequence based on the optical flow estimation network.
  • a deformation matrix determination unit configured to calculate a deformation matrix according to the optical flow using a bilinear interpolation method; and determine a deformation operation according to the deformation matrix.
  • the feature alignment module 505 of the present invention specifically includes:
  • a feature determination unit configured to extract features of each video frame in the video sequence of the first resolution by using the feature extraction network.
  • the feature alignment unit is used for aligning the features of each video frame to the features of the intermediate video frames by using the deformation operation, and obtaining the features of each video frame after alignment.
  • the present invention utilizes the potential high-resolution intermediate frame reconstruction network and the features of the aligned video frames to construct a second-resolution intermediate video frame, and the specific formula is:
  • N I ( ) is the potential high-resolution intermediate frame reconstruction network
  • C[ ] is the connection operation
  • x i is the second resolution intermediate video frame.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

L'invention concerne un procédé et un système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé, le procédé consistant à : d'abord, déterminer, au moyen d'un procédé d'apprentissage auto-supervisé, un réseau d'estimation de noyau de flou, un réseau d'estimation de flux optique, un réseau d'extraction de caractéristiques et un réseau de reconstruction de trame intermédiaire à haute résolution potentielle ; estimer un noyau de flou à l'aide d'une séquence vidéo de première résolution sur la base du réseau d'estimation de noyau de flou ; puis, déterminer une matrice de déformation sur la base du réseau d'estimation de flux optique et de la séquence vidéo de première résolution ; puis extraire, au moyen du réseau d'extraction de caractéristiques, des caractéristiques de chaque trame vidéo dans la séquence vidéo de première résolution et aligner les caractéristiques de chaque trame vidéo selon la matrice de déformation ; construire des trames vidéo intermédiaires de seconde résolution en utilisant le réseau de reconstruction de trame intermédiaire à haute résolution potentielle et les caractéristiques alignées de chaque trame vidéo ; et enfin déterminer une vidéo de seconde résolution sur la base des trames vidéo intermédiaires de seconde résolution. Dans la présente invention, au moyen du procédé d'auto-supervision, des artéfacts et des informations de structure incorrecte peuvent être efficacement améliorés pendant une reconstruction vidéo à haute résolution, ce qui permet d'améliorer encore l'effet visuel.
PCT/CN2021/074281 2021-01-19 2021-01-29 Procédé et système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé WO2022155990A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110067977.5A CN112767250B (zh) 2021-01-19 2021-01-19 一种基于自监督学习的视频盲超分辨率重建方法及系统
CN202110067977.5 2021-01-19

Publications (1)

Publication Number Publication Date
WO2022155990A1 true WO2022155990A1 (fr) 2022-07-28

Family

ID=75703085

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/074281 WO2022155990A1 (fr) 2021-01-19 2021-01-29 Procédé et système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé

Country Status (2)

Country Link
CN (1) CN112767250B (fr)
WO (1) WO2022155990A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB202017662D0 (en) * 2020-11-09 2020-12-23 Samsung Electronics Co Ltd Method and apparatus for video super resolution

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003041562A2 (fr) * 2001-11-14 2003-05-22 Whitehead Institute For Biomedical Research Diagnostic d'un cancer moleculaire a l'aide d'une signature d'expression genique tumorale
US20140354886A1 (en) * 2013-05-29 2014-12-04 Yeda Research & Development Co. Ltd. Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence
CN105913394A (zh) * 2016-04-08 2016-08-31 南京理工大学 一种基于退化图像序列的图像盲复原方法
CN112218072A (zh) * 2020-10-10 2021-01-12 南京大学 一种基于解构压缩和融合的视频编码方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106251297A (zh) * 2016-07-19 2016-12-21 四川大学 一种改进的基于多幅图像模糊核估计的盲超分辨率重建算法
CN106934769A (zh) * 2017-01-23 2017-07-07 武汉理工大学 基于近景遥感的去运动模糊方法
CN108288072A (zh) * 2018-01-26 2018-07-17 深圳市唯特视科技有限公司 一种基于生成对抗网络的面部表情合成方法
CN111275626B (zh) * 2018-12-05 2023-06-23 深圳市炜博科技有限公司 一种基于模糊度的视频去模糊方法、装置及设备
CN110634108B (zh) * 2019-08-30 2023-01-20 北京工业大学 一种基于元-循环一致性对抗网络的复合降质网络直播视频增强方法
CN112200732B (zh) * 2020-04-30 2022-10-21 南京理工大学 一种清晰特征融合的视频去模糊方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003041562A2 (fr) * 2001-11-14 2003-05-22 Whitehead Institute For Biomedical Research Diagnostic d'un cancer moleculaire a l'aide d'une signature d'expression genique tumorale
US20140354886A1 (en) * 2013-05-29 2014-12-04 Yeda Research & Development Co. Ltd. Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence
CN105913394A (zh) * 2016-04-08 2016-08-31 南京理工大学 一种基于退化图像序列的图像盲复原方法
CN112218072A (zh) * 2020-10-10 2021-01-12 南京大学 一种基于解构压缩和融合的视频编码方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Master Thesis", 1 June 2020, HARBIN INSTITUTE OF TECHNOLOGY, CN, article BI MINMIN: "Research on Image Super Resolution Technology Based on Deep Learning", pages: 1 - 66, XP055952717 *
JINSHAN PAN; SONGSHENG CHENG; JIAWEI ZHANG; JINHUI TANG: "Deep Blind Video Super-resolution", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 10 March 2020 (2020-03-10), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081618462 *

Also Published As

Publication number Publication date
CN112767250A (zh) 2021-05-07
CN112767250B (zh) 2021-10-15

Similar Documents

Publication Publication Date Title
Dong et al. Multi-scale boosted dehazing network with dense feature fusion
US20210241470A1 (en) Image processing method and apparatus, electronic device, and storage medium
Chu et al. Temporally coherent gans for video super-resolution (tecogan)
He et al. A soft MAP framework for blind super-resolution image reconstruction
Xiao et al. Space-time distillation for video super-resolution
Wang et al. Semi-supervised student-teacher learning for single image super-resolution
WO2023184913A1 (fr) Procédé et appareil d'apprentissage de modèle, procédé de reconstruction en super-résolution, dispositif et support
Bare et al. Real-time video super-resolution via motion convolution kernel estimation
WO2022155990A1 (fr) Procédé et système de reconstruction de super-résolution à l'aveugle de vidéo basés sur un apprentissage auto-supervisé
CN110689509A (zh) 基于循环多列3d卷积网络的视频超分辨率重建方法
CN113643182A (zh) 一种基于对偶学习图网络的遥感影像超分辨率重建方法
Huang et al. Learning deformable and attentive network for image restoration
Liang et al. Video super-resolution reconstruction based on deep learning and spatio-temporal feature self-similarity
Wang et al. Self-supervised deep image restoration via adaptive stochastic gradient langevin dynamics
Bai et al. Self-supervised deep blind video super-resolution
Xu et al. Deformable kernel convolutional network for video extreme super-resolution
CN117058043A (zh) 一种基于lstm的事件-图像去模糊方法
CN117196948A (zh) 一种基于事件数据驱动的视频超分辨率方法
CN112215140A (zh) 一种基于时空对抗的3维信号处理方法
CN111767679A (zh) 时变矢量场数据的处理方法及装置
Fang et al. Self-enhanced convolutional network for facial video hallucination
Zhang et al. Optimization-inspired Cumulative Transmission Network for image compressive sensing
Wan et al. Progressive convolutional transformer for image restoration
WO2022221205A1 (fr) Super-résolution vidéo à l'aide de réseaux neuronaux profonds
CN114757826A (zh) 一种基于多特征的pocs图像超分辨率重建方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21920381

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21920381

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21920381

Country of ref document: EP

Kind code of ref document: A1