CN110852944A - Multi-frame self-adaptive fusion video super-resolution method based on deep learning - Google Patents
Multi-frame self-adaptive fusion video super-resolution method based on deep learning Download PDFInfo
- Publication number
- CN110852944A CN110852944A CN201910967482.0A CN201910967482A CN110852944A CN 110852944 A CN110852944 A CN 110852944A CN 201910967482 A CN201910967482 A CN 201910967482A CN 110852944 A CN110852944 A CN 110852944A
- Authority
- CN
- China
- Prior art keywords
- resolution
- network
- frame
- super
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000004927 fusion Effects 0.000 title claims abstract description 19
- 238000013135 deep learning Methods 0.000 title claims abstract description 11
- 230000003044 adaptive effect Effects 0.000 claims abstract description 23
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 23
- 238000012549 training Methods 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 9
- 238000013178 mathematical model Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 239000006185 dispersion Substances 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention provides a multi-frame self-adaptive fusion video super-resolution method based on deep learning and electronic equipment thereof, comprising the following steps: the method comprises the following steps of firstly, constructing a data set required by training the network of the invention; and secondly, constructing a multi-frame self-adaptive fusion video super-resolution network through a deep learning framework TensorFlow, wherein the multi-frame self-adaptive fusion video super-resolution network is divided into two parts: the multi-frame adaptive registration network can distort adjacent frames of the key frames needing super resolution to enable the adjacent frames to be the same as the content of the key frames so as to provide more detail information for an algorithm, and the super resolution network super resolves the output of the multi-frame adaptive registration network into a high-resolution frame image; and (5) training.
Description
Technical Field
The invention relates to a video super-resolution algorithm based on a convolutional neural network, and relates to a multi-frame self-adaptive fusion video image registration algorithm.
Background
High-resolution video brings clearer and more comfortable visual experience to users, so that technical research related to the high-resolution video is widely regarded by students. In recent years, a rapidly developed video super-resolution technology is taken as a new technology for obtaining high-definition images at low cost, has huge commercial values in multiple industries such as security protection, finance, modern logistics and the like, and becomes a leading-edge technology for competition of large companies. The basic task of the super-resolution technology is to reconstruct a corresponding high-resolution (HR) image or video from an original low-resolution (LR) image or video, which is a typical pathological problem. Currently, some solutions have been proposed by scholars.
The existing super-resolution algorithm is mainly realized by the following two ways: 1) the method is realized by adding constraint to the reconstruction process by using the prior knowledge of the structure or the content in the picture, for example, the smoothness of the image is used for realizing the super-resolution effect; 2) the method is also an implementation mode of the current best reconstruction effect algorithm. The specific implementation modes include a dictionary learning strategy, a random forest strategy and a neural network strategy. The single frame super resolution technology refers to a super resolution technology in which an input is one image. Multi-frame super resolution refers to a technique for reconstructing a high resolution video frame from a plurality of consecutive low resolution video frames. Compared with a single-frame super-resolution technology, the multi-frame super-resolution algorithm considers that the information between the frame images is complementary, and the algorithm can utilize the redundant information to improve the super-resolution effect.
The core problem of multi-frame super-resolution algorithm design is to find an effective method for realizing registration between continuous video frames. Recent research shows that information of a plurality of adjacent low-resolution frames can be fused by combining a Convolutional Neural Network (CNN) with a motion compensation principle, and then image registration is realized. Currently, the mainstream multi-frame super-resolution algorithm usually uses a set of continuous low-resolution images with a fixed number of frames to generate a single-frame high-resolution image. However, the multi-frame super-resolution algorithm based on the fixed frame number has the following two problems: 1) when the image content difference between adjacent frames is very large, if the selected frame number is too large, great difficulty is brought to image registration, and the fused video is easy to have bad flicker phenomenon to influence the user experience; 2) when the frame number is too small, the redundant information of the adjacent frames cannot be fully utilized. How to adaptively fuse image information predicted by different frame numbers is very important.
Disclosure of Invention
Aiming at the defects of the traditional multi-frame super-resolution algorithm with the fusion of fixed frame numbers in the aspect of effective utilization of multi-frame images, the invention provides a multi-frame super-resolution algorithm with the self-adaptive fusion of predicted images with different frame numbers. The algorithm can better adapt to the fluctuation of the image content difference between adjacent frames, so that a more stable and clear super-resolution effect can be obtained. The technical scheme is as follows:
a multi-frame self-adaptive fusion video super-resolution method based on deep learning comprises the following steps:
first, a data set required to train the network of the present invention is constructed
Reading the video frames by frames in the existing video data set into images and storing the images as a high-resolution image set YHRThen, the high resolution image set Y is collectedHREach image in the image group is down-sampled to obtain a corresponding low-resolution image set YLR。
Secondly, a multi-frame self-adaptive fusion video super-resolution network is built through a deep learning framework TensorFlow
The multi-frame self-adaptive fusion video super-resolution network is divided into two parts: the multi-frame adaptive registration network and the super-resolution network, wherein the multi-frame adaptive registration network can distort the adjacent frames of the key frames needing super-resolution to make the adjacent frames and the key frames tend to have the same content so as to provide more detail information for an algorithm, and the super-resolution network super-resolves the output of the multi-frame adaptive registration network into a high-resolution frame image, comprising the following steps:
(1) the multi-frame adaptive registration network is divided into different frames according to different lengths of video framesThree subsections: respectively a key frame direct output part, a three-frame motion registration part and a five-frame motion registration part; the three-frame motion registration part and the five-frame motion registration part are respectively composed of eight layers of convolutional neural networks and recorded as FNet, a ReLU function is selected after each convolutional layer to serve as an activation function, the former three layers of convolutional neural networks realize the down-sampling function of an image through twice-maximum pooling, the latter three layers of convolutional neural networks realize the up-sampling function through bicubic interpolation, and a key frame needing super resolution is set as an nth frame and recorded as an InThe following is a mathematical model of the multi-frame adaptive registration network:
Fout＝[α·FNet(In-2,In-1,In,In+1,In+2)+β·FNet(In-1,In,In+1)+γ·FNet(In)]
wherein FoutRepresenting the output of the multi-frame adaptive registration network, α, gamma represents the weights corresponding to the five-frame motion registration part, the three-frame motion registration part and the key frame direct output part, In-2,In-1,In+1,In+2Respectively representing the first two frames of images and the second two frames of images of the key frame;
(2) super-resolution network FSRThe method comprises a plurality of convolution layers, a ReLU function is connected behind each convolution layer as an activation function, the network realizes the up-sampling of images by connecting two deconvolution layers, and the input and the output of the network are directly connected to prevent the gradient dispersion problem, IoutFor super-resolution networks FSRThe output of (2), the mathematical model of the super-resolution network is as follows:
Yout＝FSR(Fout)
third, using the high resolution image set Y obtained in the first stepHRAnd a low resolution image set YLRTraining the designed network, wherein the loss of the network is defined as L2Loss:
Loss＝(Yout-YHR)2
wherein Y isoutOutput of super-resolution network, training completionThen saving the structure and parameters of the network;
and fourthly, setting the low-resolution video needing super resolution as V, taking the low-resolution video V as the input of the network stored in the third step, and correspondingly outputting the low-resolution video as the high-resolution video needing super resolution, thereby finishing the super resolution process of the video.
Preferably, in the third step, the network optimizer is set to Adam; one training batch was set to 128 images; the initial learning rate of the network is set to 0.01; when the loss of 100 consecutive epochs does not obviously decrease, the loss is reduced by 10 times, and the final learning rate is set to be 10-5(ii) a Training epoch is set to 5000.
The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method steps when executing the program.
Compared with the traditional video super-resolution model with fixed frame number, the multi-frame self-adaptive fusion video super-resolution algorithm based on deep learning has stronger robustness under the condition that the difference of image contents between adjacent frames fluctuates greatly. The problems that image registration difficulty is increased and redundant information between adjacent frames is difficult to make full use of caused by a traditional video super-resolution algorithm with a fixed frame number are effectively solved, and flicker of a super-resolution result is effectively avoided.
The model designed by the invention can be widely used for super-resolution processing of low-quality videos, and the invention can fully consider the difference between the contents of adjacent video frames to select more appropriate network parameters to carry out super-resolution processing on the target video.
Drawings
FIG. 1 is an overall structure of a multi-frame adaptive fusion video super-resolution network
FIG. 2 is a structure of a multi-frame registration network
Fig. 3 shows the super-resolution result of the algorithm of the present invention for the same video frame, where the four images are: original low-resolution image, results of bicubic up-sampling, results of VESPCN video super-resolution network and results of the invention
FIG. 4 is a flow chart of the algorithm of the present invention
TABLE 1 parameters for a multi-frame registration network
TABLE 2 parameters of the image super-resolution network
Detailed Description
The mathematical model and the specific implementation of the deep learning based multi-frame adaptive fusion video super-resolution algorithm of the present patent are described in detail below with reference to the following examples and the accompanying drawings, and the specific flowchart is given by fig. 4:
firstly, constructing a data set required by training the network of the invention, namely reading the video in the Vimeo-90k video data set into images frame by frame and storing the images, and recording the images as a high-resolution image set YHRThen the high resolution image set Y is collected through matlabHREach image in the image group is down-sampled to obtain a corresponding low-resolution image set YLR。
And secondly, constructing a multi-frame self-adaptive fusion video super-resolution network through a deep learning framework TensorFlow. As shown in fig. 1, which is an overall framework of the network of the present invention, the multi-frame adaptive fusion video super resolution network is divided into two parts: a multi-frame adaptive registration network and a super-resolution network. The multi-frame adaptive registration network can warp the adjacent frames of the key frames needing super resolution to make the adjacent frames and the key frames tend to have the same content, so as to provide more detail information for an algorithm. The super-resolution network super-resolves the output of the multi-frame adaptive registration network into a high-resolution frame image, which is specifically as follows:
(1) the multi-frame adaptive registration network is divided into three subsections according to the length of the video frame: respectively a key frame direct output part, a three-frame motion registration part and a five-frame motion registration part. Wherein the three-frame motion registration part and the five-frame motion registration part are respectively composed of eight layers of convolutional neural networks and are marked as FNet. The structure of FNet is shown in FIG. 2, specific parameters are given in Table 1, a ReLU function is selected as an activation function after each convolution layer, the down-sampling function of an image is realized by the first three layers of convolutional neural networks through twice maximal pooling, and the next three layers of convolutional neural networks are realized by the double layers of convolutional neural networksCubic interpolation implements the upsampling function. Suppose the key frame of the required super resolution is the nth frame (denoted as I)n) The following is a mathematical model of the multi-frame adaptive registration network:
Fout＝[α·FNet(In-2,In-1,In,In+1,In+2)+β·FNet(In-1,In,In+1)+γ·FNet(In)]
wherein FoutRepresenting the output of the multi-frame adaptive registration network, α, gamma represents the weights corresponding to the five-frame motion registration part, the three-frame motion registration part and the key frame direct output part, In-2,In-1,In+1,In+2The first two images and the second two images respectively represent the key frame.
(3) Super-resolution network FSRThe structure of (1) is shown in the right half of fig. 1, specific parameters are shown in table 2, the structure comprises 12 convolutional layers, each convolutional layer is connected with a ReLU function as an activation function, and the network finally realizes the up-sampling of the image by connecting two deconvolution layers. Structurally, the present invention directly connects the inputs and outputs of the network to prevent the gradient dispersion problem from occurring during the training process. I isoutFor super-resolution networks FSRThe output of (2), the mathematical model of the super-resolution network is as follows:
Iout＝FSR(Fout)
third, using the high resolution image set Y obtained in the first stepHRAnd a low resolution image set YLRTraining the designed network, wherein the loss of the network is defined as L2The losses are specifically as follows:
Loss＝(Yout-YHR)2
wherein Y isoutAnd (4) outputting the super-resolution network. The network optimizer is set to Adam; one training batch was set to 128 images; the initial learning rate of the network is set to 0.01; when the loss of 100 consecutive epochs does not obviously decrease, the loss is reduced by 10 times, and the final learning rate is set to be 10-5(ii) a Training epoch is set to 5000. And after the training is finished, the structure and the parameters of the network are saved.
And fourthly, assuming that the low-resolution video needing super resolution is V, only the low-resolution video V is needed to be used as the input of the network stored in the third step, and the corresponding output is the high-resolution video needing super resolution, so that the video super resolution process is completed.
The invention takes the same low-resolution frame image as the input of the network, and compares the output result with other classical methods, and the comparison result is shown in fig. 3. From the results, it can be seen that our algorithm achieves better results than other algorithms.
TABLE 1
TABLE 2
Claims (3)
1. A multi-frame self-adaptive fusion video super-resolution method based on deep learning comprises the following steps:
first, a data set required to train the network of the present invention is constructed
Reading the video frames by frames in the existing video data set into images and storing the images as a high-resolution image set YHRThen, the high resolution image set Y is collectedHREach image in the image group is down-sampled to obtain a corresponding low-resolution image set YLR。
Secondly, a multi-frame self-adaptive fusion video super-resolution network is built through a deep learning framework TensorFlow
The multi-frame self-adaptive fusion video super-resolution network is divided into two parts: the multi-frame adaptive registration network and the super-resolution network, wherein the multi-frame adaptive registration network can distort the adjacent frames of the key frames needing super-resolution to make the adjacent frames and the key frames tend to have the same content so as to provide more detail information for an algorithm, and the super-resolution network super-resolves the output of the multi-frame adaptive registration network into a high-resolution frame image, comprising the following steps:
(1) the multi-frame adaptive registration network is divided into three subsections according to the length of the video frame: respectively a key frame direct output part, a three-frame motion registration part and a five-frame motion registration part; the three-frame motion registration part and the five-frame motion registration part are respectively composed of eight layers of convolutional neural networks and recorded as FNet, a ReLU function is selected after each convolutional layer to serve as an activation function, the former three layers of convolutional neural networks realize the down-sampling function of an image through twice-maximum pooling, the latter three layers of convolutional neural networks realize the up-sampling function through bicubic interpolation, and a key frame needing super resolution is set as an nth frame and recorded as an InThe following is a mathematical model of the multi-frame adaptive registration network:
Fout＝[α·FNet(In-2,In-1,In,In+1,In+2)+β·FNet(In-1,In,In+1)+γ·FNet(In)]
wherein FoutRepresenting the output of the multi-frame adaptive registration network, α, gamma represents the weights corresponding to the five-frame motion registration part, the three-frame motion registration part and the key frame direct output part, In-2,In-1,In+1,In+2Respectively representing the first two frames of images and the second two frames of images of the key frame;
(2) super-resolution network FSRThe method comprises a plurality of convolution layers, a ReLU function is connected behind each convolution layer as an activation function, the network realizes the up-sampling of images by connecting two deconvolution layers, and the input and the output of the network are directly connected to prevent the gradient dispersion problem, IoutFor super-resolution networks FSRThe output of (2), the mathematical model of the super-resolution network is as follows:
Yout＝FSR(Fout)
third, using the high resolution image set Y obtained in the first stepHRAnd a low resolution image set YLRTraining the designed network, wherein the loss of the network is defined as L2Loss:
Loss＝(Yout-YHR)2
wherein Y isoutOutputting a super-resolution network, and storing the structure and parameters of the network after training;
and fourthly, setting the low-resolution video needing super resolution as V, taking the low-resolution video V as the input of the network stored in the third step, and correspondingly outputting the low-resolution video as the high-resolution video needing super resolution, thereby finishing the super resolution process of the video.
2. The method according to claim 1, characterized in that in the third step the network optimizer is set to Adam; one training batch was set to 128 images; the initial learning rate of the network is set to 0.01; when the loss of 100 consecutive epochs does not obviously decrease, the loss is reduced by 10 times, and the final learning rate is set to be 10-5(ii) a Training epoch is set to 5000.
3. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1-2 are implemented when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910967482.0A CN110852944B (en) | 2019-10-12 | 2019-10-12 | Multi-frame self-adaptive fusion video super-resolution method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910967482.0A CN110852944B (en) | 2019-10-12 | 2019-10-12 | Multi-frame self-adaptive fusion video super-resolution method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110852944A true CN110852944A (en) | 2020-02-28 |
| CN110852944B CN110852944B (en) | 2023-11-21 |
Family
ID=69596494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910967482.0A Active CN110852944B (en) | 2019-10-12 | 2019-10-12 | Multi-frame self-adaptive fusion video super-resolution method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852944B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111586412A (en) * | 2020-05-06 | 2020-08-25 | 华为技术有限公司 | High-definition video processing method, master device, slave device and chip system |
| CN111696049A (en) * | 2020-05-07 | 2020-09-22 | 中国海洋大学 | Deep learning-based underwater distorted image reconstruction method |
| CN112019861A (en) * | 2020-07-20 | 2020-12-01 | 清华大学 | Video compression method and device based on keyframe guidance super-resolution |
| CN113592719A (en) * | 2021-08-14 | 2021-11-02 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video processing method and corresponding equipment |
| CN113610713A (en) * | 2021-08-13 | 2021-11-05 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video super-resolution method and device |
| US11954910B2 (en) | 2020-12-26 | 2024-04-09 | International Business Machines Corporation | Dynamic multi-resolution processing for video classification |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107480772A (en) * | 2017-08-08 | 2017-12-15 | 浙江大学 | A kind of car plate super-resolution processing method and system based on deep learning |
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN108921786A (en) * | 2018-06-14 | 2018-11-30 | 天津大学 | Image super-resolution reconstructing method based on residual error convolutional neural networks |
| CN109102462A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | A kind of video super-resolution method for reconstructing based on deep learning |
| CN110120011A (en) * | 2019-05-07 | 2019-08-13 | 电子科技大学 | A kind of video super resolution based on convolutional neural networks and mixed-resolution |
-
2019
- 2019-10-12 CN CN201910967482.0A patent/CN110852944B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN107480772A (en) * | 2017-08-08 | 2017-12-15 | 浙江大学 | A kind of car plate super-resolution processing method and system based on deep learning |
| CN108921786A (en) * | 2018-06-14 | 2018-11-30 | 天津大学 | Image super-resolution reconstructing method based on residual error convolutional neural networks |
| CN109102462A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | A kind of video super-resolution method for reconstructing based on deep learning |
| CN110120011A (en) * | 2019-05-07 | 2019-08-13 | 电子科技大学 | A kind of video super resolution based on convolutional neural networks and mixed-resolution |
Non-Patent Citations (1)
| Title |
|---|
| 杨学峰;王高;程耀瑜: "基于径向基函数的多帧图像超分辨重建算法", 计算机应用, vol. 34, no. 1, pages 142 - 144 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111586412A (en) * | 2020-05-06 | 2020-08-25 | 华为技术有限公司 | High-definition video processing method, master device, slave device and chip system |
| CN111696049A (en) * | 2020-05-07 | 2020-09-22 | 中国海洋大学 | Deep learning-based underwater distorted image reconstruction method |
| CN112019861A (en) * | 2020-07-20 | 2020-12-01 | 清华大学 | Video compression method and device based on keyframe guidance super-resolution |
| CN112019861B (en) * | 2020-07-20 | 2021-09-14 | 清华大学 | Video compression method and device based on keyframe guidance super-resolution |
| US11954910B2 (en) | 2020-12-26 | 2024-04-09 | International Business Machines Corporation | Dynamic multi-resolution processing for video classification |
| CN113610713A (en) * | 2021-08-13 | 2021-11-05 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video super-resolution method and device |
| CN113610713B (en) * | 2021-08-13 | 2023-11-28 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video super-resolution method and device |
| CN113592719A (en) * | 2021-08-14 | 2021-11-02 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video processing method and corresponding equipment |
| CN113592719B (en) * | 2021-08-14 | 2023-11-28 | 北京达佳互联信息技术有限公司 | Training method of video super-resolution model, video processing method and corresponding equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110852944B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110852944B (en) | Multi-frame self-adaptive fusion video super-resolution method based on deep learning | |
| CN111242846B (en) | Fine-grained scale image super-resolution method based on non-local enhancement network | |
| CN110136062B (en) | Super-resolution reconstruction method combining semantic segmentation | |
| CN113837946B (en) | Lightweight image super-resolution reconstruction method based on progressive distillation network | |
| CN112837224A (en) | Super-resolution image reconstruction method based on convolutional neural network | |
| Luo et al. | Lattice network for lightweight image restoration | |
| CN110363068A (en) | A kind of high-resolution pedestrian image generation method based on multiple dimensioned circulation production confrontation network | |
| CN111932461A (en) | Convolutional neural network-based self-learning image super-resolution reconstruction method and system | |
| CN113793286B (en) | Media image watermark removing method based on multi-order attention neural network | |
| CN113781308A (en) | Image super-resolution reconstruction method and device, storage medium and electronic equipment | |
| CN116091313A (en) | Image super-resolution network model and reconstruction method | |
| CN114881856A (en) | Human body image super-resolution reconstruction method, system, device and storage medium | |
| CN114841859A (en) | Single-image super-resolution reconstruction method based on lightweight neural network and Transformer | |
| Li et al. | High-resolution network for photorealistic style transfer | |
| Li et al. | D2c-sr: A divergence to convergence approach for real-world image super-resolution | |
| Li et al. | Image super-resolution reconstruction based on multi-scale dual-attention | |
| CN112215140A (en) | 3-dimensional signal processing method based on space-time countermeasure | |
| Peng | Super-resolution reconstruction using multiconnection deep residual network combined an improved loss function for single-frame image | |
| CN114494022B (en) | Model training method, super-resolution reconstruction method, device, equipment and medium | |
| CN107247944B (en) | Face detection speed optimization method and device based on deep learning | |
| CN112016456B (en) | Video super-resolution method and system based on adaptive back projection depth learning | |
| CN110895790B (en) | Scene image super-resolution method based on posterior degradation information estimation | |
| CN111667401B (en) | Multi-level gradient image style migration method and system | |
| CN108259779A (en) | A kind of method that quick processing large format video image is realized using part breadth data | |
| CN114862679A (en) | Single-image super-resolution reconstruction method based on residual error generation countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |