CN115546030B - Compressed video super-resolution method and system based on twin super-resolution network - Google Patents
Compressed video super-resolution method and system based on twin super-resolution network Download PDFInfo
- Publication number
- CN115546030B CN115546030B CN202211515828.1A CN202211515828A CN115546030B CN 115546030 B CN115546030 B CN 115546030B CN 202211515828 A CN202211515828 A CN 202211515828A CN 115546030 B CN115546030 B CN 115546030B
- Authority
- CN
- China
- Prior art keywords
- super
- resolution
- video
- twin
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 14
- 238000005070 sampling Methods 0.000 claims description 18
- 101100029436 Trypanosoma brucei brucei PFRB gene Proteins 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 5
- 238000007906 compression Methods 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 230000015556 catabolic process Effects 0.000 abstract description 3
- 238000006731 degradation reaction Methods 0.000 abstract description 3
- 230000000052 comparative effect Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a compressed video super-resolution method and a system based on a twin super-resolution network, wherein a low-resolution video to be processed is input into the twin super-resolution network to obtain a super-resolution video; during training, the invention compresses the original high-quality video in an M-JPEG format. And then inputting the two types of data of the compressed version and the uncompressed version into the twin super-resolution network. Finally, constraining the training of the twin super-resolution network from four aspects, and the features extracted by the two types of videos through the encoder should be as close as possible; the hyper-score results and the true values of the two types of videos form a triple together, and elements in the triple are positive samples; negative examples are the result of over-scoring of the residual between the compressed data and the conventional degradation. By means of the comparative learning mode, the twin super-resolution network can learn the characteristic parameters required by compressed video super-resolution to the maximum extent, and a more precise super-resolution reconstruction result is obtained.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence video processing, relates to a video super-resolution method and a system, and particularly relates to a compressed video super-resolution method and a system based on a twin super-resolution network.
Background
The super-resolution is a technology for processing various images and videos with poor quality into high resolution based on intelligent methods such as deep learning. In recent years, with the development of deep learning techniques, the super-resolution field based on deep learning has been developed at an unprecedented rate. At present, the application of the super-resolution technology is mainly embodied in the fields of recovering old photos and videos, recovering important ancient data, obtaining clearer monitoring videos, photographing and imaging of mobile phone cameras and the like.
When generating high and low resolution training data pairs, the traditional super-resolution method generally performs four-time down-sampling after performing gaussian blurring on a video to generate low resolution data. According to such paradigm, the hyper-molecular model has gained better and better performance in recent years. However, either applying the thus trained models directly to the compressed data or simply retraining the models on the compressed video data pairs suffers from a very severe performance penalty. Video compression is a key step of scenes such as video transmission, and therefore, super-resolution of compressed videos is of great practical significance.
The video compression process loses high-frequency detail information, so that super-resolution reconstruction of the compressed video is more difficult. The comparison learning is a typical unsupervised or self-supervised learning method, and the main idea is that the model can more fully learn the characteristics of the encoder, reduce the distance between similar samples as much as possible, and enlarge the distance between the compressed video sample and the negative sample, so that the clustering boundary is more obvious. Compared learning is used as a learning strategy, and a good effect is achieved in the fields of image super-resolution and the like. However, in general, existing research only simply uses a frame of a non-current hyper-resolution video as a negative sample, and the hyper-resolution effect of contrast learning is restricted.
Disclosure of Invention
In order to solve the technical problem, the invention provides a compressed video super-resolution method and a compressed video super-resolution system of a super-resolution network. Based on a contrast learning framework, a non-compressed video and a compressed video which are subjected to traditional Gaussian blur and bicubic down-sampling are input into a twin super-resolution network together, so that the twin super-resolution network is prompted to learn a feature expression which is insensitive to compression distortion, and the twin super-resolution network which is beneficial to compressing the video is obtained.
The method adopts the technical scheme that: a twin compressed video super-resolution method based on a super-resolution network comprises the steps that a low-resolution video to be processed is input into the twin super-resolution network to obtain a super-resolution video;
the twin super-resolution network consists of two parallel super-resolution networks; the super-resolution network comprises an encoder network and an up-sampling module;
the encoder network consists of two types of convolution layers and a plurality of PFRB modules; the first type of convolution layer takes the central frame of input continuous three-frame images as input, the size of each frame is 64 multiplied by 3, the convolution layer converts the size of the convolution layer into a Batchsize multiplied by 64 multiplied by 80 feature map, the second type of convolution layer takes other supplementary frames except the central frame as input, the size of the input data is Batchsize multiplied by 64 multiplied by 6, the convolution layer converts the data into the Batchsize multiplied by 64 multiplied by 80 feature map, and finally, the two features are input into a continuous PFRB module together with the super-separation result before the previous round of sampling; two encoder networks corresponding to the two inputs share a weight; wherein the batch size is the number of batch samples optimized by gradient;
in the PFRB module, data is firstly input into three convolution layers with the size of 3 x 3, the number of input and output channels of the three convolution layers is 80, the number of the input and output channels is recorded as x1, then the output of the convolution layers is spliced, a 1 x1 convolution layer is used for converting the total channel number 240 into 80, the obtained results are respectively spliced with three results in the x1, the obtained result is recorded as x2, finally, the three results in the x2 are respectively input into the three convolution layers, the number of channels of input data is compressed into 80, then the obtained three results and the initially input three results are correspondingly added to obtain a final result; wherein, leakyReLU activation function is used between each convolution layer to introduce nonlinear relation;
the up-sampling module inputs data with the size of batchsize multiplied by 3 multiplied by 64 multiplied by 80, firstly passes through a 3 multiplied by 3 convolutional layer, converts the number of data channels 240 into 80, continues to input into the convolutional layer, and converts the number of channels 80 into 48; upsampling the data with the scale of 2 by using a pixelshuffle function in the pytorech, and compressing the number of channels to 12; the obtained result is processed by a convolution layer, and finally, a pixelschuffle function is used for carrying out primary up-sampling with the scale of 2; the size of the output super-divided data is batch size × 3 × 256 × 256 × 3.
The technical scheme adopted by the system of the invention is as follows: a twin super-resolution network-based compressed video super-resolution system, comprising:
one or more processors;
a storage device to store one or more programs that when executed by the one or more processors cause the one or more processors to implement the twin super-resolution network-based compressed video super-resolution method.
The invention has the advantages and positive effects that:
(1) The invention defines the negative sample as the difference between the compressed data and BD data corresponding to the sample, and has better effect compared with the simple definition of other samples in the past method. In contrast to a single sample input network, negative samples, positive samples along with compressed video samples can cause the network to over-divide the compressed video samples.
(2) The method realizes the super-resolution model of the compressed video, and solves the problem that the effect of the traditional model is sharply reduced on the compressed video.
Drawings
FIG. 1 is a diagram of a twin super-resolution network structure according to an embodiment of the present invention;
FIG. 2 is a flow chart of the training of the twin super-resolution network according to the embodiment of the present invention;
FIG. 3 is a diagram of the architecture of a predecessor and successor of a twin super-resolution network according to an embodiment of the present invention;
FIG. 4 shows the subjective experimental results of the twin super-resolution network according to the embodiment of the present invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail with reference to the accompanying drawings and examples, it is to be understood that the embodiments described herein are merely illustrative and explanatory of the present invention and are not restrictive thereof.
The existing convolutional neural network structure can achieve a good effect on BD degradation, but once a scene is switched, the performance of a model is greatly reduced. And 3 coupled samples of the original frame, the BD degraded frame and the compressed frame are sent into a twin network for comparison learning, so that a super-resolution model which is good in performance on a compressed video can be obtained.
The invention provides a twin super-resolution network-based compressed video super-resolution method, which comprises the steps of inputting a low-resolution video to be processed into a twin super-resolution network of the embodiment to obtain a super-resolution video;
referring to fig. 1 and 4, the twin super-resolution network of the present embodiment is composed of two parallel super-resolution networks; the super-resolution network of the embodiment comprises an encoder network and an up-sampling module;
the encoder network of the present embodiment refers to a network structure except for the upsampling operation in the network shown in fig. 1, and is composed of two types of convolution layers and a plurality of PFRB modules; the first type of convolution layer takes the central frame of the input continuous three-frame image as input, the size of each frame is 64 multiplied by 3, the convolution layer converts the size of the convolution layer into a Batchsize multiplied by 64 multiplied by 80 feature map, the second type of convolution layer takes the supplementary frames except the central frame as input, the size of the input data is Batchsize multiplied by 64 multiplied by 6, the convolution layer converts the data into the Batchsize multiplied by 64 multiplied by 80 feature map, and finally, the two features are input into a continuous PFRB module together with the super-division result before the previous round of up-sampling; two encoder networks corresponding to the two inputs share a weight; wherein, batchsize is the batch sample number of gradient optimization, and this embodiment takes value 16.
The PFRB module of this embodiment adopts the basic framework of ResNet, data is first input into three convolutional layers of 3 × 3 size, the number of input/output channels of the three convolutional layers is all 80, the output is recorded as x1, then the outputs of the convolutional layers are spliced, then one 1 × 1 convolutional layer is used to convert the total channel number 240 into 80, and the obtained results are respectively spliced with three results in x1, the obtained result is recorded as x2, finally, the three results in x2 are respectively input into the three convolutional layers, the number of channels of input data is compressed into 80, and then the obtained three results are correspondingly summed with the initially input three results to obtain a final result; wherein, leakyReLU activation function is used between each convolution layer to introduce nonlinear relation, so that the expression capability of the model is stronger;
the up-sampling module of this embodiment is to up-sample the input data to achieve the purpose of overdividing. The size of the input data of the up-sampling module is Batchsize multiplied by 3 multiplied by 64 multiplied by 80, firstly, the data channel number 240 is converted into 80 through a 3 multiplied by 3 convolutional layer, the data channel number is continuously input into the convolutional layer, and the channel number 80 is converted into 48; upsampling the data with the scale of 2 by using a pixelshuffle function in the pytorech, and compressing the number of channels to 12; the obtained result is processed by a convolution layer, and finally, a pixelschuffle function is used for carrying out primary up-sampling with the scale of 2; in general, up-sampling with a scale of 4 is performed, the super-resolution process is realized, and the size of the outputted super-resolution data is batchsize × 3 × 256 × 256 × 3, and the batchsize is set to 16 in this embodiment.
Referring to fig. 2, the twin super-resolution network of the present embodiment is a trained twin super-resolution network; the training process comprises the following steps:
step 1: performing M-JPEG compression on a video in an original data set and bicubic downsampling processing to obtain a low-resolution compressed image which is recorded as a compressed video aiming at the original data set comprising a plurality of high-resolution continuous frames; performing Gaussian blur and bicubic downsampling processing on the video in the original data set to obtain a traditional low-quality image, and recording the image as a BD video;
the original data set name used in the present embodiment is MM522, in which 522 videos of various types are contained; in this embodiment, a video is read from the selected original data set by using a function in a python scroll library, and is written in by using an M-JEPG format to obtain a compressed video, and finally bicubic downsampling is performed on the video by using an ims function in Matlab.
Step 2: the method comprises the steps that an encoder network is utilized to carry out primary feature extraction on an input video, for input 5-dimensional video data, the encoder network outputs a 4-dimensional feature, the 4-dimensional feature is processed through a LeakyReLU activation function, the main purpose is to introduce a nonlinear function relation and restrain loss between two extracted features;
and step 3: respectively sending the extracted features into an up-sampling module to continuously perform super-division, and outputting a video which is amplified by four times compared with the input length and width; respectively constraining the relationship between the two outputs and the truth value to enable the outputs and the truth value to be as close as possible to be used as the distance between the compressed video sample and the positive sample; recording the distance between the residual error after the over-score and the difference between the true value video and the compressed video as the distance between the compressed video sample and the negative sample, wherein the distance needs to be as large as possible in a set range;
in this embodiment, the super-resolution result of the BD video and the compressed video is obtained, and the BD video and the compressed video are constrained with the true value to obtain the lossdis positive ；
Wherein the content of the first and second substances,y BD refers to the result after the super-resolution of BD video,y compress refers to the result after the super-resolution of the compressed video,gtit refers to the actual value of the light,εrepresents a constant value, set to 1e-4 in the experiment, which acts to maintain the loss value stable; will be provided withdis positive Noting the distance between the compressed video sample and the positive sample;
considering the negative sample and the distance between the negative sample and the compressed video sample, subtracting the compressed video and the BD compressed video to obtain a negative sample, and recording the negative sample asx nega (ii) a Calculating distance between negative sample and compressed video sampledis negative ；
Wherein M is: (x nega )、x nega Respectively representing the over-resolution result obtained after inputting the negative sample into the model and the negative sample itself, as used herein
In the form of (1).
And 4, step 4: calculating the total loss, and training the twin super-resolution network of the embodiment by using a reverse propagation gradient mode; and (4) circularly executing the steps 1-4, and training the twin super-resolution network to be convergent to obtain the trained twin super-resolution network.
This embodiment calculates the distance between the compressed video sample and the positive sample according to step 3dis positive And the distance between the compressed video sample and the negative sampledis negative Calculating the final loss;
wherein the content of the first and second substances,marginthe super parameter is used for setting the interval between two characteristic distances, and the interval is set to be 0.2 in the embodiment;
after the loss is calculated, the twin super-resolution network of the embodiment is subjected to back propagation and optimization by using an Adam optimizer.
The super-resolution network of the embodiment is an operation framework combining a predecessor network and a successor network, and the structures of the predecessor network and the successor network are completely the same, that is, the combination of an encoder network and an upsampling network. The antecedent and successor concepts describe the operation mode of data in a model, as shown in detail in fig. 3, the structures of a predecessor network and a successor network are the same, the predecessor network is used for extracting information of all frames behind a frame to be restored in a time sequence in advance and taking the information as a reference of final over-scoring of the successor, and HTs in the graph refer to intermediate results generated by the successor; HTp refers to intermediate results produced by the predecessor; the predecessor and successor are the same model structure, and the diagram architecture mainly embodies the processing mode of data input in the network.
This embodiment uses the training strategy to train on the MM522 training set and test on the VID4 testing set, and the test indexes are PSNR/SSIM, and the results are as follows:
| Calendar | City | Foliage | Walk | Avg | |
| compressed video super-resolution | 22.12/0.6949 | 25.55/0.6739 | 24.37/0.6186 | 27.85/0.8376 | 24.97/0.7062 |
In the aspect of generating a training data set, the experiment is different from the traditional model which trains a sample after Gaussian blur, and the original high-quality video is compressed in an M-JPEG format. Then, the two types of data, compressed and uncompressed, are input into a twin network designed as a combination of an encoder and an upsampling module. Finally, the training of the model is constrained in four aspects, and the features extracted by the two types of videos through the encoder should be as close as possible; the hyper-score results and the true values of the two types of videos form a triple together, and elements in the triple are positive samples; negative examples are the result of over-scoring of the residual between the compressed data and the conventional degradation. The subjective result obtained for the calenar video is shown in fig. 4, and it can be seen that through the comparative learning manner, the model can furthest learn the characteristic parameters required for the compressed video, and obtain a finer super-resolution reconstruction result.
It should be understood that the above description of the preferred embodiments is given for clarity and not for any purpose of limitation, and that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (5)
1. A compressed video super-resolution method based on a twin super-resolution network is characterized in that: inputting a low-resolution video to be processed into the twin super-resolution network to obtain a super-resolution video;
the twin super-resolution network consists of two parallel super-resolution networks; the super-resolution network comprises an encoder network and an up-sampling module;
the encoder network consists of two types of convolution layers and a plurality of PFRB modules; the first type of convolution layer takes the central frame of the input continuous three-frame image as input, the size of each frame is 64 multiplied by 3, the convolution layer converts the size of the convolution layer into a Batchsize multiplied by 64 multiplied by 80 feature map, the second type of convolution layer takes the supplementary frames except the central frame as input, the size of the input data is Batchsize multiplied by 64 multiplied by 6, the convolution layer converts the data into the Batchsize multiplied by 64 multiplied by 80 feature map, and finally, the two features are input into a continuous PFRB module together with the super-division result before the previous round of up-sampling; two encoder networks corresponding to the two inputs share a weight; wherein the batch size is the number of batch samples optimized by gradient;
in the PFRB module, data is firstly input into three convolution layers with the size of 3 x 3, the number of input and output channels of the three convolution layers is 80, the number of the input and output channels is recorded as x1, then the output of the convolution layers is spliced, a 1 x1 convolution layer is used for converting the total channel number 240 into 80, the obtained results are respectively spliced with three results in the x1, the obtained result is recorded as x2, finally, the three results in the x2 are respectively input into the three convolution layers, the number of channels of input data is compressed into 80, then the obtained three results and the initially input three results are correspondingly added to obtain a final result; wherein, leakyReLU activation function is used between each convolution layer to introduce nonlinear relation;
the up-sampling module inputs data with the size of batchsize multiplied by 3 multiplied by 64 multiplied by 80, firstly passes through a 3 multiplied by 3 convolutional layer, converts the number of data channels 240 into 80, continues to input into the convolutional layer, and converts the number of channels 80 into 48; upsampling the data with the scale of 2 by using a pixelshuffle function in the pytorech, and compressing the number of channels to 12; the obtained result is processed by a convolution layer, and finally, a pixelschuffle function is used for carrying out primary up-sampling with the scale of 2; the size of the output super-divided data is batch size × 3 × 256 × 256 × 3.
2. The compressed video super-resolution method based on the twin super-resolution network of claim 1, wherein: the twin super-resolution network is a trained twin super-resolution network; the training process comprises the following steps:
step 1: performing M-JPEG compression on a video in an original data set and bicubic downsampling processing to obtain a low-resolution compressed image which is recorded as a compressed video aiming at the original data set comprising a plurality of high-resolution continuous frames; performing Gaussian blur and bicubic downsampling processing on the video in the original data set to obtain a traditional low-quality image, and recording the image as a BD video;
step 2: performing initial feature extraction on an input video by using an encoder network, outputting a 4-dimensional feature by using the encoder network for input 5-dimensional video data, processing the 4-dimensional feature by using a LeakyReLU activation function, introducing a nonlinear functional relation, and constraining loss between the two extracted features;
and step 3: respectively sending the extracted features into an up-sampling module to continuously perform super-division, and outputting a video which is amplified by four times compared with the input length and width; respectively constraining the relationship between the two outputs and the truth value to enable the outputs and the truth value to be as close as possible to be used as the distance between the compressed video sample and the positive sample; recording the distance between the residual error after the over-score and the difference between the true value video and the compressed video as the distance between the compressed video sample and the negative sample, wherein the distance needs to be as large as possible in a set range;
and 4, step 4: calculating total loss, and training the twin super-resolution network in a mode of a reverse propagation gradient; and (4) circularly executing the steps 1-4, and training the twin super-resolution network to be convergent to obtain the trained twin super-resolution network.
3. The compressed video super-resolution method based on the twin super-resolution network of claim 2, wherein: in step 3, the super-resolution result of the BD video and the compressed video is obtained, and the super-resolution result and the real value are restricted to obtain the lossdis positive ；
Wherein the content of the first and second substances,y BD refers to the result after the super-resolution of BD video,y compress refers to the result after the super-resolution of the compressed video,gtit refers to the actual value of the light,εrepresents a constant; will be provided withdis positive Noting the distance between the compressed video sample and the positive sample;
the difference is made between the compressed video and the BD video to obtain a negative sample, which is recorded asx nega (ii) a Calculating distance between negative sample and compressed video sampledis negative ；
Wherein M is: (x nega )、x nega And respectively representing the over-resolution result obtained after the negative sample is input into the model and the negative sample.
4. The compressed video super-resolution method based on the twin super-resolution network of claim 2, wherein: in step 4, the distance between the compressed video sample and the positive sample obtained by the calculation in the step 3 is useddis positive And the distance between the compressed video sample and the negative sampledis negative Calculating the final loss;
wherein the content of the first and second substances,marginsetting the interval between two characteristic distances for the hyper-parameter;
and after calculating the loss, performing back propagation and optimization on the twin super-resolution network by using an Adam optimizer.
5. A compressed video super-resolution system based on a twin super-resolution network is characterized by comprising:
one or more processors;
a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the twin super resolution network based compressed video super resolution method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211515828.1A CN115546030B (en) | 2022-11-30 | 2022-11-30 | Compressed video super-resolution method and system based on twin super-resolution network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211515828.1A CN115546030B (en) | 2022-11-30 | 2022-11-30 | Compressed video super-resolution method and system based on twin super-resolution network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115546030A CN115546030A (en) | 2022-12-30 |
| CN115546030B true CN115546030B (en) | 2023-04-07 |
Family
ID=84722535
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211515828.1A Active CN115546030B (en) | 2022-11-30 | 2022-11-30 | Compressed video super-resolution method and system based on twin super-resolution network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115546030B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117058002B (en) * | 2023-10-12 | 2024-02-02 | 深圳云天畅想信息科技有限公司 | Video frame super-resolution reconstruction method and device and computer equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109118431A (en) * | 2018-09-05 | 2019-01-01 | 武汉大学 | A kind of video super-resolution method for reconstructing based on more memories and losses by mixture |
| CN114170088A (en) * | 2021-12-15 | 2022-03-11 | 中山大学 | Relational reinforcement learning system and method based on graph structure data |
| CN115409716A (en) * | 2022-11-01 | 2022-11-29 | 杭州网易智企科技有限公司 | Video processing method, device, storage medium and equipment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109345449B (en) * | 2018-07-17 | 2020-11-10 | 西安交通大学 | Image super-resolution and non-uniform blur removing method based on fusion network |
| KR20210042588A (en) * | 2019-10-10 | 2021-04-20 | 엘지전자 주식회사 | Method and apparatus for compressing or restoring image |
| CN114730372A (en) * | 2019-11-27 | 2022-07-08 | Oppo广东移动通信有限公司 | Method and apparatus for stylizing video, and storage medium |
| CN111833246B (en) * | 2020-06-02 | 2022-07-08 | 天津大学 | Single-frame image super-resolution method based on attention cascade network |
| US20220021887A1 (en) * | 2020-07-14 | 2022-01-20 | Wisconsin Alumni Research Foundation | Apparatus for Bandwidth Efficient Video Communication Using Machine Learning Identified Objects Of Interest |
| WO2022057837A1 (en) * | 2020-09-16 | 2022-03-24 | 广州虎牙科技有限公司 | Image processing method and apparatus, portrait super-resolution reconstruction method and apparatus, and portrait super-resolution reconstruction model training method and apparatus, electronic device, and storage medium |
| CN112183675B (en) * | 2020-11-10 | 2023-09-26 | 武汉工程大学 | Tracking method for low-resolution target based on twin network |
| CN113538248A (en) * | 2021-08-19 | 2021-10-22 | 南京航空航天大学 | Gamma photon image super-resolution image enhancement method based on digital twinning |
| CN115052187B (en) * | 2022-04-26 | 2024-05-03 | 复旦大学 | Super-resolution live broadcast system based on online training |
-
2022
- 2022-11-30 CN CN202211515828.1A patent/CN115546030B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109118431A (en) * | 2018-09-05 | 2019-01-01 | 武汉大学 | A kind of video super-resolution method for reconstructing based on more memories and losses by mixture |
| CN114170088A (en) * | 2021-12-15 | 2022-03-11 | 中山大学 | Relational reinforcement learning system and method based on graph structure data |
| CN115409716A (en) * | 2022-11-01 | 2022-11-29 | 杭州网易智企科技有限公司 | Video processing method, device, storage medium and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115546030A (en) | 2022-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Esrgan: Enhanced super-resolution generative adversarial networks | |
| CN108122197B (en) | Image super-resolution reconstruction method based on deep learning | |
| CN111179167B (en) | Image super-resolution method based on multi-stage attention enhancement network | |
| CN111353940B (en) | Image super-resolution reconstruction method based on deep learning iterative up-down sampling | |
| CN110675321A (en) | Super-resolution image reconstruction method based on progressive depth residual error network | |
| CN112102163B (en) | Continuous multi-frame image super-resolution reconstruction method based on multi-scale motion compensation framework and recursive learning | |
| CN109785252B (en) | Night image enhancement method based on multi-scale residual error dense network | |
| CN108989731B (en) | Method for improving video spatial resolution | |
| CN111711817B (en) | HEVC intra-frame coding compression performance optimization method combined with convolutional neural network | |
| CN115546030B (en) | Compressed video super-resolution method and system based on twin super-resolution network | |
| CN112288630A (en) | Super-resolution image reconstruction method and system based on improved wide-depth neural network | |
| CN114757828A (en) | Transformer-based video space-time super-resolution method | |
| Chen et al. | Image denoising via deep network based on edge enhancement | |
| CN113379606B (en) | Face super-resolution method based on pre-training generation model | |
| CN110177229B (en) | Video conversion method based on multi-task counterstudy, storage medium and terminal | |
| CN112348745B (en) | Video super-resolution reconstruction method based on residual convolutional network | |
| CN116309078A (en) | Single image super-resolution method based on multi-angle knowledge distillation | |
| CN115965527A (en) | Mobile terminal lightweight image super-resolution reconstruction method based on convolutional neural network | |
| Liu et al. | A new low bit-rate coding scheme for ultra high definition video based on super-resolution reconstruction | |
| Zhu et al. | MRDA-Net: Multiscale Residual Dense Attention Network for Image Denoising | |
| CN115249206A (en) | Image super-resolution reconstruction method of lightweight attention mechanism | |
| CN117528085B (en) | Video compression coding method based on intelligent feature clustering | |
| CN115601242B (en) | Lightweight image super-resolution reconstruction method suitable for hardware deployment | |
| CN112613561B (en) | EAST algorithm optimization method | |
| Liu et al. | Adaptive Texture Distillation Network for Image Hybrid Super-Resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |