CN108921791A - Lightweight image super-resolution improved method based on adaptive important inquiry learning - Google Patents
Lightweight image super-resolution improved method based on adaptive important inquiry learning Download PDFInfo
- Publication number
- CN108921791A CN108921791A CN201810711050.9A CN201810711050A CN108921791A CN 108921791 A CN108921791 A CN 108921791A CN 201810711050 A CN201810711050 A CN 201810711050A CN 108921791 A CN108921791 A CN 108921791A
- Authority
- CN
- China
- Prior art keywords
- network
- learning
- adaptive
- importance
- lightweight
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 33
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000006870 function Effects 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 5
- 238000006467 substitution reaction Methods 0.000 claims description 4
- 230000006872 improvement Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 210000003739 neck Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000011148 porous material Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
Abstract
The invention discloses a kind of lightweight image super-resolution improved methods based on adaptive important inquiry learning, improve the capability of fitting of given lightweight SISR network architecture, Learning Scheme seamlessly can assimilate knowledge from more powerful learning network, with the importance of initialisation image pixel, reach better initial capacity network, realizes super-resolution performance.
Description
Technical field
The present invention relates to the field of computer intelligence vision more particularly to a kind of light weights based on adaptive important inquiry learning
Grade image super-resolution improved method.
Background technique
In the various Video Applications such as monitoring, public safety, business, amusement and remote sensing, only increase image divides
Resolution could improve user experience.Single image super-resolution (SISR) is that one kind improves image in the absence of the additional
The method of resolution ratio, has received widespread attention.Following two mode has the characteristics that certain on this basis, is also individually present
Problem.
(1) using pixels statistics or the HR image of internal patch recurrence as priori.These methods generally can not be well
It promotes, because even a small difference between the attribute that didactic attribute and priori include leads to the HR image of reconstruct
In visible artifact.
(2) it is based on the learning method of depth convolutional neural networks (DCNN), significant effect is had been achieved for, especially exists
Some special C zoom factors.However, the structure due to them is very deep, there is significant memory and calculate demand, this is just needed
Powerful computing unit (for example, GPU) is wanted, to limit their answering on the limited many physical devices of computing capability
With especially handheld device.
Summary of the invention
The invention mainly solves the technical problem of providing a kind of lightweight image based on adaptive important inquiry learning is super
Resolution ratio improved method, improves the capability of fitting of given lightweight SISR network architecture, Learning Scheme can seamlessly from
More powerful learning network assimilation knowledge reaches better initial capacity network with the importance of initialisation image pixel, realizes
Super-resolution performance.
In order to solve the above technical problems, one technical scheme adopted by the invention is that：It provides a kind of based on adaptive weight
Want the lightweight image super-resolution improved method of inquiry learning, including step in detail below：
Step 1：A study formula is established, gives the lightweight network architecture to maximize the capability of fitting of pixel；
Step 2：Combined optimization problem is converted by network training and important inquiry learning, passes through the important compensation letter of design
Number and solve convex optimization problem to gradually increase the importance of single pixel, training process since the pixel for being easy to reconstruct, with
Fitting improvement, gradually proceed to more complicated pixel, to realize adaptive importance Learning Scheme；
Step 3：The adaptive importance Learning Scheme proposed is in a manner of the initialization of teacher's importance, from more powerful
It absorbs knowledge to teacher's network seamless, to obtain better initial capacity in a network；
Step 4：Using substitution minimum scheme, adaptive importance Learning Scheme algorithm is formed；
Step 5：Adaptive SISR model is proposed, by dynamically updating image pixel on the basis of training is lost
Importance, training is by letter to difficult example network, that is, customization lightweight SISR model.
In a preferred embodiment of the present invention, the SISR model is inspired by DCNN frame, SISR model
Basic module is convolutional layer, lightweight network is customized by directly reducing the filter in each convolutional layer, with fixed ratio
To reduce the amount of output characteristic pattern.
The beneficial effects of the invention are as follows：Lightweight image super-resolution based on adaptive important inquiry learning of the invention changes
Into method, the capability of fitting of given lightweight SISR network architecture is improved, Learning Scheme can be seamlessly from more powerful
Learning network assimilates knowledge, with the importance of initialisation image pixel, reaches better initial capacity network, realizes super-resolution
Performance.
Specific embodiment
The technical scheme in the embodiments of the invention will be clearly and completely described below, it is clear that described implementation
Example is only a part of the embodiments of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, this field is common
Technical staff's all other embodiment obtained without making creative work belongs to the model that the present invention protects
It encloses.
The embodiment of the present invention includes：
A kind of lightweight image super-resolution improved method based on adaptive important inquiry learning, including walk in detail below
Suddenly：
Step 1：A study formula is established, gives the lightweight network architecture to maximize the capability of fitting of pixel；
Step 2：Combined optimization problem is converted by network training and important inquiry learning, passes through the important compensation letter of design
Number and solve convex optimization problem to gradually increase the importance of single pixel, training process since the pixel for being easy to reconstruct, with
Fitting improvement, gradually proceed to more complicated pixel, to realize adaptive importance Learning Scheme；
Step 3：The adaptive importance Learning Scheme proposed is in a manner of the initialization of teacher's importance, from more powerful
It absorbs knowledge to teacher's network seamless, to obtain better initial capacity in a network；
Step 4：Using substitution minimum scheme, adaptive importance Learning Scheme algorithm is formed；
Step 5：Adaptive SISR model is proposed, by dynamically updating image pixel on the basis of training is lost
Importance, training is by letter to difficult example network, that is, customization lightweight SISR model.
Among the above, the SISR model is inspired by DCNN frame, and the basic module of SISR model is convolutional layer, is led to
It crosses and reduces filter in each convolutional layer directly to customize lightweight network, output characteristic pattern is reduced with fixed ratio
Amount.
Lightweight image super-resolution improved method based on adaptive important inquiry learning of the invention is developed a kind of simple
Mode of learning easy to learn, to maximize the ability of given SISR lightweight network architecture；Propose a kind of adaptive weight
Inquiry learning scheme is wanted to train lightweight SISR network, carries out enhancing training；Knowledge is freed from more powerful learning network
Out, preferably to initialize.
Specific implementation：
A, study formula
Using n LR-HG image to FXI；Can learn lightweight network s (；It is θ) as follows
Wherein indicate network parameter, l indicates loss function (for example, MSE loss or ' 1 loss ').It is optimal in the training stage
Seek to minimize desired value E (θ), wherein being trained being fed in S together with the different all pixels for rebuilding difficulty.For
Maximize the capability of fitting S of pixel, it is proposed that training has the S of adaptive importance Learning Scheme.
Wherein, WiIndicate the pixel significance vector of each training pair,Collect it is all it is important to
Amount.Due to 0≤Wi≤ 1, pixel significance can be counted as each pixel and participate in the probability of training process to be Eq. (2), example
Such as, when importance is zero, corresponding pixel will be removed from training network.h(Wi) indicate WiOn penalty, it control
Importance learning strategy and avoid WiTrivial solution (such as Wi=0).
As h (Wi) when being given as following indicator function,
The Learning Scheme proposed will degenerate in traditional Learning Scheme.Therefore, the adaptive important sexology proposed
Habit scheme is the general learning framework for being used for SISR.
B, adaptive importance Learning Scheme
The key of importance Learning Scheme is one important penalty h (W appropriate of designi).It is above-mentioned heavy in order to meet
It wants inquiry learning principle, designs penalty function H, and it is as follows in formula (2) to reconfigure Learning Scheme：
Wherein W 'iIt indicates the important vector in previous ones, and provides h (Wi, W 'i),
The w in formula (5)jtWith w 'jtThe w of denote thejiWith w 'jtMember is predefined scalar respectively.Under
Face will inquire into h (Wi, W 'i) details income.
Similar solution, formula (2) is using the alternative solution of optimization to another andw pore in formula (5).
Specifically, the solution for the problem concerning study for being can be by back-propagation algorithm when giving the important carrier of W.When it is solid
Fixed, it can be simplified in the problem of study.
Wherein w indicates the importance of the specific pixel in training sample (for example, coming from WiElement), w ' shows previous ones
In corresponding importance value (for example, corresponding element from w ').D indicates reconstruction of the network S of study in the pixel considered
Loss.In order to solve the problems in equation (6), following result is obtained.
Theorem 1：Consider constraint
It is a convex function, and f (w*) minimum value when：
w*=w '+λ .e-d (7)
According to theorem 1, the problems in equation (6) has the closed form solution of Eq. (7).In equation (7), by preceding
Increase importance value w in one iteration*Increment update importance w ' due to λ .e-d>=0, such update rule makes it possible to
Importance is gradually increased in each iteration.In addition, increment is subtracting by the reconstruction loss of the pre- learning model S in respective pixel
What few function determined, that is, provide little increment when reconstruct loss is big.
C, the initialization of teacher's importance
In order to inhibit complicated pixel well, and the prominent simple pixel when starting, establish following important function：
Wherein x teacher's networkDetermining pixel and g (x) reconstructed error (for example, ' 2 norm ') is corresponding importance value.0 and 0
Indicate the biasing and scale parameter in the function.One normalization factor, importance is set to by it
[0；1].
In view of teacher's networkWith importance function g, network s can be trained by solving following problems
Herein, it for succinct formula, usesTo indicate g being applied to xiIn in each pixel
Reconstruction error.
D, adaptive importance Learning Scheme algorithm
It, can be excellent by the totality of the adaptive importance Learning Scheme proposed in formula (2) using substitution minimum scheme
Change process is summarized as algorithm.When starting, by giving teacher's networkThe important vector W of initialization trains network S (9).
Then, existThe Learning Scheme in formula (4) is carried out in iteration, to gradually increase the capacity of S.
Algorithm
Algorithm:Adaptive importance learning(AIL)
Input:Input HR-LR training pairspre-trained teacher modelimportance function g,penalty function h and λ
1.Importance initialization from teacher:
(1)Learn importance W as Eq.(9)；
(2)Update model parameter
2.Adaptive importance learning:
For t←1to T
End for
Output:θ-parameterized model S。
E, lightweight SISR model
SISR model is inspired by DCNN frame, and wherein basic module is convolutional layer.By directly reducing each convolution
Filter in layer customizes lightweight network, and the amount of output characteristic pattern is reduced with fixed ratio (such as 0 < ρ < 1).Pass through
It does so, the lightweight network of the available different scales with different s.By the way of reducing filter be for the ease of
Verify validity of the proposed Learning Scheme in the lightweight network of enhancing different scales.
Lightweight image super-resolution improved method based on adaptive important inquiry learning of the invention, with prior art phase
Than having the following advantages that：
(1) a kind of learning strategy by letter to hardly possible is proposed, referred to as adaptive importance Learning Scheme is given light to improve
The capability of fitting of magnitude SISR network architecture；
(2) importance of image pixel is updated by dynamic, network starts from study and rebuilds easy pixel, then exposes
The pixel to become increasingly complex out is trained, and when Learning Scheme convergence, capability of fitting can be gradually increased and finally be maximized；
(3) Learning Scheme seamlessly can assimilate knowledge from more powerful learning network, with the weight of initialisation image pixel
The property wanted reaches better initial capacity network, realizes super-resolution performance.
The above description is only an embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright description is applied directly or indirectly in other relevant technology necks
Domain is included within the scope of the present invention.
Claims (2)
1. a kind of lightweight image super-resolution improved method based on adaptive important inquiry learning, which is characterized in that including with
Lower specific steps：
Step 1：A study formula is established, gives the lightweight network architecture to maximize the capability of fitting of pixel；
Step 2：Convert combined optimization problem for network training and important inquiry learning, by the important penalty function of design and
Convex optimization problem is solved to gradually increase the importance of single pixel, training process is since the pixel for being easy to reconstruct, with quasi-
The improvement of conjunction gradually proceeds to more complicated pixel, to realize adaptive importance Learning Scheme；
Step 3：The adaptive importance Learning Scheme proposed is in a manner of the initialization of teacher's importance, from more powerful teacher
It absorbs knowledge to network seamless, to obtain better initial capacity in a network；
Step 4：Using substitution minimum scheme, adaptive importance Learning Scheme algorithm is formed；
Step 5：Adaptive SISR model is proposed, by dynamically updating the important of image pixel on the basis of training is lost
Property, training is by letter to difficult example network, that is, customization lightweight SISR model.
2. the lightweight image super-resolution improved method according to claim 1 based on adaptive important inquiry learning,
It is characterized in that, the SISR model is inspired by DCNN frame, and the basic module of SISR model is convolutional layer, by direct
The filter in each convolutional layer is reduced to customize lightweight network, the amount of output characteristic pattern is reduced with fixed ratio.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810711050.9A CN108921791A (en) | 2018-07-03 | 2018-07-03 | Lightweight image super-resolution improved method based on adaptive important inquiry learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810711050.9A CN108921791A (en) | 2018-07-03 | 2018-07-03 | Lightweight image super-resolution improved method based on adaptive important inquiry learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108921791A true CN108921791A (en) | 2018-11-30 |
Family
ID=64423313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810711050.9A Pending CN108921791A (en) | 2018-07-03 | 2018-07-03 | Lightweight image super-resolution improved method based on adaptive important inquiry learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108921791A (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0532175A1 (en) * | 1991-09-11 | 1993-03-17 | Sharp Kabushiki Kaisha | Optical device utilizing superresolution effect |
| CN101226631A (en) * | 2007-12-12 | 2008-07-23 | 华为技术有限公司 | Super-resolution image reconstruction method and apparatus |
| CN101551903A (en) * | 2009-05-11 | 2009-10-07 | 天津大学 | Super-resolution image restoration method in gait recognition |
| CN101794440A (en) * | 2010-03-12 | 2010-08-04 | 东南大学 | Weighted adaptive super-resolution reconstructing method for image sequence |
| US20100211333A1 (en) * | 2009-01-14 | 2010-08-19 | Integrated Process Resources, Inc. | Leak Detection and Identification System |
| CN103530863A (en) * | 2013-10-30 | 2014-01-22 | 广东威创视讯科技股份有限公司 | Multistage reconstruction image super resolution method |
| CN103617607A (en) * | 2013-11-28 | 2014-03-05 | 天津大学 | Single image super-resolution rebuilding method |
| CN103632358A (en) * | 2013-09-27 | 2014-03-12 | 浙江师范大学 | Method for converting low-resolution image into high-resolution image |
| CN104322052A (en) * | 2012-05-09 | 2015-01-28 | 恩卡姆技术有限公司 | A system for mixing or compositing in real-time, computer generated 3D objects and a video feed from a film camera |
| CN106096538A (en) * | 2016-06-08 | 2016-11-09 | 中国科学院自动化研究所 | Face identification method based on sequencing neural network model and device |
| CN106204449A (en) * | 2016-07-06 | 2016-12-07 | 安徽工业大学 | A kind of single image super resolution ratio reconstruction method based on symmetrical degree of depth network |
| CN106228512A (en) * | 2016-07-19 | 2016-12-14 | 北京工业大学 | Based on learning rate adaptive convolutional neural networks image super-resolution rebuilding method |
| CN107194872A (en) * | 2017-05-02 | 2017-09-22 | 武汉大学 | Remote sensed image super-resolution reconstruction method based on perception of content deep learning network |
| CN107430444A (en) * | 2015-04-30 | 2017-12-01 | 谷歌公司 | For gesture tracking and the tracking of the micromotion based on RF of identification |
| CN107704857A (en) * | 2017-09-25 | 2018-02-16 | 北京邮电大学 | A kind of lightweight licence plate recognition method and device end to end |
| CN107851124A (en) * | 2015-07-27 | 2018-03-27 | 高通股份有限公司 | Media marking in self-organizing network is propagated |
| CN108235001A (en) * | 2018-01-29 | 2018-06-29 | 上海海洋大学 | A kind of deep-sea video quality objective assessment model based on space-time characteristic |
-
2018
- 2018-07-03 CN CN201810711050.9A patent/CN108921791A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0532175A1 (en) * | 1991-09-11 | 1993-03-17 | Sharp Kabushiki Kaisha | Optical device utilizing superresolution effect |
| CN101226631A (en) * | 2007-12-12 | 2008-07-23 | 华为技术有限公司 | Super-resolution image reconstruction method and apparatus |
| US20100211333A1 (en) * | 2009-01-14 | 2010-08-19 | Integrated Process Resources, Inc. | Leak Detection and Identification System |
| CN101551903A (en) * | 2009-05-11 | 2009-10-07 | 天津大学 | Super-resolution image restoration method in gait recognition |
| CN101794440A (en) * | 2010-03-12 | 2010-08-04 | 东南大学 | Weighted adaptive super-resolution reconstructing method for image sequence |
| CN104322052A (en) * | 2012-05-09 | 2015-01-28 | 恩卡姆技术有限公司 | A system for mixing or compositing in real-time, computer generated 3D objects and a video feed from a film camera |
| CN103632358A (en) * | 2013-09-27 | 2014-03-12 | 浙江师范大学 | Method for converting low-resolution image into high-resolution image |
| CN103530863A (en) * | 2013-10-30 | 2014-01-22 | 广东威创视讯科技股份有限公司 | Multistage reconstruction image super resolution method |
| CN103617607A (en) * | 2013-11-28 | 2014-03-05 | 天津大学 | Single image super-resolution rebuilding method |
| CN107430444A (en) * | 2015-04-30 | 2017-12-01 | 谷歌公司 | For gesture tracking and the tracking of the micromotion based on RF of identification |
| CN107851124A (en) * | 2015-07-27 | 2018-03-27 | 高通股份有限公司 | Media marking in self-organizing network is propagated |
| CN106096538A (en) * | 2016-06-08 | 2016-11-09 | 中国科学院自动化研究所 | Face identification method based on sequencing neural network model and device |
| CN106204449A (en) * | 2016-07-06 | 2016-12-07 | 安徽工业大学 | A kind of single image super resolution ratio reconstruction method based on symmetrical degree of depth network |
| CN106228512A (en) * | 2016-07-19 | 2016-12-14 | 北京工业大学 | Based on learning rate adaptive convolutional neural networks image super-resolution rebuilding method |
| CN107194872A (en) * | 2017-05-02 | 2017-09-22 | 武汉大学 | Remote sensed image super-resolution reconstruction method based on perception of content deep learning network |
| CN107704857A (en) * | 2017-09-25 | 2018-02-16 | 北京邮电大学 | A kind of lightweight licence plate recognition method and device end to end |
| CN108235001A (en) * | 2018-01-29 | 2018-06-29 | 上海海洋大学 | A kind of deep-sea video quality objective assessment model based on space-time characteristic |
Non-Patent Citations (2)
| Title |
|---|
| LEI ZHANG 等: "Adaptive Importance Learning for Improving Lightweight Image Super-resolution Network" * |
| 胡秋翔: "存储优化的多分辨率矢量地理数据组织研究", 《中国优秀硕士学位论文全文数据库基础科学辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jeon et al. | Ib-gan: Disentangled representation learning with information bottleneck generative adversarial networks | |
| CN109919299A (en) | A kind of meta learning algorithm based on meta learning device gradually gradient calibration | |
| CN107516129A (en) | The depth Web compression method decomposed based on the adaptive Tucker of dimension | |
| CN111932444A (en) | Face attribute editing method based on generation countermeasure network and information processing terminal | |
| CN108121975A (en) | A kind of face identification method combined initial data and generate data | |
| CN111126599B (en) | Neural network weight initialization method based on transfer learning | |
| CN109671022A (en) | A kind of picture texture enhancing super-resolution method based on depth characteristic translation network | |
| CN108535952A (en) | A kind of calculating photolithography method based on model-driven convolutional neural networks | |
| CN106951960A (en) | A kind of learning method of neutral net and the neutral net | |
| WO2022016802A1 (en) | Physical feature map- and dcnn-based computation method for machine learning-based inverse lithography technology solution | |
| Shan et al. | Residual learning of deep convolutional neural networks for image denoising | |
| CN107085835A (en) | Color image filtering method based on quaternary number Weighted Kernel Norm minimum | |
| Li et al. | Real-world image super-resolution by exclusionary dual-learning | |
| CN114897694A (en) | Image super-resolution reconstruction method based on mixed attention and double-layer supervision | |
| CN108629374A (en) | A kind of unsupervised multi-modal Subspace clustering method based on convolutional neural networks | |
| DE102022121649A1 (en) | Fast retraining of fully fused neural transceiver components | |
| CN113888399B (en) | Face age synthesis method based on style fusion and domain selection structure | |
| CN108710944A (en) | One kind can train piece-wise linear activation primitive generation method | |
| CN108921791A (en) | Lightweight image super-resolution improved method based on adaptive important inquiry learning | |
| CN110263850A (en) | Capsule network fast routing method based on Density Estimator and average drifting | |
| CN116151366A (en) | Noise tag robustness learning method based on online distillation | |
| CN110177229A (en) | Video conversion method, storage medium and terminal based on multitask confrontation study | |
| CN115705616A (en) | True image style migration method based on structure consistency statistical mapping framework | |
| CN112508792A (en) | Single-image super-resolution method and system of deep neural network integration model based on online knowledge migration | |
| CN111626917A (en) | Bidirectional image conversion system and method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181130 |