CN105046672B - A kind of image super-resolution rebuilding method - Google Patents
A kind of image super-resolution rebuilding method Download PDFInfo
- Publication number
- CN105046672B CN105046672B CN201510385633.3A CN201510385633A CN105046672B CN 105046672 B CN105046672 B CN 105046672B CN 201510385633 A CN201510385633 A CN 201510385633A CN 105046672 B CN105046672 B CN 105046672B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- mim
- dictionary
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Processing (AREA)
- Editing Of Facsimile Originals (AREA)
Abstract
The present invention discloses a kind of image super-resolution rebuilding method, under the basic framework of the sparse reconstruction of super-resolution, introduce two-dimentional sparse representation model, make it possible to efficiently use the spatial character of two dimensional image, reduce simultaneously in one-dimensional rarefaction representation there are dictionary memory space it is larger, need the parameter estimated more the problem of, so as to the memory space of reduction dictionary while image super-resolution rebuilding result is ensured.
Description
Technical field
The invention belongs to the technical field of the super-resolution of rarefaction representation, more particularly to a kind of image super-resolution rebuilding
The two-dimentional sparse representation model of method, mainly application carries out super-resolution rebuilding in single-frame images.
Background technology
Important information form of the image as the human perception world, the abundant and details of content, directly determines this mankind
Experience the level of detail of content.When the picture element density on image as unit scale is higher, then image is more clear, the details of expression
Ability is stronger, and the information of human perception is abundanter, this namely high-definition picture.The super-resolution rebuilding of image is very
Various aspects have corresponding research such as remote sensing images, satellite imagery field, field of medical images and some high-resolution display fields
Etc..
The method for improving the resolution ratio of image reduces pixel dimension or increases mainly by sensor manufacturing process is improved
Pixel quantity in unit area.But the acceptable electromagnetic energy of pixel can be reduced by reducing pixel elements, so as to cause to hold
Easily by noise jamming, cause picture quality not high.And the size for increasing integrated circuit plate can cause capacitance to increase.This easily causes electricity
Appearance is difficult to transfer charge so that its application field is reduced.Another kind of method is low to single frames or multiframe using signal processing technology
Image in different resolution is rebuild, and obtains high-definition picture, that is, the super-resolution rebuilding technology of image processing field.
The super-resolution rebuilding of image refers to eliminate due to imaging system using the method for signal processing and computer software
Deteriroation of image quality caused by focusing on the factors such as inaccurate, motion blur and non-ideal sampling is high-resolution clear so as to obtain
Clear image.Multiple image reconstruction is to merge multiple low resolution (Low Resolution, LR) figure from identical dynamic scene
The technology of picture, wherein having used the complementary information of multiple low-resolution images.And the super-resolution image reconstruction of single-frame images is then
It is the high-frequency information estimated from single-frame images other than cutoff frequency, reconstructs high-resolution image.
The concept and method of super-resolution are proposed, and propose earliest by Harris and Goodman the sixties in last century
A variety of methods, such as prolate ellipsoid Wave function method, linear extrapolation, the sinusoidal template of superposition.These methods are all based on single width
Image.Certain exploration has been carried out to super-resolution technique at present, but has been actually not very extensively.80 years generations of last century
Generation, Tsai and Huang propose multiframe super resolution ratio reconstruction method, and main thought is before existing imaging system is not changed
It puts, if there is the situation of the low-resolution image of several Same Scenes, multiple image information reconstruction high quality can be combined
Image.While multiframe super-resolution rebuilding technology develops, single-frame images method for reconstructing has also obtained tremendous development.However thing
In reality, it is difficult to obtain several qualified low-resolution images, therefore here primary concern is that the weight of single frames super-resolution
Build problem.
It is degenerated by high-resolution as follows for the model of low-resolution image：Y=SHX+N.Here, Y is low-resolution image,
S is down-sampling operator, and H is fuzzy filter, and X is high-definition picture, and N is noise image.The purpose of Super-resolution Reconstruction is exactly
How will original X images be recovered by Y.
The method of single-frame images super-resolution mainly has in the following manner：1. the method based on interpolation.Mainly pass through construction
The interpolating function of smooth curve or curved surface generates high-definition picture.That is " image interpolation ".Currently used interpolation method
Interpolation, bilinear interpolation and spline interpolation are repeated just like arest neighbors.2. the method based on reconstruct solves images above degradation model
Inverse problem.Some prioris are primarily introduced into, under the guidance of priori, solve inverse problem, and due to inverse problem
Solution is not unique, so needing centainly constraining, such as fragment linearity, edge constraint etc. is eventually found the solution minimized the error, because
This reconstructing method is a kind of method for solving optimization problem.Similar method has maximum a posteriori probability (Maximize a
Posterior, MAP) etc..But since the constraints in model needs some prioris therewith, and actually to priori
The understanding of knowledge is not necessarily accurate, so the situation of reconstruction is not fine.3. the method based on study.Freeman et al. are most
The super-resolution rebuilding of the instance-based learning early proposed, can be by the information in external image library, by learning height point
The relationship of resolution image pair, to the single frames low-resolution image of input, to estimate the detail of the high frequency in high-definition picture,
Finally obtain reconstruction image, it is clear that after the addition of external image library information, obtained reconstruction image was obtained than the past with interpolation
High-definition picture it is apparent.And with image prior information sample introducing, increase more effective constraint, carry
The high accuracy of reconstruction image.
The method of current representational study is exactly the correspondence trained between high-low resolution.Not only ensure side
Edge, and increase the detailed information of texture.Such as image block is modeled using Markov field, while uses belief propagation
The method of (Belief Propagation, BP), to eliminate the problem of boundary repeats.Its main thought is exactly low point will obtained
Then image block of the resolution image to be of moderate size is found and its most like low resolution in the database as input unit
Image block, and the high-definition picture block corresponding to the most like image block found just plays the role of supplementing high frequency detail,
Also can be used for rebuilding high-definition picture.But it is this based in learning process between high-low resolution image block, if
There are many sample, can computation complexity be increased, and there is also poor fittings if rebuilding current block using neighbouring image block
And over-fitting.
Therefore the paradigm learning super-resolution based on rarefaction representation is proposed with the development of sparse representation theory, Yang et al.
Method for reconstructing, when establishing high-low resolution dictionary, it is desirable that the corresponding sparse coefficient of high-low resolution image block is the same.In this way
For the low resolution image block of input, its rarefaction representation coefficient under low resolution base space, Ran Houfu can be obtained first
High-resolution base is used spatially, so as to fulfill the super-resolution rebuilding of image.However high-low resolution image block is required to correspond to
Sparse coefficient be that the same constraint is stronger.Actually should be there are certain mapping relations, therefore have different in succession
Person proposes the algorithm of the features such as pre-filtering interpolation, non-blind sparse deblurring enhancing, is instructed also on support vector regression model
The mapping relations practiced between high-low resolution image block carry out rebuilding super resolution.Zeyde etc. is by reducing the dimension of LR, and profit
The dictionary of HR is obtained with the dictionary calculating difference image of LR rather than directly training obtains the dictionary of LR and HR.Wang et al. is logical
Cross mapping function between establishing the corresponding sparse coefficient of LR and HR dictionaries needs consistent tight constraint to loosen original sparse coefficient.
And effective application now with non local self-similarity characteristic in image restoration problem, also there are many people will be this non local
Similitude characteristic and cluster are introduced into dictionary learning, so as to improve the quality of image super-resolution rebuilding.
The sparse reconstruction Super-Resolution Sparse of super-resolution that the present invention is mainly proposed in Yang
Under the basic framework of Representation (SRSR), two-dimentional sparse representation model is introduced, enabling efficiently use two dimension
The spatial character of image, while reduce in one-dimensional rarefaction representation there are dictionary memory space is larger, need the parameter estimated more
The problem of.The memory space of dictionary can be finally reduced while image super-resolution rebuilding result is ensured.
Invention content
The technology of the present invention solves the problems, such as：Overcome the deficiencies of the prior art and provide a kind of image super-resolution rebuilding side
Method, can efficiently use the spatial character of two dimensional image, at the same reduce in one-dimensional rarefaction representation there are dictionary memory space compared with
Greatly, the problem of needing the parameter estimated more, so as to reduce dictionary while image super-resolution rebuilding result is ensured
Memory space.
The present invention technical solution be：This image super-resolution rebuilding method, includes the following steps：
(1) RGB image of the low resolution of input is converted into YCbCr images, wherein Y is non-linear luma component, Cb
It is blue color difference component, Cr is red color difference component；Super-resolution rebuilding is carried out using bilinear interpolation to Cb, Cr；
(2) Y-component is denoted as lIm, carries out 2 times of up-samplings using bilinear interpolation, obtain corresponding intermediate resolution
Image mIm, while the characteristic image of mIm images is solved, obtain the characteristic image of different directions and different orders；
(3) using the lIm upper left corners as starting point, the image block Y of 3x3 is sampled successivelyi, wherein there is 1 pixel in each direction
Repeated region calculates the mean value M of current block；
(5) trained dictionary is utilizedWithIt solves
(6) sparse coefficient of current block is solved：
(7) the high-resolution features image block of corresponding position is rebuild
(8) by XiThe high-definition picture blocks of+M as the reconstruction of corresponding position；
(9) judge whether all to have carried out sampling and super-resolution rebuilding to low-resolution image, if low point of all 3x3
Resolution image block is complete, then performs step (10), otherwise performs step (3)-(9)；
(10) final super-resolution image X is solved according to formula (6)*：
Wherein c is the error that parameter is used for balancing global and local；
(11) the high-definition picture X for obtaining Y-component super-resolution rebuilding*It is obtained with to Cb, Cr using bilinear interpolation
To super-resolution rebuilding image combine, obtain the high-definition picture of YCbCr space；Then color space is carried out to it
Conversion, is transformed into RGB color, finally obtains colored super-resolution rebuilding image.
The present invention is in the sparse reconstruction Super-Resolution Sparse Representation (SRSR) of super-resolution
Basic framework under, introduce two-dimentional sparse representation model, enabling efficiently use the spatial character of two dimensional image, subtract simultaneously
In few one-dimensional rarefaction representation there are dictionary memory space it is larger, need the parameter estimated more the problem of, so as to ensure
The memory space of dictionary is reduced while image super-resolution rebuilding result.
Description of the drawings
Fig. 1 shows the flow chart of dictionary training method according to the present invention；
Fig. 2 shows the flow charts of image super-resolution rebuilding method according to the present invention.
Specific embodiment
As shown in Fig. 2, this image super-resolution rebuilding method, includes the following steps：
(1) RGB image of the low resolution of input is converted into YCbCr images, wherein Y is non-linear luma component, Cb
It is blue color difference component, Cr is red color difference component；Super-resolution rebuilding is carried out using bilinear interpolation to Cb, Cr；
(2) Y-component is denoted as lIm, carries out 2 times of up-samplings using bilinear interpolation, obtain corresponding intermediate resolution
Image mIm, while the characteristic image of mIm images is solved, obtain the characteristic image of different directions and different orders；
(3) using the lIm upper left corners as starting point, the image block Y of 3x3 is sampled successivelyi, wherein there is 1 pixel in each direction
Repeated region calculates the mean value M of current block；
(5) trained dictionary is utilizedWithIt solves
(6) sparse coefficient of current block is solved：
(7) the high-resolution features image block of corresponding position is rebuild
(8) by XiThe high-definition picture blocks of+M as the reconstruction of corresponding position；
(9) judge whether all to have carried out sampling and super-resolution rebuilding to low-resolution image, if low point of all 3x3
Resolution image block is complete, then performs step (10), otherwise performs step (3)-(9)；
(10) it using gradient descent method (gradient descent algorithm), is solved finally according to formula (6)
Super-resolution image X*：
Wherein c is the error that parameter is used for balancing global and local；
(11) the high-definition picture X for obtaining Y-component super-resolution rebuilding*It is obtained with to Cb, Cr using bilinear interpolation
To super-resolution rebuilding image combine, obtain the high-definition picture of YCbCr space；Then color space is carried out to it
Conversion, is transformed into RGB color, finally obtains colored super-resolution rebuilding image.
The present invention is in the sparse reconstruction Super-Resolution Sparse Representation (SRSR) of super-resolution
Basic framework under, introduce two-dimentional sparse representation model, enabling efficiently use the spatial character of two dimensional image, subtract simultaneously
In few one-dimensional rarefaction representation there are dictionary memory space it is larger, need the parameter estimated more the problem of, so as to ensure
The memory space of dictionary is reduced while image super-resolution rebuilding result.
Preferably, it is single order horizontal gradient, one different directions and the characteristic image of different orders to be obtained in the step (2)
Rank vertical gradient, second order horizontal gradient, respectively second order vertical gradient, f1=[- 10 1], f2=f1 T、f3=[1 0-2 0
1]、f4=f3 T, the characteristic image f of corresponding intermediate resolution image is obtained after filtering1MIm, f2mIm,f3MIm, and f4mIm。
Preferably, as shown in Figure 1, the dictionary of the step (5) include it is following step by step：
(5.1) high-resolution image data base is established, if image is coloured image in itself, is first turned coloured image
Turn to gray level image；
(5.2) image data base of low resolution is established：To the image in external image library, 3 times of down-samplings are carried out, are obtained
Corresponding low-resolution image lIm then to all low-resolution image lIm, carries out 2 times of up-samplings, obtains medium resolution
The image mIm of rate, while solve the characteristic image of mIm images, i.e., using different filtering process, obtain different directions and difference
The characteristic image of order obtains the characteristic image of corresponding intermediate resolution image after filtering, utilize mIm, intermediate resolution figure
Image data base of the characteristic image of picture as low resolution；
(5.3) to the image data base of the high-low resolution of foundation, sampling obtains pairs of high-definition picture block and low
The training sample set of resolution characteristics image block；(if image is coloured image in itself, needs first to turn color RGB image
Gray level image (gray) image is turned to, otherwise directly as the image Im in high resolution image data library)
(5.4) dictionary is initializedWithRandomly choose the image block X of same indexi,WithRespectively hx is obtained by row averagingi, AndIt is averaging by row and obtains vxi T, AndBy hxiAs dictionaryOne
Row primitive, and vxiAs dictionaryA row primitive, can construct respectively in this wayA row base
Member；
(5.6) to dictionaryWithCarry out dictionary updating：Dictionary is updated respectively With
(5.7) judge whether to reach iteration stopping condition：Step is jumped to if iteration stopping condition is unsatisfactory for
(5.3)；If meeting iteration stopping condition, dictionary is exportedWith
Preferably, in the step (5.2), the characteristic image of different directions and different orders include single order horizontal gradient,
Single order vertical gradient, second order horizontal gradient, respectively second order vertical gradient, f1=[- 10 1], f2=f1 T、f3=[1 0-2 0
1]、f4=f3 T, the characteristic image f of corresponding intermediate resolution image is obtained after filtering1MIm, f2mIm,f3MIm, and f4MIm, profit
With mIm, f1MIm, f2mIm,f3MIm, and f4Image data bases of the mIm as low resolution.
Preferably, in the step (5.3), on the basis of lIm, the image block of 3x3 is sampled, and is up-sampled in corresponding Im
The image block of 9x9, and corresponding mIm, f1MIm, f2mIm,f3MIm, and f4MIm then accordingly samples the image block of 6x6 respectively, point
It Cai Yang not M blocks；After image block on sampled I m and mIm, the mean value of current block is subtracted as the image block in sample, is finally obtained
Pairs of sample setWherein N=5,Wherein XiFor the 9x9 image blocks on Im images,For the 6x6 on mIm images
Image block,Respectively f1MIm, f2mIm,f3MIm, and f4The image block of 6x6 on mIm images.
Preferably, in the step (5.6), updateIt is to solve for formula (9)
DictionarySolution procedure using Aries In The Block By Block Relaxation method solve,
It solves firstUpdatePass through formula (10)：
It is givenIt solvesWhen, it solves first
Then updatePass through formula (11)：
Formula (10) and the solution procedure of (11), are solved, so as to fulfill dictionary more using Lagrange duality method
Newly.
The embodiment of this method is described in detail below.
For the ease of the explanation for facilitating understanding, first providing a little symbolic formulas here of hereinafter formula and symbol.Hereinafter
Black upper case character representing matrix：Such as matrix X, black lowercase character represents vector, such as vector x.And vec (X) is usually represented
Matrix X is by the vector form after rearrangement row.XTThe transposed matrix of representing matrix X.Representing matrix set.xjIt is in vector x
Jth element.xijThe element of the i-th row jth row of representing matrix X.The l of vector0, l1, lpNorm is respectively defined as, | | x | |0
=# { xj≠ 0 }, | | x | |1=∑j|xj| andThe l of matrix0, l1, lpNorm defines respectively | | X | |0
=# { xij≠ 0 }, | | X | |1=∑ij|xij| andSymbolRepresent Kroneker tensor operators.
Super-resolution rebuilding algorithm based on two-dimentional sparse representation model
Image super-resolution rebuilding is from fuzzy, and original high-resolution is recovered in the low-resolution image Y of down-sampling
Image X, the process that degrades can be denoted as Y=SHX, and wherein H is fuzzy filter, and S is down-sampling operator, therefore the present invention proposes
Image super-resolution rebuilding model based on two-dimentional sparse representation model is as follows：
Here the right first item is that global between low-resolution image Y and high-definition picture X rebuilds bound term, second
Item and Section 3 are the two-dimentional sparse constraint items in part, that is, ensure each image blockIt can be by two-dimensional level dictionaryWith second vertical dictionaryRarefaction representation.Wherein RiIt is i-th of image block for extracting image X
Operator,It is corresponding rarefaction representation matrix, wherein λ and γ are for balancing the two of fidelity and degree of rarefication ginsengs
Number.
The specific method for solving of the image super-resolution rebuilding algorithm for two-dimentional sparse representation model is described below.Divide such as
Lower two steps are solved.
1. given X solves a series of sparse coefficient { Bi}
2. pass through what is be obtainedSolve final X
First, we are discussed how in the case of given X, solve a series of sparse coefficient { Bi}.Direct solution mesh
Scalar functions (2) are relatively difficult, because of the X and { B in Section 2iAll it is unknown, therefore, we utilize a series of low points
The reconstruction error item of the characteristic image of resolutionTo the error term of high-definition pictureEstimated, here F(k)K-th of Linear feature extraction operator is represented, for extracting low-resolution image
The corresponding feature of block, andWithRespectively kth class low resolution feature
Image F(k)The horizontal and vertical dictionary of Y, F(k)Y includes a series of characteristic image block F(k)Yi。
Here,Represent the value of high-definition picture reconstructed on repeat region.λ is for balancing fidelity and dilute
Dredge the parameter of degree.In addition the l in formula (2)0Norm has had been replaced with l1Norm, so that original non-convex problem turns
Convex problem is turned to, consequently facilitating solving.
When given high-resolution dictionaryWith a series of low-resolution dictionariesWhen, for (4)
It solves, (4) can be converted into following problem
In above-mentioned solution procedure, local two-dimentional rarefaction representation constraint is only only accounted for, next, solving object function
(3), not only consider local sparse constraint, while consider the global restriction of image, therefore, object function (3) can be converted into
Following problem
Wherein c is the parameter for balancing global restriction and local restriction, which can directly utilize gradient descent method
It is solved, the high-definition picture X so as to finally be rebuild*。
The dictionary training algorithm of super-resolution rebuilding
In process discussed above, the calculation that Image Super-resolution Reconstruction is carried out in the case that high-low resolution dictionary gives is given
Method.How to train to obtain the planar dictionary of high-low resolution image from high-low resolution image pair next, the present invention provides.
When given pairs of sample setWhereinIt is high-definition picture block
Set,It is the set of low-resolution image (feature) block, wherein
It is an object of the present invention to the high-resolution dictionaries that training obtainsWith a series of low-resolution dictionaries
High-definition picture block and low-resolution image (feature) block is enabled to share identical sparse coefficient.Wherein training sample set
Include M high-definition picture blockWith N × M low resolution image blocksHere M1=d × M, M2
=b × M.And there are N classes low-resolution images (feature).Therefore the model of training dictionary can be defined as form：
Wherein sparse coefficient collection is combined intoM3=K2× M, whereinIt is high i-th
Image in different resolution block XiWith low-resolution image (feature) blockShared sparse coefficient.
Object function (7) is non-convex optimization problem, can be solved by two benches Aries In The Block By Block Relaxation method, two is crucial
Link：1. sparse coding：In given dictionaryWithSolve the sparse table of high-low resolution image block
Show2. dictionary updating：According to obtaining above-mentioned sparse coefficientWhen, such as and update dictionaryWith
The sparse coding stage
Given high-low resolution dictionaryWithIt can be differentiated by individually solving each height
Rate image block is to XiWithSparse coefficient, so as to obtain sparse coefficient setEach high-low resolution image block
To sparse coefficient BiIt can combine to solve by following object function and obtain：
Similar to the method for solving of formula (4), one-dimensional sparse representation model can be translated into and solved.
The dictionary updating stage
Similary dictionarySolution procedure solved using Aries In The Block By Block Relaxation method, i.e., it is fixedIt solvesOtherwise also
So.It is solved first for usTherefore it updatesSuch as lower section can be passed through
Journey：
Similarly, it givesIt solvesWhen, it solves first Then updateIt can be by solving equation below：
For problem (10) and (11), it is impossible to be used when again using the update dictionary mentioned in two-dimentional sparse representation model strange
Different value decomposes (svd) to be updated line by line to dictionary.Because the dictionary of high-definition picture and the word of low-resolution image
Allusion quotation is newer respectively, so if according to dictionary is updated line by line, can destroy pair between high-low resolution image block
It should be related to.It follows that obtained dictionary is difficult that high-low resolution is kept to share constraining for sparse coefficient again, therefore it is
The correspondence between high-low resolution image block can be kept, the present invention will(or) carry out as a whole more
Newly, therefore, the quadratically constrained quadratic programming of standard (QCQP) problem (10) and (11) is solved using Lagrange duality method.
Finally, the present invention provides the algorithm during dictionary updating and answers for training the dictionary of high-definition picture block
Miscellaneous degree, for given training setThere are M high-definition picture blocksTraining two-dimensional level dictionaryWith second vertical dictionaryThe then sparse coefficient of each blockAnd the sparse coefficient of corresponding same number in order to obtain, then one-dimensional dictionaryWherein c
=d2, K=K1×K2.Then D is updated with M image blockhAlgorithm complexity be O (K2M) (K ＜ ＜ M) updates planar dictionary
Algorithm complexity beTherefore update planar dictionaryWithRequired algorithm complexity is altogetherWork as K1=K2=K1/2When, then it is O (Kc1/2M).And one-dimensional dictionary and the required storage of planar dictionary are empty
Between be respectively c × K=c × K1×K2And c1/2×(K1+K2).No matter obviously from memory space or from algorithm complexity, this
The dictionary training method that invention proposes is less than one-dimensional sparse representation model.
In order to illustrate effectiveness of the invention, the present invention obtains high resolution graphics by trained from external image library first
The planar dictionary of the planar dictionary of picture and corresponding low-resolution image (and characteristic image).And the planar dictionary for obtaining training
Super-resolution rebuilding is carried out applied to a few width images, eventually by the subjective quality and objective quality for comparing Super-resolution Reconstruction image
To illustrate effectiveness of the invention.Wherein objective quality by Y-PSNR (Peak Signal to Noise Ratio,
PSNR it) measures, and subjective quality passes through structural similarity amount (structural similarity (SSIM) index
Measurement, SSIM) measurement.
In order to verify the quality to images above super-resolution rebuilding, mainly pass through Y-PSNR (Peak Signal
To Noise Ratio, PSNR) measurement, unit is decibel (dB).Its calculation formula is as follows：
Two width sizes are that the mean square error MSE of the image of m*n is defined as follows：
Wherein I, J represent the image of original not Noise and utilize sparse coding method reconstruction image respectively, and I (x, y),
J (x, y) is corresponding to the pixel value at position (x, y), then mean square error is smaller, then PSNR is higher, then the denoising effect of this method
Fruit is higher.
In addition the judgment criteria of subjective quality is:Structural similarity amount (structural similarity (SSIM)
index measurement,SSIM).It is a kind of evaluation method based on structure distortion picture quality, this method by brightness and
Contrast is detached from image structure information, and integrated structure information carries out picture quality evaluation and defines：
SSIM (i, j)=[L (i, j)]α·[C(i,j)]β[S(i,j)]γ
Wherein：
L represents brightness (Lightness), with mean value (μi,μj) estimation as brightness, C represents contrast
(Contrast), with standard deviation (σi,σj) estimation spent as a comparison, covariance sigmaijMeasurement as structure similarity degree.α,
Beta, gamma is the weight for adjusting brightness, contrast and structural information, and denominator occurs zero or close to zero in order to prevent, and generates
Wild effect, and introduce λ1,λ2,λ3.As α=β=γ=1, λ3=λ2When/2, simplified formula is
SSIM has symmetry, boundedness, uniqueness, therefore it can reflect the subjective quality of image well, especially
It is since observed value at certain is in a flash more paid close attention to the details of some regional area, SSIM combinations PSNR can more reflect figure
The quality of picture.The discrimination of SSIM is smaller when generally PSNR is larger, i.e. PSNR high, SSIM is also high, and when PSNR is smaller, SSIM
Just there is preferable discrimination.
Present invention employs the external image library identical with SRSR methods and amplify 3 times of basic reconstruction parameters of grade, then
100000 high-definition picture blocks and low-resolution image (feature) block are sampled, dictionary primitive of the invention is 32.This
Invention has selected 15 width standard testing image single frames low-resolution images as input, respectively includes：Flower, Kodim07,
Pallon, Kodim19, Barche, Parrots, Donna, Girl, Parthenon, Raccon, Lena, Tulips,
Peppers, Voit and Papav.
What the SRSR algorithms and Elad that the main and scholars such as bicubic interpolation algorithms, Yang propose in the present invention proposed
Scale-up algorithms are compared.Wherein, in order to illustrate the validity of dictionary memory space, the one-dimensional dictionary size of Yang is set
One-dimensional dictionary size for scholars such as 225x60, Zeyde is (324*30+111*5), and this dictionary size is 2x39x32,.
That is the dictionary size that SRSR and Scale-up methods use is 5 times and 4 times of this dictionary respectively.Finally in order to preferably compare
Compared with the quality of super-resolution rebuilding, the comparison result of the PSNR and SSIM of the gray level image of reconstruction are only provided here：
The PSNR results contrasts of 1 super-resolution rebuilding image of table
The SSIM results contrasts of 2 super-resolution rebuilding image of table
The above is only presently preferred embodiments of the present invention, not makees limitation in any form to the present invention, it is every according to
According to any simple modification, equivalent change and modification that the technical spirit of the present invention makees above example, still belong to the present invention
The protection domain of technical solution.
Claims (7)
1. a kind of image super-resolution rebuilding method, it is characterised in that：Include the following steps：
(1) RGB image of the low resolution of input is converted into YCbCr images, wherein Y is non-linear luma component, and Cb is blue
Color color difference components, Cr are red color difference components；Super-resolution rebuilding is carried out using bilinear interpolation to Cb, Cr；
(2) Y-component is denoted as lIm, carries out 2 times of up-samplings using bilinear interpolation, obtain the image of corresponding intermediate resolution
MIm, while the characteristic image of mIm images is solved, obtain the characteristic image of different directions and different orders；
(3) using the lIm upper left corners as starting point, the image block Y of 3x3 is sampled successivelyi, wherein there is the repeatability of 1 pixel in each direction
Region calculates the mean value M of current block；
(4) respectively from the characteristic image of mIm and different directions and different orders, the image of the 6x6 on corresponding position is extracted
BlockAnd it solvesWhereinRepresent high-resolution reconstructed on repeated region
The value of rate image, YiFor the image block of 3x3, F(k)K-th of Linear feature extraction operator is represented, for extracting low-resolution image
The corresponding feature of block, F(k)YiRepresent the characteristic image block of extraction；vec(Zi) it is the new vector constructed；
(5) trained dictionary is utilizedWithIt solves
Wherein D represents one-dimensional dictionary；WithRespectively kth class low resolution characteristic image F(k)The horizontal and vertical word of Y
Allusion quotation；WithRepresent the horizontal and vertical dictionary of high-definition picture X；
(6) sparse coefficient of current block is solved：
Rebuild the high-resolution features image block of corresponding positionWherein λ is for balancing fidelity and degree of rarefication
Parameter,It is i-th of high-definition picture block XiWith low-resolution image blockShared sparse coefficient；
(7) by XiThe high-definition picture blocks of+M as the reconstruction of corresponding position；
(8) judge whether all to have carried out sampling and super-resolution rebuilding to low-resolution image, if the low resolution of all 3x3
Image block is complete, then performs step (9), otherwise performs step (3)-(8)；
(9) final high-definition picture X is solved according to formula (6)*：
Wherein c is the error that parameter is used for balancing global and local；Y is low-resolution image；S is down-sampling operator；H is fuzzy
Wave filter；X is high-definition picture；When solution obtains BiAfterwards, high-definition picture X is rebuild0；
The high-definition picture X that Y-component super-resolution rebuilding is obtained*With the super-resolution obtained to Cb, Cr using bilinear interpolation
Rate reconstruction image combines, and obtains the high-definition picture of YCbCr space；Then color space conversion is carried out to it, is converted
To RGB color, colored super-resolution rebuilding image is finally obtained.
2. image super-resolution rebuilding method according to claim 1, it is characterised in that：It is obtained not in the step (2)
The characteristic image of equidirectional and different orders is that single order horizontal gradient, single order vertical gradient, second order horizontal gradient, second order are vertically terraced
Degree, respectively f1=[- 10 1], f2=f1 T、f3=[1 0-2 0 1], f4=f3 T, corresponding medium resolution is obtained after filtering
The characteristic image f of rate image1MIm, f2mIm,f3MIm, and f4mIm。
3. image super-resolution rebuilding method according to claim 2, it is characterised in that：The dictionary packet of the step (5)
Include it is following step by step：
(5.1) high-resolution image data base is established, if image is coloured image in itself, is first converted into coloured image
Gray level image；
(5.2) image data base of low resolution is established：To the image in external image library, 3 times of down-samplings are carried out, are corresponded to
Low-resolution image lIm, then to all low-resolution image lIm, carry out 2 times of up-samplings, obtain intermediate resolution
Image mIm, while the characteristic image of mIm images is solved, the characteristic image of different directions and different orders is obtained, is obtained after filtering
The characteristic image of corresponding intermediate resolution image, using mIm, the characteristic image of intermediate resolution image is as low resolution
Image data base；
(5.3) to the image data base of the high-low resolution of foundation, sampling obtains pairs of high-definition picture block and low resolution
The training sample set of rate characteristic image block；
(5.4) dictionary is initializedWithRandomly choose the image block X of same indexi, Yi (1), Yi (2), Yi (3), Yi (4)And Yi (5), respectively hx is obtained by row averagingi, AndIt is averaging by row
To vxi T, AndBy hxiAs dictionaryA row primitive, and vxiAs word
Allusion quotationA row primitive, can construct respectively in this wayA row primitive；
(5.5) to dictionaryWithCarry out sparse coding：First withWithPoint
Other fabric tensor dictionary, so as to obtain the dictionary rebuild for sparse codingAnd to D each column signal into
Row normalization operation, while to sample setIn every a pair of sample, construct new vector
Then i-th of sparse coefficient B is solved using formula (5)i, all sparse coefficient set are then obtained successively
WhereinFor paired samples collection,It is the collection of high-definition picture block
It closes,It is the set of low-resolution image characteristic block, and
(5.6) to dictionaryWithCarry out dictionary updating：Dictionary is updated respectively With
(5.7) judge whether to reach iteration stopping condition：Step (5.3) is jumped to if iteration stopping condition is unsatisfactory for；Such as
Fruit meets iteration stopping condition, then exports dictionaryWith
4. image super-resolution rebuilding method according to claim 3, it is characterised in that：It is different in the step (5.2)
It is vertically terraced that the characteristic image of direction and different orders includes single order horizontal gradient, single order vertical gradient, second order horizontal gradient, second order
Degree, respectively f1=[- 10 1], f2=f1 T、f3=[1 0-2 0 1], f4=f3 T, corresponding medium resolution is obtained after filtering
The characteristic image f of rate image1MIm, f2mIm,f3MIm, and f4MIm utilizes mIm, f1MIm, f2mIm,f3MIm, and f4MIm conducts
The image data base of low resolution.
5. image super-resolution rebuilding method according to claim 4, it is characterised in that：In the step (5.3), with
On the basis of lIm, the image block of 3x3 is sampled, and in the image block of corresponding Im up-samplings 9x9, and corresponding mIm, f1MIm,
f2mIm,f3MIm, and f4MIm then accordingly samples the image block of 6x6 respectively, samples M blocks respectively；Image block on sampled I m and mIm
Afterwards, the mean value of current block is subtracted as the image block in sample, finally obtains pairs of sample set Wherein N=5,Wherein
XiFor the 9x9 image blocks on Im images,For the image block of the 6x6 on mIm images, Yi (2),Yi (3),Yi (4),Yi (5)Respectively
f1MIm, f2mIm,f3MIm, and f4The image block of 6x6 on mIm images；Im is the image in high resolution image data library.
6. image super-resolution rebuilding method according to claim 5, it is characterised in that：In the step (5.6), updateIt is to solve for formula (9)
DictionarySolution procedure using Aries In The Block By Block Relaxation method solve,
It solves firstUpdatePass through formula (10)：
It is givenIt solvesWhen, it solves first
Then updatePass through formula (11)：
Formula (10) and the solution procedure of (11), are solved using Lagrange duality method, so as to fulfill the update of dictionary；
Wherein r, t represent dictionary respectivelyWithColumn vector index.
7. image super-resolution rebuilding method according to claim 6, it is characterised in that：In the step (5.7), iteration stopping item
Part is：Iterations reach upper limit num；Or fidelity error reaches
ε1To preset error；Or degree of rarefication reaches | | Bi||1≤ε2, ε2To preset error.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510385633.3A CN105046672B (en) | 2015-06-30 | 2015-06-30 | A kind of image super-resolution rebuilding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510385633.3A CN105046672B (en) | 2015-06-30 | 2015-06-30 | A kind of image super-resolution rebuilding method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105046672A CN105046672A (en) | 2015-11-11 |
| CN105046672B true CN105046672B (en) | 2018-06-22 |
Family
ID=54453191
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510385633.3A Active CN105046672B (en) | 2015-06-30 | 2015-06-30 | A kind of image super-resolution rebuilding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105046672B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105678697B (en) * | 2015-12-30 | 2018-10-12 | 北京工业大学 | A kind of face image super-resolution reconstruction method based on DCT domain eigentransformation |
| CN105657263B (en) * | 2015-12-31 | 2018-11-02 | 杭州卓腾信息技术有限公司 | A kind of super-resolution digital slices scan method based on area array cameras |
| CN105957029B (en) * | 2016-04-25 | 2019-06-04 | 南方医科大学 | MR image reconstruction method based on tensor dictionary learning |
| CN105976410A (en) * | 2016-05-05 | 2016-09-28 | 南京信息工程大学 | Method for rapid super-resolution reconstruction of single image based on non-linear prediction sparse coding |
| CN106097278B (en) * | 2016-06-24 | 2021-11-30 | 北京工业大学 | Sparse model, reconstruction method and dictionary training method of multi-dimensional signal |
| CN107977929B (en) * | 2016-10-21 | 2021-02-23 | 中国电信股份有限公司 | Image super-resolution processing method and device |
| CN108369725A (en) * | 2017-03-13 | 2018-08-03 | 深圳市大疆创新科技有限公司 | Handle method, chip, processor, computer system and the mobile device of image |
| CN107239781B (en) * | 2017-05-03 | 2020-07-28 | 北京理工大学 | Hyperspectral reflectivity reconstruction method based on RGB image |
| CN108734675B (en) * | 2018-05-17 | 2021-09-28 | 西安电子科技大学 | Image restoration method based on mixed sparse prior model |
| CN108765297B (en) * | 2018-06-14 | 2020-07-17 | 厦门大学 | Super-resolution reconstruction method based on cyclic training |
| CN109102463B (en) * | 2018-08-13 | 2023-01-24 | 苏州飞搜科技有限公司 | Super-resolution image reconstruction method and device |
| CN109816600A (en) * | 2018-12-21 | 2019-05-28 | 西北工业大学 | Confocal microscopy image restored method based on rarefaction representation |
| CN110264407B (en) * | 2019-06-28 | 2023-01-03 | 北京金山云网络技术有限公司 | Image super-resolution model training and reconstruction method, device, equipment and storage medium |
| CN111210390A (en) * | 2019-10-15 | 2020-05-29 | 杭州电子科技大学 | Motion blur restoration method based on Golay sequence complementary code word set |
| CN113689341B (en) * | 2020-05-18 | 2024-08-23 | 京东方科技集团股份有限公司 | Image processing method and training method of image processing model |
| CN112801919B (en) * | 2021-03-22 | 2021-06-29 | 恒生电子股份有限公司 | Image defogging model training method, defogging processing method and device and storage medium |
| CN114972038B (en) * | 2022-06-20 | 2024-09-03 | 中国科学院自动化研究所 | End-to-end multi-frame super-resolution method and system based on correlation dynamic filtering |
| CN116309067B (en) * | 2023-03-21 | 2023-09-29 | 安徽易刚信息技术有限公司 | Light field image space super-resolution method |
-
2015
- 2015-06-30 CN CN201510385633.3A patent/CN105046672B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| SINGLE IMAGE SUPER-RESOLUTION VIA 2D SPARSE REPRESENTATION;Qi Na et al;《IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA & EXPO（ICME）》;20150629;第3、4节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105046672A (en) | 2015-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105046672B (en) | A kind of image super-resolution rebuilding method | |
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| Xu et al. | EMFusion: An unsupervised enhanced medical image fusion network | |
| CN109035142B (en) | Satellite image super-resolution method combining countermeasure network with aerial image prior | |
| CN109741256A (en) | Image super-resolution rebuilding method based on rarefaction representation and deep learning | |
| CN112465827A (en) | Contour perception multi-organ segmentation network construction method based on class-by-class convolution operation | |
| CN106204447A (en) | The super resolution ratio reconstruction method with convolutional neural networks is divided based on total variance | |
| CN109214989B (en) | Single image super resolution ratio reconstruction method based on Orientation Features prediction priori | |
| CN109389552A (en) | A kind of Image Super-resolution based on context-sensitive multitask deep learning | |
| CN110047138A (en) | A kind of magnetic resonance thin layer image rebuilding method | |
| CN106920214B (en) | Super-resolution reconstruction method for space target image | |
| CN106157244A (en) | A kind of QR Code Image Super-resolution Reconstruction method based on rarefaction representation | |
| CN104835130A (en) | Multi-exposure image fusion method | |
| CN110634147A (en) | Image matting method based on bilateral boot up-sampling | |
| CN105631807A (en) | Single-frame image super resolution reconstruction method based on sparse domain selection | |
| Yang et al. | Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction | |
| Guo et al. | Multiscale semilocal interpolation with antialiasing | |
| Yang et al. | Image super-resolution based on deep neural network of multiple attention mechanism | |
| Kang et al. | Fusion of brain PET and MRI images using tissue-aware conditional generative adversarial network with joint loss | |
| CN109559278B (en) | Super resolution image reconstruction method and system based on multiple features study | |
| CN107845065A (en) | Super-resolution image reconstruction method and device | |
| CN112163998A (en) | Single-image super-resolution analysis method matched with natural degradation conditions | |
| Hu et al. | An efficient fusion algorithm based on hybrid multiscale decomposition for infrared-visible and multi-type images | |
| Yan et al. | Md³net: Integrating model-driven and data-driven approaches for pansharpening | |
| Mansour et al. | Image-to-image MLP-mixer for image reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |