CN104008539B - Image super-resolution rebuilding method based on multiscale geometric analysis - Google Patents
Image super-resolution rebuilding method based on multiscale geometric analysis Download PDFInfo
- Publication number
- CN104008539B CN104008539B CN201410234469.1A CN201410234469A CN104008539B CN 104008539 B CN104008539 B CN 104008539B CN 201410234469 A CN201410234469 A CN 201410234469A CN 104008539 B CN104008539 B CN 104008539B
- Authority
- CN
- China
- Prior art keywords
- image
- frequency coefficient
- tau
- resolution
- high frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 90
- 238000004458 analytical method Methods 0.000 title claims abstract description 17
- 230000009466 transformation Effects 0.000 claims abstract description 42
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 10
- 230000004927 fusion Effects 0.000 claims abstract description 8
- 238000002474 experimental method Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 238000006731 degradation reaction Methods 0.000 claims description 8
- 230000015556 catabolic process Effects 0.000 claims description 7
- 238000010168 coupling process Methods 0.000 claims description 7
- 238000005859 coupling reaction Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000002146 bilateral effect Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008878 coupling Effects 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 abstract description 11
- 230000008859 change Effects 0.000 abstract description 3
- 238000002059 diagnostic imaging Methods 0.000 abstract description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 abstract description 2
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 7
- 230000007850 degeneration Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000006866 deterioration Effects 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 238000013519 translation Methods 0.000 description 3
- 238000000701 chemical imaging Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000003325 tomography Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000013401 experimental design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000002969 morbid Effects 0.000 description 1
- 230000004899 motility Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses an image super-resolution rebuilding method based on multiscale geometric analysis. The method includes the steps that (1) a low-resolution image is generated by means of a blurring model, (2) image registration is carried out, (3) image fusion is carried out, and (4) a rebuilt image is obtained. The method has good invariance for horizontal moving, rotation, size scaling, noise and the like, stability of affine transformation, view change and the like remains to a certain extent, and accordingly edge information keeps clear and relatively smooth; an algorithm is stable and has low calculation complexity and the method has wide application prospects in the fields of space target imaging, remote sensing imaging, medical imaging, tomographic imaging and the like.
Description
Technical field
The invention belongs to image super-resolution rebuilding technical field, it is related to a kind of image based on multi-scale geometric analysis and surpasses
Resolution reconstruction method.
Background technology
During digital image acquisition, due to the restriction of imaging device performance and the impact of shooting condition, can make to adopt
The image collecting assumes relatively low resolution.Image super-resolution rebuilding (Super-Resolution Image
Reconstruction, SR) it is that one kind obtains high-resolution (HP) image or figure from single frames or multiframe low resolution (LP) image
As sequence, thus improving picture quality, improving the signal processing method of image visual effect.
Image super-resolution rebuilding technology, for such as motion deformation, optical dimming, low sampling in image imaging process
The image deterioration problem that the various degeneration factor such as rate, random noise causes is studied, extraterrestrial target imaging, remotely sensed image,
The various fields such as medical imaging, tomography are with a wide range of applications, and for various microwave imagings, multispectral imaging, surpass
The HD image of acoustic imaging etc. recovers also will there is reference role, additionally, in medical domain, computer generated image, financial sector
The aspects such as HD image recovery, the identification of special objective and the evidence obtaining of recording monitor data Deng important events also have important
Application.
In terms of mathematical angle, image super-resolution rebuilding is that non-the fitting under Hardmard meaning determines mathematical anti-problem, therefore
Become applied mathematics, image procossing, computer vision, calculate the numerous researcher concerns in the world of the multidisciplinary fields such as harmonic analyses
Hot issue.
Image super-resolution rebuilding method can be largely classified into following a few class：Frequently (rate) domain method, sky () domain method, base
Method in study.The basic thought of frequency domain method is：With image conversion, low-resolution image is transformed to frequency domain, adopt in frequency
Remove spectral aliasing with suitable method, then again spatial domain is changed to by inversion, obtain the result of super-resolution rebuilding.Frequency domain side
Law theory is simple, computational complexity low it is easy to realize parallel processing.But the shortcoming of this kind of method is the theoretical premise being based on
Excessively idealize, global translation motion and the constant model that degrades of linear space can only be confined to it is impossible to be effectively applied to majority
Occasion.A kind of super resolution ratio reconstruction method of anti-aliasing profile wave structure is utilized in the current research result [1] of such method.Should
Method substitutes the tower wave filter group of Laplce in profile wave convert with anti-aliasing tower wave filter group and carries out chi to image
Degree conversion, reduces spectral aliasing；According to the similarity between the high-frequency sub-band of different scale, high fdrequency components are carried out with subband and inserts
Value, and decomposed by anisotropic filter group travel direction, high-frequency information is carried out with selectivity fusion treatment and profile ripple inverse transformation,
Thus obtaining reconstruction image.
Spatial domain method have stronger comprise the prior-constrained ability in spatial domain, main method includes：
Nonuniform sample interpolation：The theoretical basiss of this method are that low-resolution image is after motion estimation algorithm
It is heterogeneous when being mapped on high-definition picture grid, therefore by height be can be obtained by the interpolation of nonuniform sampling point
The pixel value of the sampled point on image in different resolution grid.The utilization multiaspect matching proposing in Typical Representative such as [2] is realized being based on and is inserted
The image super-resolution rebuilding method of value, the method is realized to space by the pixel of low-resolution image is carried out with multiple sampling
The making full use of of structural information, the also image super-resolution rebuilding method based on sparse neighbor interpolation in [3].
Inverse iteration sciagraphy：It is to propose in [4] earliest, its principle is using the high-definition picture simulation estimated
Image deterioration process, generates the low-resolution image estimated；Then calculate the difference with the low-resolution image of observation；Residual error is thrown
Shadow is in the high-definition picture estimated.Repeat this process, the high-definition picture until estimating meets stopping criterion for iteration.
Projections onto convex sets：A kind of image super-resolution rebuilding method based on set theory, the method have motility and
Extensibility, can preferably preserve the detailed information such as the edge of image, but computation complexity is high, is affected by initial estimate
Larger, and, there is no the best initial estimate of effective method choice, in addition, the solution of this method is not unique, much should
With in be difficult to meet the requirement of particular problem.
Statistical method-maximum a-posteriori estimation and maximal possibility estimation：Super-resolution rebuilding problem is a morbid state
Problem, ill-conditioning problem to be made is converted into solvable problem, needs previously-introduced priori additional conditions.The thought of maximum a posteriori probability is just
It is under conditions of known sequence of low resolution pictures so that the posterior probability of high-definition picture is maximum.Such method also may be used
So that with reference to image deblurring and denoising etc., so that reconstructed results have uniqueness and stability, its defect is that amount of calculation is huge.
Super resolution ratio reconstruction method based on study：Such method is proposed in document [5] by Freeman first, its base
This thought is the relation first learning between low-resolution image and high-definition picture, then instructs super-resolution using this relation
Rate is rebuild.One kind is proposed based on non-local mean and adjustment kernel regression (Steering Kernel in current research result [6]
Regression single-frame image super-resolution reconstruction method), the method pass through learn low-resolution image in non local and
Local rule priori is realizing super-resolution reconstruction.Document [7] proposes a kind of sparse volume of cluster based on multiple geometry dictionary
Code scheme, for the super-resolution rebuilding of image.The method is supported to concentrate from training image and is extracted a large amount of high resolution graphics at random
Picture, and it is clustered into geometric areas, from learning corresponding geometry dictionary, each of sparse coding low-resolution image local
Block.Document [8], using nonparametric Bayes inference method based on numerical integration, is referred to as in statistical literature that " integrated nesting is drawn
This is approximate for pula ", construct super resolution ratio reconstruction method.
The final goal of image super-resolution rebuilding algorithm is to improve the quality of image as far as possible.In view of frequency domain method reason
By simple, computational complexity is low to be therefore a kind of very practical method for reconstructing it is easy to realize and be easy to parallel processing, but lacks
Point is can only to be confined to global translation motion and the constant model that degrades of linear space it is impossible to be effectively applied to most occasions.For
Make up this deficiency of frequency domain method, it is contemplated that with multi-scale geometric analysis instrument profile wave converts
(Contourlet) multiple dimensioned, the multi-direction, multiresolution, being had using profile wave convert and anisotropic feature, are come
Effectively catch the singular point in image, and by suitably being chosen to profile wave system number, to make full use of the space of image
Dimensional information and directional information, reconstructed high frequency information, improves the quality of image.
Content of the invention
It is an object of the invention to provide a kind of image super-resolution rebuilding method based on multi-scale geometric analysis, solve
From multiframe low-resolution image, characteristic information extraction to rebuild the radio-frequency component of loss, leads to asking of image resolution ratio reduction
Topic.
The technical solution adopted in the present invention, a kind of image super-resolution rebuilding method based on multi-scale geometric analysis,
Specifically implement according to following steps：
Step 1, using degrade model generate low-resolution image
Continuous three frame normal resolution image I are chosen from observed image1、I2And I3, entered at row degradation using following model
Reason, obtains corresponding low-resolution image T1、T2And T3：
Tk=D B M Ik+ n=H Ik+ n,
In formula, k=1,2,3, D, B, M, n be respectively down-sampling operator, obscure operator, motion blur operator, noise operator；
Step 2, image registration
For image T subject to registrationk, k=1,2,3, select the wherein larger two field picture T of signal to noise ratio1As reference frame, profit
Extract image characteristic point with SIFT algorithm, if TkSet of characteristic points be respectively：
Wherein, mkIt is corresponding to TkFeature count out,K=1,2,3, j=1,
2,...,max(m1,m2,m3),
Set of computations P respectively1With P2、P1With P3In characteristic point between Euclidean distance：
j1=1,2 ..., m1, j2=1,2 ..., m2
Wherein, j1=1,2 ..., m1, j2=1,2 ..., m2, j3=1,2 ..., m3,
Find out P1With P2Minimum Eustachian distance d between middle characteristic point1With secondary little Euclidean distance d2, definition distance rates are r=
d1/d2, defining registering criterion is：
Wherein η is the threshold value being obtained by experiment, if more than threshold value, distance rates think that the match is successful, otherwise it is assumed that
Join unsuccessfully；To T3Do same process, obtain images after registration and be respectively：
T2'=M1·T2,
T3'=M2·T3,
Wherein, M1、M2It is motion compensation matrix；
Step 3, image co-registration
Contourlet transformation is carried out to the image after registration, i.e. Ck=C (Tk'), wherein, C () is profile ripple computing,
Assume image T to be reconstructed after couplingk' the low frequency coefficient after contourlet transformation is designated asIt is shown below：
Wherein k=2,3, the number of low frequency coefficient is l=i*j, the high frequency coefficient of note τ directional subband of s-th yardstick
ForTotal high frequency coefficient h=p*q,
For the low frequency coefficient of gained after contourlet transformation, processed using weighted mean method, obtained inverse transformation
Low frequency coefficient, kth frame coupling after image T to be reconstructedk' the low frequency coefficient after contourlet transformation isωkFor power
Weight coefficient, and meetThen final choice as the low frequency coefficient of Contourlet inverse transformation is
HFS for gained after contourlet transformation is merged by following principle：
It is respectively compared the size of corresponding high frequency coefficient variance on corresponding each directional subband of each yardstick, take variance
Little coefficient as the high frequency coefficient of corresponding scale respective direction inverse transformation, that is, adoptsRepresent the τ direction of s-th yardstick
The meansigma methodss of subband high frequency coefficient, this average value expression is：
UsingRepresent the variance of the τ directional subband high frequency coefficient of s-th yardstick, this variance expression formula is：
It is the height of the τ directional subband Contourlet inverse transformation of s-th yardstick
Frequency coefficient, order：
Wherein Gs,τRepresent final choice as the high frequency coefficient of the τ directional subband of s-th yardstick of inverse transformation, G is each
Direction each scale coefficient matrix, wherein row represent the direction index of directional subband, and row represent yardstick index,
Carry out Contourlet inverse transformation with the high frequency coefficient G of low frequency coefficient L and each each direction of yardstick, i.e. I'=C-1
(L, G), C-1() is profile ripple transform operation, and I' is fusion image；
Step 4, acquisition reconstruction image
Bicubic difference operation is carried out to fusion image I' obtaining and obtains I'', bilateral filtering process is carried out to I'', obtains
To reconstruction image II.
The present invention has the advantages that：
1) the image registration step of the inventive method, using the method for registering images based on Sift, Sift feature belongs to figure
As local feature, oneself is proved there is excellent invariance to translation, rotation, scaling and noise etc., to affine transformation,
Visual angle change etc. also keeps certain stability.Using multi-scale geometric analysis instrument profile wave convert (Contourlet)
Multiple dimensioned, multi-direction, multiresolution and anisotropic feature, effectively catch the singular point in image, and by profile ripple
Coefficient is suitably chosen, and makes full use of space scale information and the directional information of image, recovers the radio-frequency component lost, carries
The quality of hi-vision.The inventive method has taken into full account marginal information in image so that marginal information keeps clear, relatively flat
Sliding.
2) method of the present invention is applied to gray level image and coloured image.Test result indicate that, the method for the present invention reaches
Obvious reconstruction effect, algorithm has stability, and have relatively low computation complexity.
3) method of the present invention has in various fields such as extraterrestrial target imaging, remotely sensed image, medical imaging, tomographies
It is widely applied prospect；HD image recovery for various microwave imagings, multispectral imaging, ultra sonic imaging etc. also will have
Reference role；Additionally, the high-resolution of the recording monitor data in important events such as medical domain, computer generated image, financial sectors
The aspects such as image recovery, the identification of special objective and evidence obtaining also have important application.
Brief description
Fig. 1 is the FB(flow block) of the inventive method；
Fig. 2 is the original Lena image of the inventive method embodiment；
Fig. 3 is the broad image one of Fig. 2；
Fig. 4 is the broad image two of Fig. 2；
The original color image of Tu5Shi embodiment bookstore.
Fig. 6 is the broad image three of Fig. 5；
Fig. 7 is the broad image four of Fig. 5；
Fig. 8 is the gray level image registration effect that with Fig. 2 for reference frame, Fig. 3 is carried out with registration；
Fig. 9 is the coloured image registration effect that with Fig. 6 for reference frame, Fig. 7 is carried out with registration.
Figure 10 is the original-gray image of embodiment cartoon；
Figure 11 is left one side of something degraded image of Figure 10；
Figure 12 is right one side of something degraded image of Figure 10；
Figure 13 is the reconstructed results with the method for the present invention to Figure 11 and Figure 12；
Figure 14 is the original color image of embodiment river；
Figure 15 is left one side of something degraded image of Figure 14；
Figure 16 is right one side of something degraded image of Figure 14；
Figure 17 is the reconstructed results using the inventive method to Figure 15 and Figure 16；
Figure 18 is that the MSE in the inventive method embodiment minimum direction compares cartogram；
Figure 19 is that the MSE in direction in the middle of the inventive method embodiment compares cartogram；
Figure 20 is that the MSE in the inventive method embodiment maximum direction compares cartogram.
Specific embodiment
The present invention is described in detail with reference to the accompanying drawings and detailed description.
Referring to Fig. 1, the image super-resolution rebuilding method based on multi-scale geometric analysis for the present invention, including the low resolution of generation
Rate image (degraded image), image registration, image co-registration and acquisition reconstruction image step, specifically implement according to following steps：
Step 1, using degrade model generate low-resolution image (degraded image)
Continuous three frame normal resolution image I are chosen from observed image1、I2And I3, entered at row degradation using following model
Reason, obtains corresponding low-resolution image T1、T2And T3：
Tk=D B M Ik+ n=H Ik+ n,
In formula, k=1,2,3, D, B, M, n be respectively down-sampling operator, obscure operator, motion blur operator, noise operator.
During this utilize point spread function PSF (Point Spread Function) produce motion blur function and under
Sample operator, this function has two parameters theta and len, represents shooting object counter clockwise direction with theta angular movement
Len pixel, changes the fog-level of image by adjusting this two parameters.
Step 2, image registration
In view of having reasonable registration effect based on the matching algorithm of SIFT feature point, this step adopts this kind of method
Carry out image registration.
For image T subject to registrationk, k=1,2,3, select the wherein larger two field picture T of signal to noise ratio1(without loss of generality, if
For T1) as reference frame, extract image characteristic point using SIFT algorithm, if TkSet of characteristic points be respectively：
Wherein, mkIt is corresponding to TkFeature count out,K=1,2,3, j=1,
2,...,max(m1,m2,m3),
Set of computations P respectively1With P2、P1With P3In characteristic point between Euclidean distance：
j1=1,2 ..., m1, j2=1,2 ..., m2
Wherein, j1=1,2 ..., m1, j2=1,2 ..., m2, j3=1,2 ..., m3,
Find out P1With P2Minimum Eustachian distance d between middle characteristic point1With secondary little Euclidean distance d2, definition distance rates are r=
d1/d2, defining registering criterion (matching principle) is：
Wherein η is the threshold value being obtained by experiment, if more than threshold value, distance rates think that the match is successful, otherwise it is assumed that
Join unsuccessfully；To T3Do same process, obtain images after registration and be respectively：
T2'=M1·T2,
T3'=M2·T3,
Wherein, M1、M2It is motion compensation matrix.
Step 3, image co-registration
Contourlet transformation is carried out to the image after registration, i.e. Ck=C (Tk'), wherein, C () is profile ripple computing,
Assume image T to be reconstructed after couplingk' the low frequency coefficient after contourlet transformation is designated asIt is shown below：
Wherein k=2,3, the number of low frequency coefficient is l=i*j, the high frequency coefficient of note τ directional subband of s-th yardstick
ForTotal high frequency coefficient h=p*q,
For the low frequency coefficient of gained after contourlet transformation, processed using weighted mean method, obtained inverse transformation
Low frequency coefficient, kth frame coupling after image T to be reconstructedk' the low frequency coefficient after contourlet transformation isωkFor power
Weight coefficient, and meetThen final choice as the low frequency coefficient of Contourlet inverse transformation is
HFS for gained after contourlet transformation is merged by following principle：
It is respectively compared the size of corresponding high frequency coefficient variance on corresponding each directional subband of each yardstick, take variance
Little coefficient as the high frequency coefficient of corresponding scale respective direction inverse transformation, that is, adoptsRepresent the τ direction of s-th yardstick
The meansigma methodss of subband high frequency coefficient, this average value expression is：
UsingRepresent the variance of the τ directional subband high frequency coefficient of s-th yardstick, this variance expression formula is：
It is the height of the τ directional subband Contourlet inverse transformation of s-th yardstick
Frequency coefficient, order：
Wherein Gs,τRepresent final choice as the high frequency coefficient of the τ directional subband of s-th yardstick of inverse transformation, G is each
Direction each scale coefficient matrix, wherein row represent the direction index of directional subband, and row represent yardstick index,
Carry out Contourlet inverse transformation with the high frequency coefficient G of low frequency coefficient L and each each direction of yardstick, i.e. I'=C-1
(L, G), C-1() is profile ripple transform operation, and I' is fusion image.
Step 4, acquisition reconstruction image
Bicubic difference operation is carried out to fusion image I' obtaining and obtains I'', bilateral filtering process is carried out to I'', obtains
To reconstruction image II.
The emulation experiment of the inventive method
The purpose of emulation experiment is to analyze the effectiveness of the inventive method by visual effect and numerical result.Emulation is real
Test and complete under MATLAB2010a environment, the computer processor of use is Pentium (R) Dual-Core CPU,
E5400 2.70GHz, 2.70GHz, inside save as 2.00GB.
Used in emulation experiment, image is derived from http://image.baidu.com and UCID image library [9].These figures
As containing the various natural image such as plant, animal, personage, landscape, there is different structure feature, different photoenvironment, difference
Brightness, different texture feature.
1) visual effect is shown
The purpose designing this experiment is visual effect in order to show the method, and the test image in experiment is all from using
State the degraded image collection of method generation.
1.1) visual effect of image registration：
Tested according to the algorithm described in step 2, Fig. 2 is original Lena image (gray level image), and Fig. 3 is by Fig. 2
The low-resolution image generating through degeneration Fuzzy Processing, Fig. 4 is through degeneration Fuzzy Processing by Fig. 2 with bigger degradation factor
The low-resolution image generating.Fig. 8 represents the gray level image registration effect that with Fig. 2 for reference frame, Fig. 3 is carried out with registration.Fig. 5 is
The original color image of bookstore, Fig. 6 is the low-resolution image being generated through degeneration Fuzzy Processing by Fig. 5, and Fig. 7 is by Fig. 5 warp
Cross the low-resolution image of degeneration Fuzzy Processing generation, Fig. 9 represents the coloured image that with Fig. 6 for reference frame, Fig. 7 is carried out with registration
Registration effect.
1.2) visual effect of image reconstruction：
In order to show reconstruction effect, this experimental design is degenerated the left-half of same two field picture and right half part respectively, so
Tested according to the method described in step 2, step 3 and step 4 afterwards.Figure 10 is the original gradation artwork of cartoon, Tu11Shi
Degraded image after row degradation is processed is entered to left one side of something of Figure 10, Figure 12 is that the right one side of something to Figure 10 enters the fall after row degradation is processed
Matter image, Figure 13 is gray level image reconstructed results.Figure 14 is the original color image of river, and Figure 15 is that the left one side of something to Figure 14 enters
Degraded image after row degradation process, Figure 16 is that the right one side of something to Figure 14 enters the degraded image after row degradation is processed, and Figure 17 is coloured silk
The reconstructed results of color image.
2) Numerical Value Result Analysis
2.1) peak noise ratio (PSNR) and signal noise ratio (snr) of image (ISNR) analysis
Table 1, the comparative result of PSNR and ISNR
The purpose of this experiment is the quality that the checking method of the present invention can improve image, for this purpose, using pass
It is analyzed in objective evaluation standard PSNR of picture quality and ISNR.On the premise of randomly choosing Degenerate Graphs image set, select
500 width images of image library, for different gray level images and coloured image, are respectively compared absolute-value scheme [1] and variance method (this
Inventive method) PSNR and ISNR, the value of variance method is better than absolute-value scheme, and detailed results are as shown in table 1.
2.2) mean square error (Mean Squared Error, MSE) analysis
This experiment is passed through to calculate the stability of the objective evaluation standard MSE explanation method of the present invention of picture quality.At random
Select Degenerate Graphs image set, table 2 is that it is carried out use respectively after the contourlet transformation of different series with variance method [1] and the present invention
Method calculate MSE.
Table 2, MSE compare (unit：1×109db)
The every level of decomposition direction decomposed in view of Contourlet is alternatively, we select respectively to each level
Minimum direction, middle direction and maximum its MSE of direction calculating, result shows, increases with decomposed class, the inventive method
MSE value shows obvious stability, and Figure 18, Figure 19 and Figure 20 sets forth the comparison cartogram in three directions.
Table 3, calculating time compare (unit：s)
2.3) efficiency analysiss
This experiment statistics completes the calculating time of different decomposition series, result such as table for two methods of same two field picture
Shown in 3.
Test result indicate that the method for the present invention has higher computational efficiency.
The inventive method utilizes the multiple dimensioned, multi-party of multi-scale geometric analysis instrument profile wave convert (Contourlet)
To, multiresolution and anisotropic feature, effectively catch the singular point in image, and by carrying out properly to profile wave system number
Selection, make full use of space scale information and the directional information of image, recover the radio-frequency component lost, improve the matter of image
Amount.Firstly for reference picture, using image deterioration model, obscured by diffusion, motion blur, a Gaussian noise etc. and
Degeneration is processed, and obtains multiframe low-resolution image；Image registration is carried out using SIFT algorithm, the mapping of multiframe low-resolution image
To the same grid of reference；Contourlet transformation is carried out to the image after registration；For obtained after contourlet transformation
Low frequency part, obtains the low frequency coefficient of its inverse transformation using weighted mean method, for the height being obtained after contourlet transformation
Frequency part, compares the variance size of corresponding high frequency coefficient on same scale equidirectional, selects variance less coefficient conduct
The high frequency coefficient of Contourlet inverse transformation；Carry out Contourlet inversion using the low frequency coefficient newly obtaining and high frequency coefficient
Change；Again bicubic difference and bilateral filtering are carried out again to the image obtaining, the higher new images of final acquisition resolution.
List of references
[1] Zhao Peng. the super-resolution rebuilding [D] based on anti-aliasing profile ripple. Qinhuangdao. University On The Mountain Of Swallows .2012.
[2]Fei Zhou,Wenming Yang,and Qingmin Liao.Interpolation-Based Image
Super-Resolution Using Multisurface Fitting[J].IEEE Trans.Image Process.,
2012,pp.3312-3318.
[3]Xinbo Gao,Kaibing Zhang,Dacheng Tao,Xuelong Li,Image Super-
Resolution With Sparse Neighbor Embedding.IEEE Transactions on Image
Processing,Vol.21(7),July 2012,pp.3194-3205.
[4]M Irani,S peleg.Super resolution from image sequences[C].In
proceedings of International Conference on Pattern Recognition.Piscataway,NJ,
USA.1990,pp.115-120.
[5]W.T.Freeman,T.R.Jones,and E.C.Pasztor.Example-Based Super-
Resolution [J] .IEEE.Computer Graphics and Applications, Vol.22, pp.56-65,2002.
[6]Kaibing Zhang,Xinbo Gao,Dacheng Tao,Xuelong Li,Single Image Super-
Resolution With Non-Local Means and Steering Kernel Regression,IEEE
Transactions on Image Processing,Vol.21(11),Nov.2012,pp.4544-4556.
[7]Shuyuan Yang,Min Wang,Yiguang Chen,and Yaxin Sun,Single-Image
Super-Resolution Reconstruction via Learned Geometric Dictionaries and
Clustered Sparse Coding,IEEE Transactions on Image Processing,Vol.21(9),2012,
pp.4016-4028.
[8]Marcelo Oliveira Camponez,Evandro Ottoni Teatini Salles,and Mário
Sarcinelli-Filho,Super-Resolution Image Reconstruction Using Nonparametric
Bayesian INLA Approximation,IEEE Transactions on Image Processing,Vol.21(8),
2012,pp.3491-3501.
[9]UCID,http://www-staff.lboro.ac.uk/～cogs/datasets/UCID/ucid.html.
Claims (2)
1. a kind of image super-resolution rebuilding method based on multi-scale geometric analysis is it is characterised in that specifically according to following step
Rapid enforcement：
Step 1, using degrade model generate low-resolution image
Continuous three frame normal resolution image I are chosen from observed image1、I2And I3, enter row degradation using following model and process,
Obtain corresponding low-resolution image T1、T2And T3：
Tk=D B M Ik+ n=H Ik+ n,
In formula, k=1,2,3, D, B, M, n be respectively down-sampling operator, obscure operator, motion blur operator, noise operator；
Step 2, image registration
For image T subject to registrationk, k=1,2,3, the maximum two field picture T of choosing wherein signal to noise ratio1As reference frame, utilize
SIFT algorithm extracts image characteristic point, if TkSet of characteristic points be respectively：
Wherein, mkIt is corresponding to TkFeature count out,
Set of computations P respectively1With P2、P1With P3In characteristic point between Euclidean distance：
Wherein, j1=1,2 ..., m1, j2=1,2 ..., m2, j3=1,2 ..., m3,
Find out P1With P2Minimum Eustachian distance d between middle characteristic point1With secondary little Euclidean distance d2, definition distance rates are r=d1/
d2, defining registering criterion is：
Wherein η is the threshold value being obtained by experiment, if more than threshold value, distance rates think that the match is successful, otherwise it is assumed that coupling is lost
Lose；To T3Do same process, obtain images after registration and be respectively：
T2'=M1·T2,
T3'=M2·T3,
Wherein, M1、M2It is motion compensation matrix；
Step 3, image co-registration
Contourlet transformation is carried out to the image after registration, i.e. Ck=C (Tk'), wherein, C () be profile ripple computing it is assumed that
Image T to be reconstructed after couplingk' the low frequency coefficient after contourlet transformation is designated asIt is shown below：
Wherein k=2,3, the number of low frequency coefficient is l=i*j, and the high frequency coefficient of note τ directional subband of s-th yardstick is
Total high frequency coefficient h=p*q,
For the low frequency coefficient of gained after contourlet transformation, processed using weighted mean method, obtained the low of inverse transformation
Frequency coefficient, image T to be reconstructed after kth frame couplingk' the low frequency coefficient after contourlet transformation isωkFor weight system
Number, and meetThen final choice as the low frequency coefficient of Contourlet inverse transformation is
HFS for gained after contourlet transformation is merged by following principle：
It is respectively compared the size of corresponding high frequency coefficient variance on corresponding each directional subband of each yardstick, take variance minimum
Coefficient as the high frequency coefficient of corresponding scale respective direction inverse transformation, that is, adoptsRepresent the τ directional subband of s-th yardstick
The meansigma methodss of high frequency coefficient, this average value expression is：
UsingRepresent the variance of the τ directional subband high frequency coefficient of s-th yardstick, this variance expression formula is：
It is the high frequency system of the τ directional subband Contourlet inverse transformation of s-th yardstick
Number, order：
Wherein Gs,τRepresent final choice as the high frequency coefficient of the τ directional subband of s-th yardstick of inverse transformation, G is all directions
Each scale coefficient matrix, wherein row represent the direction index of directional subband, and row represent yardstick index,
Carry out Contourlet inverse transformation with the high frequency coefficient G of low frequency coefficient L and each each direction of yardstick, i.e. I'=C-1(L,
G), C-1() is profile ripple transform operation, and I' is fusion image；
Step 4, acquisition reconstruction image
Bicubic interpolation computing is carried out to fusion image I' obtaining and obtains I ", to I " carry out bilateral filtering process, rebuild
Image II.
2. the image super-resolution rebuilding method based on multi-scale geometric analysis according to claim 1 it is characterised in that：
In described step 1, produce motion blur function and down-sampling operator using point spread function PSF, point spread function PSF has two
Individual parameter theta and len, represent shooting object counter clockwise direction with theta angular movement len pixel, by adjusting this
Two parameters are changing the fog-level of image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410234469.1A CN104008539B (en) | 2014-05-29 | 2014-05-29 | Image super-resolution rebuilding method based on multiscale geometric analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410234469.1A CN104008539B (en) | 2014-05-29 | 2014-05-29 | Image super-resolution rebuilding method based on multiscale geometric analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104008539A CN104008539A (en) | 2014-08-27 |
| CN104008539B true CN104008539B (en) | 2017-02-15 |
Family
ID=51369180
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410234469.1A Expired - Fee Related CN104008539B (en) | 2014-05-29 | 2014-05-29 | Image super-resolution rebuilding method based on multiscale geometric analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104008539B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104268829B (en) * | 2014-10-17 | 2017-09-29 | 中国科学院地理科学与资源研究所 | A kind of Super-resolution Mapping based on multiscale space regularization model |
| CN104680502B (en) * | 2015-03-19 | 2017-09-22 | 四川大学 | Infrared image super-resolution reconstruction method based on sparse dictionary and non-downsampling Contourlet conversion |
| CN104933678B (en) * | 2015-06-30 | 2018-04-10 | 西安理工大学 | A kind of image super-resolution rebuilding method based on image pixel intensities |
| CN105976316B (en) * | 2016-04-27 | 2018-12-07 | 武汉卓目科技有限公司 | Multispectral image method for reconstructing and system based on contact-type image sensor |
| CN106447717B (en) * | 2016-09-30 | 2019-05-03 | 中国科学院自动化研究所 | A kind of method for reconstructing of the light selective film illumination micro-imaging based on multi-angle |
| CN106940895B (en) * | 2017-01-12 | 2021-06-29 | 北京理工大学 | Estimation method of degradation function applied to wireless tomography system |
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN108537731B (en) * | 2017-12-29 | 2020-04-14 | 西安电子科技大学 | Image super-resolution reconstruction method based on compressed multi-scale feature fusion network |
| CN108391060B (en) * | 2018-03-26 | 2021-02-09 | 华为技术有限公司 | Image processing method, image processing device and terminal |
| CN109215122B (en) * | 2018-09-12 | 2022-09-16 | 井冈山大学 | Streetscape three-dimensional reconstruction system and method and intelligent trolley |
| CN109345497B (en) * | 2018-09-14 | 2021-12-03 | 淮阴师范学院 | Image fusion processing method and system based on fuzzy operator and computer program |
| CN109447931B (en) * | 2018-10-26 | 2022-03-15 | 深圳市商汤科技有限公司 | Image processing method and device |
| CN111344736A (en) * | 2018-12-18 | 2020-06-26 | 深圳市大疆创新科技有限公司 | Image processing method, image processing device and unmanned aerial vehicle |
| CN109886872B (en) * | 2019-01-10 | 2023-05-16 | 深圳市重投华讯太赫兹科技有限公司 | Security inspection equipment and image detection method thereof |
| CN113222814B (en) * | 2021-04-22 | 2023-08-04 | 深圳赛安特技术服务有限公司 | Resolution processing method, device, equipment and storage medium for image |
| CN114146890B (en) * | 2021-12-03 | 2022-09-13 | 深圳先进技术研究院 | Ultrasonic sound control method and sound tweezers device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103236041A (en) * | 2013-04-25 | 2013-08-07 | 北京交通大学 | Image super resolution reconstruction method on basis of Contourlet transformation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8755636B2 (en) * | 2011-09-14 | 2014-06-17 | Mediatek Inc. | Method and apparatus of high-resolution image reconstruction based on multi-frame low-resolution images |
-
2014
- 2014-05-29 CN CN201410234469.1A patent/CN104008539B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103236041A (en) * | 2013-04-25 | 2013-08-07 | 北京交通大学 | Image super resolution reconstruction method on basis of Contourlet transformation |
Non-Patent Citations (3)
| Title |
|---|
| Interpolation-Based Image Super-Resolution Using Multisurface Fitting;Fei Zhou et al.;《IEEE Transactions on Image Processing》;20120731;第21卷(第7期);第3312-3318页 * |
| Single-Image Super-Resolution Reconstruction via Learned Geometric Dictionaries and Clustered Sparse Coding;Shuyuan Yang et al.;《IEEE Transactions on Image Processing》;20120930;第21卷(第9期);第4016-4028页 * |
| 基于Contourlet的改进双线性插值图像超分辨率算法;蔡念 等;《传感技术学报》;20110131;第24卷(第1期);第59-64页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104008539A (en) | 2014-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104008539B (en) | Image super-resolution rebuilding method based on multiscale geometric analysis | |
| Suryanarayana et al. | Accurate magnetic resonance image super-resolution using deep networks and Gaussian filtering in the stationary wavelet domain | |
| CN110119780B (en) | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network | |
| CN110570353A (en) | Dense connection generation countermeasure network single image super-resolution reconstruction method | |
| CN106952228A (en) | The super resolution ratio reconstruction method of single image based on the non local self-similarity of image | |
| CN104408700A (en) | Morphology and PCA (principal component analysis) based contourlet fusion method for infrared and visible light images | |
| CN102629374B (en) | Image super resolution (SR) reconstruction method based on subspace projection and neighborhood embedding | |
| CN106952317B (en) | Hyperspectral image reconstruction method based on structure sparsity | |
| CN106485671A (en) | Multi-direction Weighted T V based on edge and self-similarity constraint image deblurring method | |
| Min et al. | Blind deblurring via a novel recursive deep CNN improved by wavelet transform | |
| CN116029902A (en) | Knowledge distillation-based unsupervised real world image super-resolution method | |
| Chen et al. | Image denoising via deep network based on edge enhancement | |
| Shao et al. | Uncertainty-guided hierarchical frequency domain transformer for image restoration | |
| CN104200439B (en) | Image super-resolution method based on adaptive filtering and regularization constraint | |
| CN115578262A (en) | Polarization image super-resolution reconstruction method based on AFAN model | |
| Li et al. | Image reflection removal using end‐to‐end convolutional neural network | |
| Jia et al. | Dual-complementary convolution network for remote-sensing image denoising | |
| Suryanarayana et al. | Single image super-resolution algorithm possessing edge and contrast preservation | |
| Qin et al. | Remote sensing image super-resolution using multi-scale convolutional neural network | |
| Zhang et al. | Multisensor Infrared and Visible Image Fusion via Double Joint Edge Preservation Filter and Nonglobally Saliency Gradient Operator | |
| Gao et al. | Algorithms of image fusion based on wavelet transform | |
| Xu et al. | FEMRNet: Feature-enhanced multi-scale residual network for image denoising | |
| Ma et al. | Noisy remote sensing image fusion based on JSR | |
| Sun et al. | A novel multi-focus image fusion algorithm based on NSST-FRFT | |
| Li et al. | A novel multi-frame color images super-resolution framework based on deep convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190123 Address after: Room 6113, 6th floor, 999 Changning Road, Changning District, Shanghai 200050 Patentee after: DEEPBLUE TECHNOLOGY (SHANGHAI) Co.,Ltd. Address before: 710048 No. 5 Jinhua South Road, Shaanxi, Xi'an Patentee before: Xi'an University of Technology |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220106 Address after: 315000 No. 138-1, Zhongshan West Road, Fenghua District, Ningbo City, Zhejiang Province (self declaration) Patentee after: Shenlan industrial intelligent Innovation Research Institute (Ningbo) Co.,Ltd. Address before: 200050 room 6113, 6th floor, 999 Changning Road, Changning District, Shanghai Patentee before: DEEPBLUE TECHNOLOGY (SHANGHAI) Co.,Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170215 |