CN104008539A - Image super-resolution rebuilding method based on multiscale geometric analysis - Google Patents

Abstract

The invention discloses an image super-resolution rebuilding method based on multiscale geometric analysis. The method includes the steps that (1) a low-resolution image is generated by means of a blurring model, (2) image registration is carried out, (3) image fusion is carried out, and (4) a rebuilt image is obtained. The method has good invariance for horizontal moving, rotation, size scaling, noise and the like, stability of affine transformation, view change and the like remains to a certain extent, and accordingly edge information keeps clear and relatively smooth; an algorithm is stable and has low calculation complexity and the method has wide application prospects in the fields of space target imaging, remote sensing imaging, medical imaging, tomographic imaging and the like.

Description

Based on the image super-resolution rebuilding method of multi-scale geometric analysis

Technical field

The invention belongs to image super-resolution rebuilding technical field, relate to a kind of image super-resolution rebuilding method based on multi-scale geometric analysis.

Background technology

In digital image acquisition process, due to the restriction of imaging device performance and the impact of shooting condition, can make the image collecting present lower resolution.Image super-resolution rebuilding (Super-Resolution Image Reconstruction, SR) be a kind of to obtain high resolving power (HP) image or image sequence from single frames or multiframe low resolution (LP) image, thereby improve picture quality, improve the signal processing method of image visual effect.

Image super-resolution rebuilding technology, for in image imaging process such as motion deformation, optical dimming, low sampling rate, the image deterioration problem that the various degeneration factors such as random noise cause is studied, in extraterrestrial target imaging, remotely sensed image, medical imaging, the various fields such as tomography are with a wide range of applications, for various microwave imagings, multispectral imaging, the HD image of ultrasonic imaging etc. recovers also will have reference role, in addition, at medical domain, computer generated image, the HD image of the recording monitor data of the important events such as financial sector is recovered, the aspect such as identification and evidence obtaining of special objective also has important application.

From mathematical angle, image super-resolution rebuilding is that the non-suitable fixed number under Hardmard meaning is learned indirect problem, therefore becomes the multidisciplinary fields such as applied mathematics, image processing, computer vision, the calculating harmonic analysis hot issue that numerous researchers pay close attention in the world.

Image super-resolution rebuilding method mainly can be divided into following a few class: frequently (rate) territory method, sky () territory method, method based on study.The basic thought of frequency domain method is: low-resolution image is transformed to frequency domain with image conversion, adopt suitable method to remove spectral aliasing, and then change to spatial domain by inversion in frequency, obtain the result of super-resolution rebuilding.Frequency domain method is theoretical simple, and computational complexity is low, is easy to realize parallel processing.But the shortcoming of these class methods be based on theoretical premise too idealized, can only be confined to global translation motion and the constant model that degrades of linear space, can not be effectively applied to most occasions.A kind of super resolution ratio reconstruction method that utilized anti-aliasing profile wave structure in the current research result [1] of these class methods.The method is carried out change of scale by the tower bank of filters of Laplce that anti-aliasing tower bank of filters substitutes in profile wave convert to image, reduces spectral aliasing; According to the similarity between the high-frequency sub-band of different scale, high fdrequency component is carried out to subband interpolation, and decompose by anisotropic filter group travel direction, high-frequency information is carried out to selectivity and merge processing and the inverse transformation of profile ripple, rebuild image thereby obtain.

Spatial domain method has the stronger prior-constrained ability in spatial domain that comprises, and main method comprises:

Nonuniform sample interpolation method: the theoretical foundation of this method is, low-resolution image is heterogeneous while being mapped on high-definition picture grid after motion estimation algorithm, therefore by just can obtain the pixel value of the sampled point on high-definition picture grid to the interpolation of nonuniform sampling point.The Typical Representative multiaspect matching that utilizes proposing as middle in [2] realizes the image super-resolution rebuilding method based on interpolation, the method, by the pixel of low-resolution image being carried out to multiple sampling realization to the making full use of of space structure information, also has the image super-resolution rebuilding method based on sparse neighborhood interpolation in [3].

Inverse iteration sciagraphy: propose in [4] the earliest, its principle is to utilize the high-definition picture analog image process that degrades of estimating, generates the low-resolution image of estimating; Then the low-resolution image of calculating and observation is poor; Residual projection to estimate high-definition picture in.Repeat this process, until the high-definition picture of estimating meets stopping criterion for iteration.

Convex set projection method: be a kind of image super-resolution rebuilding method based on set theory, the method has dirigibility and extensibility, can preserve preferably the detailed information such as the edge of image, but computation complexity is high, be subject to the impact of initial estimate larger, and, do not have effective method to select best initial estimate, in addition, the solution of this method is not unique, is difficult to meet the requirement of particular problem in a lot of application.

Statistical method-maximum a posteriori probability is estimated and maximal possibility estimation: super-resolution rebuilding problem is an ill-conditioning problem, makes ill-conditioning problem be converted into solvable problem, need to introduce in advance priori subsidiary condition.The thought of maximum a posteriori probability is exactly under the condition of known sequence of low resolution pictures, makes the posterior probability maximum of high-definition picture.The all right combining image deblurring of these class methods and denoising etc., make reconstructed results have uniqueness and stability, and its defect is that calculated amount is huge.

Super resolution ratio reconstruction method based on study: these class methods are proposed in document [5] by Freeman first, and its basic thought is the relation of first learning between low-resolution image and high-definition picture, then utilize this relation to instruct super-resolution rebuilding.In current research result [6], proposed a kind of single-frame images super resolution ratio reconstruction method based on non-local mean and adjustment core recurrence (Steering Kernel Regression), the method realizes super-resolution reconstruction by the non local and local rule priori in study low-resolution image.Document [7] has proposed a kind of cluster sparse coding scheme based on multiple how much dictionaries, for the super-resolution rebuilding of image.The method is supported to concentrate a large amount of high-definition pictures of random extraction from training image, and is clustered into geometric areas, from how much dictionaries corresponding to learning, each localized mass in sparse coding low-resolution image.Document [8] uses the nonparametric Bayes inference method based on numerical integration, is called as " integrated nested Laplce is approximate " in statistical literature, structure super resolution ratio reconstruction method.

The final goal of image super-resolution rebuilding algorithm is to improve as far as possible the quality of image.Consider that frequency domain method is theoretical simple, computational complexity is low, is easy to realize and be easy to parallel processing, is therefore a kind of very practical method for reconstructing, but shortcoming is to be confined to global translation motion and the constant model that degrades of linear space, can not be effectively applied to most occasions.In order to make up this deficiency of frequency domain method, we consider with multi-scale geometric analysis instrument-profile wave convert (Contourlet), multiple dimensioned, multi-direction, the multiresolution that utilizes that profile wave convert has and anisotropic feature, effectively catch the singular point in image, and by profile wave system number is carried out to suitable choosing, to make full use of space scale information and the directional information of image, reconstructed high frequency information, the quality of raising image.

Summary of the invention

The object of this invention is to provide a kind of image super-resolution rebuilding method based on multi-scale geometric analysis, solved from multiframe low-resolution image characteristic information extraction and come the radio-frequency component of reconstructing lost, the problem that causes image resolution ratio to reduce.

The technical solution adopted in the present invention, a kind of image super-resolution rebuilding method based on multi-scale geometric analysis, specifically implement according to following steps:

Step 1, utilize the model generation low-resolution image that degrades

From observed image, choose the common image in different resolution I of continuous three frame ₁, I ₂and I ₃, utilize the processing of degenerating of following model, obtain corresponding low-resolution image T ₁, T ₂and T ₃:

T _k＝D·B·M·I _k+n＝H·I _k+n，

In formula, k=1,2,3, D, B, M, n are respectively down-sampling operator, obscure operator, motion blur operator, noise operator;

Step 2, image registration

For image T subject to registration _k, k=1,2,3, select the wherein larger two field picture T of signal to noise ratio (S/N ratio) ₁as with reference to frame, utilize SIFT algorithm to extract image characteristic point, establish T _kunique point set be respectively:

$P_1=\{p_{\mathrm{1,1}},p_{\mathrm{1,2}},...,p_{{1,m}_1}\},$

$P_2=\{p_{\mathrm{2,1}},p_{\mathrm{2,2}},...,p_{{2,m}_2}\},$

$P_3=\{p_{\mathrm{3,1}},p_{\mathrm{3,2}},...,p_{{3,m}_3}\},$

Wherein, m _kfor corresponding to T _kunique point number, k=1,2,3, j=1,2 ..., max (m ₁, m ₂, m ₃),

Set of computations P respectively ₁with P ₂, P ₁with P ₃in unique point between Euclidean distance:

$d(p_{{1,j}_1},p_{{2,j}_2})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{2,j}_2}^i)}^2},$ j ₁＝1,2,...,m ₁，j ₂＝1,2,...,m ₂

$d(p_{{1,j}_1},p_{{3,j}_3})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{3,j}_3}^i)}^2},$

Wherein, j ₁=1,2 ..., m ₁, j ₂=1,2 ..., m ₂, j ₃=1,2 ..., m ₃,

Find out P ₁with P ₂minimum Eustachian distance d between middle unique point ₁with inferior little Euclidean distance d ₂, definition distance rates is r=d ₁/ d ₂, definition registration criterion is:

Wherein η serves as reasons and tests the threshold value obtaining, and thinks that the match is successful, otherwise think that it fails to match if distance rates is greater than threshold value; To T ₃do same processing, obtain images after registration and be respectively:

T ₂′＝M ₁·T ₂，

T ₃′＝M ₂·T ₃，

Wherein, M ₁, M ₂be motion compensation matrix;

Step 3, image co-registration

Image after registration is carried out to Contourlet conversion, i.e. C _k=C (T _k'), wherein, C () is the computing of profile ripple, supposes the rear image T to be reconstructed of coupling _k' be designated as through the low frequency coefficient after Contourlet conversion be shown below:

$X_k^g=\left[\begin{array}{cccc}{x}_{k,\mathrm{1,1}}^g& x_{k,\mathrm{1,2}}^g& ...& x_{k,1,j}^g\\ x_{k,\mathrm{2,1}}^g& x_{k,\mathrm{2,2}}^g& ...& x_{k,2,j}^g\\ ...& ...& ...& ...\\ x_{k,i,1}^g& x_{k,i,2}^g& ...& x_{k,i,j}^g\end{array}\right],$

Wherein k=2,3, the number of low frequency coefficient is l=i*j, the high frequency coefficient of τ directional subband of s yardstick of note is total high frequency coefficient h=p*q,

$Y_k^{s,\mathrm{\τ}}=\left[\begin{array}{cccc}{y}_{k,\mathrm{1,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{1,2}}^{s,\mathrm{\τ}}& ...& y_{k,1,q}^{s,\mathrm{\τ}}\\ y_{k,\mathrm{2,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{2,2}}^{s,\mathrm{\τ}}& ...& y_{k,2,q}^{s,\mathrm{\τ}}\\ ...& ...& ...& ...\\ y_{k,p,1}^{s,\mathrm{\τ}}& y_{k,p,2}^{s,\mathrm{\τ}}& ...& y_{k,p,q}^{s,\mathrm{\τ}}\end{array}\right],s=\mathrm{1,2},...,g$

For the low frequency coefficient of gained after Contourlet conversion, adopt method of weighted mean to process, obtain the low frequency coefficient of inverse transformation, image T to be reconstructed after k frame coupling _k' through the low frequency coefficient after Contourlet conversion be ω _kfor weight coefficient, and meet final selection as the low frequency coefficient of Contourlet inverse transformation is

HFS for gained after Contourlet conversion merges by following principle:

Institute's corresponding high frequency coefficient side extent on the each directional subband of more corresponding each yardstick, gets the coefficient of variance minimum as the high frequency coefficient of corresponding scale respective direction inverse transformation respectively, adopts represent the mean value of τ directional subband high frequency coefficient of s yardstick, this mean value expression formula is:

$\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{Y}_{k,i,j}^{s,\mathrm{\τ}},$

Adopt represent the variance of τ directional subband high frequency coefficient of s yardstick, this variance expression formula is: $V_k^{s,\mathrm{\τ}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{(Y_{k,i,j}^{s,\mathrm{\τ}}-\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}})}^2,$

be the high frequency coefficient of τ directional subband Contourlet of s yardstick inverse transformation, order:

$G=\left[\begin{array}{ccccccc}{G}^{\mathrm{1,1}}& G^{2^1,1}& 0& 0& 0& ...& 0\\ G^{\mathrm{1,2}}& G^{\mathrm{2,2}}& G^{\mathrm{3,2}}& G^{2^2,2}& 0& & 0\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ G^{1,g}& G^{2,g}& G^{3,g}& G^{4,g}& G^{5,g}& & G^{2^g,g}\end{array}\right],$

Wherein G ^{s, τ}represent the final high frequency coefficient of selecting as τ the directional subband of s yardstick of inverse transformation, G is the each scale coefficient matrixes of all directions, the wherein direction index of line display directional subband, and yardstick index is shown in list,

Carry out Contourlet inverse transformation, i.e. I'=C with the high frequency coefficient G of low frequency coefficient L and the each direction of each yardstick ^-1(L, G), C ^-1() is profile ripple inverse transformation computing, and I' is fused images;

Image is rebuild in step 4, acquisition

The fused images I' obtaining is carried out to the computing of bicubic difference and obtain I'', I'' is carried out to bilateral filtering processing, obtain rebuilding image I I.

The present invention has following beneficial effect:

1) the image registration step of the inventive method, use the method for registering images based on Sift, Sift feature belongs to image local feature, oneself is proved has good unchangeability to translation, rotation, yardstick convergent-divergent and noise etc., and affined transformation, visual angle change etc. are also kept to certain stability.Utilize multiple dimensioned, multi-direction, multiresolution and the anisotropic feature of multi-scale geometric analysis instrument-profile wave convert (Contourlet), effectively catch the singular point in image, and by profile wave system number is carried out to suitable choosing, make full use of space scale information and the directional information of image, recover the radio-frequency component of losing, improve the quality of image.The inventive method has taken into full account the marginal information in image, makes marginal information keep clear, relatively level and smooth.

2) method of the present invention is applicable to gray level image and coloured image.Experimental result shows, method of the present invention has reached obvious reconstruction effect, and algorithm has stability, and has lower computation complexity.

3) method of the present invention is with a wide range of applications in various fields such as extraterrestrial target imaging, remotely sensed image, medical imaging, tomographies; HD image for various microwave imagings, multispectral imaging, ultrasonic imaging etc. recovers also will have reference role; In addition recover in the HD image of the recording monitor data of the important events such as medical domain, computer generated image, financial sector,, the aspect such as identification and evidence obtaining of special objective also has important application.

Brief description of the drawings

Fig. 1 is the FB(flow block) of the inventive method;

Fig. 2 is the original Lena image of the inventive method embodiment;

Fig. 3 is the blurred picture one of Fig. 2;

Fig. 4 is the blurred picture two of Fig. 2;

The original color image of Tu5Shi embodiment bookstore.

Fig. 6 is the blurred picture three of Fig. 5;

Fig. 7 is the blurred picture four of Fig. 5;

Fig. 8 is the gray level image registration effect of taking Fig. 2 as reference frame, Fig. 3 being carried out registration;

Fig. 9 is the coloured image registration effect of taking Fig. 6 as reference frame, Fig. 7 being carried out registration.

Figure 10 is the original-gray image of embodiment cartoon;

Figure 11 is the left half of degraded image of Figure 10;

Figure 12 is the right half of degraded image of Figure 10;

Figure 13 is the reconstructed results to Figure 11 and Figure 12 by method of the present invention;

Figure 14 is the original color image of embodiment river;

Figure 15 is the left half of degraded image of Figure 14;

Figure 16 is the right half of degraded image of Figure 14;

Figure 17 adopts the reconstructed results of the inventive method to Figure 15 and Figure 16;

Figure 18 is the MSE comparative statistics figure of the minimum direction of the inventive method embodiment;

Figure 19 be the inventive method embodiment third side to MSE comparative statistics figure;

Figure 20 is the MSE comparative statistics figure of the maximum direction of the inventive method embodiment.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.

Referring to Fig. 1, the present invention is based on the image super-resolution rebuilding method of multi-scale geometric analysis, comprise that generating low-resolution image (degraded image), image registration, image co-registration and acquisition rebuilds image step, specifically implements according to following steps:

Step 1, utilize the model generation low-resolution image (degraded image) that degrades

From observed image, choose the common image in different resolution I of continuous three frame ₁, I ₂and I ₃, utilize the processing of degenerating of following model, obtain corresponding low-resolution image T ₁, T ₂and T ₃:

T _k＝D·B·M·I _k+n＝H·I _k+n，

In formula, k=1,2,3, D, B, M, n are respectively down-sampling operator, obscure operator, motion blur operator, noise operator.

In this process, utilize point spread function PSF (Point Spread Function) to produce motion blur function and down-sampling operator, this function has two parametric t heta and len, represent shooting object counterclockwise with theta angular movement len pixel, change the fog-level of image by adjusting these two parameters.

Step 2, image registration

Consider that the matching algorithm based on SIFT unique point has reasonable registration effect, this step adopts this kind of method to carry out image registration.

For image T subject to registration _k, k=1,2,3, select the wherein larger two field picture T of signal to noise ratio (S/N ratio) ₁(without loss of generality, be made as T ₁) as with reference to frame, utilize SIFT algorithm to extract image characteristic point, establish T _kunique point set be respectively:

$P_1=\{p_{\mathrm{1,1}},p_{\mathrm{1,2}},...,p_{{1,m}_1}\},$

$P_2=\{p_{\mathrm{2,1}},p_{\mathrm{2,2}},...,p_{{2,m}_2}\},$

$P_3=\{p_{\mathrm{3,1}},p_{\mathrm{3,2}},...,p_{{3,m}_3}\},$

Wherein, m _kfor corresponding to T _kunique point number, k=1,2,3, j=1,2 ..., max (m ₁, m ₂, m ₃),

Set of computations P respectively ₁with P ₂, P ₁with P ₃in unique point between Euclidean distance:

$d(p_{{1,j}_1},p_{{2,j}_2})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{2,j}_2}^i)}^2},$ j ₁＝1,2,...,m ₁，j ₂＝1,2,...,m ₂

$d(p_{{1,j}_1},p_{{3,j}_3})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{3,j}_3}^i)}^2},$

Wherein, j ₁=1,2 ..., m ₁, j ₂=1,2 ..., m ₂, j ₃=1,2 ..., m ₃,

Find out P ₁with P ₂minimum Eustachian distance d between middle unique point ₁with inferior little Euclidean distance d ₂, definition distance rates is r=d ₁/ d ₂, definition registration criterion (matching principle) is:

Wherein η serves as reasons and tests the threshold value obtaining, and thinks that the match is successful, otherwise think that it fails to match if distance rates is greater than threshold value; To T ₃do same processing, obtain images after registration and be respectively:

T ₂′＝M ₁·T ₂，

T ₃′＝M ₂·T ₃，

Wherein, M ₁, M ₂be motion compensation matrix.

Step 3, image co-registration

Image after registration is carried out to Contourlet conversion, i.e. C _k=C (T _k'), wherein, C () is the computing of profile ripple, supposes the rear image T to be reconstructed of coupling _k' be designated as through the low frequency coefficient after Contourlet conversion be shown below:

$X_k^g=\left[\begin{array}{cccc}{x}_{k,\mathrm{1,1}}^g& x_{k,\mathrm{1,2}}^g& ...& x_{k,1,j}^g\\ x_{k,\mathrm{2,1}}^g& x_{k,\mathrm{2,2}}^g& ...& x_{k,2,j}^g\\ ...& ...& ...& ...\\ x_{k,i,1}^g& x_{k,i,2}^g& ...& x_{k,i,j}^g\end{array}\right],$

Wherein k=2,3, the number of low frequency coefficient is l=i*j, the high frequency coefficient of τ directional subband of s yardstick of note is total high frequency coefficient h=p*q,

$Y_k^{s,\mathrm{\τ}}=\left[\begin{array}{cccc}{y}_{k,\mathrm{1,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{1,2}}^{s,\mathrm{\τ}}& ...& y_{k,1,q}^{s,\mathrm{\τ}}\\ y_{k,\mathrm{2,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{2,2}}^{s,\mathrm{\τ}}& ...& y_{k,2,q}^{s,\mathrm{\τ}}\\ ...& ...& ...& ...\\ y_{k,p,1}^{s,\mathrm{\τ}}& y_{k,p,2}^{s,\mathrm{\τ}}& ...& y_{k,p,q}^{s,\mathrm{\τ}}\end{array}\right],s=\mathrm{1,2},...,g$

For the low frequency coefficient of gained after Contourlet conversion, adopt method of weighted mean to process, obtain the low frequency coefficient of inverse transformation, image T to be reconstructed after k frame coupling _k' through the low frequency coefficient after Contourlet conversion be ω _kfor weight coefficient, and meet final selection as the low frequency coefficient of Contourlet inverse transformation is

HFS for gained after Contourlet conversion merges by following principle:

Institute's corresponding high frequency coefficient side extent on the each directional subband of more corresponding each yardstick, gets the coefficient of variance minimum as the high frequency coefficient of corresponding scale respective direction inverse transformation respectively, adopts represent the mean value of τ directional subband high frequency coefficient of s yardstick, this mean value expression formula is:

$\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{Y}_{k,i,j}^{s,\mathrm{\τ}},$

Adopt represent the variance of τ directional subband high frequency coefficient of s yardstick, this variance expression formula is: $V_k^{s,\mathrm{\τ}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{(Y_{k,i,j}^{s,\mathrm{\τ}}-\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}})}^2,$

be the high frequency coefficient of τ directional subband Contourlet of s yardstick inverse transformation, order:

$G=\left[\begin{array}{ccccccc}{G}^{\mathrm{1,1}}& G^{2^1,1}& 0& 0& 0& ...& 0\\ G^{\mathrm{1,2}}& G^{\mathrm{2,2}}& G^{\mathrm{3,2}}& G^{2^2,2}& 0& & 0\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ G^{1,g}& G^{2,g}& G^{3,g}& G^{4,g}& G^{5,g}& & G^{2^g,g}\end{array}\right],$

Wherein G ^{s, τ}represent the final high frequency coefficient of selecting as τ the directional subband of s yardstick of inverse transformation, G is the each scale coefficient matrixes of all directions, the wherein direction index of line display directional subband, and yardstick index is shown in list,

Carry out Contourlet inverse transformation, i.e. I'=C with the high frequency coefficient G of low frequency coefficient L and the each direction of each yardstick ^-1(L, G), C ^-1() is profile ripple inverse transformation computing, and I' is fused images.

Image is rebuild in step 4, acquisition

The fused images I' obtaining is carried out to the computing of bicubic difference and obtain I'', I'' is carried out to bilateral filtering processing, obtain rebuilding image I I.

The emulation experiment of the inventive method

The object of emulation experiment is to analyze the validity of the inventive method by visual effect and numerical result.Emulation experiment completes under MATLAB2010a environment, and the computer processor of use is Pentium (R) Dual-Core CPU, E5400 2.70GHz, and 2.70GHz, inside saves as 2.00GB.

The image using in emulation experiment is from http://image.baidu.com and UCID image library [9].These images have comprised the various natural images such as plant, animal, personage, landscape, have different structure feature, different light environment, different brightness, different texture feature.

1) visual effect is shown

The object that designs this experiment is the visual effect in order to show the method, and the test pattern in experiment is all from the degraded image collection generating with said method.

1.1) visual effect of image registration:

Test according to the algorithm of describing in step 2, Fig. 2 is original Lena image (gray level image), Fig. 3 is the low-resolution image being generated through degeneration Fuzzy Processing by Fig. 2, and Fig. 4 is the low-resolution image being generated through degeneration Fuzzy Processing with larger degradation factor by Fig. 2.Fig. 8 represents taking Fig. 2 as reference frame, Fig. 3 to be carried out the gray level image registration effect of registration.Fig. 5 is the original color image of bookstore, Fig. 6 is the low-resolution image being generated through degeneration Fuzzy Processing by Fig. 5, Fig. 7 is the low-resolution image being generated through degeneration Fuzzy Processing by Fig. 5, and Fig. 9 represents taking Fig. 6 as reference frame, Fig. 7 to be carried out the coloured image registration effect of registration.

1.2) visual effect of image reconstruction:

In order to show reconstruction effect, this experimental design degenerate respectively left-half and the right half part of same two field picture, then test according to the method for describing in step 2, step 3 and step 4.Figure 10 is the former figure of original gray scale of cartoon, and Figure 11 is to left one side of something of Figure 10 degraded image after treatment of degenerating, and Figure 12 is that Figure 13 is gray level image reconstructed results to right one side of something of Figure 10 degraded image after treatment of degenerating.Figure 14 is the original color image of river, and Figure 15 is to left one side of something of Figure 14 degraded image after treatment of degenerating, and Figure 16 is to right one side of something of Figure 14 degraded image after treatment of degenerating, the reconstructed results that Figure 17 is coloured image.

2) Numerical Value Result Analysis

2.1) peak noise is analyzed than (PSNR) and signal noise ratio (snr) of image (ISNR)

The comparative result of table 1, PSNR and ISNR

The object of this experiment is the quality that checking method of the present invention can improve image, for this object, utilizes and analyzes about objective evaluation standard P SNR and the ISNR of picture quality.Under the random prerequisite of selecting Degenerate Graphs image set, select 500 width images of image library, for different gray level images and coloured image, compare respectively PSNR and the ISNR of absolute-value scheme [1] and variance method (the inventive method), the value of variance method is all better than absolute-value scheme, and detailed results is as shown in table 1.

2.2) square error (Mean Squared Error, MSE) is analyzed

This experiment illustrates the stability of method of the present invention by the objective evaluation standard MSE of computed image quality.The random Degenerate Graphs image set of selecting, table 2 is that it is carried out to the MSE that uses respectively variance method [1] and method of the present invention to calculate after the Contourlet conversion of different progression.

Table 2, MSE comparison (unit: 1 × 10 ⁹db)

Consider Contourlet decompose each grade of decomposition direction can select, we to each level select respectively minimum direction, third side to its MSE of maximum direction calculating, result shows, along with decomposed class increases, the MSE value of the inventive method shows obvious stability, and Figure 18, Figure 19 and Figure 20 have provided respectively the comparative statistics figure of three directions.

Table 3, comparison computing time (unit: s)

2.3) efficiency analysis

This experiment statistics the computing time that completes different decomposition progression for two kinds of methods of same two field picture, result is as shown in table 3.

Experimental result shows that method of the present invention has higher counting yield.

The inventive method is utilized multiple dimensioned, multi-direction, multiresolution and the anisotropic feature of multi-scale geometric analysis instrument-profile wave convert (Contourlet), effectively catch the singular point in image, and by profile wave system number is carried out to suitable choosing, make full use of space scale information and the directional information of image, recover the radio-frequency component of losing, improve the quality of image.First for reference picture, utilize image deterioration model, obscure and the processing of degenerating by a diffusion, motion blur, Gaussian noise etc., obtain multiframe low-resolution image; Adopt SIFT algorithm to carry out image registration, multiframe low-resolution image is mapped to the same grid of reference; Image after registration is carried out to Contourlet conversion; For the low frequency part obtaining after Contourlet conversion, adopt method of weighted mean to obtain the low frequency coefficient of its inverse transformation, for the HFS obtaining after Contourlet conversion, the relatively variance size of corresponding high frequency coefficient on same scale equidirectional, selects coefficient that variance the is less high frequency coefficient as Contourlet inverse transformation; Utilize the low frequency coefficient and the high frequency coefficient that newly obtain to carry out Contourlet inverse transformation; Again the image obtaining is carried out to bicubic difference and bilateral filtering again, finally obtain the new images that resolution is higher.

List of references

[1] Zhao Peng. the super-resolution rebuilding [D] based on anti-aliasing profile ripple. Qinhuangdao. the .2012. of University On The Mountain Of Swallows

[2]Fei?Zhou,Wenming?Yang,and?Qingmin?Liao.Interpolation-Based?Image?Super-Resolution?Using?Multisurface?Fitting[J].IEEE?Trans.Image?Process.,2012,pp.3312-3318.

[3]Xinbo?Gao,Kaibing?Zhang,Dacheng?Tao,Xuelong?Li,Image?Super-Resolution?With?Sparse?Neighbor?Embedding.IEEE?Transactions?on?Image?Processing,Vol.21(7),July?2012,pp.3194-3205.

[4]M?Irani,S?peleg.Super?resolution?from?image?sequences[C].In?proceedings?of?International?Conference?on?Pattern?Recognition.Piscataway,NJ,USA.1990,pp.115-120.

[5]W.T.Freeman,T.R.Jones,and?E.C.Pasztor.Example-Based?Super-Resolution[J].IEEE.Computer?Graphics?and?Applications,Vol.22，pp.56-65,2002.

[6]Kaibing?Zhang,Xinbo?Gao,Dacheng?Tao,Xuelong?Li,Single?Image?Super-Resolution?With?Non-Local?Means?and?Steering?Kernel?Regression,IEEE?Transactions?on?Image?Processing,Vol.21(11),Nov.2012,pp.4544-4556.

[7]Shuyuan?Yang,Min?Wang,Yiguang?Chen,and?Yaxin?Sun,Single-Image?Super-Resolution?Reconstruction?via?Learned?Geometric?Dictionaries?and?Clustered?Sparse?Coding,IEEE?Transactions?on?Image?Processing,Vol.21(9),2012,pp.4016-4028.

[8]Marcelo?Oliveira?Camponez,Evandro?Ottoni?Teatini?Salles,and?Mário?Sarcinelli-Filho,Super-Resolution?Image?Reconstruction?Using?Nonparametric?Bayesian?INLA?Approximation,IEEE?Transactions?on?Image?Processing,Vol.21(8),2012,pp.3491-3501.

[9]UCID, http://www-staff.lboro.ac.uk/～cogs/datasets/UCID/ucid.html。

Claims (2)

1. the image super-resolution rebuilding method based on multi-scale geometric analysis, is characterized in that, specifically implements according to following steps:

Step 1, utilize the model generation low-resolution image that degrades

From observed image, choose the common image in different resolution I of continuous three frame ₁, I ₂and I ₃, utilize the processing of degenerating of following model, obtain corresponding low-resolution image T ₁, T ₂and T ₃:

T _k＝D·B·M·I _k+n＝H·I _k+n，

In formula, k=1,2,3, D, B, M, n are respectively down-sampling operator, obscure operator, motion blur operator, noise operator;

Step 2, image registration

For image T subject to registration _k, k=1,2,3, select the wherein larger two field picture T of signal to noise ratio (S/N ratio) ₁as with reference to frame, utilize SIFT algorithm to extract image characteristic point, establish T _kunique point set be respectively:

$P_1=\{p_{\mathrm{1,1}},p_{\mathrm{1,2}},...,p_{{1,m}_1}\},$

$P_2=\{p_{\mathrm{2,1}},p_{\mathrm{2,2}},...,p_{{2,m}_2}\},$

$P_3=\{p_{\mathrm{3,1}},p_{\mathrm{3,2}},...,p_{{3,m}_3}\},$

Wherein, m _kfor corresponding to T _kunique point number, k=1,2,3, j=1,2 ..., max (m ₁, m ₂, m ₃),

Set of computations P respectively ₁with P ₂, P ₁with P ₃in unique point between Euclidean distance:

$d(p_{{1,j}_1},p_{{2,j}_2})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{2,j}_2}^i)}^2},$ j ₁＝1,2,...,m ₁，j ₂＝1,2,...,m ₂

$d(p_{{1,j}_1},p_{{3,j}_3})=\sqrt{\underset{i=1}{\overset{128}{\mathrm{\Σ}}}{(p_{{1,j}_1}^i-p_{{3,j}_3}^i)}^2},$

Wherein, j ₁=1,2 ..., m ₁, j ₂=1,2 ..., m ₂, j ₃=1,2 ..., m ₃,

Find out P ₁with P ₂minimum Eustachian distance d between middle unique point ₁with inferior little Euclidean distance d ₂, definition distance rates is r=d1/d2, definition registration criterion is:

Wherein η serves as reasons and tests the threshold value obtaining, and thinks that the match is successful, otherwise think that it fails to match if distance rates is greater than threshold value; To T ₃do same processing, obtain images after registration and be respectively:

T ₂′＝M ₁·T ₂，

T ₃′＝M ₂·T ₃，

Wherein, M ₁, M ₂be motion compensation matrix;

Step 3, image co-registration

Image after registration is carried out to Contourlet conversion, i.e. C _k=C (T _k'), wherein, C () is the computing of profile ripple, supposes the rear image T to be reconstructed of coupling _k' be designated as through the low frequency coefficient after Contourlet conversion be shown below:

$X_k^g=\left[\begin{array}{cccc}{x}_{k,\mathrm{1,1}}^g& x_{k,\mathrm{1,2}}^g& ...& x_{k,1,j}^g\\ x_{k,\mathrm{2,1}}^g& x_{k,\mathrm{2,2}}^g& ...& x_{k,2,j}^g\\ ...& ...& ...& ...\\ x_{k,i,1}^g& x_{k,i,2}^g& ...& x_{k,i,j}^g\end{array}\right],$

Wherein k=2,3, the number of low frequency coefficient is l=i*j, the high frequency coefficient of τ directional subband of s yardstick of note is total high frequency coefficient h=p*q,

$Y_k^{s,\mathrm{\τ}}=\left[\begin{array}{cccc}{y}_{k,\mathrm{1,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{1,2}}^{s,\mathrm{\τ}}& ...& y_{k,1,q}^{s,\mathrm{\τ}}\\ y_{k,\mathrm{2,1}}^{s,\mathrm{\τ}}& y_{k,\mathrm{2,2}}^{s,\mathrm{\τ}}& ...& y_{k,2,q}^{s,\mathrm{\τ}}\\ ...& ...& ...& ...\\ y_{k,p,1}^{s,\mathrm{\τ}}& y_{k,p,2}^{s,\mathrm{\τ}}& ...& y_{k,p,q}^{s,\mathrm{\τ}}\end{array}\right],s=\mathrm{1,2},...,g$

For the low frequency coefficient of gained after Contourlet conversion, adopt method of weighted mean to process, obtain the low frequency coefficient of inverse transformation, image T to be reconstructed after k frame coupling _k' through the low frequency coefficient after Contourlet conversion be ω _kfor weight coefficient, and meet final selection as the low frequency coefficient of Contourlet inverse transformation is

HFS for gained after Contourlet conversion merges by following principle:

Institute's corresponding high frequency coefficient side extent on the each directional subband of more corresponding each yardstick, gets the coefficient of variance minimum as the high frequency coefficient of corresponding scale respective direction inverse transformation respectively, adopts represent the mean value of τ directional subband high frequency coefficient of s yardstick, this mean value expression formula is:

$\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{Y}_{k,i,j}^{s,\mathrm{\τ}},$

Adopt represent the variance of τ directional subband high frequency coefficient of s yardstick, this variance expression formula is: $V_k^{s,\mathrm{\τ}}=\frac{1}{p\·q}\underset{i=1}{\overset{p}{\mathrm{\Σ}}}\underset{j=1}{\overset{q}{\mathrm{\Σ}}}{(Y_{k,i,j}^{s,\mathrm{\τ}}-\stackrel{\‾}{Y_k^{s,\mathrm{\τ}}})}^2,$

be the high frequency coefficient of τ directional subband Contourlet of s yardstick inverse transformation, order:

$G=\left[\begin{array}{ccccccc}{G}^{\mathrm{1,1}}& G^{2^1,1}& 0& 0& 0& ...& 0\\ G^{\mathrm{1,2}}& G^{\mathrm{2,2}}& G^{\mathrm{3,2}}& G^{2^2,2}& 0& & 0\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ .& .& .& .& .& & .\\ G^{1,g}& G^{2,g}& G^{3,g}& G^{4,g}& G^{5,g}& & G^{2^g,g}\end{array}\right],$

Wherein G ^{s, τ}represent the final high frequency coefficient of selecting as τ the directional subband of s yardstick of inverse transformation, G is the each scale coefficient matrixes of all directions, the wherein direction index of line display directional subband, and yardstick index is shown in list,

Carry out Contourlet inverse transformation, i.e. I'=C with the high frequency coefficient G of low frequency coefficient L and the each direction of each yardstick ^-1(L, G), C ^-1() is profile ripple inverse transformation computing, and I' is fused images;

Image is rebuild in step 4, acquisition

The fused images I' obtaining is carried out to the computing of bicubic difference and obtain I'', I'' is carried out to bilateral filtering processing, obtain rebuilding image I I.

2. the image super-resolution rebuilding method based on multi-scale geometric analysis according to claim 1, it is characterized in that: in described step 1, utilize point spread function PSF to produce motion blur function and down-sampling operator, this function has two parametric t heta and len, represent shooting object counterclockwise with theta angular movement len pixel, change the fog-level of image by adjusting these two parameters.