CN103295196A - Super-resolution image reconstruction method based on non-local dictionary learning and biregular terms - Google Patents

Abstract

The invention discloses a super-resolution image reconstruction method based on non-local dictionary learning and biregular terms, and mainly aims to solve the problem that reconstructed images are unnatural due to the fact that prior information of ultralow-resolution images cannot be fully utilized in existing dictionary learning methods. The method includes the main steps: (1), obtaining an initial high-resolution image; (2) training an initial residual dictionary set d0 and an initial expected dictionary set D0; (3) computing an initial non-local regular weight matrix W0 and an initial local kernel regression regular weight matrix K0 on the initial high-resolution image; (4) performing regular optimization processing on an inputted initial high-resolution image to obtain an optimized image; and (5) applying the initial residual dictionary set d0 and the initial expected dictionary set D0 for reconstructing the optimized image to obtain a reconstructed image. The method is capable of reconstructing remote sensing images and effectively maintaining marginal and texture information of the images, and can be used for satellite monitoring and remote-sensing imagery.

Description

Image super-resolution rebuilding method based on non local dictionary study and biregular item

Technical field

The invention belongs to technical field of image processing, relate to a kind of super resolution ratio reconstruction method of image, can be used for the super-resolution rebuilding of natural image, medical image and remote sensing images.

Background technology

Image super-resolution rebuilding is to utilize single width or several low-resolution images to reconstruct the inverse problem of a panel height image in different resolution.In general, the method for image super-resolution rebuilding is divided three classes: based on the method for interpolation, based on the method for Model Reconstruction and based on the method for learning.Wherein, the method based on interpolation comprises classical bilinear interpolation, cubic spline interpolation etc.These class methods are simply quick, but cause edge fog easily, the reconstruction effect that does not reach; Method based on Model Reconstruction comprises the iterative backprojection method, the maximum a posteriori probability method, convex set sciagraphy etc., mostly these class methods are the reconstruction at several low-resolution images, though can produce and rebuild effect preferably, but the parameter of reconstruction model is difficult to estimate, improves the limited in one's ability of image resolution ratio.Along with the development of machine learning techniques, a large amount of super resolution ratio reconstruction methods based on study are suggested.Such method is learnt known high-definition picture and the relation between the low-resolution image, and instructs the super-resolution of low-resolution image to rebuild as a kind of priori this relation.Popular in recent years is the dictionary study super resolution ratio reconstruction method based on rarefaction representation that people such as Yang proposes, and they suppose that most natural images have similar structure and represent primitive, for example edge, texture etc.Since then, many method for reconstructing based on dictionary study are developed.

Though most method for reconstructing based on dictionary study can effectively utilize the prior imformation of external high-resolution sample image, these class methods have been ignored the priori for the treatment of ultra-low resolution image self, and the image after causing rebuilding exists pseudomorphism and distortion.

Summary of the invention

The objective of the invention is to the shortcoming at the image super-resolution method for reconstructing of learning based on dictionary, a kind of image super-resolution rebuilding method based on non local dictionary study and biregular item is proposed, treat the self-similarity information of ultra-low resolution image redundancy with utilization, introduce improved non local regular terms and improved karyomerite and return regular terms, improve the quality of image reconstruction.

For achieving the above object, technical scheme of the present invention is the adaptive doubledictionary of structure from treat the ultra-low resolution image at first, then according to the modelling objective optimization function that degrades of image, obtains high-definition picture by iteration repeatedly.Concrete steps comprise as follows:

(1) input low-resolution image I _l, to this low-resolution image I _lMake bilinear interpolation and handle, obtain initial high-definition picture

(2) to initial high-definition picture

Carry out the training of adaptive cluster dictionary, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R}, initial expectation dictionary set D ₀With initial residual error dictionary set d ₀

(3) at initial high-definition picture

The initial non local canonical weight matrix W of last calculating ₀With initial karyomerite canonical weight matrix K ₀

(4) establishing iterations is n, n=0, and 1,2 ..., 299, primary iteration n=0, the rebuilding super resolution image:

4a) input primary iteration image

According to non local canonical weight matrix W ₀With initial karyomerite canonical weight matrix K ₀, to this primary iteration image

Carry out canonical optimization, image is optimized

4b) use initial expectation wordbook D ₀, initial residual error wordbook d ₀, R cluster centre C _Center={ C _i, i=1,2 ..., the primary iteration image of R} and input

To optimizing image

Rebuild the image after obtaining rebuilding

4c) establish error precision ε=2e-9, N is reconstructed image

In total image block number, according to described parameter ε, N,

Judge whether iteration stops, if Then termination of iterations is exported reconstructed image Otherwise with reconstructed image

As new iteration input picture, iterations adds 1, i.e. n=n+1, and establish indication parameter p1=120;

4d) according to parameter p 1, n judges whether to upgrade initial expectation dictionary set D ₀, initial residual error allusion quotation set d ₀, initial non local canonical weight matrix W ₀And initial karyomerite canonical weight matrix K ₀, if n is the integral multiple of p1, then with initial expectation dictionary set D ₀Be updated to D _n, with initial residual error dictionary set d ₀Be updated to d _n, with initial non local canonical weight matrix W ₀Be updated to W _n, with initial karyomerite canonical weight matrix K ₀Be updated to K _nOtherwise do not upgrade, return step 4a).

The present invention has the following advantages compared with prior art:

1) the present invention has taken full advantage of the prior imformation for the treatment of ultra-low resolution image self, overcome the shortcoming that traditional method for reconstructing based on dictionary study can't effectively replenish low-resolution image disappearance information, the information of additional low-resolution image disappearance that can be more efficient and directed;

2) the present invention introduces improved non local bilateral canonical bound term in process of reconstruction, when carrying out similar coupling, characteristic distance, brightness distance and the geometric position distance of image block have been taken all factors into consideration, overcome traditional non local canonical technology and only utilized brightness apart from the shortcoming of weighing the image block similarity, effectively improved the precision of similar coupling;

3) the present invention introduces improved karyomerite and returns canonical bound term in process of reconstruction, utilize high-frequency characteristic figure to ask for weight, taken full advantage of the local feature prior imformation of image, overcome traditional nuclear and returned the shortcoming that the canonical technology is only utilized monochrome information calculating weight, can try to achieve accurate more partial weight coefficient, effectively improve quality of reconstructed images.

Experiment shows that the present invention is applicable to the super-resolution rebuilding of remote sensing images, has kept edge of image and grain details information preferably.

Description of drawings

Fig. 1 is general flow chart of the present invention;

Fig. 2 is the original high resolution figure of two width of cloth test patterns used of the present invention;

Fig. 3 is with the present invention and has two kinds of methods now to the reconstruction design sketch of test pattern IM1;

Fig. 4 is with the present invention and has two kinds of methods now to the reconstruction design sketch of test pattern IM2.

Embodiment

With reference to accompanying drawing 1, concrete steps of the present invention comprise:

Step 1. pair initial high-definition picture

Carry out the training of adaptive cluster dictionary, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R}, initial expectation dictionary set D ₀With initial residual error dictionary set d ₀

1a) extract the initial high resolution image

High-frequency characteristic, obtain high-frequency characteristic figure G;

1b) respectively at the initial high resolution image

With get 7 * 7 piece on the high-frequency characteristic image G, with this initial high resolution image

On all images piece that obtains be arranged in order with the form of column vector, form the image block set

All characteristic blocks of obtaining on this high-frequency characteristic image G form with column vector is arranged in order, forms the characteristic block set

1c) utilize the method for K mean cluster that characteristic block is gathered

Carry out cluster, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R};

1d) according to R cluster centre C _Center={ C _i, i=1,2 ..., R} and characteristic block set

Image block is gathered

Be divided into R image block subclass:

1e) establish image block u _iIt is the initial high resolution image

On i image block, establish this image block u _iBelong to X class image block subclass

X ∈ 1,2 ..., R} utilizes the method for non-local mean filtering, obtains image block u _i25 similar image piece u ' ₁, u' ₂..., u' ₂₅, obtain this 25 similar image piece u ' simultaneously ₁, u' ₂..., u' ₂₅With image block u _iWeight coefficient w ₁, w ₂..., w ₂₅

1f) according to 25 weight coefficient w ₁, w ₂..., w ₂₅With 25 similar image piece u ' ₁, u' ₂..., u' ₂₅, computed image piece u _iThe estimated image piece $\stackrel{\‾}{u_i}:\stackrel{\‾}{u_i}=\underset{m=1}{\overset{25}{\mathrm{\Σ}}}{w}_m{u}_m^{\text{'}};$

1g) according to 25 similar image piece u ' ₁, u' ₂..., u' ₂₅With the estimated image piece

Calculate this 25 similar image pieces and estimated image piece

Residual image piece Δ ₁, Δ ₂..., Δ ₂₅, wherein,

${\mathrm{\Δ}}_2=u_2^{\text{'}}-\stackrel{\‾}{u_i},...,{\mathrm{\Δ}}_{25}=u_{25}^{\text{'}}-\stackrel{\‾}{u_i};$

1h) repeating step 1e) to step 1g), obtain the image block subclass

All similar image pieces and image block subclass

All residual image pieces; All similar image pieces are arranged in turn, formed similar set of blocks U, all residual image pieces are arranged in turn, form residual block set Δ;

1i) utilize the method for principal component analysis (PCA) that similar set of blocks U is trained, obtain the image block subclass The expectation dictionary

Utilize the method for principal component analysis (PCA) that residual block set Δ is trained, obtain the image block subclass The residual error dictionary

1j) repeating step 1e) to step 1i), obtain R expectation dictionary

With R residual error dictionary $d_0^1,d_0^2,...,d_0^R;$

1k) with this R expectation dictionary

Order is arranged, and constitutes the expectation dictionary set

With this R residual error dictionary Order is arranged, and constitutes the residual error dictionary set $d_0=\{d_0^1,d_0^2,...,d_0^R\}.$

Above-mentioned steps 1c) used clustering method comprises K mean cluster method in, fuzzy C-means clustering method etc., this example adopts but is not limited to K mean cluster method, the routine call of K mean cluster method from " Weisheng Dong; Guangming Shi; Lei Zhang; and Xiaolin Wu; Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[J] .IEEE Transactions on image processing; vol.20, no.7, pp.1838-1857, July2011. ".

Above-mentioned steps 1i) method of used training dictionary comprises the K-singular value decomposition method in, principal component analysis (PCA) etc., this example adopts but is not limited to principal component analysis (PCA), the routine call of principal component analysis (PCA) from " Weisheng Dong; Guangming Shi; Lei Zhang; and Xiaolin Wu; Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[J] .IEEE Transactions on image processing; vol.20, no.7, pp.1838-1857, July2011. ".

Step 2. is at the initial high resolution image

The initial non local canonical weight matrix W of last calculating ₀

2a) to the initial high resolution image

Extract high-frequency characteristic, obtain high-frequency characteristic figure G;

2b) establish image block u _iBe the initial high resolution image

I image block, establish pixel c _iBe image block u _iCentral pixel point; If image block g _iFor i the image block of high-frequency characteristic figure G, establish pixel d _iBe image block g _iCentral pixel point;

2c) at the initial high resolution image On, with pixel c _iCentered by get the similar window Q of 12 * 12 sizes ₁, this similar window Q ₁The interior pixel c that gets rid of _i, 143 pixel c ' are arranged ₁, c' ₂..., c ' ₁₄₃On high-frequency characteristic figure G, with pixel d _iCentered by get the similar window Q of 12 * 12 sizes ₂, this similar window Q ₂The interior pixel d that gets rid of _i, 143 pixel d ' are arranged ₁, d' ₂..., d ' ₁₄₃

2d) establish image block u _jBe with similar window Q ₁Interior j pixel c' _jCentered by image block, establish image block g _jBe with similar window Q ₂Interior j pixel d' _jCentered by image block, j=1,2 ..., 143, establish pixel c _iAt the initial high resolution image

On two-dimensional coordinate be Z _i(x y), establishes pixel c' _jAt the initial high resolution image

On two-dimensional coordinate be Z _j(x, y);

2e) according to coordinate Z _i(x is y) with coordinate Z _j(x y), calculates described pixel c' _jWith described pixel c _iThe brightness value Euclidean distance

The geometric position Euclidean distance The feature Euclidean distance

With total distance

$d_{{1c}_j^{\text{'}}}={\left|\right|u_i-u_j\left|\right|}_2^2/49,$

$d_{{2c}_j^{\text{'}}}={\left|\right|Z_i(x,y)-Z_j(x,y)\left|\right|}_2^2/49,$

$d_{{3c}_j^{\text{'}}}={\left|\right|g_i-g_j\left|\right|}_2^2/49,$

$d_{c_j^{\text{'}}}=d_{{1c}_j^{\text{'}}}+d_{{2c}_j^{\text{'}}}+d_{{3c}_j^{\text{'}}};$

2f) according to step 2e) described in method, obtain and pixel c _i25 nearest pixel { c' _Pm, m=1,2 ..., 25}, this 25 pixels and pixel c _iThe brightness value Euclidean distance be

This 25 pixels and pixel c _iThe geometric position Euclidean distance be

This 25 pixels and pixel c _iThe feature Euclidean distance be

2g) establish brightness and smoothly control parameter h ₁=300, establish level and smooth control parameter h how much ₂=100, establish feature and smoothly control parameter h ₃=100, smoothly control parameter h according to brightness ₁, how much level and smooth control parameter h ₂Smoothly control parameter h with feature ₃, with above-mentioned brightness value Euclidean distance

The geometric position Euclidean distance

The feature Euclidean distance

Substitution gaussian kernel function exp (.), calculating pixel point c' _PmWith pixel c _iWeight coefficient w (c _i, c' _Pm): $w(c_i,c_{\mathrm{pm}}^{\text{'}})=\exp (-\frac{d_{{1c}_{\mathrm{pm}}^{\text{'}}}}{h_1})\exp (-\frac{d_{{2c}_{\mathrm{pm}}^{\text{'}}}}{h_2})\exp (-\frac{d_{3c_{\mathrm{pm}}^{\text{'}}}}{h_3});$

2h) repeating step 2a) to step 2g), obtain the initial high resolution image In the weight coefficient of whole pixels, all weight coefficients are arranged according to the coordinate order, form initial canonical weight matrix W ₀: $W_0=\left\{\begin{array}{c}w(c_i,c_{\mathrm{pm}}^{\text{'}}),c_i\∈I_h^0,c_{\mathrm{pm}}^{\text{'}}\∈I_h^0,m=\mathrm{1,2},...,25\\ 0\end{array}\right..$

Step 3. is at the initial high resolution image The initial karyomerite canonical weight matrix K of last calculating ₀

3a) to the initial high resolution image

Extract high-frequency characteristic, obtain high-frequency characteristic figure G;

3b) establish pixel a _iBe i pixel among the high-frequency characteristic figure G, establish this pixel a _iCoordinate vector be t, establish this pixel a _iBrightness value be F (t), with this pixel a _iCentered by get 7 * 7 window;

3c) have 49 pixels in this window, these 49 pixels are arranged in turn, form column vector b=[b ₁, b ₂..., b ₄₉] ^T, establish j pixel b _jCoordinate vector be t _j, establish pixel b _jBrightness value be q _j

3d) establish matrix S _jBe pixel b _jLocal covariance matrix, establish overall smooth parameter h=0.005, the sampling density μ of portion=0.5 of setting a trap, according to above-mentioned parameter, calculating pixel point b _jRotation matrix

3e) according to above-mentioned parameter, calculating pixel point b _jThe control nuclear matrix

$W_{H_{b_j}^{\mathrm{steer}}}(t_i-t)=\frac{\sqrt{\det \left(S_j\right)}}{2\mathrm{\π}{h}^2{\mathrm{\μ}}^2}\exp \{-\frac{{(t_j-t)}^T{S}_j(t_j-t)}{2h^2{\mathrm{\μ}}^2}\},$ J=1 ... 49, wherein, exp (.) is gaussian kernel function, det (.) is for asking the function of matrix determinant.

3f) according to above-mentioned parameter, obtain 49 pixel { b _j, j=1,2 ..., the overhead control nuclear matrix W of 49} _t: $W_t=\mathrm{diag}[{\mathrm{<figure-callout\; id="1"\; label="W\; H\; b"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">W</figure-callout>}}_{H_{b_{}}^{}}(t_1-t),W_{H_{b_2}^{\mathrm{steer}}}(t_2-t)...,W_{H_{b_{49}}^{\mathrm{steer}}}(t_{49}-t)],$ Wherein, diag (.) gets the diagonal angle operation;

3g) establish total coordinates matrix $E_t=\left[\begin{array}{cc}1{(t_1-t)}^T{\mathrm{ltr}}^T\left\{(t-t_1){(t_1-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\\ 1{(t_2-t)}^T{\mathrm{ltr}}^T\left\{(t_2-t){(t_2-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ 1{(t_{49}-t)}^T{\mathrm{ltr}}^T\left\{(t_{49}-t){(t_{49}-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\end{array}\right],$ According to total coordinates matrix E _tWith overhead control nuclear matrix W _t, utilize the method for least square, calculate the estimated value of brightness value F (t)

Wherein, ltr (.) represents the following triangle element of a matrix is lined up the operation of a column vector, e ₁Represent that first element is 1, other element all is 0 column vector;

3h) establishing variable element N' is the Taylor expansion exponent number, will Be rewritten as the form of linear weighted function: $\hat{F}\left(t\right)=\underset{j=1}{\overset{49}{\mathrm{\&Sigma;}}}{W}_{b_j}(t;H_{b_j}^{\mathrm{steer}},N^{\text{'}})b_j,$ Matrix wherein

J pixel b in the expression window _jWith pixel a _iWeight coefficient:

3i) establish variable element N'=2, repeating step 3a) to step 3h), obtain the initial high resolution image In the weight coefficient of whole pixels, all weight coefficients are arranged according to the coordinate order, form initial karyomerite function canonical weight matrix K ₀:

$K_0=\left\{\begin{array}{c}{W}_{b_j}(t;H_{b_j}^{\mathrm{steer}},2),b_j\&Element;b\\ 0,b_j\&NotElement;b\end{array}\right..$

Above-mentioned steps 3e) used nuclear comprises bilateral nuclear in, control nuclear etc., this example adopts but is not limited to control nuclear, and the routine call of control nuclear is from " H.Takeda, S.Farsiu; and P.Milanfar; Kernel regression for image processing and reconstruction.IEEE Transactions on image processing, vol.16, no.2; pp.349 – 366,2007. ".

Step 4. input primary iteration image

According to non local canonical weight matrix W ₀With initial karyomerite canonical weight matrix K ₀, to this primary iteration image Carry out canonical optimization, image is optimized

4a) establishing Matrix C is sampling matrix, and establishing low resolution image is I _l, establishing unknown high-definition picture is I _h, establish regular parameter β ₁=0.14, β ₂=0.46, establishing matrix E is unit matrix;

4b) according to non local canonical weight matrix W ₀, initial karyomerite canonical weight matrix K ₀With step 4a) in parameter, obtain canonical and optimize formula and be: $I_h=\underset{I_h}{\arg \min }({\left|\right|I_l-{\mathrm{CI}}_h\left|\right|}_2^2+{\mathrm{\&beta;}}_1{\left|\right|(E-W_0)I_h\left|\right|}_2^2+{\mathrm{\&beta;}}_2{\left|\right|(E-K_0)I_h\left|\right|}_2^2),$ Wherein

Be l ₂The norm value finding function;

4c) establish the constant matrix ${\tilde{I}}_l=\left[\begin{array}{c}{I}_l\\ 0\\ 0\end{array}\right],$ If operation matrix $\tilde{K}=\left[\begin{array}{c}C\\ {\mathrm{\&beta;}}_1(E-W_0)\\ {\mathrm{\&beta;}}_2(E-K_0)\end{array}\right],$ According to the constant matrix

Operation matrix

With step 4b) in canonical optimize simplified formula and be:

4d) according to input picture

And formula

Utilize gradient descent method calculation optimization image

${\mathrm{<figure-callout\; id="1"\; label="I\; h"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">I</figure-callout>}}_h^{}=I_h^0+K^T({\tilde{I}}_l-{\mathrm{KI}}_h^0)=I_h^0+C^T{I}_l-C^T{\mathrm{CI}}_h^0$

$-{\mathrm{\&beta;}}_1^2{(E-W_0)}^T(E-W_0)I_h^0-{\mathrm{\&beta;}}_2^2{(E-K_0)}^T(E-K_0)I_h^0.$

This step 4d) l in ₂The used method for solving of norm formula comprises the gradient descent method, soft-threshold shrinkage method etc., this example adopts but is not limited to the gradient descent method, the routine call of gradient descent method from " Weisheng Dong; Guangming Shi; Lei Zhang; and Xiaolin Wu; Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[J] .IEEE Transactions on image processing; vol.20, no.7, pp.1838-1857, July2011. ".

Step 5. is used initial expectation wordbook D ₀, initial residual error wordbook d ₀, R cluster centre C _Center={ C _i, i=1,2 ..., the primary iteration image of R} and input

To optimizing image Rebuild the image after obtaining rebuilding

5a) extract the optimization image

High-frequency characteristic, obtain high-frequency characteristic image G;

5b) respectively to optimizing image

G gets 7 * 7 piece with the high-frequency characteristic image, and this optimizes image

On all images piece that obtains be arranged in order with the form of column vector, form the image block set

All characteristic blocks that obtain on this high-frequency characteristic figure G are arranged in order with the form of column vector, form the characteristic block set

5c) according to R cluster centre C _Center={ C _i, i=1,2 ..., R} and characteristic block set

Image block is gathered

Be divided into R image block subclass:

5d) establish image block u _iIt is the initial high resolution image

On i image block, establish this image block u _iBelong to X class image block subclass

X ∈ 1,2 ..., R} is according to classification X, at expectation dictionary set D ₀In selected dictionary

At residual error dictionary set d ₀In selected dictionary

5e) utilize the non-local mean technology, obtain image block u _iThe estimation piece

And residual block

5f) utilize the estimation piece The residual block Δ _i, the expectation dictionary The residual error dictionary

Calculate this image block u _iAt the expectation dictionary

On the rarefaction representation factor alpha _DWith image block u _iAt the residual error dictionary On the rarefaction representation factor alpha _d:

${\mathrm{\&alpha;}}_D={\left(D_0^X\right)}^T\stackrel{\&OverBar;}{u_i},$

${\mathrm{\&alpha;}}_d={\left(d_0^X\right)}^T{\mathrm{\&Delta;}}_i;$

5g) establish the initial high resolution image In i image block be u ' _i, according to step 5a) to 5f) described in method, establish this image block u ' _iExpectation dictionary rarefaction representation coefficient be α ' _D, establish this image block u ' _iResidual error dictionary rarefaction representation factor alpha ' _d

5h) establish threshold parameter ε 2=0.35, according to threshold parameter ε 2, expectation dictionary rarefaction representation factor alpha ' _D, residual error dictionary rarefaction representation factor alpha ' _d, utilize the rarefaction representation method of iteration weighting, compute sparse is represented regular parameter λ _DAnd λ _d:

${\mathrm{\&lambda;}}_D=\frac{1}{{\mathrm{\&alpha;}}_D^{\text{'}}+\mathrm{\&epsiv;}2},$

${\mathrm{\&lambda;}}_d=\frac{1}{{\mathrm{\&alpha;}}_d^{\text{'}}+\mathrm{\&epsiv;}2};$

5i) according to above-mentioned parameter α _D, α _d, λ _D, λ _d, utilize the soft-threshold shrinkage method, with image block u _iExpectation dictionary rarefaction representation factor alpha _DBe updated to With residual error dictionary rarefaction representation factor alpha _dBe updated to

${\mathrm{\&alpha;}}_{\mathrm{<figure-callout\; id="1"\; label="D"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">D</figure-callout>}}^1=\mathrm{sign}\left({\mathrm{\&alpha;}}_D\right)\&times;\max (\mathrm{abs}\left({\mathrm{\&alpha;}}_D\right)-{\mathrm{\&lambda;}}_D,0),$

${\mathrm{\&alpha;}}_{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">d</figure-callout>}}^1=\mathrm{sign}\left({\mathrm{\&alpha;}}_d\right)\&times;\max (\mathrm{abs}\left({\mathrm{\&alpha;}}_d\right)-{\mathrm{\&lambda;}}_d,0),$

Wherein, sign (.) expression sign function, abs (.) represents to take absolute value function;

5j) according to above-mentioned parameter

The expectation dictionary

The residual error dictionary Computed image piece u _iImage block after the reconstruction

$\stackrel{\&OverBar;}{u_i}=D_0^X{\mathrm{\&alpha;}}_{\mathrm{<figure-callout\; id="1"\; label="D"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">D</figure-callout>}}^1+d_0^X{\mathrm{\&alpha;}}_{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA00003218872000011.png"\; state="\{\{state\}\}">d</figure-callout>}}^1;$

5k) repeating step 5d) to step 5j), rebuild and optimize image

In all image blocks, the image after obtaining rebuilding

Above-mentioned steps 5h) China and foreign countries' dictionary regular parameter λ _DWith interior dictionary regular parameter λ _dUsed method for solving comprises the rarefaction representation method of iteration weighting, the rarefaction representation method of overall weighting etc.This example adopts but is not limited to the rarefaction representation method of iteration weighting, and the routine call of the rarefaction representation method of iteration weighting is from " E.J.Candes, M.B.Wakin, and S.Boyd, Enhancing Sparsity by Reweighted l ₁Minimization[J], Journal of Fourier Analysis and Applications, vol.14, pp.877 – 905,2008. ".

Above-mentioned steps 5i) the used method for solving of rarefaction representation coefficient update comprises the gradient descent method, soft-threshold shrinkage method etc. in.This example adopts but is not limited to the soft-threshold shrinkage method, the routine call of soft-threshold shrinkage method is from " I.Daubechies; M.Defriese; and C.DeMol; An iterative thresholding algorithm forlinear inverse problems with a sparsity constraint[J], Commun.Pure Appl.Math., vol.57; pp.1413～1457,2004. ".

Effect of the present invention can further specify by following experiment:

1) experiment condition

The test pattern that this experiment is adopted is IM1 and IM2, as shown in Figure 2, adopt MATLAB7.10.0 software as emulation tool, computing machine is configured to Intel (R) Core (TM) 2/2.33G/1.99G, in this experiment, the down-sampling factor in the image deterioration process is made as 3.

2) experiment content

Experiment 1, utilize Bicubic interpolation algorithm, SCDL algorithm and the inventive method, the IM1 image is rebuild, the result as shown in Figure 3, wherein Fig. 3 (a) is the high-definition picture of IM1 image, and Fig. 3 (b) is the low-resolution image of input, and Fig. 3 (c) is the reconstructed results of Bicubic interpolation algorithm, Fig. 3 (d) is the reconstructed results of SCDL algorithm, and Fig. 3 (e) is reconstructed results of the present invention;

Experiment 2, utilize Bicubic interpolation algorithm, SCDL algorithm and the inventive method, the IM2 image is rebuild, the result as shown in Figure 3, wherein Fig. 4 (a) is the high-definition picture of IM2 image, and Fig. 4 (b) is the low-resolution image of input, and Fig. 4 (c) is the reconstructed results of Bicubic interpolation algorithm, Fig. 4 (d) is the reconstructed results of SCDL algorithm, and Fig. 4 (e) is reconstructed results of the present invention.

Interpretation

From Fig. 3-Fig. 4 as can be seen, the visual effect of reconstructed image of the present invention is better than other two kinds of method for reconstructing, and for two width of cloth test pattern IM1 and IM2 of input, it is relatively good that edge of image and grain details information all keep.

Claims (6)

1. the image super-resolution rebuilding method based on non local dictionary study and biregular item comprises the steps:

(1) input low-resolution image I _l, to this low-resolution image I _lMake bilinear interpolation and handle, obtain initial high-definition picture

(2) to initial high-definition picture

Carry out the training of adaptive cluster dictionary, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R}, initial expectation dictionary set D ₀With initial residual error dictionary set d ₀

(3) at initial high-definition picture

The initial non local canonical weight matrix W of last calculating ₀With initial karyomerite canonical weight matrix K ₀

(4) establishing iterations is n, n=0, and 1,2 ..., 299, primary iteration n=0, the rebuilding super resolution image:

4a) input primary iteration image

According to non local canonical weight matrix W ₀With initial karyomerite canonical weight matrix K ₀, to this primary iteration image Carry out canonical optimization, image is optimized

4b) use initial expectation wordbook D ₀, initial residual error wordbook d ₀, R cluster centre C _Center={ C _i, i=1,2 ..., the primary iteration image of R} and input

To optimizing image Rebuild the image after obtaining rebuilding

4c) establish error precision ε=2e-9, N is reconstructed image

In total image block number, according to described parameter

Judge whether iteration stops, if

Then termination of iterations is exported reconstructed image

Otherwise with reconstructed image

As new iteration input picture, iterations adds 1, i.e. n=n+1, and establish indication parameter p1=120;

4d) according to parameter p 1, n judges whether to upgrade initial expectation dictionary set D ₀, initial residual error allusion quotation set d ₀, initial non local canonical weight matrix W ₀And initial karyomerite canonical weight matrix K ₀, if n is the integral multiple of p1, then with initial expectation dictionary set D ₀Be updated to D _n, with initial residual error dictionary set d ₀Be updated to d _n, with initial non local canonical weight matrix W ₀Be updated to W _n, with initial karyomerite canonical weight matrix K ₀Be updated to K _nOtherwise do not upgrade, return step 4a).

2. the image super-resolution rebuilding method based on the study of non local dictionary and biregular item according to claim 1, wherein step (2) is described to initial high-definition picture

Carry out the training of adaptive cluster dictionary, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R}, initial expectation dictionary set D ₀With initial residual error dictionary set d ₀, carry out as follows:

2a) extract the initial high resolution image

High-frequency characteristic, obtain high-frequency characteristic figure G;

2b) respectively at the initial high resolution image

With get 7 * 7 piece on the high-frequency characteristic image G, with this initial high resolution image On all images piece that obtains be arranged in order with the form of column vector, form the image block set

All characteristic blocks of obtaining on this high-frequency characteristic image G form with column vector is arranged in order, forms the characteristic block set

2c) utilize the method for K mean cluster that characteristic block is gathered Carry out cluster, obtain R cluster centre C _Center={ C _i, i=1,2 ..., R};

2d) according to R cluster centre C _Center={ C _i, i=1,2 ..., R} and characteristic block set

Image block is gathered

Be divided into R image block subclass:

2e) establish image block u _iIt is the initial high resolution image

On i image block, establish this image block u _iBelong to X class image block subclass

X ∈ 1,2 ..., R} utilizes the method for non-local mean filtering, obtains image block u _i25 similar image piece u ' ₁, u' ₂..., u' ₂₅, obtain this 25 similar image piece u ' simultaneously ₁, u' ₂..., u' ₂₅With image block u _iWeight coefficient w ₁, w ₂..., w ₂₅

2f) according to 25 weight coefficient w ₁, w ₂..., w ₂₅With 25 similar image piece u ' ₁, u' ₂..., u' ₂₅, computed image piece u _iThe estimated image piece $\stackrel{\&OverBar;}{u_i}:\stackrel{\&OverBar;}{u_i}=\underset{m=1}{\overset{25}{\mathrm{\&Sigma;}}}{w}_m{u}_m^{\text{'}};$

2g) according to 25 similar image piece u ' ₁, u' ₂..., u' ₂₅With the estimated image piece

Calculate this 25 similar image pieces and estimated image piece Residual image piece Δ ₁, Δ ₂..., Δ ₂₅, wherein,

${\mathrm{\&Delta;}}_2=u_2^{\text{'}}-\stackrel{\&OverBar;}{u_i},...,{\mathrm{\&Delta;}}_{25}=u_{25}^{\text{'}}-\stackrel{\&OverBar;}{u_i};$

2h) repeating step 2e) to step 2g), obtain the image block subclass

All similar image pieces and image block subclass

All residual image pieces; All similar image pieces are arranged in turn, formed similar set of blocks U, all residual image pieces are arranged in turn, form residual block set Δ;

2i) utilize the method for principal component analysis (PCA) that similar set of blocks U is trained, obtain the image block subclass

The expectation dictionary

Utilize the method for principal component analysis (PCA) that residual block set Δ is trained, obtain the image block subclass

The residual error dictionary

2j) repeating step 2e) to step 2i), obtain R expectation dictionary

With R residual error dictionary $d_0^1,d_0^2,...,d_0^R;$

2k) with this R expectation dictionary Order is arranged, and constitutes the expectation dictionary set With this R residual error dictionary

Order is arranged, and constitutes the residual error dictionary set $d_0=\{d_0^1,d_0^2,...,d_0^R\}.$

3. the image super-resolution rebuilding method based on the study of non local dictionary and biregular item according to claim 1, wherein step (3) is described at the initial high resolution image

The initial non local canonical weight matrix W of last calculating ₀, carry out as follows:

3.1) to the initial high resolution image

Extract high-frequency characteristic, obtain high-frequency characteristic figure G;

3.2) establish image block u _iBe the initial high resolution image

I image block, establish pixel c _iBe image block u _iCentral pixel point; If image block g _iFor i the image block of high-frequency characteristic figure G, establish pixel d _iBe image block g _iCentral pixel point;

3.3) at the initial high resolution image

On, with pixel c _iCentered by get the similar window Q of 12 * 12 sizes ₁, this similar window Q ₁The interior pixel c that gets rid of _i, 143 pixel c ' are arranged ₁, c' ₂..., c ' ₁₄₃On high-frequency characteristic figure G, with pixel d _iCentered by get the similar window Q of 12 * 12 sizes ₂, this similar window Q ₂The interior pixel d that gets rid of _i, 143 pixel d ' are arranged ₁, d' ₂..., d ' ₁₄₃

3.4) establish image block u _jBe with similar window Q ₁Interior j pixel c' _jCentered by image block, establish image block g _jBe with similar window Q ₂Interior j pixel d' _jCentered by characteristic block, j=1,2 ..., 143, establish pixel c _iAt the initial high resolution image

On two-dimensional coordinate be Z _i(x y), establishes pixel c' _jAt the initial high resolution image

On two-dimensional coordinate be Z _j(x, y);

3.5) according to coordinate Z _i(x is y) with coordinate Z _j(x y), calculates described pixel c' _jWith described pixel c _iThe brightness value Euclidean distance The geometric position Euclidean distance The feature Euclidean distance

With total distance

$d_{{1c}_j^{\text{'}}}={\left|\right|u_i-u_j\left|\right|}_2^2/49,$

$d_{{2c}_j^{\text{'}}}={\left|\right|Z_i(x,y)-Z_j(x,y)\left|\right|}_2^2/49,$

$d_{{3c}_j^{\text{'}}}={\left|\right|g_i-g_j\left|\right|}_2^2/49,$

$d_{c_j^{\text{'}}}=d_{{1c}_j^{\text{'}}}+d_{{2c}_j^{\text{'}}}+d_{{3c}_j^{\text{'}}};$

3.6) according to step 3.5) and described in method, obtain and pixel c _i25 nearest pixel { c' _Pm, m=1,2 ..., 25}, this 25 pixels and pixel c _iThe brightness value Euclidean distance be

This 25 pixels and pixel c _iThe geometric position Euclidean distance be

This 25 pixels and pixel c _iThe feature Euclidean distance be $\{d_{{3c}_{\mathrm{pm}}^{\text{'}}},m=\mathrm{1,2},...,25\};$

3.7) establish brightness and smoothly control parameter h ₁=300, establish level and smooth control parameter h how much ₂=100, establish feature and smoothly control parameter h ₃=100, smoothly control parameter h according to brightness ₁, how much level and smooth control parameter h ₂Smoothly control parameter h with feature ₃, with above-mentioned brightness value Euclidean distance The geometric position Euclidean distance

The feature Euclidean distance

Substitution gaussian kernel function exp (.), calculating pixel point c' _PmWith pixel c _iWeight coefficient w (c _i, c' _Pm): $w(c_i,c_{\mathrm{pm}}^{\text{'}})=\exp (-\frac{d_{{1c}_{\mathrm{pm}}^{\text{'}}}}{h_1})\exp (-\frac{d_{{2c}_{\mathrm{pm}}^{\text{'}}}}{h_2})\exp (-\frac{d_{3c_{\mathrm{pm}}^{\text{'}}}}{h_3});$

3.8) repeating step 3.1) to step 3.7), obtain the initial high resolution image

In the weight coefficient of whole pixels, all weight coefficients are arranged according to the coordinate order, form initial canonical weight matrix W ₀: $W_0=\left\{\begin{array}{c}w(c_i,c_{\mathrm{pm}}^{\text{'}}),c_i\&Element;I_h^0,c_{\mathrm{pm}}^{\text{'}}\&Element;I_h^0,m=\mathrm{1,2},...,25\\ 0\end{array}\right..$

4. the image super-resolution rebuilding method based on the study of non local dictionary and biregular item according to claim 1, wherein step (3) is described at the initial high resolution image

The initial karyomerite canonical weight matrix K of last calculating ₀, carry out as follows:

3A) to the initial high resolution image

Extract high-frequency characteristic, obtain high-frequency characteristic figure G;

3B) establish pixel a _iBe i pixel among the high-frequency characteristic figure G, establish this pixel a _iCoordinate vector be t, establish this pixel a _iBrightness value be F (t), with this pixel a _iCentered by get 7 * 7 window;

3C) have 49 pixels in this window, these 49 pixels are arranged in turn, form column vector b=[b ₁, b ₂..., b ₄₉] ^T, establish j pixel b _jCoordinate vector be t _j, establish pixel b _jBrightness value be q _j

3D) establish matrix S _jBe pixel b _jLocal covariance matrix, establish overall smooth parameter h=0.005, the sampling density μ of portion=0.5 of setting a trap, according to above-mentioned parameter, calculating pixel point b _jRotation matrix

3E) according to above-mentioned parameter, calculating pixel point b _jThe control nuclear matrix

$W_{H_{b_j}^{\mathrm{steer}}}(t_i-t)=\frac{\sqrt{\det \left(S_j\right)}}{2\mathrm{\&pi;}{h}^2{\mathrm{\&mu;}}^2}\exp \{-\frac{{(t_j-t)}^T{S}_j(t_j-t)}{2h^2{\mathrm{\&mu;}}^2}\},$ J=1 ... 49, wherein, exp (.) is gaussian kernel function, det (.) is for asking the function of matrix determinant.

3F) according to above-mentioned parameter, obtain 49 pixel { b _j, j=1,2 ..., the overhead control nuclear matrix W of 49} _t: $W_t=\mathrm{diag}[W_{H_{b_1}^{\mathrm{steer}}}(t_1-t),W_{H_{b_2}^{\mathrm{steer}}}(t_2-t)...,W_{H_{b_{49}}^{\mathrm{steer}}}(t_{49}-t)],$ Wherein, diag (.) gets the diagonal angle operation;

3G) establish total coordinates matrix $E_t=\left[\begin{array}{cc}1{(t_1-t)}^T{\mathrm{ltr}}^T\left\{(t-t_1){(t_1-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\\ 1{(t_2-t)}^T{\mathrm{ltr}}^T\left\{(t_2-t){(t_2-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ \&CenterDot;& \&CenterDot;\\ 1{(t_{49}-t)}^T{\mathrm{ltr}}^T\left\{(t_{49}-t){(t_{49}-t)}^T\right\}& \&CenterDot;\&CenterDot;\&CenterDot;\end{array}\right],$ According to total coordinates matrix E _tWith overhead control nuclear matrix W _t, utilize the method for least square, calculate the estimated value of brightness value F (t)

Wherein, ltr (.) represents the following triangle element of a matrix is lined up the operation of a column vector, e ₁Represent that first element is 1, other element all is 0 column vector;

3H) establishing variable element N' is the Taylor expansion exponent number, will Be rewritten as the form of linear weighted function: $\hat{F}\left(t\right)=\underset{j=1}{\overset{49}{\mathrm{\&Sigma;}}}{W}_{b_j}(t;H_{b_j}^{\mathrm{steer}},N^{\text{'}})b_j,$ Matrix wherein J pixel b in the expression window _jWith pixel a _iWeight coefficient:

3I) establish variable element N'=2, repeating step 3A) to step 3H), obtain the initial high resolution image

In the weight coefficient of whole pixels, all weight coefficients are arranged according to the coordinate order, form initial karyomerite function canonical weight matrix K ₀:

$K_0=\left\{\begin{array}{c}{W}_{b_j}(t;H_{b_j}^{\mathrm{steer}},2),b_j\&Element;b\\ 0,b_j\&NotElement;b\end{array}\right..$

5. the image super-resolution rebuilding method based on the study of non local dictionary and biregular item according to claim 1, wherein step 4a) described input primary iteration image

According to non local canonical weight matrix W ₀With initial karyomerite canonical weight matrix K ₀, to this primary iteration image Carry out canonical optimization, image is optimized

Carry out as follows:

4a1) establishing Matrix C is sampling matrix, and establishing low resolution image is I _l, establishing unknown high-definition picture is I _h, establish regular parameter β ₁=0.14, β ₂=0.46, establishing matrix E is unit matrix;

4a2) according to non local canonical weight matrix W ₀, initial karyomerite canonical weight matrix K ₀With step 4a1) in parameter, obtain canonical and optimize formula and be: $I_h=\underset{I_h}{\arg \min }({\left|\right|I_l-{\mathrm{CI}}_h\left|\right|}_2^2+{\mathrm{\&beta;}}_1{\left|\right|(E-W_0)I_h\left|\right|}_2^2+{\mathrm{\&beta;}}_2{\left|\right|(E-K_0)I_h\left|\right|}_2^2),$ Wherein

Be l ₂The norm value finding function;

4a3) establish the constant matrix ${\tilde{I}}_l=\left[\begin{array}{c}{I}_l\\ 0\\ 0\end{array}\right],$ If operation matrix $\tilde{K}=\left[\begin{array}{c}C\\ {\mathrm{\&beta;}}_1(E-W_0)\\ {\mathrm{\&beta;}}_2(E-K_0)\end{array}\right],$ According to the constant matrix

Operation matrix

With step 4a2) in canonical optimize simplified formula and be:

4a4) according to input picture And formula

Utilize gradient descent method calculation optimization image

$I_h^{1/2}=I_h^0+K^T({\tilde{I}}_l-{\mathrm{KI}}_h^0)=I_h^0+C^T{I}_l-C^T{\mathrm{CI}}_h^0$

$-{\mathrm{\&beta;}}_1^2{(E-W_0)}^T(E-W_0)I_h^0-{\mathrm{\&beta;}}_2^2{(E-K_0)}^T(E-K_0)I_h^0.$

6. the image super-resolution rebuilding method based on the study of non local dictionary and biregular item according to claim 1, wherein step 4b) the initial expectation wordbook D of described application ₀, initial residual error wordbook d ₀, R cluster centre C _Center={ C _i, i=1,2 ..., the primary iteration image of R} and input

To optimizing image

Rebuild the image after obtaining rebuilding

Carry out as follows:

4b1) extract the optimization image

High-frequency characteristic, obtain high-frequency characteristic image G;

4b2) respectively to optimizing image

G gets 7 * 7 piece with the high-frequency characteristic image, and this optimizes image

On all images piece that obtains be arranged in order with the form of column vector, form the image block set

All characteristic blocks that obtain on this high-frequency characteristic figure G are arranged in order with the form of column vector, form the characteristic block set

4b3) according to R cluster centre C _Center={ C _i, i=1,2 ..., R} and characteristic block set Image block is gathered

Be divided into R image block subclass:

4b4) establish image block u _iIt is the initial high resolution image

On i image block, establish this image block u _iBelong to X class image block subclass

, X ∈ 1,2 ..., R} is according to classification X, at expectation dictionary set D ₀In selected dictionary

At residual error dictionary set d ₀In selected dictionary

4b5) utilize the non-local mean technology, obtain image block u _iThe estimation piece

And residual block

4b6) utilize the estimation piece

The residual block Δ _i, the expectation dictionary The residual error dictionary

Calculate this image block u _iAt the expectation dictionary

On the rarefaction representation factor alpha _DWith image block u _iAt the residual error dictionary

On the rarefaction representation factor alpha _d:

${\mathrm{\&alpha;}}_D={\left(D_0^X\right)}^T\stackrel{\&OverBar;}{u_i},$

${\mathrm{\&alpha;}}_d={\left(d_0^X\right)}^T{\mathrm{\&Delta;}}_i;$

4b7) establish the initial high resolution image

In i image block be u ' _i, according to step 4b1) to 4b6) described in method, establish this image block u ' _iExpectation dictionary rarefaction representation coefficient be α ' _D, establish this image block u ' _iResidual error dictionary rarefaction representation factor alpha ' _d

4b8) establish threshold parameter ε 2=0.35, according to threshold parameter ε 2, expectation dictionary rarefaction representation factor alpha ' _D, residual error dictionary rarefaction representation factor alpha ' _d, utilize the rarefaction representation method of iteration weighting, compute sparse is represented regular parameter λ _DAnd λ _d:

${\mathrm{\&lambda;}}_D=\frac{1}{{\mathrm{\&alpha;}}_D^{\text{'}}+\mathrm{\&epsiv;}2},$

${\mathrm{\&lambda;}}_d=\frac{1}{{\mathrm{\&alpha;}}_d^{\text{'}}+\mathrm{\&epsiv;}2};$

4b9) according to above-mentioned parameter α _D, α _d, λ _D, λ _d, utilize the soft-threshold shrinkage method, with image block u _iExpectation dictionary rarefaction representation factor alpha _DBe updated to

With residual error dictionary rarefaction representation factor alpha _dBe updated to

${\mathrm{\&alpha;}}_D^1=\mathrm{sign}\left({\mathrm{\&alpha;}}_D\right)\&times;\max (\mathrm{abs}\left({\mathrm{\&alpha;}}_D\right)-{\mathrm{\&lambda;}}_D,0),$

${\mathrm{\&alpha;}}_d^1=\mathrm{sign}\left({\mathrm{\&alpha;}}_d\right)\&times;\max (\mathrm{abs}\left({\mathrm{\&alpha;}}_d\right)-{\mathrm{\&lambda;}}_d,0),$

Wherein, sign (.) expression sign function, abs (.) represents to take absolute value function;

4b10) according to above-mentioned parameter

The expectation dictionary

The residual error dictionary

Computed image piece u _iImage block after the reconstruction

$\stackrel{\&OverBar;}{u_i}=D_0^X{\mathrm{\&alpha;}}_D^1+d_0^X{\mathrm{\&alpha;}}_d^1;$

4b11) repeating step 4b4) to step 4b10), rebuild and optimize image

In all image blocks, the image after obtaining rebuilding

Images