CN101950365A - Multi-task super-resolution image reconstruction method based on KSVD dictionary learning - Google Patents

Abstract

The invention discloses a multi-task super-resolution image reconstruction method based on KSVD dictionary learning, which mainly solves the problem of relatively serious quality reduction of the reconstructed image under high amplification factors in the existing method. The method mainly comprises the following steps: firstly, inputting a training image, and filtering the training image to extract features; extracting image blocks to construct a matrix M, and dividing the matrix M into K classes to acquire K pairs of initial dictionaries H1, H2...Hk and L1, L2...Lk; then, training the K pairs of initial dictionaries H1, H2...Hk and L1, L2...Lk into K pairs of new dictionaries Dh1, Dh2...Dhk and Dl1, Dl2...Dlk by utilizing a KSVD method; and finally, carrying out super-resolution reconstruction on the input low-resolution image by utilizing a multi-task algorithm and the dictionaries Dh1, Dh2...Dhk and Dl1, Dl2...Dlk to acquire a final reconstructed image. The invention can reconstruct various natural images containing non-texture images such as animals, plants, people and the like and images with stronger texture features such as buildings and the like, and can effectively improve the quality of the reconstructed image under high amplification factors.

Description

Multitask super-resolution image reconstruction method based on the study of KSVD dictionary

Technical field

The invention belongs to technical field of image processing, the ultra-resolution ratio reconstructing method that relates to a kind of image, promptly by the relational model between study low-resolution image and the high-definition picture, predict the high-definition picture that it is corresponding from the low-resolution image of single width input, can be used for the super-resolution reconstruction of all kinds of natural images.

Background technology

Super-resolution image reconstruction can be regarded an inverse problem that recovers high-definition picture from the one or more low-resolution image as.In order to address this problem, some traditional methods based on model are suggested: such as MAP (maximum a-posteriori) method, the maximum likelihood estimation technique, convex set sciagraphy (POCS) etc.But these traditional methods can produce level and smooth phenomenon and sawtooth effect, and the quality suppression ratio of reconstructed image is more serious under high amplification factor condition.Therefore, Freeman etc. have proposed a kind of reconstructing method based on study, it mainly is to learn relation between low-resolution image and the high-definition picture by Markov probabilistic model and priori, reconstructs its corresponding high-definition picture from a width of cloth low-resolution image.People such as Sun then expand this method, it mainly is by using some original profile prioris to remove the level and smooth phenomenon of border and details, but this method still needs a large amount of low-resolution images and high-definition picture piece to guarantee the adequacy of priori profile detailed information, calculated amount is huge, the image reconstruction time is long, causes efficient on the low side.These algorithm all is based on the restructing algorithm of single task in addition, compares with multitasked algorithm, can not carry out the information sharing between a plurality of tasks, causes image reconstruction effect deviation.

Summary of the invention

The objective of the invention is to overcome the shortcoming of above-mentioned prior art, propose a kind of multitask super-resolution image reconstruction method,, improve the efficient and the quality of reconstructed image to shorten the image reconstruction time based on the study of KSVD dictionary.

For achieving the above object, technical scheme of the present invention is at first to introduce tens various types of other training images, comprises plant, animal, people and building, and all corresponding width of cloth low-resolution image of each panel height image in different resolution; Utilize the KSVD algorithm that these images are trained to K to dictionary; Utilize the method for multitask reconstruct then, the low-resolution image of importing is reconstituted high-definition picture.Concrete steps comprise:

(1) input training image carries out filtering to it and extracts feature, extracts matrix M of 100,000 pairs of little block structures of image again, is divided into the K class, obtains K to initial dictionary H ₁, H ₂... H _kAnd L ₁, L ₂... L _k, and K cluster centre C ₁, C ₂... C _k

(2) utilize the KSVD algorithm to separate formula:

Subject to To K to initial dictionary H ₁, H ₂... H _kAnd L ₁, L ₂... L _kTrain, obtain K new dictionary D _H1, D _H2... D _HkAnd D _L1, D _L2... D _Lk,

Wherein, Y is the initial dictionary of input, and D is that target is trained dictionary and is initialized as a dictionary at random, and X is the Sparse Decomposition matrix,

Be any i row, || X _i|| ₀Be X _i0 norm,

For finding the solution 2 norms of Y-DX, T ₀Be the degree of rarefication control coefrficient;

(3) input one secondary low-resolution image Q after its filtering extraction feature, appoints a fritter q who gets characteristic image, calculates K the cluster centre C that obtains in q and the step (1) respectively ₁, C ₂... C _kBetween distance be: d ₁, d ₂... d _k, to d ₁, d ₂... d _kGet reciprocal and normalization, obtain K weight w ₁, w ₂... w _k

(4) utilize dictionary D _L1, D _L2... D _LkWith low-resolution image fritter q, find the solution dictionary D respectively _L1, D _L2... D _LkCorresponding Sparse Decomposition coefficient a ₁, a ₂... a _k, its solution formula is: q=D _L1A, i=1 wherein, 2....k;

(5) utilize dictionary D _H1, D _H2... D _HkWith coefficient a ₁, a ₂... a _k, find the solution high-definition picture fritter x respectively ₁, x ₂... x _k, its solution formula is x _i=D _Hia _i, i=1,2....k;

(6) utilize weight w ₁, W ₂... w _kTo x ₁, x ₂... x _kCarry out the weights addition and obtain the high-definition picture fritter:

$x=\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{w}_i{x}_i;$

(7) repeating step (3) is handled all input picture fritters successively to step (6), obtains initial reconstitution image Z, at last initial reconstitution image Z is carried out global optimization, obtains final reconstructed image Z ^*

The present invention has the following advantages compared with prior art:

The present invention extracts the various characteristic informations of training image owing to utilize KSVD algorithm training dictionary, thereby has reduced the number of dictionary atom, shortens the image reconstruction time, has improved reconstruct efficient; Simultaneously since the present invention when utilizing multitask learning algorithm reconstructed image, can learn mutually between a plurality of tasks, thereby improved the quality of reconstructed image; Because the present invention can be reconstructed various natural images, comprise plant, animal, people, buildings in addition, thus overcome tradition based on the algorithm of model for the not strong problem of various input picture compatibility.

Experiment showed, that the present invention is applicable to the super-resolution reconstruction of various natural images, after being reconstructed by this algorithm, it is relatively good that the detail textures information of reconstructed image all keeps.

Description of drawings

Fig. 1 is a general flow chart of the present invention;

Fig. 2 is that weights of the present invention calculate sub-process figure;

Fig. 3 is the plant training image that the present invention adopts;

Fig. 4 is the animal training image that the present invention adopts;

Fig. 5 is people's training image that the present invention adopts;

Fig. 6 is the building training image that the present invention adopts;

Fig. 7 is with the reconstruct design sketch of the present invention to test Lena image;

Fig. 8 is with the reconstruct design sketch of the present invention to test Pepper image;

Fig. 9 is with the reconstruct design sketch of the present invention to test b obcat image;

Figure 10 is with the reconstruct design sketch of the present invention to test Athens image;

Figure 11 is the effect contrast figure with the present invention and other the whole bag of tricks reconstructed image.

Specific implementation method

With reference to accompanying drawing 1, concrete steps of the present invention comprise:

Step 1. pair training image carries out pre-service and classification

1a) input training image carries out filtering to it and extracts feature, and the wave filter that is adopted is f ₁=[1,0,1],

f ₃=[1,0 ,-2,0,1],

The training image that is adopted such as Fig. 3, Fig. 4, Fig. 5 and shown in Figure 6;

1b) randomly draw about 100,000 pairs of image fritters, construct a matrix M, utilize the K-means algorithm that the image fritter in the matrix M is divided into 5 classes, but be not limited to 5 classes, obtain 5 couples of initial dictionary H ₁, H ₂, H ₃, H ₄, H ₅And L ₁, L ₂, L ₃, L ₄, L ₅And 5 cluster centre C ₁, C ₂, C ₃, C ₄, C ₅, but be not limited to 5 pairs of initial dictionaries and 5 cluster centres.

Step 2. utilize the KSVD algorithm to selected to initial dictionary H ₁, H ₂, H ₃, H ₄, H ₅And L ₁, L ₂, L ₃, L ₄, L ₅Train

This example is to selected 5 pairs of initial dictionary training, but is not limited to 5 pairs of initial dictionaries, also can be to the initial dictionary training of other number, and training step is as follows:

2a) from the KSVD algorithm, provide total optimization formula

Subject to

Wherein Y represents that the initial dictionary imported, D represent target training dictionary, and X represents the Sparse Decomposition coefficient;

2b) in total optimization formula Be out of shape and obtain:

${\left|\right|Y-\mathrm{DX}\left|\right|}_F^2={\left|\right|Y-\underset{j=1}{\overset{K}{\mathrm{\Σ}}}{d}_j{x}_T^j\left|\right|}_F^2={\left|\right|(Y-\underset{j\≠k}{\mathrm{\Σ}}{d}_j{x}_T^j)-d_k{x}_T^k\left|\right|}_F^2={\left|\right|E_k-d_k{x}_T^k\left|\right|}_F^2$

Wherein, Y is the initial dictionary of input, and D is target training dictionary and is initialized as a dictionary at random that X is the Sparse Decomposition matrix, d _jBe the j row atom of D, For the j of X is capable, N is total columns of D, d _kBe the k row atom of D,

For the k of X is capable, E _kFor not using the k row atom d of D _kCarry out the error matrix that the signal Sparse Decomposition is produced;

2c) to the formula after the distortion

Multiply by matrix Ω _k, obtain the target decomposition formula

Wherein

Ω _kSize be P ^*| ω _k|, P is the columns of the initial dictionary Y of input,

| ω _k| be ω _kThe mould value, and Ω _kAt (ω _k(j), j) locating is 1, and other place is 0 entirely, wherein 1≤j≤| ω _k|, ω _k(j) be ω _kThe j number;

2d) to the target decomposition formula In error matrix

Carrying out the SVD decomposition obtains

Wherein U is, V ^TBe right singular matrix, Δ is a singular value matrix;

2e) first row with left singular matrix U remove the more k row atom d of fresh target train word allusion quotation D _k

2f) repeating step 2c) to step 2e) all atoms among the D are upgraded processing, obtain 5 couples of new dictionary D _H1, D _H2, D _H3, D _H4, D _H5And D _L1, D _L2, D _L3, D _L4, D _L5

Step 3. is imported low-resolution image and is calculated the weights of each image fritter correspondence

With reference to Fig. 2, being implemented as follows of this step:

3a) input one secondary low-resolution image Q carries out filtering and feature extraction to it, obtains characteristic image, and the wave filter that is adopted is f ₁=[1,0,1], f ₃=[1,0 ,-2,0,1],

3b) appoint a fritter q who gets characteristic image, its size is 3*3, calculates 5 cluster centre C that obtain in q and the step (1) respectively ₁, C ₂, C ₃, C ₄, C ₅Between distance: d ₁, d ₂, d ₃, d ₄, d ₅, to d ₁, d ₂, d ₃, d ₄, d ₅Get inverse and obtain 1/d ₁, 1/d ₂, 1/d ₃, 1/d ₄, 1/d ₅, to 1/d ₁, 1/d ₂, 1/d ₃, 1/d ₄, 1/d ₅Carry out normalized and obtain 5 weight w ₁, w ₂, w ₃, w ₄, w ₅

Step 4. multitask image reconstruction

4a) utilize dictionary D _L1, D _L2, D _L3, D _L4, D _L5With low-resolution image fritter q, find the solution dictionary D respectively _L1, D _L2, D _L3, D _L4, D _L5Corresponding Sparse Decomposition coefficient a ₁, a ₂, a ₃, a ₄, a ₅, its solution formula is: q=D _L1A, i=1 wherein, 2,3,4,5;

4b) utilize dictionary D _H1, D _H2, D _H3, D _H4, D _H5With coefficient a ₁, a ₂, a ₃, a ₄, a ₅, find the solution high-definition picture fritter x respectively ₁, x ₂, x ₃, x ₄, x ₅, its solution formula is x ₁=D _H1a ₁, i=1,2,3,4,5;

4c) utilize weight w ₁, w ₂, w ₃, w ₄, w ₅To x ₁, x ₂, x ₃, x ₄, x ₅Carry out the weights addition and obtain the high-definition picture fritter:

4d) repeating step 4a) to step 4c), all input low-resolution image fritters are handled successively, obtain initial reconstitution image Z.

Step 5. couple initial reconstitution image Z carries out global optimization

By following formula initial reconstitution image Z is carried out global optimization, obtains final reconstructed image:

$Z^{*}=\arg \min {\left|\right|Z-Z_0\left|\right|}_F^2$ s.t.DHZ＝Q

Wherein, Z ₀Be original high resolution image, Z ^*Be final reconstructed image, DH is following operation, the low-resolution image of Q for importing adopted to initial reconstitution image Z.

Effect of the present invention can further specify by following experiment:

1) experiment condition

This experiment adopts standard testing image Lena, Pepper, Bobcat, Athens, Girl as experimental data, adopts software MATLAB 7.0 as emulation tool, and computer configuration is Intel Core2/1.8G/1G.

2) experiment content

2a) utilize Bicubic interpolation algorithm, MaYi algorithm and algorithm of the present invention respectively, all kinds of input test images be reconstructed:

At first, the Lena image is reconstructed, the result as shown in Figure 7, wherein Fig. 7 (a) be that Bicubic interpolation algorithm result, Fig. 7 (c) are result of the present invention for MaYi arithmetic result, Fig. 7 (d) for input low-resolution image, Fig. 7 (b);

Secondly, the Pepper image is reconstructed, the result as shown in Figure 8, wherein Fig. 8 (a) be that Bicubic interpolation algorithm result, Fig. 8 (c) are result of the present invention for MaYi arithmetic result, Fig. 8 (d) for input low-resolution image, Fig. 8 (b);

Once more, the Bobcat image is reconstructed, the result as shown in Figure 9, wherein Fig. 9 (a) be that Bicubic interpolation algorithm result, Fig. 9 (c) are result of the present invention for MaYi arithmetic result, Fig. 9 (d) for input low-resolution image, Fig. 9 (b);

At last, the Athens image is reconstructed, the result as shown in figure 10, wherein Figure 10 (a) be that Bicubic interpolation algorithm result, Figure 10 (c) are result of the present invention for MaYi arithmetic result, Figure 10 (d) for input low-resolution image, Figure 10 (b).

2b) utilize S.Dai algorithm, MaYi algorithm, single task algorithm and the present invention that test pattern Girl is reconstructed respectively, obtain result as shown in figure 11, wherein Figure 11 (a) is that MaYi arithmetic result, Figure 11 (d) are original high-definition picture for result, Figure 11 (e) of single task algorithm for result of the present invention, Figure 11 (f) for input low-resolution image, Figure 11 (b) S.Dai arithmetic result, Figure 11 (c).

3) interpretation

From Fig. 7, Fig. 8, Fig. 9, Figure 10 as can be seen, the present invention is better than other method on the visual effect of reconstructed image, it is relatively good that the grain details information of image all keeps, and, can obtain good reconstruct effect for various types of input pictures such as Lena, Pepper, Bobcat, Athens;

No matter be to compare as can be seen from Figure 11, still compare that the present invention is better than them on visual effect with nearest MaYi algorithm based on study with traditional S.Dai algorithm based on model.

Claims (3)

1. the multitask super-resolution image reconstruction method based on the study of KSVD dictionary comprises the steps:

(1) input training image carries out filtering to it and extracts feature, extracts matrix M of 100,000 pairs of little block structures of image again, is divided into the K class, obtains K to initial dictionary H ₁, H ₂... H _kAnd L ₁, L ₂... L _k, and K cluster centre C ₁, C ₂... C _k

(2) utilize the KSVD algorithm to separate formula:

Subject to

To K to initial dictionary H ₁, H ₂... H _kAnd L ₁, L ₂... L ₃Train, obtain K new dictionary D _H1, D _H2... D _HkAnd D _L1, D _L2... D _Lk,

Wherein, Y is the initial dictionary of input, and D is that target is trained dictionary and is initialized as a dictionary at random, and X is the Sparse Decomposition matrix,

Be any i row, || X _i|| 0 is X _i0 norm,

For finding the solution 2 norms of Y-DX, T ₀Be the degree of rarefication control coefrficient;

(3) input one secondary low-resolution image Q after its filtering extraction feature, appoints a fritter q who gets characteristic image, calculates K the cluster centre C that obtains in q and the step (1) respectively ₁, C ₂... C _kBetween distance be: d ₁, d ₂... d _k, to d ₁, d ₂... d _kGet reciprocal and normalization, obtain K weight w ₁, w ₂... w _k

(4) utilize dictionary D _L1, D _L2... D _LkWith low-resolution image fritter q, find the solution dictionary D respectively _L1, D _L2... D _LkCorresponding Sparse Decomposition coefficient a ₁, a ₂... a _k, its solution formula is: q=D _LiA, i=1 wherein, 2....k;

(5) utilize dictionary D _H1, D _H2... D _HkWith coefficient a ₁, a ₂... a _k, find the solution high-definition picture fritter x respectively ₁, x ₂... x _k, its solution formula is x _i=D _Hia ₁, i=1,2...k;

(6) utilize weight w ₁, w ₂... w _kTo x ₁, x ₂... x _kCarry out the weights addition and obtain the high-definition picture fritter:

(7) repeating step (3) is handled all input picture fritters successively to step (6), obtains initial reconstitution image Z, at last initial reconstitution image Z is carried out global optimization, obtains final reconstructed image Z ^*

2. the multitask super-resolution image reconstruction method based on KSVD dictionary study according to claim 1, wherein step (2) described to K to initial dictionary H ₁, H ₂... H _kAnd L ₁, L ₂... L _kTrain, carry out as follows:

2a) to total optimization formula:

Subject to In

Be out of shape:

${\left|\right|Y-\mathrm{DX}\left|\right|}_F^2={\left|\right|Y-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{d}_j{x}_T^j\left|\right|}_F^2={\left|\right|(Y-\underset{j\≠k}{\mathrm{\Σ}}{d}_j{x}_T^j)-d_k{x}_T^k\left|\right|}_F^2={\left|\right|E_k-d_k{x}_T^k\left|\right|}_F^2$

Wherein, Y is the initial dictionary of input, and D is target training dictionary and is initialized as a dictionary at random that X is the Sparse Decomposition matrix, d _jBe the j row atom of D,

For the j of X is capable, N is total columns of D, d _kBe the k row atom of D,

For the k of X is capable, E _kFor not using the k row atom d of D _kCarry out the error matrix that the signal Sparse Decomposition is produced;

2b) to the formula after the distortion

Multiply by matrix Ω _k, obtain the target decomposition formula

Wherein

Ω _kSize be P ^*| ω _k|, P is the columns of the initial dictionary Y of input,

| ω _k| be ω _kThe mould value, and Ω _kAt (ω _k(j), j) locating is 1, and other place is 0 entirely, wherein 1≤j≤| ω _k|, ω _k(j) be ω _kThe j number;

2c) to the target decomposition formula

In error matrix

Carrying out the SVD decomposition obtains Wherein U is a left singular matrix, V ^TBe right singular matrix, Δ is a singular value matrix;

2d) be listed as the k row atom d that removes to upgrade D with first of left singular matrix U _k, in like manner successively the k value is upgraded processing from 1 to N to all atoms the D, obtain K to new dictionary D _H1, D _H2... D _HkAnd D _L1, D _L2... D _Lk

3. the multitask super-resolution image reconstruction method based on the study of KSVD dictionary according to claim 1, wherein step (7) is described carries out global optimization to initial reconstitution image Z, is undertaken by following formula:

$Z^{*}=\arg \min {\left|\right|Z-Z_0\left|\right|}_F^2$ s.t.DHZ＝Q

Wherein, Z ₀Be original high resolution image, Z ^*Be final reconstructed image, DH is the down-sampling operation to initial reconstitution image Z, and Q is the low-resolution image of input.

Images