CN105976410A - Method for rapid super-resolution reconstruction of single image based on non-linear prediction sparse coding - Google Patents

Abstract

The invention discloses a method for super-resolution reconstruction of an image based on non-linear prediction sparse coding. In a training process, a non-linear predictive coding item is superposed based on criterion function of a classic sparse coding algorithm, a unique optimization strategy is designed to minimize the object function. In the reconstruction process, only one non-linear iteration step is employed for input low resolution image blocks and a low resolution dictionary obtained through training in advance to directly approach sparse coding required, and solving sparse expression coefficient for each small image block is avoided. Experiment results show that the method, compared with a classic image super-resolution algorithm based on sparse coding, ensures that the reconstruction result has full competitiveness and substantially reduces experiment time.

Description

A kind of single image super-resolution based on nonlinear prediction sparse coding is quickly rebuild Method

Technical field

Invention relates to technical field of image information processing, particularly relates to a kind of single image based on prediction sparse coding Quick super-resolution method for reconstructing.

Background technology

Image super-resolution problem is always a basic problem of digital image processing field, is also that current computer regards One of hot issue being concerned by people in feel field.Image super-resolution refers to by width low-resolution image or an image sequence Recover the process of high-definition picture.Height due to image resolution ratio is often demonstrated by the height of picture quality, and a width High-definition picture is provided that more image information, and image recognition, image understanding play very important effect, therefore from Low-resolution image recovers high-definition picture and is necessary.Owing to having limitation on hardware device, and use Algorithm realizes somewhat complex design operation and the huge consuming cost that image super-resolution technology can be alleviated in hardware technology, thus No matter in theory or reality, suffer from wide research and application prospect.At present, image super-resolution technology is visited in remote sensing The fields such as survey, medical imaging, public safety, man-machine interaction suffer from being widely applied.

Owing to the low-resolution image under same scene loses more high-frequency information with thin than high-definition picture Joint information, so the high-definition picture reconstructed under same scene by low-resolution image is always an ill-posed Problem.At present, image super-resolution method is broadly divided into three major types: based on interpolation, based on rebuilding and method based on study.Base In the Super-Resolution of Images Based of interpolation by several low-resolution images under Same Scene are mapped to reference picture to obtain Obtain corresponding sub-pix information, then by these sub-pixs calculate the sample prime information obtaining sampled point, its essence is adopted exactly The pixel of sampling point is substituted by the pixel of surrounding and obtains.Algorithm based on interpolation has the spy that realization is simple, computation complexity is low Point, but interpolation method can not increase the information of image, and nature can not model the details that image is abundant exactly.Based on reconstruction Super-resolution method be divided into two steps: registrate and rebuild.The purpose of registration is to obtain low-resolution image and with reference to low point The relative displacement in sub-pix rank of the resolution image, the purpose of reconstruction is exactly on the basis of making full use of priori, excellent Change and solve target image.Based in the method rebuild, estimation, registration between frame serve vital effect, but It is extremely difficult for going for the other registration information of accurate sub-pixel, so the result rebuild also can be not fully up to expectations. Additionally, based on the method rebuild to noise also inadequate robust.In recent years, the super-resolution method of main flow is all based on the side of study Method.Outside high-resolution and low-resolution training set of images, by machine learning knowledge, is learnt a mapping by method based on study Relation, then by study to mapping relations and low-resolution image (or image block) reconstruct high-definition picture (figure As block).

Image super-resolution rebuilding algorithm based on rarefaction representation is a kind of the most reasonable method based on study. Full resolution pricture and low-resolution image are the most spatially divided into one and are by image super-resolution model based on rarefaction representation The sub-block that row are corresponding, uses the strategy of joint training to be high-definition picture block and low-resolution image block trains high-resolution Dictionary and low-resolution dictionary, when training dictionary, it shares same based on the high-resolution and low-resolution image block on correspondence position The hypothesis of sparse coding.When rebuilding, for the low resolution input block of input, first obtain one for each input block Good sparse coding, then utilizes the sparse coding asked and high-resolution dictionary to reconstruct high-definition picture block.Due to base Method in sparse coding needs each image block solves a sparse coefficient problem at test phase, if piece image The block number got is too much, and the computing cost of whole process is the biggest.Use the mutation method of sparse coding herein: predict dilute Dredge coding, single image is carried out super-resolution rebuilding work, test result indicate that, with classical algorithm based on sparse coding Comparing, this algorithm greatly reduces the time that phase of regeneration spends while reconstructed results has sufficient competition power.

Summary of the invention

Goal of the invention: the present invention, compared with classical algorithm based on sparse coding, is ensureing that experimental result has fully On the premise of competitiveness, it is substantially reduced the time required for reconstruction.

1) technical solution of the present invention is as follows:

Classical generally requires iterative based on sparse coding process, therefore the most more under big data environment.Make A mutation for sparse coding, it was predicted that sparse coding is using a non-iterative approximation step to replace iterative coding process, Eliminate test phase and solve the computing cost of coding, significantly reduce amount of calculation.Prediction sparse coding is at image with regard Object identification achieves well application frequently.In this programme, the training block of given sampling is to { X_h,X_l, X here_hIt it is high-resolution Rate image block, X_lIt it is the low-resolution image block obtained through corresponding high-definition picture block down-sampling.Consider as follows herein Criterion function:

In above formula, i represents the i-th row of dictionary, A code sparse coding.Wherein:

In above formula, N and M is the dimension that high-resolution and low-resolution image block is expressed with column vector form respectively, and W, B are non-linear The parameter of forecast model.Compared with classical sparse coding image super-resolution training criterion, herein on the basis of it respectively Add high-resolution and low-resolution non-linear decoded item, so that the dictionary that training is out has the merit of coded linear prediction Energy.

2) criterion function of this programme is a convex problem, uses herein and fixes some parameters, optimizes one of them parameter Method alternative optimization, optimize following problem herein:

Whole optimization process is as follows:

2-1) make t=0.Use gaussian random matrix to dictionary D_lAnd D_hInitialization processes, and by each row of dictionary All office's normalization, random initializtion W and B.

2-2) fixing B^(t)、W^(t)、WithUse ADMM method, update A^(t):

2-3) fixing A^(t),WithUse gradient descent method, renewal W and B:

2-4) fixing A^(t), W, B, updateWith

This step utilizes associating dictionary training thinking to be optimized.

2-5) make t:=t+1, iteration 2) and 4), until convergence.

3), when utilizing nonlinear prediction sparse coding to carry out image super-resolution rebuilding, need not be each again at phase of regeneration Individual image block solves a rarefaction representation problem, if the image of input is Y, the image block of each input is y.Each image block Sparse coding can approach as follows:

A=tanh (Wy+B) (4)

It can be seen that this process relates only to the computing that matrix is most basic, ask relative to solving the sparse of a belt restraining item For topic, computing cost greatly reduces.It is as follows that high-resolution block rebuilds expression formula:

x_h=D_hA=D_h(tanh(Wy+B)) (5)

Shown below is the super-resolution flow process of nonlinear prediction sparse coding.

Super-Resolution of Images Based based on nonlinear prediction sparse coding:

3-1) input: the dictionary D that pre-training obtains_hAnd D_l, a width low-resolution image Y, parameter W and B.

3-2) For is for each 5*5 fritter y in Y, from left to right, from top to bottom, ensures 4 in each direction

Pixel is overlapping:

1. calculating the average pixel value m in each piece, the pixel value of each piece deducts average；

2. high-resolution block x is calculated_h=D_h(tanh(Wy+B))；

3. x is calculated_h+ m, inserts high-definition picture X₀。

End

3-3) in solution space, find and X₀The image mated most:

H is a fuzzy operator, and S is a down-sampling operator, and c is balance parameters.

3-4) output: super-resolution image X^*。

Table 1 below lists the evaluation index PSNR value that super-resolution result is conventional, and table 2 is the comparison of experimental period.Can To find out, single from PSNR value, on the whole algorithm is compared quite with based on Corresponding Sparse Algorithm herein, but during reconstruction used herein Between greatly reduce.It addition, the image varied in size input herein is tested, its reconstruction time is as shown in table 3, can see Go out, especially when input picture more significantly time, algorithm is reconstructed results is competitive while herein, and advantage in time is more Add substantially.

PSNR value contrast unit: the dB of 1 three kinds of methods of table

Image	Interpolation method	Corresponding Sparse Algorithm	Algorithm herein
				Girl	34.856	36.657	36.574
Baby	32.514	34.233	34.139

Table 2 reconstruction time contrasts, unit: second

Image	Corresponding Sparse Algorithm	Algorithm herein
			Girl	101.542	12.13
Baby	100.478	12.34

Table 3 varies in size unit of time used by image: the second

Accompanying drawing explanation

Fig. 1 is the present invention program logic diagram.

Beneficial effect

The present invention is while ensure that experimental index has sufficient competition power, needed for greatly reducing whole process of reconstruction The time wanted.Especially when the image of input is bigger, the advantage in present invention face in time becomes apparent from.

Detailed description of the invention

1 couple of present invention is described in further detail below in conjunction with the accompanying drawings.

Use super-resolution to test 69 common images herein and be about 100,000 as training set, the image block of whole training Individual block.Training process is as follows:

1) make t=0, use gaussian random matrix to dictionary D_lAnd D_hInitialization processes, and by each row of dictionary Office's normalization, random initializtion W and B；

2) fixingW^(t)、B^(t), use ADMM method, update A^(t):

3) fixing A^(t),WithUse gradient descent method, renewal W and B:

4) fixing A^(t), W^(t),B^(t), updateWith

This step utilizes associating dictionary training thinking to be optimized；

5) t:=t+1 is made, iteration 2) to 4), until convergence；

After training terminates, preserve D_h、D_l、W、B.Obtain dictionary to and model parameter after, low-resolution image is carried out Reconstruction, rebuilds flow process as follows:

1) input: the dictionary D that pre-training obtains_hAnd D_l, a width low-resolution image Y, parameter W and B.

2) For is for each 5*5 fritter y in Y, from left to right, from top to bottom, ensures 4 pictures in each direction

Element is overlapping:

1. calculating the average pixel value m in each piece, the pixel value of each piece deducts average；

2. high-resolution block x is calculated_h=D_h(tanh(Wy+B))；

3. x is calculated_h+ m, inserts high-definition picture X₀。

End

3) in solution space, find and X₀The image mated most:

H is a fuzzy operator, and S is a down-sampling operator, and c is balance parameters.

4) output: super-resolution image X^*。

Accompanying drawing 1 gives the present invention recovery process figure at each image block of phase of regeneration.For input each low point Resolution image block, first obtains the expression coefficient in each comfortable low-resolution dictionary, then profit for each low-resolution image block Represent that coefficient and high-resolution dictionary reconstruct high-definition picture block by this group.

The allocation of computer of the present embodiment experiment is: the operating system of 64, the internal memory of 16GB, Intel process Device, software runtime environment is MATLAB R2011a version, and experiment input picture is 128*128, amplifies twice.

Claims (1)

1. an image super-resolution rebuilding method based on nonlinear prediction sparse coding, it is characterised in that: phase of regeneration is not Need each image block is solved a rarefaction representation problem, and only with the low-resolution image block inputted and training in advance Obtain low-resolution dictionary and directly dope sparse coding, employing following steps:

1) the training block of sampling given below is to { X_h,X_l, X here_hIt is high-definition picture block, X_lIt is through corresponding high-resolution The low-resolution image block that rate image block down-sampling obtains.Consider following criterion function herein:

$\begin{array}{c}\min f(D_h,D_l,A)\\ \begin{array}{cc}s.t.& \left|\right|D_{h,i}|{|}_2^2\≤1,|\left|D_{l,i}\right|{|}_2^2\≤1\end{array}\end{array}---\left(1\right)$

In above formula, i represents the i-th row of dictionary, A code sparse coding；Wherein:

$\begin{array}{c}f(D_h,D_l,A)=\frac{1}{N}\left|\right|X_h-D_{h}A|{|}_2^2+\frac{1}{N}||\tanh ({\mathrm{WX}}_h+B)-A|{|}_2^2\\ +\frac{1}{M}\left|\right|X_l-D_{l}A|{|}_2^2+\frac{1}{M}||\tanh \left({\mathrm{WX}}_l+B\right)-A|{|}_2^2+\mathrm{\λ}\left(\frac{1}{N}+\frac{1}{M}\right)\left|\right|A|{|}_1\end{array}---\left(2\right)$

In above formula, N and M is the dimension that high-resolution and low-resolution image block is expressed with column vector form respectively, and W, B are nonlinear predictions The parameter of model；

2) following problem is optimized:

$\begin{array}{c}\underset{\left\{D_h,D_l,A,W,B\right\}}{\min }=\frac{1}{N}\left|\right|X_h-D_{h}A|{|}_2^2+\frac{1}{N}||\tanh \left({\mathrm{WX}}_h+B\right)-A|{|}_2^2+\frac{1}{M}\left|\right|X_l-D_{l}A|{|}_2^2+\frac{1}{M}||\tanh \left({\mathrm{WX}}_l+B\right)-A|{|}_2^2+\mathrm{\λ}\left(\frac{1}{N}+\frac{1}{M}\right)\left|\right|A|{|}_1\\ \begin{array}{cc}s.t.& \left|\right|D_{h,i}|{|}_2^2\≤1,|\left|D_{l,i}\right|{|}_2^2\≤1\end{array}\end{array}---\left(3\right)$

Whole optimization process is as follows:

2-1) make t=0, use gaussian random matrix to dictionary D_lAnd D_hInitialization processes, and is made by each row of dictionary Unit-normalization, random initializtion W and B；

2-2) fixingUse ADMM method, update A^(t):

$A^{\left(t\right)}=\underset{A}{\mathrm{argmin}}f(B^{\left(t\right)},W^{\left(t\right)},D_h^{\left(t\right)},D_l^{\left(t\right)},A)$

2-3) fixingAnd A^(t)Use gradient descent method, renewal W and B:

$W^{(t+1)},B^{(t+1)}=\underset{W,B}{\mathrm{argmin}}f(D_h^{\left(t\right)},D_l^{\left(t\right)},A^{\left(t\right)},W^{\left(t\right)},B^{\left(t\right)})$

2-4) fixing A^(t), W^(t),B^(t), updateWith

$\begin{array}{c}{D}_h^{(t+1)},D_l^{(t+1)}=\underset{D_h,D_l}{\mathrm{argmin}}f(A^{\left(t\right)},W^{(t+1)},B^{(t+1)},D_h^{\left(t\right)},D_l^{\left(t\right)})\\ \begin{array}{cc}s.t.& \left|\right|D_{h,i}|{|}_2^2\≤1,|\left|D_{l,i}\right|{|}_2^2\≤1\end{array}\end{array}$

This step utilizes associating dictionary training thinking to be optimized；

2-5) make t:=t+1, iteration 2-2) arrive 2-4), until convergence；

3), when utilizing nonlinear prediction sparse coding to carry out image super-resolution rebuilding, need not be each figure again at phase of regeneration As block solves a rarefaction representation problem, if the image of input is Y, the image block of each input is y, each image block sparse Coding can approach as follows:

A=tanh (Wy+B) (4)

It is as follows that high-resolution block rebuilds expression formula:

x_h=D_hA=D_h(tanh(Wy+B)) (5)

Shown below is the super-resolution flow process of nonlinear prediction sparse coding；

Step 3) in Super-Resolution of Images Based based on nonlinear prediction sparse coding:

3-1) input: the dictionary D that pre-training obtains_hAnd D_l, a width low-resolution image Y, parameter W and B；

3-2) For is for each 5*5 fritter y in Y, from left to right, from top to bottom, ensures 4 in each direction；

Individual pixel is overlapping:

1. calculating the average pixel value m in each piece, the pixel value of each piece deducts average；

2. high-resolution block x is calculated_h=D_h(tanh(Wy+B))；

3. x is calculated_h+ m, inserts high-definition picture X₀；

End

3-3) in solution space, find and X₀The image mated most:

$X^{*}=\underset{X}{\mathrm{argmin}}\left|\right|SHX-Y|{|}_2^2+c||X-X_0|{|}_2^2$

H is a fuzzy operator, and S is a down-sampling operator, and c is balance parameters；

3-4) output: super-resolution image X^*。