CN105844590A - Image super-resolution reconstruction method and system based on sparse representation - Google Patents

Abstract

The invention discloses an image super-resolution reconstruction method based on sparse representation and a system. The method comprises following steps: a complete high-low resolution dictionary pair is obtained by means of an image training library; bicubic amplification is performed on a low resolution image to be reconstructed to obtain an initial image of the super-resolution algorithm; the first order and second order gradient features of the initial image are extracted; overlapping partition is performed on the extracted features and sparse representation of the low frequency image blocks is obtained by means of the low resolution dictionary; the sparse representation coefficient of the obtained low resolution image blocks is approximately equal to the sparse representation coefficient of the high resolution image blocks to be solved in the high resolution dictionary and the initial estimate of a corresponding high resolution image is estimated by means of the sparse representation coefficient; reconstruction error of the high resolution image is reduced by means of a back projection filter.

Description

Image super-resolution reconstruction method and system based on sparse representation

Technical Field

The invention relates to image processing, in particular to a method and a system for reconstructing super-resolution images based on sparse representation.

Background

In the image super-resolution reconstruction algorithm, firstly, the relationship between high-resolution images is established. Generally speaking, in a single-frame image super-resolution image reconstruction algorithm, geometric distortion can be ignored, so that a degradation phenomenon in an image acquisition process can be simulated as an original high-resolution image through a series of processes such as optical blurring, down-sampling and noise interference, and the existing single-frame image super-resolution algorithm mainly comprises an image super-resolution algorithm based on learning and an image super-resolution algorithm based on interpolation reconstruction.

The super-resolution reconstruction technology based on learning firstly utilizes a high-resolution image training library and an image degradation model to obtain an image training set with high and low resolutions, then obtains a mapping relation between the images with the high and low resolutions through a certain learning algorithm, and finally utilizes an optimization algorithm to optimize the images with the low resolutions to be reconstructed, so as to estimate corresponding images with the high resolutions.

Classical interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation methods. Of the three interpolation methods, the calculation complexity of the nearest neighbor and bilinear interpolation is relatively low, and the requirements of real-time operation can be met in most occasions, so that the three interpolation methods are widely applied. However, undesirably, the two methods have little contribution to the improvement of the resolution, and especially, enough high-frequency information cannot be obtained, so that the recovered image still has large distortion. Of the three methods, the bicubic interpolation is most suitable for enhancing the detail information of the high-frequency part, and the disadvantage is that the calculation complexity is relatively large.

Although bicubic interpolation provides more detail information, it still has the same problems as the former two methods, especially when high resolution is required, and it cannot provide enough detail information. Interpolation methods either exhibit aliasing or edge blurring. The nearest neighbor interpolation edge step sawtooth image distortion is very obvious, the bilinear interpolation edge step sawtooth image distortion is very obvious, certain edge blurring exists, the bicubic interpolation edge step distortion is weak, and the edge blurring is very obvious.

The traditional interpolation reconstruction method is simple in calculation, easy to implement and wide in application. However, in the case of high magnification, the reconstructed high resolution image is often too smooth at the edges and lacks sufficient detail information due to the large loss of original information of the image.

The learning-based super-resolution algorithm utilizes the training library to learn the prior information of the image before the high-resolution image reconstruction is carried out, so that more image detail information can be obtained and better image reconstruction quality can be obtained. However, in the super-resolution algorithm based on learning, the number of neighbors is not properly selected, so that the details and edges of the reconstructed high-resolution image are blurred, and the reconstruction quality of the image is affected.

Disclosure of Invention

Aiming at the problems, the invention provides an image super-resolution reconstruction method and system based on sparse representation.

The technical scheme adopted by the invention for solving the technical problems is as follows:

the image super-resolution reconstruction method based on sparse representation comprises the following steps:

s1, obtaining a complete high-low resolution dictionary pair by using the image training library, wherein the complete high-low resolution dictionary pair comprises a low resolution dictionary Dl and a high resolution dictionary Dh;

s2, carrying out bicubic amplification on the low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, extracting first-order and second-order gradient features of the initial image, carrying out overlapped blocking on the extracted features to obtain low-resolution image blocks, combining the features corresponding to each low-resolution image block into a vector, setting the vector as y, and then obtaining a sparse representation a of y by using a low-resolution dictionary Dl^*：

$a^{*}=\arg \min \left|\right|y-Dla|{|}_2^2+\mathrm{\β}||PDha-w|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1=\arg \min ||\tilde{y}-\tilde{D}a|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1$

Wherein,p is an overlapping area, w is a reconstructed pixel value in the overlapping area, and β is a constant used for adjusting the influence of the reconstructed pixel value on the reconstruction of the block to be reconstructed;

s3, the obtained sparse representation coefficient of the low-resolution image block is approximately equal to the sparse representation coefficient of the high-resolution image block to be solved in the high-resolution dictionary Dh, and the corresponding high-resolution image block is estimated by utilizing the sparse representation coefficient

S4, reducing the reconstruction error of the high-resolution image by using a back projection filter to obtain a filtered high-resolution image block, wherein the back projection filter is expressed asWherein S represents a sampling factor, H represents a blurring operator, c is a constant, and X₀An initial estimation value of the high-resolution image; the gradient descent method is adopted to solve:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X)]

where Xt represents the estimate of the high resolution image after t iterations and v represents the step size in the gradient descent method.

In the method of the present invention, step S1 specifically includes: extracting high and low resolution image block information in an image training library, and training the obtained high and low resolution image blocks by using a K-singular value decomposition algorithm to obtain sparsely represented high and low resolution dictionary pairs.

In the method, the extracted first-order and second-order gradient features are HOG direction gradient histogram features.

The invention also provides an image super-resolution reconstruction system based on sparse representation, which comprises:

the training module is used for obtaining a complete high-low resolution dictionary pair by utilizing the image training library, wherein the complete high-low resolution dictionary pair comprises a low resolution dictionary Dl and a high resolution dictionary Dh;

the sparse representation module is used for carrying out bicubic amplification on a low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, extracting first-order and second-order gradient features of the initial image, carrying out overlapped blocking on the extracted features, combining the features corresponding to each image block into a vector, setting the vector as y, and then obtaining a sparse representation α of y by using a low-resolution dictionary Dl^*：

$a^{*}=\arg \min \left|\right|y-Dla|{|}_2^2+\mathrm{\β}||PDha-w|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1=\arg \min ||\tilde{y}-\tilde{D}a|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1$

Wherein,p is an overlapping area, w is a reconstructed pixel value in the overlapping area, and β is a constant used for adjusting the influence of the reconstructed pixel value on the reconstruction of the block to be reconstructed;

a high resolution image reconstruction module, configured to approximately equal the obtained sparse representation coefficient of the low resolution dictionary Dl to the sparse representation coefficient of the high resolution image block to be solved in the high resolution dictionary Dh, and estimate a corresponding high resolution image block using the sparse representation coefficient

An error elimination module for reducing the reconstruction error of the high resolution image by using a backward projection filter to obtain a filtered high resolution image block, wherein the backward projection filter is expressed asWherein S represents a sampling factor, H represents a blurring operator, X₀For the initial estimation value of the high-resolution image, c is a constant, and a gradient descent method is adopted to solve the following problems:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X)]

where Xt represents the estimate of the high resolution image after t iterations and v represents the step size in the gradient descent method.

In the system of the present invention, the training module is specifically configured to: extracting high and low resolution image block information in an image training library, and training the obtained high and low resolution image blocks by using a K-singular value decomposition algorithm to obtain sparsely represented high and low resolution dictionary pairs.

In the system, the extracted first-order and second-order gradient features are HOG direction gradient histogram features.

The super-resolution reconstruction method has the advantages that the parameter β is introduced in the super-resolution reconstruction estimation process and used for adjusting the influence of the reconstructed pixel value on the reconstruction of a block to be reconstructed, the introduction of the parameter β, namely the introduction of uncertainty can prevent the expansion of estimation errors among different variables, the high-resolution image and unknown parameters in an algorithm are modeled under a frame, the unknown variables are jointly estimated, and the super-parameter X is calculated and analyzed₀The value of (2) improves the stability to noise without parameter adjustment.

Drawings

The invention will be further described with reference to the accompanying drawings and examples, in which:

FIG. 1 is a flow chart of a super-resolution image reconstruction method based on sparse representation according to an embodiment of the present invention;

FIG. 2 is a dictionary training process diagram of an image super-resolution reconstruction algorithm based on sparse representation according to an embodiment of the present invention;

fig. 3 is a diagrammatic view of image reconstruction in accordance with an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The image super-resolution reconstruction algorithm based on sparse representation comprises two stages of overcomplete dictionary construction and high-resolution image reconstruction. As shown in fig. 1, the method specifically comprises the following steps:

s1, obtaining a complete high-low resolution dictionary pair by using the image training library, wherein the complete high-low resolution dictionary pair comprises a low resolution dictionary Dl and a high resolution dictionary Dh;

s2, carrying out bicubic amplification on the low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, extracting first-order and second-order gradient features of the initial image, carrying out overlapping blocking on the extracted features to obtain low-resolution image blocks, combining the features corresponding to each low-resolution image block into a vector, and obtaining sparse representation of y by using a low-resolution dictionary Dl;

s3, enabling the obtained sparse representation coefficient of the low-resolution image block to be approximately equal to the sparse representation coefficient of the high-resolution image block to be solved in the high-resolution dictionary, and estimating the initial estimation value of the corresponding high-resolution image by utilizing the sparse representation coefficient;

and S4, reducing the reconstruction error of the high-resolution image by using a back projection filter.

The solution of the sparse representation coefficient is involved in both the dictionary construction and the image reconstruction process, and therefore, the sparse representation theoretical model of the image, the overcomplete dictionary construction and the high-resolution image reconstruction method will be described in detail below.

(1) Sparse representation theoretical model of image

Sparse representation of an image means that the image or image signal can be approximated by a linear combination of a suitable series of atoms in an overcomplete atom library. This atomic pool is overcomplete because the number of atoms used in the linear combination is much less than the total number of atoms in the atomic pool. The sparse representation principle of the image is as follows:

given an image signal Y and an overcomplete dictionary D, Y may be represented by a suitable series of baseline combinations in D. The mathematical relationship can be expressed as:

min||α||₀s.t.y＝Dα

wherein α is a sparse representation coefficient, | | · | | non-volatile memory₀Represents the norm of 0, because the expression is used for solving the problem of non-calculation₀Is a difficult problem of NP, adopts | | · |. non woven phosphor in the practical calculation₁The solving process of (a) is converted into a convex optimization problem:

$a^{*}=\arg \min \left|\right|y-D\mathrm{\α}|{|}_2^2+\mathrm{\λ}|\left|\mathrm{\α}\right|{|}_1$

where λ is the regularization parameter (i.e., the linear parameter value of a x and a).

(2) Overcomplete dictionary construction

The first step of the image super-resolution reconstruction algorithm based on sparse representation is to use an image training library to obtain an over-complete sparse representation dictionary. The overcomplete dictionary training is to obtain sparse representation high and low resolution dictionary pairs by extracting high and low resolution image block information in an image training library and training the obtained high and low resolution image blocks by using a K-singular value decomposition algorithm.

Fig. 2 shows a specific process of dictionary training in the image super-resolution reconstruction algorithm based on sparse representation. The method specifically comprises the following steps:

first, an image in a low-resolution image set is divided into image blocks of N × N sizes, and features are extracted. The specific method is to adopt 4 one-dimensional filters:

f₁＝[-1,0,1]

f₃＝[1,0,-2,0,1]

where T denotes transposition. And taking the convolution result of the low-resolution image and the four filter operators as the characteristic of the low-resolution image. The four feature blocks obtained by blocking and the corresponding high-resolution image blocks are combined to be used as the input of a dictionary training algorithm, and the method takes the neighborhood information of the low-resolution image into consideration, so that the compatibility among the reconstructed high-resolution image blocks is improved.

And training a low-resolution dictionary. And (4) solving by using a sparse K-SVD algorithm according to the result of the step (i) to obtain a low-resolution dictionary Dl.

And thirdly, interpolating the low-resolution dictionary Dl to obtain an interpolation image set Dh with the same size as the high-resolution image set of the previous stage.

The technology for performing dictionary training by using the image super-resolution reconstruction algorithm based on sparse representation comprises a training sample extraction method and a K-SVD algorithm for performing dictionary training.

a. Extraction of training samples

In the image super-resolution reconstruction algorithm based on sparse representation, the processing of the image is performed in blocks, training samples are randomly extracted from high-resolution images in an image training library, and the number of the training samples is usually far larger than the number of atoms in a dictionary. In order to obtain a dictionary with wide applicability, as many natural high-resolution images as possible should be selected as an image training library, and the image information in the image training library is as many as possible. In a specific dictionary training process, a low-resolution image is degraded from a high-resolution image, and multiple features of the low-resolution image are often required to be extracted, and the features are partitioned into blocks and form a vector with corresponding high-resolution image blocks to perform dictionary training. Since the first and second order gradients of an image can effectively express the features of the image and the extraction algorithm is simple, they are commonly used to express the features of the image. The first-order and second-order gradients of an image are Histogram of Oriented Gradient (HOG) features, which are feature descriptors used for object detection in computer vision and image processing. The first and second order gradient filter operators of an image can be expressed as:

f₁＝[-1,0,1]，f₂＝f₁ ^T

f₃＝[1,0,-2,0,1]，f₄＝f₃ ^T

where T denotes transposition. And taking the convolution result of the low-resolution image and the four filter operators as the characteristic of the low-resolution image. And then combining the four feature blocks obtained by blocking with the corresponding high-resolution image blocks to be used as the input of a dictionary training algorithm. The method considers neighborhood information of the low-resolution image, thereby being beneficial to improving the compatibility between reconstructed high-resolution image blocks.

b.K-SVD dictionary training method

The K-SVD algorithm is also called as a generalized K-means clustering algorithm and is greatly connected with the K-means clustering algorithm, when each signal in the K-SVD algorithm is only approximately represented by one atom, the K-SVD algorithm is degraded into the K-means clustering algorithm, the main idea is to solve the continuous alternation of the sparse representation of an input sample in a current dictionary and the updating process of the dictionary, update each column of the dictionary according to the sparse representation result, assume that N training samples are output, and use D ∈ R to update each column of the dictionary according to the sparse representation result^n*kDenotes an overcomplete dictionary, Y ═ Y_i∈RⁿI 1, 2...... N } represents a training sample set, and a ═ α ·_i∈R^kAnd i ═ 1, 2.... N } represents a sparse representation coefficient set of training samples, the K-SVD dictionary training algorithm can be equivalent to a solving equation

$\begin{array}{ccc}\underset{D.A}{\min }\left\{\right||Y-DA|{|}_F^2\}& s.t.& \∀i,\left|\right|a_i|{|}_0\≤T_0\end{array}$

Where T is an upper limit value of the number of non-zeros in the sparse representation coefficient.

(3) Image reconstruction process

See figure 3 for details. After dictionary training, the image super-resolution reconstruction algorithm based on sparse representation carries out sparse representation and reconstruction on a known low-resolution image in a trained high-low resolution dictionary, and therefore an expected high-resolution image is estimated. And (3) a high-resolution image reconstruction process based on the image super-resolution reconstruction algorithm of sparse representation.

Firstly, carrying out bicubic amplification on a low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, then extracting first-order and second-order gradient features of the initial image, carrying out overlapped blocking on the features, combining the features corresponding to each image block into a vector, setting the vector as y, and then obtaining a sparse representation system alpha by utilizing y obtained from a low-resolution dictionary Dl.

$a^{*}=\arg \min \left|\right|y-Dl\mathrm{\α}|{|}_2^2+\mathrm{\β}||PDh\mathrm{\α}-w|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1=\arg \min ||\tilde{y}-\tilde{D}\mathrm{\α}|{|}_2^2+\mathrm{\λ}\left|\right|\mathrm{\α}|{|}_1$

Wherein,

$\tilde{D}=\left[\begin{array}{c}{D}_l\\ {\mathrm{PD}}_h\end{array}\right],\tilde{y}=\left[\begin{array}{c}y\\ \mathrm{\β}w\end{array}\right]$

p is the overlap region, w is the reconstructed pixel value in the overlap region, β is a constant used to adjust the reconstructed pixel value's effect on the reconstruction of the block to be reconstructed β is determined by using the characteristics of the picture itself Represents the square of 2 norm, | ·| non-woven phosphor₁Representing a 1 norm. λ is a and a linear parameter value.

Then, according to the characteristic that the high-resolution image blocks and the low-resolution image blocks have similar geometric manifold in local parts, the high-resolution image blocks and the low-resolution image blocks can be considered to be consistent to sparse representation coefficients in a training set, so that the obtained sparse representation coefficients of the low-resolution image blocks are approximately equal to the sparse representation coefficients of the high-resolution image blocks to be solved in a high-resolution dictionary, and the corresponding high-resolution image blocks are estimated by utilizing the sparse representation coefficients

$\tilde{x}={\mathrm{Dh\α}}^{*}$

Using the above method, an initial estimate X of a high resolution image can be obtained₀. However, the obtained high-resolution image does not accurately represent the actual high-resolution image, and the process of degrading the high-resolution image into the low-resolution image cannot be well reflected, and meanwhile, interference (such as noise and the like) may exist in the low-resolution image. Therefore, at the end of the algorithm, the sparse representation-based image super-resolution reconstruction algorithm applies a backward projection filter to reduce the reconstruction error of the high-resolution image. The backward projection filter is expressed by a mathematical formula of

$X^{*}=\arg \min \left|\right|SHY-y|{|}_2^2+c||X-X_0|{|}_2^2$

Wherein S represents a sampling factor, H represents a blurring operator, and c is a constant. In practical calculations, a gradient descent method is often used to solve:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X₀)]

wherein, Xt represents the estimation of the high resolution image after T iterations, T represents transposition, v represents the step size in the gradient descent method, lm +1 represents the (m + 1) th parameter in the low resolution dictionary Dl, and Y represents the corresponding parameter in the high resolution dictionary Dh.

The image super-resolution reconstruction method based on sparse representation can obtain better image reconstruction results, but has the defects that the running time of the algorithm is longer compared with that of bi-quadratic interpolation, and meanwhile, compared with the methods of bi-quadratic interpolation and neighborhood embedding, the image reconstruction of the algorithm at the edge is not smooth enough. But the image super-resolution reconstruction algorithm based on sparse representation can better retain the information of the original high-resolution image.

It will be understood that modifications and variations can be made by persons skilled in the art in light of the above teachings and all such modifications and variations are intended to be included within the scope of the invention as defined in the appended claims.

Claims (6)

1. A super-resolution image reconstruction method based on sparse representation is characterized by comprising the following steps:

s1, obtaining a complete high-low resolution dictionary pair by using the image training library, wherein the complete high-low resolution dictionary pair comprises a low resolution dictionary Dl and a high resolution dictionary Dh;

s2, carrying out bicubic amplification on the low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, extracting first-order and second-order gradient features of the initial image, carrying out overlapped blocking on the extracted features to obtain low-resolution image blocks, and carrying out double blocking on each low-resolution image blockCombining the corresponding features of the image block into a vector, setting the vector as y, and then obtaining a sparse representation a of y by using a low-resolution dictionary Dl^*：

$a^{*}=\arg \min \left|\right|y-Dla|{|}_2^2+\mathrm{\β}||PDha-w|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1=\arg \min ||\tilde{y}-\tilde{D}a|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1$

Wherein,p is the overlapping area, w is the reconstructed pixel value in the overlapping area, β is a constant used to adjust the influence of the reconstructed pixel value on the reconstruction of the block to be reconstructed, and λ is a^*Linear parameter values with a;

s3, the obtained sparse representation coefficient of the low-resolution image feature block is approximately equal to the sparse representation coefficient of the high-resolution image block to be solved in the high-resolution dictionary Dh, and the corresponding high-resolution image block is estimated by utilizing the sparse representation coefficient

S4, reducing the reconstruction error of the high-resolution image by using a back projection filter to obtain a filtered high-resolution image block, and back projectingThe filter is expressed by the mathematical formulaWherein S represents a sampling factor, H represents a blurring operator, c is a constant, and X₀An initial estimation value of the high-resolution image; the gradient descent method is adopted to solve:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X)]

wherein, T represents transposition, Xt represents estimation of a high resolution image after T iterations, v represents a step size in a gradient descent method, lm +1 represents an m +1 th parameter in a low resolution dictionary Dl, and m is an integer.

2. The method according to claim 1, wherein step S1 is specifically: extracting high and low resolution image block information in an image training library, and training the obtained high and low resolution image blocks by using a K-singular value decomposition algorithm to obtain sparsely represented high and low resolution dictionary pairs.

3. The method of claim 1, wherein the extracted first-order and second-order gradient features are HOG histogram of oriented gradient features.

4. An image super-resolution reconstruction system based on sparse representation is characterized by comprising:

the training module is used for obtaining a complete high-low resolution dictionary pair by utilizing the image training library, wherein the complete high-low resolution dictionary pair comprises a low resolution dictionary Dl and a high resolution dictionary Dh;

the sparse representation module is used for carrying out bicubic amplification on a low-resolution image to be reconstructed to obtain an initial image of a super-resolution algorithm, extracting first-order and second-order gradient features of the initial image, carrying out overlapped blocking on the extracted features, combining the features corresponding to each image block into a vector, setting the vector as y, and then obtaining a sparse representation α of y by using a low-resolution dictionary Dl^*：

$a^{*}=\arg \min \left|\right|y-Dla|{|}_2^2+\mathrm{\β}||PDha-w|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1=\arg \min ||\tilde{y}-\tilde{D}a|{|}_2^2+\mathrm{\λ}\left|\right|a|{|}_1$

Wherein,p is an overlapping area, w is a reconstructed pixel value in the overlapping area, and β is a constant used for adjusting the influence of the reconstructed pixel value on the reconstruction of the block to be reconstructed;

a high resolution image reconstruction module, configured to approximately equal the obtained sparse representation coefficient of the low resolution dictionary Dl to the sparse representation coefficient of the high resolution image block to be solved in the high resolution dictionary Dh, and estimate a corresponding high resolution image block using the sparse representation coefficient

An error elimination module for reducing the reconstruction error of the high resolution image by using a backward projection filter to obtain a filtered high resolution image block, wherein the backward projection filter is expressed asWherein S represents a sampling factor, H represents a blurring operator, X₀For the initial estimation value of the high-resolution image, c is a constant, and a gradient descent method is adopted to solve the following problems:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X)]

where Xt represents the estimate of the high resolution image after t iterations and v represents the step size in the gradient descent method.

5. The system of claim 4, wherein the training module is specifically configured to: extracting high and low resolution image block information in an image training library, and training the obtained high and low resolution image blocks by using a K-singular value decomposition algorithm to obtain sparsely represented high and low resolution dictionary pairs.

6. The system of claim 4, wherein the extracted first and second order gradient features are HOG histogram of oriented gradient features.