CN105844590A

CN105844590A - Image super-resolution reconstruction method and system based on sparse representation

Info

Publication number: CN105844590A
Application number: CN201610167515.XA
Authority: CN
Inventors: 熊盛武; 郑文博; 曹旺; 许开弦; 瞿毅力; 韩恩浩
Original assignee: Wuhan University of Technology WUT
Current assignee: Wuhan University of Technology WUT
Priority date: 2016-03-23
Filing date: 2016-03-23
Publication date: 2016-08-10

Abstract

The invention discloses an image super-resolution reconstruction method based on sparse representation and a system. The method comprises following steps: a complete high-low resolution dictionary pair is obtained by means of an image training library; bicubic amplification is performed on a low resolution image to be reconstructed to obtain an initial image of the super-resolution algorithm; the first order and second order gradient features of the initial image are extracted; overlapping partition is performed on the extracted features and sparse representation of the low frequency image blocks is obtained by means of the low resolution dictionary; the sparse representation coefficient of the obtained low resolution image blocks is approximately equal to the sparse representation coefficient of the high resolution image blocks to be solved in the high resolution dictionary and the initial estimate of a corresponding high resolution image is estimated by means of the sparse representation coefficient; reconstruction error of the high resolution image is reduced by means of a back projection filter.

Description

Image super-resolution reconstruction method and system based on sparse representation

技术领域technical field

本发明涉及图像处理，尤其涉及一种基于稀疏表示的图像超分辨率重建方法及系统。The present invention relates to image processing, in particular to an image super-resolution reconstruction method and system based on sparse representation.

背景技术Background technique

在图像超分辨率重建算法中，首先要建立高分辨率图像之间的关系。一般来说，在单帧图像超分辨图像重建算法中，几何扭曲可以忽略不计，因此可以将图像获取过程中的退化现象模拟为原始高分辨率图像经过光学模糊、下采样和噪声干扰等一系列过程，现有的单帧图像超分辨率算法主要包括基于学习的图像超分辨率算法和基于插值重建的图像超分辨率算法。In the image super-resolution reconstruction algorithm, the relationship between high-resolution images must first be established. Generally speaking, in the single-frame image super-resolution image reconstruction algorithm, geometric distortion can be ignored, so the degradation phenomenon in the image acquisition process can be simulated as the original high-resolution image after a series of optical blurring, downsampling, and noise interference. The existing single-frame image super-resolution algorithms mainly include image super-resolution algorithms based on learning and image super-resolution algorithms based on interpolation reconstruction.

基于学习的超分辨率重建技术首先利用高分辨图像训练库和图像退化模型获得一个高低分辨率的图像训练集，然后通过一定的学习算法获得高低分辨率图像之间的映射关系，最后利用优化算法对待重建的低分辨率图像进行优化，估计出相应的高分辨率图像。The learning-based super-resolution reconstruction technology first uses the high-resolution image training library and image degradation model to obtain a high- and low-resolution image training set, and then obtains the mapping relationship between high and low-resolution images through a certain learning algorithm, and finally uses the optimization algorithm Optimize the low-resolution image to be reconstructed and estimate the corresponding high-resolution image.

经典的插值方法包括最近邻插值、双线性插值和双三次插值方法。三种插值方法中，最近邻域和双线性插值计算复杂度相对较低，在多数场合能满足实时运算的需求，因此它们被广泛应用。但是，不如人意的是，这两种方法对分辨率的提高贡献很小，尤其是无法得到足够的高频信息，而使得恢复出来的图像仍然存在较大的失真。这三种方法中，双立方插值最适合增强高频部分的细节信息，它的不足就是其计算复杂度比较大。Classical interpolation methods include nearest neighbor interpolation, bilinear interpolation and bicubic interpolation methods. Among the three interpolation methods, nearest neighbor and bilinear interpolation have relatively low computational complexity, and can meet the needs of real-time calculations in most occasions, so they are widely used. However, what is unsatisfactory is that these two methods have little contribution to the improvement of resolution, especially they cannot obtain enough high-frequency information, so that the restored image still has a large distortion. Among these three methods, bicubic interpolation is most suitable for enhancing the detail information of high-frequency parts, but its disadvantage is that its computational complexity is relatively large.

尽管双立方插值的能提供更多的细节信息，不过，它仍然存在与前两种方法相同的问题，尤其是需要高分辨率的场合，它无法给出足够多的细节信息。插值方法或者出现锯齿效应或者出现边缘模糊现象。最近邻插值边缘阶梯锯齿图像失真非常明显，双线性插值边缘阶梯锯齿图像失真较为明显，有一定的边缘模糊，双立方插值边缘阶梯失真较弱，边缘模糊非常明显。Although bicubic interpolation can provide more detailed information, it still has the same problems as the first two methods, especially when high resolution is required, it cannot give enough detailed information. Interpolation methods either suffer from aliasing or blurred edges. The edge step aliasing image distortion of the nearest neighbor interpolation is very obvious, the edge step aliasing image distortion of the bilinear interpolation is more obvious, and there is a certain edge blur, and the edge step distortion of the bicubic interpolation is weak, and the edge blur is very obvious.

传统的插值重建方法计算简单容易实现，应用也较为广泛。但是在高放大倍数的情况下，由于图像原始信息的大量丢失，会使得重建出来的高分辨率图像在边缘处常常过于平滑，而且缺少足够的细节信息。The traditional interpolation reconstruction method is simple and easy to implement, and it is widely used. However, in the case of high magnification, due to the loss of a large amount of original information of the image, the reconstructed high-resolution image is often too smooth at the edge and lacks sufficient detail information.

基于学习的超分辨率算法，在进行高分辨图像重建之前，利用训练库，对图像的先验信息进行了学习，所以能获得更多的图像细节信息，获得更好的图像重建质量。然而在基于学习一类的超分辨率算法中，近邻数目选择不当，会使重建出来的高分辨率图像细节和边缘模糊，从而影响图像的重建质量。Based on the learning super-resolution algorithm, before the high-resolution image reconstruction, the prior information of the image is learned by using the training library, so more image detail information can be obtained, and better image reconstruction quality can be obtained. However, in super-resolution algorithms based on learning, improper selection of the number of neighbors will blur the details and edges of the reconstructed high-resolution image, thus affecting the reconstruction quality of the image.

发明内容Contents of the invention

本发明针对上述问题，提供一种基于稀疏表示的图像超分辨率重建方法及系统。Aiming at the above problems, the present invention provides an image super-resolution reconstruction method and system based on sparse representation.

本发明解决其技术问题所采用的技术方案是：The technical solution adopted by the present invention to solve its technical problems is:

提供一种基于稀疏表示的图像超分辨率重建方法，包括以下步骤：An image super-resolution reconstruction method based on sparse representation is provided, comprising the following steps:

S1、利用图像训练库得到完备的高低分辨率字典对，包括低分辨率字典Dl和高分辨率字典Dh；S1, using the image training library to obtain a complete pair of high- and low-resolution dictionaries, including a low-resolution dictionary Dl and a high-resolution dictionary Dh;

S2、将待重建的低分辨图像进行双三次放大，获得超分辨率算法的初始图像，提取该初始图像的一阶、二阶梯度特征，对提取的特征进行重叠分块，得到低分辨率图像块，将每一个低分辨率图像块对应的特征组合成一个向量，设为y，然后利用低分辨率字典Dl获得y的稀疏表示a^*：S2. Perform bicubic amplification on the low-resolution image to be reconstructed to obtain the initial image of the super-resolution algorithm, extract the first-order and second-order gradient features of the initial image, and overlap and block the extracted features to obtain a low-resolution image Block, combine the features corresponding to each low-resolution image block into a vector, set it as y, and then use the low-resolution dictionary Dl to obtain the sparse representation a ^* of y:

${a a}^{* *} = = arg arg min min | | | | y the y - - D D. l l a a | | {| |}_{22}^{22} + + β β | | | | P P D D. h h a a - - w w | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11} = = arg arg min min | | | | \overset{~ ~}{y the y} - - \overset{~ ~}{D D.} a a | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11}$

其中，P为重叠区域，w为重叠区域中已重建出来的像素值，β是一个常数，用来调整已重建出来的像素值对待重建块重建的影响；in, P is the overlapping area, w is the reconstructed pixel value in the overlapping area, and β is a constant used to adjust the influence of the reconstructed pixel value on the reconstruction of the reconstruction block;

S3、将获得的低分辨率图像块的稀疏表示系数近似地等于待求的高分辨率图像块在高分辨率字典Dh中的稀疏表示系数，并利用其估计出相应的高分辨率图像块 S3. The sparse representation coefficient of the obtained low-resolution image block is approximately equal to the sparse representation coefficient of the high-resolution image block to be found in the high-resolution dictionary Dh, and use it to estimate the corresponding high-resolution image block

S4、利用后向投影滤波器来减小高分辨率图像的重建误差，得到滤波后的高分辨率图像块，后向投影滤波器用数学式表达为其中，S表示采样因子，H表示模糊算子，c为常数，X₀为高分辨率图像的初始估计值；采用梯度下降法来求解:S4, using the back projection filter to reduce the reconstruction error of the high-resolution image, and obtain the filtered high-resolution image block, the back projection filter is expressed mathematically as Among them, S represents the sampling factor, H represents the fuzzy operator, c is a constant, and _X0 is the initial estimated value of the high-resolution image; the gradient descent method is used to solve:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X)]X _lm+1 ＝X _t +v[H ^T S ^T (Y-SHX _t )+c(X _t -X)]

其中，Xt表示t次迭代后对高分辨率图像的估计，v表示梯度下降法中的步长。where Xt represents the estimation of the high-resolution image after t iterations, and v represents the step size in the gradient descent method.

本发明所述的方法中，步骤S1具体为：提取图像训练库中的高低分辨率图像块信息，并利用K-奇异值分解算法对获得的高低分辨率图像块进行训练得到稀疏表示的高低分辨率字典对。In the method of the present invention, step S1 specifically includes: extracting the high and low resolution image block information in the image training library, and using the K-singular value decomposition algorithm to train the obtained high and low resolution image blocks to obtain the high and low resolution of the sparse representation rate dictionary pair.

本发明所述的方法中，所提取的一阶、二阶梯度特征为HOG方向梯度直方图特征。In the method of the present invention, the extracted first-order and second-order gradient features are HOG direction gradient histogram features.

本发明还提供一种基于稀疏表示的图像超分辨率重建系统包括：The present invention also provides a sparse representation-based image super-resolution reconstruction system comprising:

训练模块，用于利用图像训练库得到完备的高低分辨率字典对，包括低分辨率字典Dl和高分辨率字典Dh；The training module is used to obtain a complete high and low resolution dictionary pair using the image training library, including a low resolution dictionary D1 and a high resolution dictionary Dh;

稀疏表示模块，用于将待重建的低分辨图像进行双三次放大，获得超分辨率算法的初始图像，提取该初始图像的一阶、二阶梯度特征，对提取的特征进行重叠分块，将每一个图像块对应的特征组合成一个向量，设为y，然后利用低分辨率字典Dl获得y的稀疏表示α^*：The sparse representation module is used to double-cubic magnify the low-resolution image to be reconstructed to obtain the initial image of the super-resolution algorithm, extract the first-order and second-order gradient features of the initial image, and overlap and block the extracted features. The features corresponding to each image block are combined into a vector, set to y, and then use the low-resolution dictionary Dl to obtain the sparse representation α ^* of y:

高分辨率图像重建模块，用于将获得的低分辨率字典Dl的稀疏表示系数近似地等于待求的高分辨率图像块在高分辨率字典Dh中的稀疏表示系数，并利用其估计出相应的高分辨率图像块 The high-resolution image reconstruction module is used to approximately equal the sparse representation coefficient of the obtained low-resolution dictionary Dl to the sparse representation coefficient of the high-resolution image block to be obtained in the high-resolution dictionary Dh, and use it to estimate the corresponding high-resolution image blocks of

误差消除模块，用于利用后向投影滤波器来减小高分辨率图像的重建误差，得到滤波后的高分辨率图像块，后向投影滤波器用数学式表达为其中，S表示采样因子，H表示模糊算子，，X₀为高分辨率图像的初始估计值，c为常数，采用梯度下降法来求解:The error elimination module is used to reduce the reconstruction error of the high-resolution image by using the back-projection filter to obtain the filtered high-resolution image block, and the back-projection filter is expressed as Among them, S represents the sampling factor, H represents the fuzzy operator, _X0 is the initial estimated value of the high-resolution image, c is a constant, and the gradient descent method is used to solve:

本发明所述的系统中，训练模块具体用于：提取图像训练库中的高低分辨率图像块信息，并利用K-奇异值分解算法对获得的高低分辨率图像块进行训练得到稀疏表示的高低分辨率字典对。In the system of the present invention, the training module is specifically used to: extract the high and low resolution image block information in the image training library, and use the K-singular value decomposition algorithm to train the obtained high and low resolution image blocks to obtain the height of the sparse representation Resolution dictionary pair.

本发明所述的系统中，所提取的一阶、二阶梯度特征为HOG方向梯度直方图特征。In the system of the present invention, the extracted first-order and second-order gradient features are HOG direction gradient histogram features.

本发明产生的有益效果是：本发明在进行超分辨率重建的估计过程中引入了参数β，用来调整已重建出来的像素值对待重建块重建的影响，参数β的引入也就是引入了不确定性，可阻止不同变量间估计误差的扩大；将高分辨率图像和算法中的未知参数在一个框架下建模，对未知变量进行联合估计，计算出解析出超参数X₀的值，提高了对噪声的稳定性，不需要进行参数调整。The beneficial effects produced by the present invention are: the present invention introduces a parameter β in the estimation process of super-resolution reconstruction, which is used to adjust the influence of the reconstructed pixel value on the reconstruction of the reconstruction block. Determinism, which can prevent the expansion of estimation errors between different variables; model the high-resolution images and unknown parameters in the algorithm under one framework, jointly estimate the unknown variables, calculate and analyze the value of the hyperparameter X ₀ , and improve In order to be stable against noise, no parameter adjustment is required.

附图说明Description of drawings

下面将结合附图及实施例对本发明作进一步说明，附图中：The present invention will be further described below in conjunction with accompanying drawing and embodiment, in the accompanying drawing:

图1是本发明实施例基于稀疏表示的图像超分辨率重建方法的流程图；Fig. 1 is a flowchart of an image super-resolution reconstruction method based on sparse representation according to an embodiment of the present invention;

图2是本发明实施例基于稀疏表示的图像超分辨率重建算法的字典训练过程图；Fig. 2 is the dictionary training process diagram of the image super-resolution reconstruction algorithm based on sparse representation in the embodiment of the present invention;

图3是本发明实施例图像重建简图。Fig. 3 is a schematic diagram of image reconstruction according to an embodiment of the present invention.

具体实施方式detailed description

为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

本发明基于稀疏表示的图像超分辨率重建算法包括过完备字典构建和高分辨率图像重建两个阶段。如图1所示，具体包括以下步骤：The image super-resolution reconstruction algorithm based on sparse representation of the present invention includes two stages of over-complete dictionary construction and high-resolution image reconstruction. As shown in Figure 1, it specifically includes the following steps:

S2、将待重建的低分辨图像进行双三次放大，获得超分辨率算法的初始图像，提取该初始图像的一阶、二阶梯度特征，对提取的特征进行重叠分块，得到低分辨率图像块，将每一个低分辨率图像块对应的特征组合成一个向量，利用低分辨率字典Dl获得y的稀疏表示；S2. Perform bicubic amplification on the low-resolution image to be reconstructed to obtain the initial image of the super-resolution algorithm, extract the first-order and second-order gradient features of the initial image, and overlap and block the extracted features to obtain a low-resolution image Block, combine the features corresponding to each low-resolution image block into a vector, and use the low-resolution dictionary Dl to obtain the sparse representation of y;

S3、将获得的低分辨率图像块的稀疏表示系数近似地等于待求的高分辨率图像块在高分辨率字典中的稀疏表示系数，并利用其估计出相应的高分辨率图像的初始估计值；S3. The sparse representation coefficient of the obtained low-resolution image block is approximately equal to the sparse representation coefficient of the high-resolution image block to be obtained in the high-resolution dictionary, and use it to estimate the initial estimate of the corresponding high-resolution image value;

S4、利用后向投影滤波器来减小高分辨率图像的重建误差。S4. Using a back-projection filter to reduce a reconstruction error of the high-resolution image.

在字典构建和图像重建过程中均涉及到稀疏表示系数的求解，因此，下文将对图像的稀疏表示理论模型、过完备字典构建和高分辨率图像重建方法进行详细说明。Both dictionary construction and image reconstruction involve the solution of sparse representation coefficients. Therefore, the theoretical model of image sparse representation, over-complete dictionary construction and high-resolution image reconstruction methods will be described in detail below.

(1)图像的稀疏表示理论模型(1) Sparse representation theory model of image

图像的稀疏表示指的是，图像或图像信号可以利用过完备原子库中一系列合适的原子进行线性组合来近似逼近。由于该线性组合用到的原子数目远远小于原子库中原子的总数目，因此，这个原子库是过完备的。图像的稀疏表示原理如下:The sparse representation of an image means that an image or image signal can be approximated by a linear combination of a series of suitable atoms in an over-complete atomic library. Since the number of atoms used in this linear combination is much smaller than the total number of atoms in the atomic library, the atomic library is over-complete. The principle of image sparse representation is as follows:

给定图像信号Y和过完备字典D，可以利用D中一系列合适的基线性组合来表示Y。其中的数学关系可表示为：Given an image signal Y and an overcomplete dictionary D, Y can be represented by a series of suitable base linear combinations in D. The mathematical relationship can be expressed as:

min||α||₀s.t.y＝Dαmin||α|| ₀ sty＝Dα

其中，α是稀疏表示系数，||·||₀表示0范数，由于利用表达式求解||·||₀是一个NP难的问题，实际计算中采用||·||₁的求解过程转化为凸优化问题：Among them, α is a sparse representation coefficient, and ||·|| ₀ represents the 0 norm. Since it is an NP-hard problem to solve ||·|| ₀ using the expression, the solution process of ||·|| ₁ is used in the actual calculation. Converted to a convex optimization problem:

${a a}^{* *} = = arg arg min min | | | | y the y - - D D. α α | | {| |}_{22}^{22} + + λ λ | | | | α α | | {| |}_{11}$

其中，λ是正则化参数(即a*与a的线性参数值)。Among them, λ is the regularization parameter (that is, the linear parameter value of a* and a).

(2)过完备字典构建(2) Complete dictionary construction

基于稀疏表示的图像超分辨率重建算法的第一步是利用图像训练库来获得过完备的稀疏表示字典。过完备字典的训练是通过提取图像训练库中的高低分辨率图像块信息，并利用K-奇异值分解算法对获得的高低分辨率图像块进行训练得到稀疏表示的高低分辨率字典对。The first step of the image super-resolution reconstruction algorithm based on sparse representation is to use the image training library to obtain an over-complete sparse representation dictionary. The training of the complete dictionary is to extract the high and low resolution image block information in the image training library, and use the K-singular value decomposition algorithm to train the obtained high and low resolution image blocks to obtain sparsely represented high and low resolution dictionary pairs.

图2给出了基于稀疏表示的图像超分辨率重建算法中字典训练的具体过程。具体为：Figure 2 shows the specific process of dictionary training in image super-resolution reconstruction algorithm based on sparse representation. Specifically:

①将低分辨率图像集中的图像划分为N×N大小的图像块，并提取特征。具体方法是采用4个一维滤波器：① Divide the images in the low-resolution image set into N×N size image blocks, and extract features. The specific method is to use four one-dimensional filters:

f₁＝[-1,0,1] f ₁ =[-1,0,1]

f₃＝[1,0,-2,0,1] f ₃ =[1,0,-2,0,1]

其中，T表示转置。将低分辨图像与这四个滤波器算子的卷积结果作为低分辨图像的特征。将分块得到的四个特征块与其对应的高分辨率图像块联合起来作为字典训练算法的输入，这种方法考虑了低分辨率图像的邻域信息，因此有利于提高重建的高分辨率图像块间的兼容性。Among them, T means transpose. The convolution results of the low-resolution image and these four filter operators are used as the features of the low-resolution image. The four feature blocks obtained by block segmentation and their corresponding high-resolution image blocks are combined as the input of the dictionary training algorithm. This method considers the neighborhood information of the low-resolution image, so it is beneficial to improve the reconstruction of the high-resolution image. Compatibility between blocks.

②训练低分辨率字典。根据①的结果，利用稀疏K-SVD算法求解，得到低分辨率字典Dl。② Training low-resolution dictionary. According to the results of ①, the sparse K-SVD algorithm is used to solve the problem, and the low-resolution dictionary Dl is obtained.

③将低分辨率字典Dl通过插值得到与其上一级的高分辨率图像集的尺寸相同的插值图像集Dh。③ The low-resolution dictionary Dl is interpolated to obtain an interpolated image set Dh with the same size as the upper-level high-resolution image set.

基于稀疏表示的图像超分辨率重建算法进行字典训练所用的技术包括训练样本的提取方法和进行字典训练的K-SVD算法。The techniques used for dictionary training in the image super-resolution reconstruction algorithm based on sparse representation include the extraction method of training samples and the K-SVD algorithm for dictionary training.

a.训练样本的提取a. Extraction of training samples

在基于稀疏表示的图像超分辨率重建算法中，对图像的处理是分块进行的，训练样本由图像训练库中的高分辨图像随机抽取而来，通常训练样本数要远远大于字典中的原子个数。为了获得一个具有广泛适用性的字典，应尽可能选择较多的自然高分辨率图像作为图像训练库，且图像训练库中的图像信息种类要尽可能多。在具体的字典训练过程中，低分辨率图像是由高分辨图像退化而来，常常需要提取低分辨率图像的多种特征，并将其特征进行分块与对应的高分辨图像块组成一个向量进行字典的训练。由于图像的一阶、二阶梯度能有效地表达图像的特征，且提取算法简单，因此常用它们来表示图像的特征。图像的一阶、二阶梯度为方向梯度直方图(Histogram of Oriented Gradient,HOG)特征，HOG特征是一种在计算机视觉和图像处理中用来进行物体检测的特征描述子。图像的一阶、二阶梯度滤波器算子可以表示为：In the image super-resolution reconstruction algorithm based on sparse representation, the image is processed in blocks, and the training samples are randomly selected from the high-resolution images in the image training library. Usually, the number of training samples is much larger than that in the dictionary. number of atoms. In order to obtain a dictionary with wide applicability, as many natural high-resolution images as possible should be selected as the image training library, and the types of image information in the image training library should be as large as possible. In the specific dictionary training process, the low-resolution image is degenerated from the high-resolution image, and it is often necessary to extract various features of the low-resolution image, and divide its features into blocks and form a vector with the corresponding high-resolution image blocks Conduct dictionary training. Since the first-order and second-order gradients of an image can effectively express the features of the image, and the extraction algorithm is simple, they are often used to represent the features of the image. The first-order and second-order gradients of the image are Histogram of Oriented Gradient (HOG) features, and the HOG feature is a feature descriptor used for object detection in computer vision and image processing. The first-order and second-order gradient filter operators of the image can be expressed as:

f₁＝[-1,0,1]，f₂＝f₁ ^T f ₁ =[-1,0,1], f ₂ =f ₁ ^T

f₃＝[1,0,-2,0,1]，f₄＝f₃ ^T f ₃ =[1,0,-2,0,1], f ₄ =f ₃ ^T

其中，T表示转置。将低分辨图像与这四个滤波器算子的卷积结果作为低分辨图像的特征。然后将分块得到的四个特征块与其对应的高分辨率图像块联合起来作为字典训练算法的输入。这种方法考虑了低分辨率图像的邻域信息，因此有利于提高重建的高分辨率图像块间的兼容性。Among them, T means transpose. The convolution results of the low-resolution image and these four filter operators are used as the features of the low-resolution image. Then the four feature blocks obtained by block segmentation are combined with their corresponding high-resolution image blocks as the input of the dictionary training algorithm. This method takes into account the neighborhood information of the low-resolution image, so it is beneficial to improve the compatibility between the reconstructed high-resolution image patches.

b.K-SVD字典训练法b. K-SVD dictionary training method

K-SVD算法是基于稀疏表示的图像超分辨率重建算法中字典训练过程的核心算法。K-SVD算法又被称为广义K一均值聚类算法，它与K-均值聚类算法有很大的联系，当K-SVD算法中的每个信号只用一个原子来近似表示时，K-SVD算法就退化为K一均值聚类算法。它的主要思想是:求解输入样例在当前字典中的稀疏表示以及字典更新这两个过程的不断交替，根据稀疏表示结果对字典的每一列进行更新。假设有N个训练样本输出，用D∈R^n*k表示过完备词典，Y＝{y_i∈Rⁿ,i＝1,2,........,N}表示训练样本集合，A＝{α_i∈R^k,i＝1,2,......,N}表示训练样本的稀疏表示系数集合，则K-SVD字典训练算法可以等价为求解式The K-SVD algorithm is the core algorithm of the dictionary training process in the image super-resolution reconstruction algorithm based on sparse representation. The K-SVD algorithm is also called the generalized K-means clustering algorithm, which has a great relationship with the K-means clustering algorithm. When each signal in the K-SVD algorithm is approximately represented by only one atom, K -SVD algorithm degenerates into K-means clustering algorithm. Its main idea is: solve the sparse representation of the input sample in the current dictionary and the continuous alternation of the dictionary update, and update each column of the dictionary according to the result of the sparse representation. Assuming that there are N training sample outputs, use D∈R ^n*k to represent the overcomplete dictionary, and Y={y _i ∈R ⁿ ,i=1,2,...,N} to represent the training sample set , A={α _i ∈ R ^k ,i=1,2,...,N} represents the sparse representation coefficient set of training samples, then the K-SVD dictionary training algorithm can be equivalent to the solution formula

$\begin{matrix} \underset{D D. . . A A}{min min} {{| | | | Y Y - - D D. A A | | {| |}_{F f}^{22}}} & s the s . . t t . . & &ForAll; &ForAll; i i,, | | | | {a a}_{i i} | | {| |}_{00} \leq \leq {T T}_{00} \end{matrix}$

其中，T为稀疏表示系数中非零数目的上限值。Among them, T is the upper limit value of the non-zero number in the sparse representation coefficient.

(3)图像重建过程(3) Image reconstruction process

详见图3。基于稀疏表示的图像超分辨率重建算法在经过字典训练之后，通过将己知的低分辨率图像在训练得到的高低分辨率字典中进行稀疏表示和重建，从而估计出期望的高分辨率图像。基于稀疏表示的图像超分辨率重建算法的高分辨率图像重建过程。See Figure 3 for details. The image super-resolution reconstruction algorithm based on sparse representation estimates the desired high-resolution image by sparsely representing and reconstructing known low-resolution images in the trained high- and low-resolution dictionary after dictionary training. High-Resolution Image Reconstruction Process with Sparse Representation-Based Image Super-Resolution Reconstruction Algorithms.

首先将待重建的低分辨图像进行双三次放大，获得超分辨率算法的初始图像，接着提取该初始图像的一阶、二阶梯度特征，对特征进行重叠分块，将每一个图像块对应的特征组合成一个向量，设为y，然后利用在低分辨率字典Dl中获得y得稀疏表示系α*。First, the low-resolution image to be reconstructed is bicubically enlarged to obtain the initial image of the super-resolution algorithm, and then the first-order and second-order gradient features of the initial image are extracted, and the features are overlapped and divided into blocks, and each image block corresponds to The features are combined into a vector, set to y, and then use the sparse representation system α* obtained in the low-resolution dictionary Dl for y.

${a a}^{* *} = = arg arg min min | | | | y the y - - D D. l l α α | | {| |}_{22}^{22} + + β β | | | | P P D D. h h α α - - w w | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11} = = arg arg min min | | | | \overset{~ ~}{y the y} - - \overset{~ ~}{D D.} α α | | {| |}_{22}^{22} + + λ λ | | | | α α | | {| |}_{11}$

其中，in,

$\overset{~ ~}{D D.} = = [\begin{matrix} {D D.}_{l l} \\ {PD PD}_{h h} \end{matrix}],, \overset{~ ~}{y the y} = = [\begin{matrix} y the y \\ β β w w \end{matrix}]$

P为重叠区域，w为重叠区域中已重建出来的像素值，β是一个常数，用来调整已重建出来的像素值对待重建块重建的影响。β的值可根据利用图片自身的特征来确定表示2范数的平方，||·||₁表示1范数。λ为a*与a的线性参数值。P is the overlapping area, w is the reconstructed pixel value in the overlapping area, and β is a constant used to adjust the influence of the reconstructed pixel value on the reconstructed block. The value of β can be determined according to the characteristics of the picture itself Indicates the square of the 2-norm, and ||·|| ₁ indicates the 1-norm. λ is the linear parameter value of a* and a.

然后根据高低分辨率图像块在局部具有相似的几何流形这一特性，可认为高低分辨率图像块对在训练集中的稀疏表示系数是一致的，因此将获得的低分辨图像块的稀疏表示系数近似地等于待求的高分辨率图像块在高分辨率字典中的稀疏表示系数，并利用其估计出相应的高分辨率图像块 Then, according to the characteristic that the high and low resolution image blocks have similar geometric manifolds locally, it can be considered that the sparse representation coefficients of the high and low resolution image blocks in the training set are consistent, so the sparse representation coefficients of the obtained low resolution image blocks It is approximately equal to the sparse representation coefficient of the high-resolution image block to be sought in the high-resolution dictionary, and uses it to estimate the corresponding high-resolution image block

$\overset{~ ~}{x x} = = {Dhα Dhα}^{* *}$

利用上述方法，可以获得高分辨率图像的初始估计值X₀。但是，得到的高分辨率图像并不能精确地代表实际的高分辨率图像，且不能很好地体现高分辨率图像退化成低分辨率图像的过程，同时低分辨率图像中可能存在干扰(如噪声等)。因此在算法的最后，基于稀疏表示的图像超分辨率重建算法运用后向投影滤波器来减小高分辨率图像的重建误差。后向投影滤波器用数学式表达为Using the above method, the initial estimated value X ₀ of the high-resolution image can be obtained. However, the obtained high-resolution image cannot accurately represent the actual high-resolution image, and cannot well reflect the process of degrading the high-resolution image into a low-resolution image, and there may be interference in the low-resolution image (such as noise, etc.). Therefore, at the end of the algorithm, the image super-resolution reconstruction algorithm based on sparse representation uses a back-projection filter to reduce the reconstruction error of high-resolution images. The backprojection filter is expressed mathematically as

${X x}^{* *} = = arg arg min min | | | | S S H h Y Y - - y the y | | {| |}_{22}^{22} + + c c | | | | X x - - {X x}_{00} | | {| |}_{22}^{22}$

其中，S表示采样因子，H表示模糊算子，c为常数。在实际计算中，常采用梯度下降法来求解:Among them, S represents the sampling factor, H represents the fuzzy operator, and c is a constant. In actual calculation, the gradient descent method is often used to solve:

X_lm+1＝X_t+v[H^TS^T(Y-SHX_t)+c(X_t-X₀)]X _lm+1 ＝X _t +v[H ^T S ^T (Y-SHX _t )+c(X _t -X ₀ )]

其中，Xt表示t次迭代后对高分辨率图像的估计，T表示转置，v表示梯度下降法中的步长，lm+1表示低分辨率字典Dl中的第m+1个参数，Y表示高分辨率字典Dh中对应的参数。Among them, Xt represents the estimation of the high-resolution image after t iterations, T represents the transpose, v represents the step size in the gradient descent method, lm+1 represents the m+1th parameter in the low-resolution dictionary Dl, Y Indicates the corresponding parameters in the high-resolution dictionary Dh.

基于稀疏表示的图像超分辨率重建方法可以获得更好的图像重建结果,但缺点是相对于双二次插值来说，该算法的运行时间比较长，同时与双二次插值和邻域嵌入的方法相比，该算法在边缘处图像重建得不够平滑。但是基于稀疏表示的图像超分辨率重建算法能更好的保留原始高分辨率图像的信息。The image super-resolution reconstruction method based on sparse representation can obtain better image reconstruction results, but the disadvantage is that compared with biquadratic interpolation, the running time of this algorithm is relatively long, and it is compatible with biquadratic interpolation and neighborhood embedding. Compared with other methods, this algorithm does not reconstruct the image smoothly at the edge. But the image super-resolution reconstruction algorithm based on sparse representation can better preserve the information of the original high-resolution image.

应当理解的是，对本领域普通技术人员来说，可以根据上述说明加以改进或变换，而所有这些改进和变换都应属于本发明所附权利要求的保护范围。It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for image super-resolution reconstruction based on sparse representation, comprising the following steps:

S1, using the image training library to obtain a complete pair of high- and low-resolution dictionaries, including a low-resolution dictionary Dl and a high-resolution dictionary Dh;

S2. Perform bicubic amplification on the low-resolution image to be reconstructed to obtain the initial image of the super-resolution algorithm, extract the first-order and second-order gradient features of the initial image, and overlap and block the extracted features to obtain a low-resolution image Block, combine the features corresponding to each low-resolution image block into a vector, set it as y, and then use the low-resolution dictionary Dl to obtain the sparse representation a ^* of y:

{a a}^{* *} = = arg arg min min | | | | y the y - - D D. l l a a | | {| |}_{22}^{22} + + β β | | | | P P D D. h h a a - - w w | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11} = = arg arg min min | | | | \overset{~ ~}{y the y} - - \overset{~ ~}{D D.} a a | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11}

in, P is the overlapping area, w is the reconstructed pixel value in the overlapping area, β is a constant used to adjust the influence of the reconstructed pixel value on the reconstruction of the reconstruction block, and λ is the linear parameter value of a ^* and a;

S3. The sparse representation coefficient of the obtained low-resolution image feature block is approximately equal to the sparse representation coefficient of the high-resolution image block to be obtained in the high-resolution dictionary Dh, and use it to estimate the corresponding high-resolution image block

S4, using the back projection filter to reduce the reconstruction error of the high-resolution image, and obtain the filtered high-resolution image block, the back projection filter is expressed mathematically as Among them, S represents the sampling factor, H represents the fuzzy operator, c is a constant, and _X0 is the initial estimated value of the high-resolution image; the gradient descent method is used to solve:

X _lm+1 ＝X _t +v[H ^T S ^T (Y-SHX _t )+c(X _t -X)]

Among them, T represents the transpose, Xt represents the estimation of the high-resolution image after t iterations, v represents the step size in the gradient descent method, lm+1 represents the m+1th parameter in the low-resolution dictionary Dl, m is an integer.

2. The method according to claim 1, wherein step S1 is specifically: extracting the high and low resolution image block information in the image training library, and utilizing the K-singular value decomposition algorithm to process the obtained high and low resolution image blocks Train to get sparsely represented high- and low-resolution dictionary pairs.

3. The method according to claim 1, wherein the extracted first-order and second-order gradient features are HOG direction gradient histogram features.

4. An image super-resolution reconstruction system based on sparse representation, characterized in that, comprising:

The training module is used to obtain a complete high and low resolution dictionary pair using the image training library, including a low resolution dictionary D1 and a high resolution dictionary Dh;

The sparse representation module is used to double-cubic magnify the low-resolution image to be reconstructed to obtain the initial image of the super-resolution algorithm, extract the first-order and second-order gradient features of the initial image, and overlap and block the extracted features. The features corresponding to each image block are combined into a vector, set to y, and then use the low-resolution dictionary Dl to obtain the sparse representation α ^* of y:

{a a}^{* *} = = arg arg min min | | | | y the y - - D D. l l a a | | {| |}_{22}^{22} + + β β | | | | P P D D. h h a a - - w w | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11} = = arg arg min min | | | | \overset{~ ~}{y the y} - - \overset{~ ~}{D D.} a a | | {| |}_{22}^{22} + + λ λ | | | | a a | | {| |}_{11}

in, P is the overlapping area, w is the reconstructed pixel value in the overlapping area, and β is a constant used to adjust the influence of the reconstructed pixel value on the reconstruction of the reconstruction block;

The high-resolution image reconstruction module is used to approximately equal the sparse representation coefficient of the obtained low-resolution dictionary Dl to the sparse representation coefficient of the high-resolution image block to be obtained in the high-resolution dictionary Dh, and use it to estimate the corresponding high-resolution image blocks of

The error elimination module is used to reduce the reconstruction error of the high-resolution image by using the back-projection filter to obtain the filtered high-resolution image block, and the back-projection filter is expressed as Among them, S represents the sampling factor, H represents the fuzzy operator, X ₀ is the initial estimated value of the high-resolution image, c is a constant, and the gradient descent method is used to solve it:

X _lm+1 ＝X _t +v[H ^T S ^T (Y-SHX _t )+c(X _t -X)]

where Xt represents the estimation of the high-resolution image after t iterations, and v represents the step size in the gradient descent method.

5. The system according to claim 4, wherein the training module is specifically used for: extracting high and low resolution image block information in the image training library, and utilizing the K-singular value decomposition algorithm to obtain high and low resolution image blocks Perform training to obtain sparsely represented high- and low-resolution dictionary pairs.

6. The system according to claim 4, wherein the extracted first-order and second-order gradient features are HOG direction gradient histogram features.