CN110675347B - Image blind restoration method based on group sparse representation - Google Patents

Abstract

The invention discloses an image blind restoration method based on group sparse representation, which is used for realizing blind restoration of a blurred image. The method comprises the following steps: constructing an image pyramid, and estimating a fuzzy kernel from coarse to fine; searching similar image blocks in the down-sampled image, combining the current block and the cross-scale similar block into a similar image block group, and establishing a group sparse representation regular term in a target function; and alternately updating the fuzzy core and the clear image by iteration, and in the clear image updating step, reconstructing the clear image estimated in the previous iteration by using the group sparse representation to be used as a reference image for updating the clear image at the next time. Because the edge of the downsampled image has stronger similarity with the clear image, the low rank of the group matrix is realized by using the association of the group sparse representation and the low rank matrix and restricting the sparsity of the representation coefficients, and the edge of the reconstructed image is forced to be close to the edge of the clear image. The method disclosed by the invention improves the robustness to noise and can process the fuzzy kernel estimation with large scale.

Description

A Blind Image Restoration Method Based on Group Sparse Representation

technical field

The present invention relates to the field of image restoration, and more particularly, to an image blind restoration method based on group sparse representation.

Background technique

In the process of image acquisition and transmission, due to the imaging system, recording equipment, transmission medium and other reasons, the digital images obtained by the imaging equipment usually suffer from various distortions, resulting in the degradation of the image quality. This phenomenon is called image degradation. Image restoration aims to model the image degradation process, solve the inverse process of the degradation model, and restore the original clear image from the degraded image.

Blur is a common image degradation phenomenon. In the process of image acquisition by digital imaging equipment, the shaking of the equipment, defocusing and the motion of the object itself will lead to blurring. In some application scenarios, when the acquired image is blurred, the cost of re-acquiring the image is high or not feasible. When the image is re-acquired in the area, the surface conditions of the area may have changed when the waiting time is long and the image is re-imaged. In addition, in the imaging process, the degree of blurring can be avoided or reduced with the assistance of some special devices, such as the use of fixed devices such as tripods or optical image stabilization devices, but tripods are generally inconvenient to carry and have limited applications, while optical image stabilization The device can only resolve the blur caused by small jitter, and can't do anything about the larger blur.

Blind restoration of a single image is a technology that effectively removes the blur in the image by restoring the original clear image from a single blurred image only when the degradation model or degradation parameters are unknown by means of image processing. The process of acquiring the image by the imaging system is a positive problem, and the use of the degraded image to restore the potential original image is a typical image inverse problem. Typically, the degraded matrix is singular and the problem is ill-conditioned. According to whether the degradation model is available, it is divided into two categories: if the degradation matrix is known, it is called a non-blind restoration method; if the degradation matrix is unknown, it is called a blind restoration problem. In practical applications, it is usually impossible to know the degradation model or degradation parameters of the observed image, so the blind image restoration is more practical.

Since the degradation process of image blur is generally expressed as a two-dimensional convolution of a blur kernel and a clear image, the deblurring problem is also called a deconvolution problem. Image blind deconvolution is a serious underdetermined problem, the number of unknown variables to be solved is greater than the number of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce prior knowledge about the image, called the image prior model, and add it to the reconstruction process as a regularization constraint. This process is called regularization. Regularization constraints provide additional additional information for solving ill-conditioned problems, constraining the space of feasible solutions. Existing blind deconvolution algorithms all use various prior knowledge directly or indirectly. These algorithms can be roughly divided into two categories. One type of algorithm uses heuristic edge enhancement methods, and the other type of algorithm directly establishes blur kernels or sharp edges. A prior probability distribution model for the image.

The blind deconvolution method based on heuristic edge enhancement assumes that there are sufficient edges in the blurred image, approximates the clear image by enhancing the edges in the image, and finally estimates the blur kernel by using the approximate estimation of edge enhancement and the corresponding relationship of the blurred image. Due to the limited capability of the edge enhancement algorithm, it is generally impossible to obtain a clear edge through only one enhancement process. Therefore, continuous iteration is required in the blind deconvolution process, that is, the edge enhancement is continued on the estimated clear image each time to gradually approach Original clear image. Because this type of blind deconvolution method directly uses heuristic methods to enhance the edge of the blurred image, there is generally no unified objective function, and the algorithm complexity is low, but in the iterative solution process, in order to avoid the phenomenon of excessive edge enhancement, etc. , it is generally necessary to continuously adjust the parameters of the edge enhancement algorithm according to the number of iterations, so this type of method is more sensitive to parameter settings.

Another class of blind deconvolution methods builds a prior probability distribution model for the blur kernel or sharp image, and estimates the blur kernel and sharp image by solving the maximum a posteriori problem. The optimization objective function contains regularization constraints on sharp images and blur kernels, which should help solve the underdetermination of blind deconvolution problems and limit the space of feasible solutions. How to choose an appropriate regularization constraint is the key to the study of blind deconvolution methods. Previous work mainly utilizes prior knowledge about image gradients, which represent the relationship between adjacent pixels in an image and are not sufficient to represent larger-scale image structures. In recent years, some blind restoration methods utilize prior information about image blocks. Compared with algorithms that utilize gradient priors, blind restoration methods utilizing prior knowledge of image blocks have greater improvements in accuracy and robustness.

The methods based on sparse representation take advantage of the sparse representation of images under a certain set of bases (or dictionaries), and add this sparseness as a regularization constraint to the objective function. The key to the blind image restoration method based on sparse representation is to construct a dictionary representing clear images, so that the image block has a sparse representation under the dictionary. There are two obvious problems with sparse representation: 1) dictionary learning requires a large amount of computation; 2) it ignores the correlation between different blocks. The K-SVD algorithm under the sparse framework is the most commonly used adaptive dictionary learning method. This kind of dictionary learning usually requires a certain scale of data set to construct a dictionary, and uses the data set as a sample for dictionary learning. On the one hand, in the construction of this global dictionary, in order to make various types of image patches sparsely represented under the dictionary, a large number of samples must be used for dictionary learning, which leads to low dictionary learning efficiency; on the other hand, When the samples of dictionary learning cannot accurately provide similar information to the low-resolution images to be processed, it is difficult for this method to guarantee the effect of image reconstruction. In addition, the solution of the sparse representation coefficients under the dictionary for each data vector in the sparse representation is independent of each other, which lacks the problem of constraints on the global structure of the reconstructed image.

Although many blind image restoration methods have been proposed, most of them assume low noise levels. Blind image restoration is an ill-conditioned inverse problem. In the absence of prior knowledge of noise, blind image restoration will amplify the noise, resulting in the inability to accurately estimate the blur kernel. For the widely studied blind image restoration method based on image gradient sparse prior, image noise will destroy the prior prediction of gradient distribution, resulting in a certain bias in the estimation of blur kernel. In addition, the blind restoration model of heuristic edge enhancement usually looks for salient edges to estimate the blur kernel, and the noise in the blurred image will cause the error of edge localization, which will cause the bias of blur kernel estimation and lead to the failure of image blind restoration.

SUMMARY OF THE INVENTION

The invention discloses an image blind restoration method based on group sparse representation. By constraining the overall sparsity constraint of each group sparse representation coefficient under a specific dictionary, the rank of each group matrix is minimized, and the edge of the current image is forced to be more near the edge of a sharp image. The group sparse representation can fuse the local sparse and non-local self-similar priors of the image, and the rank minimization under a specific dictionary can better represent the global structural properties of the data. The method disclosed in the invention improves the robustness to noise, can process blurred images disturbed by noise, and can estimate large-scale blur kernels to restore large-scale blurred images.

The technical solution adopted in the present invention is a method for blind restoration of images based on group sparse representation, which is characterized in that it comprises the following steps:

Step 1. Read in the blurred image and build an image pyramid;

Read in the blurred image, set the initial size of the blur kernel, and downsample the blurred image layer by layer to construct an image pyramid. The size of the blur kernel corresponding to the image pyramids at different levels is different. Less than 3×3, that stops the construction of the image pyramid. In the estimation process, the clear image estimated on the image pyramid of the coarse layer is used as the initial estimate of the clear image on the image pyramid of the thin layer after image interpolation. At each layer of the image pyramid, the established blind deconvolution objective function is the same. The following steps are the solution process of the blind deconvolution objective function on each layer of the image pyramid.

Step 2. Initialize clear image;

设置

否则粗一层金字塔估计的清晰图像的插值结果作为当前层清晰图像的初始估计

Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image

set up

Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer

Step 3. Screen the image blocks and estimate the marker matrix M;

的背景平滑区域受到的约束较少，从而可能导致复原图像的平滑区域含有较多的噪声，为减小噪声对边缘估计造成的干扰，首先对复原图像

进行高斯滤波，然后对滤波后的图像估计边缘。Set the marker matrix to M, and M is also a binary image. If the value of a pixel in M is 1, it indicates that the corresponding image block participates in blur kernel estimation and group sparse representation constraints. Since this method only restricts the group sparsity regularization constraint to the edge region of the image, the restored image

The background smooth area is less constrained, which may cause the smooth area of the restored image to contain more noise. In order to reduce the interference of noise on edge estimation, first of all

Gaussian filtering is performed, and then edges are estimated on the filtered image.

Step 4. Estimate the blur kernel;

用下式更新下一次迭代的模糊核

fix the estimate for the current image

Update the blur kernel for the next iteration with

为图像的梯度算子，λ_h为正则化参数，⊙表示逐元素相乘，

表示傅里叶变换，

表示傅里叶逆变换，

表示傅里叶变化的复共轭。将复原图像

中梯度

小于一定阈值的像素点的梯度置为0。记梯度阈值为τ，模糊核大小为N_h，则τ的选取方式为：首先将图像梯度根据其方向分为四组，然后设置τ的取值以保证每一组都保留至少有

个像素点用于估计模糊核。由于随着迭代次数的增加，复原图像越来越清晰，为了使复原图像中更多的像素点逐步加入模糊核估计的过程中，在每一次迭代时将τ的值缩小为上一次迭代时的1.1倍。where y represents the blurred image,

is the gradient operator of the image, λ _h is the regularization parameter, ⊙ means element-wise multiplication,

represents the Fourier transform,

represents the inverse Fourier transform,

Represents the complex conjugate of the Fourier transform. will restore the image

medium gradient

The gradient of pixels less than a certain threshold is set to 0. Denote the gradient threshold as τ and the size of the blur kernel as N _h , then the selection method of τ is as follows: first, the image gradients are divided into four groups according to their directions, and then the value of τ is set to ensure that each group retains at least some

pixels are used to estimate the blur kernel. Since the restored image becomes clearer with the increase of the number of iterations, in order to gradually add more pixels in the restored image to the blur kernel estimation process, the value of τ is reduced to the value of the previous iteration at each iteration. 1.1 times.

Step 5. Estimate the clear image;

固定下一次迭代的模糊核估计

更新下一次迭代的图像

an estimate given the current image

Fixed blur kernel estimation for next iteration

Update the image for the next iteration

Step 5.1 Construct similar image block groups;

和

其中N为清晰图像的像素数目，a表示降采样因子，

为N维列向量，

为N/a²维列向量。从清晰图像X和降采样图像X^a中抽取的图像块分别表示为Q_jX和R_iX^a，其中

和

为抽取矩阵，分别用于从清晰图像和降采样图像中抽取第j个和第i个图像块，抽取的图像块尺寸为n。对于图像中的任意图像块Q_jX，在降采样图像X^a中搜索相似图像块R_iX^a。由于图像的不同尺度间广泛存在着图像块的相似性，即对于Q_jX，在降采样图像中的一定尺寸搜索窗口内利用图像块匹配方法寻找多个与其相似的图像块，设在X^a中搜索m-1个与Q_jX最相似的图像块，并按列表示为

Q_jX与这些在降采样图像中的非局部相似图像块聚合构成一个相似图像块组P_j，表示为：Let the clear image and its downsampled image be represented by columns as

and

where N is the number of pixels in the clear image, a is the downsampling factor,

is an N-dimensional column vector,

is an N/a ² -dimensional column vector. The image patches extracted from the clear image X and the downsampled image X ^a are denoted as Q _j X and R _i X ^a , respectively, where

and

is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image, respectively, and the size of the extracted image block is n. For any image block Q _j X in the image, a similar image block R _i X ^a is searched in the down-sampled image X ^a . Due to the widespread similarity of image blocks between different scales of the image, that is, for Q _j X, the image block matching method is used to find a number of similar image blocks in a certain size search window in the down-sampled image, set at X ^a Search for m-1 image patches that are most similar to Q _j X in the

Q _j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P _j , which is expressed as:

Among them, n is the size of the image block, and m is the number of similar image blocks.

Step 5.2 Reconstruct the reference image z _s by the group sparse representation;

Using the singular value threshold operator to reconstruct the group matrix P _j , its closed solution is

是以β≥0为参数的∑_j软阈值算子，定义为

in,

is the ∑ _j soft threshold operator with β≥0 as the parameter, defined as

的秩越小。用

近似

中的第一列即重建图像块

D_j为自适应字典。令

在交替迭代求解的过程中，z_s作为下一次迭代更新待估计清晰图像

的参考图像。From the above formula, the larger the β, the reconstruction of the group matrix

the smaller the rank. use

approximate

The first column in the reconstructed image block

D _j is an adaptive dictionary. make

In the process of alternate iterative solution, z _s is used as the next iteration to update the clear image to be estimated

reference image.

Step 5.3 Solve

之后，通过求解如下线性方程组更新对

的估计：in getting

After that, update the pair by solving the following system of linear equations

Estimate:

为y的一维列向量，

为点扩散函数

的矩阵形式，即模糊矩阵，

为梯度算子

的矩阵形式。由于M的作用，无法通过傅里叶变换在频域中快速求解，采取双共轭梯度法来求解如上线性方程组，获得

where |M| is the number of non-zero elements in M, N is the number of image pixels,

is a one-dimensional column vector of y,

is the point spread function

The matrix form of , the fuzzy matrix,

is the gradient operator

matrix form. Due to the effect of M, it cannot be quickly solved in the frequency domain through Fourier transform, and the double conjugate gradient method is used to solve the above linear equation system, and obtain

Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel;

并对清晰图像的估计

进行更新，更新为

如果此时的迭代达到最大迭代次数或者迭代收敛，则停止迭代。否则，令k＝k+1，k代表迭代次数，然后重复步骤3、步骤4和步骤5。Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel

and estimates for sharp images

to update, update to

If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped. Otherwise, let k=k+1, where k represents the number of iterations, and then repeat step 3, step 4, and step 5.

Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image.

在

的基础上采用有效的非盲复原算法获得最终的清晰图像估计。非盲复原算法有全变分正则化方法、稀疏非盲复原方法和EPLL算法等。Through steps 1 to 6, the estimation results of the blur kernel are obtained

exist

On the basis of the effective non-blind restoration algorithm, the final clear image estimation is obtained. Non-blind restoration algorithms include total variation regularization method, sparse non-blind restoration method and EPLL algorithm.

Preferably, the specific implementation method of screening the image blocks and eliminating the interference of smooth regions on the estimation of the blur kernel is: performing edge estimation on the current image estimation by an edge detection method, and using the image blocks corresponding to the edge pixels to estimate the blur kernel and sharpness. image.

Preferably, the size n of the image block is 5×5.

Preferably, the singular value threshold β is 0.2.

Preferably, the number m of the similar image blocks is 19.

Preferably, the size of the search window is 25×25.

Preferably, the downsampling factor a is 4/3.

Preferably, the number of iterations of the loop is 14, and no iteration convergence threshold is set.

Preferably, the size of the blur kernel is 51×51.

Since blind image restoration is a serious ill-conditioned inverse problem, in the absence of prior knowledge of noise, blind image restoration will amplify the noise, resulting in the inability to accurately estimate the blur kernel. The sparse representation uses the linear combination of as few dictionary elements as possible to represent each image block, and the solution of the sparse representation coefficients of each image block under the dictionary is independent of each other, so the sparse representation lacks the overall constraints on the data representation coefficients under the dictionary . Due to the self-similarity of images, the group sparse representation can constrain the sparsity of the group sparse representation coefficients as a whole. The present invention proposes an image prior constraint term of group sparse representation, which is used for the solution of regularized image blind restoration problem. Since the degree of blurring decreases with image downsampling, the edge of the downsampled image is more similar to the clear image. Therefore, the present invention searches for a plurality of similar image blocks in the downsampled image by using cross-scale self-similarity. , constitute a similar block image group. Since the group matrix of these similar image blocks merged should have low rank, the low rank of the group matrix is achieved by constraining the sparsity of the representation coefficient by using the association between the group sparse representation and the low rank matrix. Consider the problem of group sparse representation dictionary construction. The low-rank constraint can avoid the interference of the blind restoration model by noise, and at the same time, through the overall constraint with the cross-scale similar blocks in the down-sampled image, the edges of the current image are forced to be clearer in the iteration, so that the edges of the reconstructed image are close to the edges of the clear image.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic diagram of blind image restoration provided by an embodiment of the present invention;

2 is a flowchart of an image blind restoration method based on group sparse representation provided by an embodiment of the present invention;

3 is an image pyramid structure diagram provided by an embodiment of the present invention;

4 is a schematic diagram of constructing similar image block groups according to an embodiment of the present invention;

5 is a schematic diagram of similarity between a clear image and a blurred image and a down-sampled image of the blurred image provided by an embodiment of the present invention;

6 is an example of an edge image block marking matrix provided by an embodiment of the present invention;

FIG. 7 is a comparison of the average PSNR of various methods on the Koehler dataset provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The degradation process from a clear image to a blurred image is usually expressed as a convolutional model as shown below:

y=h*x+n (1)

Among them, y represents the blurred image, x represents the clear image, * represents the convolution operation, h is the blur kernel, and n is the noise. Here, the blur kernel h and noise n contain all the information of the degraded model. Under the convolution model, the blind image restoration method studies how to simultaneously estimate the blur kernel h and the clear image x from the blurred image y, as shown in Figure 1. Since the blurring process is modeled as a form of convolution, the blind deblurring problem is also known as the blind deconvolution problem.

Image blind deconvolution is an underdetermined problem, the number of unknown variables to be solved is greater than the number of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce prior knowledge about the image, called the image prior model, denoted as Ψ(x), and add it to the objective function as a regularization constraint. This process is called regularization. Regularization constraints provide additional additional information for solving ill-conditioned problems and constrain the solution space of the objective function. Regularization methods are usually expressed as the following optimization problems:

where Ψ _x (x) and Ψ _h (h) are the prior constraints of the clear image x and the blur kernel h, respectively, and the constants λ _x and λ _h are Lagrange multipliers, also known as regularization parameters. The first term represents the data fidelity term, and the last two terms represent the regularization constraint term. The role of the regularization parameters λx and _{λh is} to balance the proportion of reconstruction error and prior constraints in the objective function.

Image prior model occupies a very important position in the problem of blind image deconvolution. How to design effective regularization constraints to describe the prior knowledge or additional information of images is the key issue of blind image deconvolution methods. The invention mines prior knowledge from the multi-scale self-similarity of the image itself, and effectively adds the additional information contained in the multi-scale self-similar structure to the reconstructed image by constructing regularization constraints, so as to solve the problem of blind deconvolution of the image. Converted to an optimization problem that finds a solution that satisfies certain constraints.

从中抽取出图像块并按列表示为Q_jX，其中

为抽取矩阵，通过图像块匹配方法在图像中搜索Q_jX的相似图像块并按列表示为

将这些非局部相似图像块聚合为一个相似图像组，记为

其中，m为相似图像块的个数。与稀疏表示类似，设计一个字典

这里t为字典大小，相似图像块组

的稀疏表示通过求解如下最小化问题：Multi-scale self-similar structures widely exist in images, and multi-scale self-similarity can provide necessary additional information for blind image restoration. Whether it is the apparent similarity of aerial photography planes, ships, etc., or the potential similarity of houses, road edges, mountains and rivers, etc., this self-similarity is embodied in the image with the same scale and different scales. Similar image blocks . Let the one-dimensional column vector composed of clear images be expressed as

Image patches are extracted from them and denoted by columns as Q _j X, where

In order to extract the matrix, the similar image blocks of Q _j X are searched in the image by the image block matching method and expressed in columns as

Aggregate these non-locally similar image patches into a similar image group, denoted as

Among them, m is the number of similar image blocks. Similar to sparse representation, design a dictionary

Here t is the dictionary size, similar image block group

The sparse representation of is solved by solving the following minimization problem:

In the formula, ||·|| _F represents the Frobenius norm of the matrix, card(A _j ) represents the number of non-zero elements in the coefficient matrix A _j , and K represents the sparse constraint on the represented coefficient A _j . Equation (3) represents minimizing the group sparse representation error under the premise of constraining the sparsity of the representation coefficients.

为组稀疏表示系数。通过选取合适的字典保证表示系数的稀疏性，这种稀疏性可以作为对图像的正则化约束加入到图像盲复原的目标函数中，目标函数可表示为：The group matrix P _j is sparsely represented by the dictionary D _j , that is,

Coefficients are sparsely represented for groups. By selecting a suitable dictionary to ensure the sparsity of the representation coefficients, this sparsity can be added to the objective function of image blind restoration as a regularization constraint on the image. The objective function can be expressed as:

为图像的梯度算子，*表示卷积操作，||·||₂表示向量l₂范数，||·||_F表示矩阵Frobenius范数，P_j表示图像块Q_jX与其相似图像块构成的组矩阵，D_j为自适应字典，A_j为组稀疏表示系数，λ_g、λ_s、λ_h为正则化参数。式(4)中第一项为数据保真项，第二项为组稀疏表示正则化约束项，第三项为梯度约束项，第四项为模糊核的正则化约束项。数据保真项的形式是由噪声的概率模型决定的，本发明将模糊图像的梯度图像中含有的噪声建模为常见的加性高斯白噪声，于是得到了如式(4)所示的由2范数平方约束的数据保真项。数据保真项可以定义在图像域中，也可以定义在图像的梯度域中，由于本发明通过二维快速傅里叶变换(FFT)来加快目标函数的求解速度，而梯度图像在图像边界处的取值为0，可以更好地满足FFT要求的周期性边界条件，因而本发明将数据保真项定义在图像的梯度域中，以减小复原图像在边界处的失真。对于模糊核，本发明采取简单的2范数约束以简化求解过程。where y is the blurred image, x is the clear image, h is the blur kernel,

is the gradient operator of the image, * represents the convolution operation, ||·|| ₂ represents the vector l ₂ norm, ||·|| _F represents the matrix Frobenius norm, P _j represents the image block Q _j X and its similar image blocks The formed group matrix, D _j is the adaptive dictionary, A _j is the group sparse representation coefficient, λ _g , λ _s , λ _h are regularization parameters. In formula (4), the first term is the data fidelity term, the second term is the group sparse representation regularization constraint term, the third term is the gradient constraint term, and the fourth term is the regularization constraint term of the fuzzy kernel. The form of the data fidelity term is determined by the probability model of noise. In the present invention, the noise contained in the gradient image of the blurred image is modeled as a common additive Gaussian white noise, so the equation (4) is obtained by 2-norm squared constraint data fidelity term. The data fidelity term can be defined in the image domain or in the gradient domain of the image, because the present invention uses two-dimensional fast Fourier transform (FFT) to speed up the solution speed of the objective function, and the gradient image is at the image boundary. The value of is 0, which can better meet the periodic boundary conditions required by FFT, so the present invention defines the data fidelity item in the gradient domain of the image to reduce the distortion of the restored image at the boundary. For the fuzzy kernel, the present invention adopts a simple 2-norm constraint to simplify the solution process.

更新对模糊核的估计结果

然后在固定

的基础上更新

如此循环迭代直到估计结果收敛或者迭代次数达到某一预设的阈值。The present invention solves the estimation of the fuzzy kernel through the optimization problem shown in Equation (4). Since Equation (4) is non-convex and has no closed solution, the present invention adopts an alternate iteration method to solve the optimization problem shown in Equation (4). Solve, that is, fix the current estimation result for the clear image

Update the estimation result of the blur kernel

then fixed

update based on

Iterates in this way until the estimation result converges or the number of iterations reaches a preset threshold.

According to the multi-scale self-similarity of the image itself, this embodiment discloses an image blind restoration method based on group sparse representation, so as to realize the blind restoration of the blurred image. Referring to Figure 2, the above method at least includes the following steps:

Step 1. Read in the blurred image and build an image pyramid;

The invention adopts the image pyramid method commonly used in image blind deconvolution, and estimates the clear image and the blur kernel layer by layer from coarse to fine. The initial estimates of sharp images on the image pyramid are shown in Figure 3. Since the downsampling operation reduces the blurriness of the blurred image, and the larger the downsampling factor is, the smaller the blurriness is. Therefore, the blurriness of the blurred image at the bottom of the image pyramid is smaller, and the clearer image is estimated by blind deconvolution. Therefore, if the estimation result is interpolated as the initial estimation of the clear image in the image pyramid of the thin layer, the initial estimation of the clear image on the image pyramid of the thin layer is closer to the real clear image to be estimated, which can speed up the refinement process. The estimation process on one layer of image pyramids, and helps to improve the accuracy of estimation results on a thin layer of image pyramids. The image pyramid is constructed by downsampling the fuzzy observation image layer by layer. The size of the blur kernel corresponding to the image pyramid of different levels is different. When constructing the image pyramid, if the size of the blur kernel corresponding to the image pyramid of the current layer is less than 3×3, the image pyramid is stopped. structure. At each layer of the image pyramid, the established blind deconvolution objective function is the same. The following steps are the solution process of the blind deconvolution objective function on each layer of the image pyramid.

Step 2. Initialize clear image;

设置

否则粗一层金字塔估计的清晰图像的插值结果作为当前层清晰图像的初始估计

Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image

set up

Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer

Step 3. Screen the image blocks and estimate the marker matrix M;

Although multi-scale self-similarity exists widely in images, not all image blocks can provide effective additional information for image restoration. Pixels in different regions of the restored image have different effects on blur kernel estimation. The pixel value of an image area is constant, then the blurred pixel value of this area is still the same constant, the clear image and the blurred image are exactly the same in this area, so this area cannot provide effective information for the estimation of the blur kernel . Therefore, only the image edges play a key role in the estimation of blur kernels, and multi-scale self-similarity widely exists in the edges, which makes it easier to search for similar blocks.

The present invention proposes three screening schemes to screen image blocks, so as to eliminate the interference of smooth regions on blur kernel estimation. The first is to calculate the variance of the image blocks, and select the image blocks with larger variances, which correspond to the edges in the image that change sharply; the second is to use the edge detection method to estimate the current image. The image block estimates the blur kernel; the third is to calculate the image gradient. The gradient is a measure of the gray level change in the local area of the image, and the image block corresponding to the pixel whose gradient is greater than a certain threshold is used to estimate the blur kernel.

的背景平滑区域受到的约束较少，从而可能导致复原图像的平滑区域含有较多的噪声，为了减小噪声对边缘估计造成的干扰，本发明首先对复原图像

进行高斯滤波，然后对滤波后的图像估计边缘。Set the labeling matrix to M, where M is a binary image, as shown in Figure 5, if a pixel in M takes a value of 1, it indicates that the corresponding image block participates in blur kernel estimation and group sparse representation constraints. Since the present invention only restricts the group sparsity regularization constraint to the edge region of the image, the restored image

The background smooth area is less constrained, which may cause the smooth area of the restored image to contain more noise. In order to reduce the interference caused by noise to edge estimation, the present invention firstly adjusts

Gaussian filtering is performed, and then edges are estimated on the filtered image.

Step 4. Estimate the blur kernel;

更新

目标函数简化为：fixed

renew

The objective function simplifies to:

In the formula, ⊙ represents element-wise multiplication. According to Parseval's theorem, the energy of the image in the air domain is equivalent to the frequency domain energy of the Fourier transform, then the minimization problem shown in equation (5) in the air domain is equivalent to the minimization problem in the frequency domain as follows:

的二次函数，因此存在闭合解。令上式对

的导数为零，可得闭合解：Equation (6) is about

, so there is a closed solution. Let the above formula be correct

The derivative of is zero, the closed solution can be obtained:

表示傅里叶变换，

表示傅里叶逆变换，

表示傅里叶变化的复共轭。在式(7)中，本发明采取一种常用的方式排除图像平滑区域对模糊核估计的干扰，即将复原图像

中梯度

小于一定阈值的像素点的梯度置为0。记梯度阈值为τ，模糊核大小为N_h，则τ的选取方式为：首先将图像梯度根据其方向分为4组，然后设置τ的取值以保证每一组都保留至少有

个像素点用于估计模糊核。由于随着迭代次数的增加，复原图像越来越清晰，为了使复原图像中更多的像素点逐步加入到估计模糊核的过程中，在每一次迭代时将τ的值缩小为上一次迭代时的1.1倍。In the formula,

represents the Fourier transform,

represents the inverse Fourier transform,

Represents the complex conjugate of the Fourier transform. In the formula (7), the present invention adopts a common method to eliminate the interference of the image smooth area to the blur kernel estimation, that is, to restore the image

medium gradient

The gradient of pixels less than a certain threshold is set to 0. Denote the gradient threshold as τ and the size of the blur kernel as N _h , then the selection method of τ is as follows: first, the image gradients are divided into 4 groups according to their directions, and then the value of τ is set to ensure that each group retains at least some

pixels are used to estimate the blur kernel. Since the restored image becomes clearer with the increase of the number of iterations, in order to gradually add more pixels in the restored image to the process of estimating the blur kernel, in each iteration, the value of τ is reduced to the value of the previous iteration. 1.1 times.

Step 5. Estimate the clear image;

固定

更新

根据参与计算的图像块个数调整组稀疏表示正则化约束项的权重，式(4)所示的目标函数简化为：given

fixed

renew

According to the number of image blocks involved in the calculation, the weight of the regularization constraint term of the group sparse representation is adjusted, and the objective function shown in equation (4) is simplified as:

为y的一维列向量，将式(8)转换为矩阵向量乘积的形式：Among them, |M| is the number of non-zero elements in M, and N is the number of image pixels. Assume

is a one-dimensional column vector of y, convert equation (8) into the form of matrix-vector product:

为梯度算子

的矩阵形式，

为点扩散函数

的矩阵形式，即模糊矩阵。In the formula,

is the gradient operator

in matrix form,

is the point spread function

The matrix form of , the fuzzy matrix.

Step 5.1 Construct similar image block groups;

由于图像具有跨尺度自相似性，且降采样中的模糊程度小于当前层的模糊程度，其边缘更加清晰，与清晰图像具有更强的相似性。因此，本发明提出在降采样图像中搜索多个相似图像块从而构成相似图像块组，在整体上约束相似图像块组的稀疏性，迫使当前图像的边缘更接近清晰图像的边缘。According to the multi-scale self-similarity of the image itself, including the same-scale and cross-scale self-similarity, the present invention designs a regularization constraint term for group sparse representation, namely

Since the image has cross-scale self-similarity, and the blurring degree in downsampling is smaller than that of the current layer, its edges are sharper and have stronger similarity to the clear image. Therefore, the present invention proposes to search for a plurality of similar image blocks in the down-sampled image to form a similar image block group, constrain the sparsity of the similar image block group as a whole, and force the edge of the current image to be closer to the edge of the clear image.

The present invention gives a loose proof of this conclusion. Denote the two-dimensional coordinates as ξ, a certain image block in the clear image is expressed as f(ξ), and the blur kernel is h(ξ). Under the action of the blur kernel h(ξ), the clear image block f(ξ) corresponds to The fuzzy image block is expressed as q(ξ), then there is

q(ξ)=h(ξ)*f(ξ) (10)

Since multi-scale self-similarity generally exists in clear images, assuming that there is an image block similar to f(ξ) in the clear image, and its scale is a times the scale of f(ξ) (a>1), then the similarity An image block can be represented as f(ξ/a). Under the action of the blur kernel h(ξ), the blurred image block corresponding to the clear image block f(ξ/a) is expressed as r(ξ), then we have

r(ξ)=h(ξ)*f(ξ/a) (11)

If the blurred image is downsampled and its downsampling factor is a, the reduced image block corresponding to the blurred image block r(ξ) can be expressed as

r ^a (ξ)=r(aξ)=h(aξ)*f(ξ) (12)

According to formula (12), the image block ra (ξ) can be considered as the result of convolution of the clear image block ^f (ξ) and the blur kernel h(aξ). Since the size of h(aξ) is 1/a times the size of h(ξ), h(aξ) causes less blur to the image than h(ξ). By comparing equations (10) and (12), it can be known that the blur degree of ra (ξ) is less than q(ξ), that is, compared with the blurred image block ^q (ξ), the image block r in the down-sampled image of the blurred image ^a (ξ) is more similar to the clear image block f(ξ) and can provide more accurate additional information for the restoration of f(ξ). Fig. 6 intuitively illustrates the relationship of each image block in the above proof process.

和

其中N为清晰图像的大小，a表示降采样因子。从清晰图像X和降采样图像X^a中抽取的图像块分别表示为Q_jX和R_iX^a，其中

和

为抽取矩阵，分别用于从清晰图像和降采样图像中抽取第j个和第i个图像块，抽取的图像块尺寸为n。对于图像中的任意图像块Q_jX，在降采样图像X^a中搜索其相似图像块R_iX^a。由于图像的不同尺度间广泛存在着图像块的相似性，即对于Q_jX，可以在降采样图像中寻找多个与其相似的图像块，设在X^a中搜索m-1个与Q_jX最相似的图像块，并按列表示为

Q_jX与这些在降采样图像中的非局部相似图像块聚合构成一个相似图像块组P_j，可表示为：Because similar image blocks often appear in adjacent areas, the image block matching method is used to search for similar image blocks within a certain size search window in the down-sampled image. Let the clear image and its downsampled image be represented by columns as

and

where N is the size of the clear image and a is the downsampling factor. The image patches extracted from the clear image X and the downsampled image X ^a are denoted as Q _j X and R _i X ^a , respectively, where

and

is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image, respectively, and the size of the extracted image block is n. For any image block Q _j X in the image, search for its similar image block R _i X ^{a in the down-sampled image X a .} Due to the widespread similarity of image blocks between different scales of the image, that is, for Q _j X, multiple image blocks similar to it can be found in the down-sampled image, and it is assumed that m-1 in X ^a are searched for with Q _j X the most similar image patches, and are represented in columns as

Q _j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P _j , which can be expressed as:

Among them, n is the size of the image block, and m is the number of similar image blocks.

对于每一个输入图像块Q_jX，在

中找到对应位置的图像块

阈值Δd的计算式为：There are various metric criteria for the similarity of image blocks, such as Euclidean distance, correlation coefficient, etc. The present invention uses Euclidean distance as the basis for measuring the similarity between image blocks, and searches for similarity whose Euclidean distance is less than the set threshold Δd. image patches instead of searching for a fixed number of similar patches for each patch. For the search of similar image blocks, the present invention adopts an adaptive threshold method for judging the similarity of image blocks, performs interpolation shift on the original image X, and generates an image with 1/2 sub-pixel displacement.

For each input image patch Q _j X, in

Find the corresponding image block in

The calculation formula of the threshold Δd is:

Among them, γ is the control coefficient. It can be seen from equation (14) that for smooth image blocks, the threshold Δd is smaller, and for edge image blocks, the threshold Δd is larger; in other words, the more severe the content of the image block changes, the larger the threshold Δd; otherwise, the more the content of the image block changes. Flat, the smaller the threshold Δd, the purpose is to balance the number of similar image blocks searched for smooth image blocks and edge image blocks. In addition, the present invention sets the lower limit L _low and the upper limit L _high of the number of similar blocks to be searched, so that the number of similar blocks L _low ≤m≤L _high . If the number of searched similar blocks is less than L _low , this input image block is not used when establishing the equation system of equation (9). If the number of searched similar blocks is greater than L _high , then only the first L _high similar image blocks are selected.

写为

的形式，可表示为It can be seen from equation (13) that P _j is a similar image block group, and the first column is the image block Q _j X. In order to establish the relationship with the image X in the expression, in equation (9), the

written as

form, which can be expressed as

Let its derivative with respect to X be 0, the following equation can be obtained:

in,

Step 5.2 Reconstruct the reference image z _s by the group sparse representation;

因而该方程没有闭合解。本发明用

近似

求解字典D_j下的组稀疏表示系数，利用组稀疏表示对上一次迭代中估计的清晰图像

进行重建，作为更新

的参考图像。令

在交替迭代求解的过程中，z_s可以视为下一次迭代更新待估计清晰图像

的参考图像，其中，z_s是由当前对清晰图像的估计结果

及其降采样图像中的相似图像块通过组稀疏重建得到，由于降采样操作降低了图像的模糊程度，通过对相似图像块组矩阵进行整体的组稀疏表示，迫使重建图像的边缘更加清晰，z_s的模糊程度更小，用模糊程度更小的z_s作为参考图像，可使下一次迭代得到的对清晰图像的估计

比当前估计

更清晰，因此随着迭代次数的增加，清晰图像估计结果的模糊程度越来越小。这正是本发明利用跨尺度而非同尺度结构自相似性作为正则化约束的原因。若采用相同尺度结构自相似性作为正则化约束，则参考图像z_s是依据

中的相似图像块重建得到的，其模糊程度与

相当，因此不能作为更清晰的参考图像，从而不能在迭代过程中为待估计的清晰图像提供有效的信息。Since the unknown sparse representation coefficient α _j on the right-hand side of the equal sign of equation (16) depends on the solution of the equation

Therefore, the equation has no closed solution. for the present invention

approximate

Solve the group sparse representation coefficients under the dictionary D _j , and use the group sparse representation for the clear image estimated in the previous iteration

Rebuild as an update

reference image. make

In the process of alternate iterative solution, z _s can be regarded as the next iteration to update the clear image to be estimated

The reference image of , where z _s is the result of the current estimation of the sharp image

and the similar image blocks in the down-sampled image are obtained through group sparse reconstruction. Since the down-sampling operation reduces the blurring degree of the image, the overall group sparse representation of the similar image block group matrix forces the edges of the reconstructed image to be clearer, z The blur degree of _s is smaller, and using z _s with a smaller blur degree as the reference image can make the estimation of the clear image obtained in the next iteration

than current estimates

Sharper, so as the number of iterations increases, the blurriness of the result of the sharp image estimation becomes less and less. This is why the present invention utilizes cross-scale rather than intra-scale structural self-similarity as a regularization constraint. If the same-scale structure self-similarity is used as the regularization constraint, the reference image z _s is based on

It is reconstructed from similar image blocks in

Therefore, it cannot be used as a sharper reference image, and thus cannot provide effective information for the sharp image to be estimated in the iterative process.

中的每一个图像块，在其下采样图像

中搜索相似图像块，构成组矩阵。由于这些相似图像块合并的组矩阵应是低秩矩阵，本发明利用组稀疏表示与低秩矩阵的关联，通过约束表示系数的稀疏性来实现组矩阵的低秩性，无需考虑组稀疏表示字典构建的问题。稀疏表示中的各个数据向量在字典下的稀疏表示系数的求解是相互独立的，缺乏对数据在字典下表示系数的整体约束，而特定字典下秩最小化可以更好地表示数据的全局结构特征，以解决稀疏表示缺乏对重建图像全局结构约束的问题，可以提高对噪声的鲁棒性，进而提高模糊核估计的准确性。The approximate solution process of equation (16) will be described in detail below. for the current image

for each image block in which downsamples the image

Search for similar image blocks in the group to form a group matrix. Since the group matrix for merging these similar image blocks should be a low-rank matrix, the present invention utilizes the association between the group sparse representation and the low-rank matrix, and realizes the low-rank property of the group matrix by constraining the sparsity of the representation coefficients, without considering the group sparse representation dictionary build problem. The solution of the sparse representation coefficients of each data vector in the sparse representation under the dictionary is independent of each other, lacking the overall constraints on the coefficients of the data representation under the dictionary, and the rank minimization under a specific dictionary can better represent the global structural features of the data , to solve the problem that the sparse representation lacks constraints on the global structure of the reconstructed image, which can improve the robustness to noise, thereby improving the accuracy of blur kernel estimation.

First perform singular value decomposition on the group matrix P _j :

为秩1矩阵。Among them, ∑ _j =diag(σ _j,1 ,...,σ _j,r ) is the singular value diagonal matrix, σ _j,i ,i=1,...,r is the singular value of the matrix P _j , r=min( n, m), m and n represent the dimensions of the group matrix P _j . u _{j, i} and v _{j, i} are the column vectors of U _j and V _j , respectively,

is a rank 1 matrix.

按列排列为列向量

每一个相似图像块组P_j的自适应字典D_j由r个d_j，i构成，即

其中，d_j，i为字典D_j中的原子。将P_j按列排列为列向量

奇异值向量表示为

于是，式(17)可写为：Zha et al. propose an adaptive dictionary construction method as an explanation for the association of group sparse representations with low-rank matrices. Will

Arrange by column as a column vector

The adaptive dictionary D _j of each similar image block group P _j is composed of r d _{j, i} , that is,

Among them, d _{j, i} are atoms in the dictionary D _j . Arrange P _j column by column as a column vector

The singular value vector is represented as

So, equation (17) can be written as:

P _j =D _j A _j (18)

It can be seen from equation (18) that the rank function of the group matrix P _j is equivalent to the l ₀ norm of its singular value vector A _j , that is, rank(P _j )=||A _j || ₀ , ||·|| ₀ is the l ₀ norm of the vector. That is, the low rank of the group matrix is equivalent to the sparsity of the matrix singular value vector. In this way, the constrained sparsity represents the sparsity of the coefficients A _j , which is equivalent to the low rank of the constraint group matrix P _j . Since the l ₀ norm is a non-convex function, the l ₁ norm is used to perform convex relaxation on the l ₀ norm. Since the nuclear norm of the group matrix P _j is equivalent to the l ₁ norm of its singular value vector A _j , that is | |P _j || _* =||A _j || ₁ , using the nuclear norm of the matrix to perform convex relaxation on the rank function of the matrix to constrain the low rank of the group matrix P _j . Using the singular value threshold operator to solve the minimization problem of the group matrix P _j kernel norm ||P _j || _* , its closed solution is

是以β≥0为参数的∑_j软阈值算子，定义为in,

is the ∑ _j soft threshold operator with β≥0 as the parameter, defined as

的秩越小。

中的第一列即重建图像块

由于降采样中图像块的边缘更加清晰，与清晰图像具有更强的相似性，通过约束相似图像块组成矩阵的秩，迫使当前图像的边缘更接近清晰图像的边缘。It can be seen from equation (20) that the larger the β is, the more the group matrix is reconstructed.

the smaller the rank.

The first column in the reconstructed image block

Since the edge of the image block in downsampling is clearer and has a stronger similarity with the clear image, by constraining the rank of the matrix composed of similar image blocks, the edge of the current image is forced to be closer to the edge of the clear image.

Step 5.3 Solve

之后，对上一次迭代中估计的清晰图像

进行重建，作为更新

的参考图像。将通过求解线性方程组更新对

的估计，此时方程(16)可以近似表示为Obtaining approximate group sparse reconstruction blocks

After that, for the sharp image estimated in the previous iteration

Rebuild as an update

reference image. will update the pair by solving a system of linear equations

, then equation (16) can be approximately expressed as

Due to the effect of M, it cannot be quickly solved in the frequency domain by Fourier transform. The present invention adopts the Bi-Conjugate Gradient (BICG) method to solve the formula (21), and obtains

Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel;

并对清晰图像的估计

进行更新，更新为

如果此时的迭代达到最大迭代次数或者迭代收敛，则停止迭代。否则，令k＝k+1，然后重复步骤3、步骤4和步骤5，直至迭代达到最大迭代次数或者迭代收敛。Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel

and estimates for sharp images

to update, update to

If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped. Otherwise, let k=k+1, and then repeat step 3, step 4 and step 5 until the iteration reaches the maximum number of iterations or the iteration converges.

Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image.

在

的基础上采用有效的非盲复原算法获得最终的清晰图像估计。非盲复原算法有全变分正则化方法、稀疏非盲复原方法和EPLL算法等。Fuzzy kernel estimation corresponds to solving the optimization problem shown in equation (4), and obtains the estimation result of the fuzzy kernel

exist

On the basis of the effective non-blind restoration algorithm, the final clear image estimation is obtained. Non-blind restoration algorithms include total variation regularization method, sparse non-blind restoration method and EPLL algorithm.

Preferably, the present invention sets the image block size n=5×5, the singular value threshold β=0.2, the regularization parameters λ _s =0.0008, λ _g =0.002 and λ _h =0.0003N, the number of similar image blocks m=19 , the size of the search window is 25×25. The larger the downsampling factor a, the clearer the image blocks in the downsampling image of the blurred image, but at the same time the number of similar image blocks between images of different scales is smaller, so it is necessary to comprehensively consider setting the value of the downsampling factor. Set the scaling factor between pyramids to 4/3. Since blind restoration does not know the real size of the blur kernel, it is necessary to estimate the size of the blur kernel in other ways during restoration, or pre-set the size of the blur kernel. If the estimated or preset size of the blur kernel is too large, it is not easy to solve the small size of the blur kernel; if the estimated or preset size of the blur kernel is small, the complete blur kernel cannot be solved. The present invention sets the size of the blur kernel to be 51×51. Experiments on simulation and real data show that the blur kernel size of most blurred images is not greater than 51×51, and when the real blur kernel is small, the present invention can still obtain accurate estimation when the blur kernel size is set to 51×51 result. In the present invention, the iteration number of the loop is set to 14, and the iteration convergence threshold is not set.

The present invention verifies the disclosed method on the Koehler data set. The Koehler data set includes 4 images, each image has 12 types of blur kernels (the last 5 are large-sized blur kernels), and 48 blurred images are generated in total. For large size blur kernel, its initial size is set to 151×151. By comparing the peak signal-to-noise ratio (PSNR) of each deblurred result with 199 unblurred images captured along the camera motion trajectory, and recording the maximum PSNR as an indicator for quantitative evaluation. The larger the PSNR between the restored image and the ground-truth image, the closer the restored image is to the ground-truth image. The methods proposed by Pan et al. and Yan et al. are currently two widely recognized blind deblurring methods with the best performance. Among them, Pan et al. introduced a dark channel prior based on image block statistics in the kernel estimation, while Yan et al. Image regularization term for the patch bright channel prior. On the Koehler dataset, Figure 7 quantitatively compares the average PSNR of the method disclosed in the present invention and these two methods on each image. It can be seen that the method disclosed in the present invention is superior to the average PSNR of all images. The method of Yan et al. outperformed the method of Pan et al. on the average PSNR of the latter two images.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of this invention. Thus, the present invention is not intended to be limited to the embodiments of the present invention shown, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (6)

1. an image blind restoration method based on group sparse representation, is characterized in that, comprises the following steps: Step 1. Read in the blurred image and build an image pyramid; Read in the blurred image, set the initial size of the blur kernel, and downsample the blurred image layer by layer to construct an image pyramid. The size of the blur kernel corresponding to the image pyramids at different levels is different. Less than 3 × 3, that is, stop the construction of the image pyramid; Step 2. Initialize clear image; Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image

set up

Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer

Step 3. Screen the image blocks and estimate the marker matrix M; Set the marker matrix to M, and M is also a binary image. If the value of a pixel in M is 1, it indicates that the corresponding image block participates in the blur kernel estimation and group sparse representation constraints; for the restored image

Perform Gaussian filtering, and then estimate the edge of the filtered image; Step 4. Estimate the blur kernel; fix the estimate for the current image

Update the blur kernel for the next iteration with

where y represents the blurred image,

is the gradient operator of the image, λ _h is the regularization parameter, ⊙ means element-wise multiplication,

represents the Fourier transform,

represents the inverse Fourier transform,

Represents the complex conjugate of the Fourier transform; will restore the image

medium gradient

The gradient of pixels less than a certain threshold is set to 0; the gradient threshold is recorded as

The size of the blur kernel is N _h , then

The selection method is: first divide the image gradients into four groups according to their directions, and then set

value to ensure that each group retains at least

pixels are used to estimate the blur kernel; as the number of iterations increases, the restored image becomes clearer and clearer, in order to gradually add more pixels in the restored image to the process of blur kernel estimation, in each iteration,

The value of is reduced to 1.1 times of the previous iteration; Step 5. Estimate the clear image; an estimate given the current image

Fixed blur kernel estimation for next iteration

Update the image for the next iteration

Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel; Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel

and estimates for sharp images

to update, update to

If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped; otherwise, let k=k+1, k represents the number of iterations, and then repeat steps 3, 4 and 5; Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image; Through steps 1 to 6, the estimation results of the blur kernel are obtained

exist

On the basis of , an effective non-blind restoration algorithm is used to obtain the final clear image estimation; non-blind restoration algorithms include total variational regularization method, sparse non-blind restoration method and EPLL algorithm. 2. a kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, Step 5.1 Construct similar image block groups; Let the clear image and its downsampled image be represented by columns as

and

where N is the number of pixels in the clear image, a is the downsampling factor,

is an N-dimensional column vector,

is an N/a ² -dimensional column vector; the image blocks extracted from the clear image X and the down-sampled image X ^a are denoted as Q _j X and R _i X ^a , respectively, where

and

is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image respectively, and the size of the extracted image block is n; for any image block Q _j X in the image, in the down-sampled image X Search for similar image blocks R _i X ^a in ^a ; because the similarity of image blocks widely exists between different scales of the image, that is, for Q _j X, the image block matching method is used to search for multiple image blocks in a certain size search window in the down-sampled image. There are image blocks similar to it, suppose m-1 image blocks that are most similar to Q _j X are searched in X ^a , and they are represented by columns as

Q _j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P _j , which is expressed as:

Among them, n is the size of the image block, and m is the number of similar image blocks; Step 5.2 Reconstruct the reference image z _s by the group sparse representation; Using the singular value threshold operator to reconstruct the group matrix P _j , its closed solution is

Among them, S _β (∑ _j ) is the ∑ _j soft threshold operator with β>0 as the parameter, which is defined as S _β (∑ _j )=soft(∑ _j ; β)=max(∑ _j −β; 0) From the above formula, the larger the β is, the reconstruction of the group matrix

The smaller the rank is; the

approximate

The first column in the reconstructed image block

D _j is an adaptive dictionary; let

In the process of alternate iterative solution, z _s is used as the next iteration to update the clear image to be estimated

the reference image; Step 5.3 Solve

in getting

After that, update the pair by solving the following system of linear equations

Estimate:

where |M| is the number of non-zero elements in M, N is the number of image pixels,

is a one-dimensional column vector of y,

is the point spread function

The matrix form of , the fuzzy matrix,

is the gradient operator

The matrix form of ; due to the effect of M, it cannot be quickly solved in the frequency domain by Fourier transform, and the biconjugate gradient method is used to solve the above linear equation system, and obtain

3. a kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, The size n of the image block is 5×5. 4. a kind of image blind restoration method based on group sparse representation according to claim 2, is characterized in that, The downsampling factor a is 4/3. 5. A kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, The iteration number of the loop is 14, and no iteration convergence threshold is set. 6. A kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, The size of the blur kernel is 51×51.

Images