CN110675347B - Image blind restoration method based on group sparse representation - Google Patents

Image blind restoration method based on group sparse representation Download PDF

Info

Publication number
CN110675347B
CN110675347B CN201910938167.5A CN201910938167A CN110675347B CN 110675347 B CN110675347 B CN 110675347B CN 201910938167 A CN201910938167 A CN 201910938167A CN 110675347 B CN110675347 B CN 110675347B
Authority
CN
China
Prior art keywords
image
group
clear
estimate
sparse representation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910938167.5A
Other languages
Chinese (zh)
Other versions
CN110675347A (en
Inventor
禹晶
彭天奇
董醒儒
肖创柏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201910938167.5A priority Critical patent/CN110675347B/en
Publication of CN110675347A publication Critical patent/CN110675347A/en
Application granted granted Critical
Publication of CN110675347B publication Critical patent/CN110675347B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/10Image enhancement or restoration using non-spatial domain filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20016Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20048Transform domain processing
    • G06T2207/20056Discrete and fast Fourier transform, [DFT, FFT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an image blind restoration method based on group sparse representation, which is used for realizing blind restoration of a blurred image. The method comprises the following steps: constructing an image pyramid, and estimating a fuzzy kernel from coarse to fine; searching similar image blocks in the down-sampled image, combining the current block and the cross-scale similar block into a similar image block group, and establishing a group sparse representation regular term in a target function; and alternately updating the fuzzy core and the clear image by iteration, and in the clear image updating step, reconstructing the clear image estimated in the previous iteration by using the group sparse representation to be used as a reference image for updating the clear image at the next time. Because the edge of the downsampled image has stronger similarity with the clear image, the low rank of the group matrix is realized by using the association of the group sparse representation and the low rank matrix and restricting the sparsity of the representation coefficients, and the edge of the reconstructed image is forced to be close to the edge of the clear image. The method disclosed by the invention improves the robustness to noise and can process the fuzzy kernel estimation with large scale.

Description

一种基于组稀疏表示的图像盲复原方法A Blind Image Restoration Method Based on Group Sparse Representation

技术领域technical field

本发明涉及图像复原领域,更具体地说,涉及一种基于组稀疏表示的图像盲复原方法。The present invention relates to the field of image restoration, and more particularly, to an image blind restoration method based on group sparse representation.

背景技术Background technique

在图像采集和传输的过程中,由于成像系统、记录设备、传输介质等原因,成像设备获取的数字图像通常发生各种失真,造成图像质量降低,这种现象称为图像降质。图像复原旨在对图像降质过程进行建模,求解降质模型的逆过程,从降质的图像中恢复出原始的清晰图像。In the process of image acquisition and transmission, due to the imaging system, recording equipment, transmission medium and other reasons, the digital images obtained by the imaging equipment usually suffer from various distortions, resulting in the degradation of the image quality. This phenomenon is called image degradation. Image restoration aims to model the image degradation process, solve the inverse process of the degradation model, and restore the original clear image from the degraded image.

模糊是一种常见的图像降质现象,在数字成像设备采集图像的过程中,设备的抖动、散焦以及物体本身的运动都会导致模糊的产生。在某些应用场景下,当获取的图像存在模糊现象时,重新获取图像的成本较高或者根本不可行,例如非地球同步轨道卫星搭载的对地成像设备只有当卫星再一次经过某一地理区域时,才能对该区域重新获取图像,等待时间较长且再次成像时,该区域地表情况可能已经发生变化。此外,在成像过程中可以通过一些特殊器件的辅助来避免或减小模糊程度,例如采用三脚架等固定设备或光学防抖等器件,但三脚架一般不便于携带,应用比较受限,而光学防抖器件只能解决较小的抖动引起的模糊,对于较大的模糊无能为力。Blur is a common image degradation phenomenon. In the process of image acquisition by digital imaging equipment, the shaking of the equipment, defocusing and the motion of the object itself will lead to blurring. In some application scenarios, when the acquired image is blurred, the cost of re-acquiring the image is high or not feasible. When the image is re-acquired in the area, the surface conditions of the area may have changed when the waiting time is long and the image is re-imaged. In addition, in the imaging process, the degree of blurring can be avoided or reduced with the assistance of some special devices, such as the use of fixed devices such as tripods or optical image stabilization devices, but tripods are generally inconvenient to carry and have limited applications, while optical image stabilization The device can only resolve the blur caused by small jitter, and can't do anything about the larger blur.

单幅图像盲复原是通过图像处理的手段,在降质模型或降质参数未知的情况下,仅从单幅模糊图像中复原出原始的清晰图像,有效去除图像中模糊的技术。由成像系统获取图像的过程为正问题,利用降质图像中恢复潜在的原始图像是典型的图像逆问题。通常情况下,降质矩阵是奇异的,该问题是病态的。根据降质模型是否可用分为两类:若降质矩阵已知,则称为非盲复原方法;若降质矩阵未知,则称为盲复原问题。在实际应用中,通常无法获知观测图像的降质模型或降质参数,因此,图像盲复原更具有实用性。Blind restoration of a single image is a technology that effectively removes the blur in the image by restoring the original clear image from a single blurred image only when the degradation model or degradation parameters are unknown by means of image processing. The process of acquiring the image by the imaging system is a positive problem, and the use of the degraded image to restore the potential original image is a typical image inverse problem. Typically, the degraded matrix is singular and the problem is ill-conditioned. According to whether the degradation model is available, it is divided into two categories: if the degradation matrix is known, it is called a non-blind restoration method; if the degradation matrix is unknown, it is called a blind restoration problem. In practical applications, it is usually impossible to know the degradation model or degradation parameters of the observed image, so the blind image restoration is more practical.

由于图像模糊的降质过程一般表示为模糊核与清晰图像的二维卷积,因而去模糊问题也称为解卷积问题。图像盲解卷积是一个严重的欠定问题,待求解的未知变量数目大于已知变量的数目,解不惟一。为了寻找准确解,有必要引入关于图像的先验知识,称为图像先验模型,将其作为正则化约束项加入到重建过程中,这个过程称为正则化。正则化约束为解决病态问题提供额外的附加信息,约束可行解的空间。现有的盲解卷积算法都直接或间接地利用各种先验知识,这些算法大致可以分为两类,一类算法利用启发式的边缘增强方法,另一类算法直接建立模糊核或清晰图像的先验概率分布模型。Since the degradation process of image blur is generally expressed as a two-dimensional convolution of a blur kernel and a clear image, the deblurring problem is also called a deconvolution problem. Image blind deconvolution is a serious underdetermined problem, the number of unknown variables to be solved is greater than the number of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce prior knowledge about the image, called the image prior model, and add it to the reconstruction process as a regularization constraint. This process is called regularization. Regularization constraints provide additional additional information for solving ill-conditioned problems, constraining the space of feasible solutions. Existing blind deconvolution algorithms all use various prior knowledge directly or indirectly. These algorithms can be roughly divided into two categories. One type of algorithm uses heuristic edge enhancement methods, and the other type of algorithm directly establishes blur kernels or sharp edges. A prior probability distribution model for the image.

基于启发式边缘增强的盲解卷积方法假设模糊图像中有充分的边缘,通过增强图像中的边缘对清晰图像进行近似估计,最后利用边缘增强的近似估计和模糊图像的对应关系估计模糊核。由于边缘增强算法的能力有限,一般不可能仅通过一次增强过程就得到清晰的边缘,因而在盲解卷积过程中需要不断迭代,即每一次对估计的清晰图像继续进行边缘增强,以逐步逼近原始清晰图像。由于这一类盲解卷积方法直接用启发式的方法增强模糊图像的边缘,一般没有统一的目标函数,算法复杂度较低,但在迭代求解的过程中,为了避免出现边缘过增强等现象,一般需要根据迭代次数不断调节边缘增强算法的参数,因而这一类方法对参数设置较为敏感。The blind deconvolution method based on heuristic edge enhancement assumes that there are sufficient edges in the blurred image, approximates the clear image by enhancing the edges in the image, and finally estimates the blur kernel by using the approximate estimation of edge enhancement and the corresponding relationship of the blurred image. Due to the limited capability of the edge enhancement algorithm, it is generally impossible to obtain a clear edge through only one enhancement process. Therefore, continuous iteration is required in the blind deconvolution process, that is, the edge enhancement is continued on the estimated clear image each time to gradually approach Original clear image. Because this type of blind deconvolution method directly uses heuristic methods to enhance the edge of the blurred image, there is generally no unified objective function, and the algorithm complexity is low, but in the iterative solution process, in order to avoid the phenomenon of excessive edge enhancement, etc. , it is generally necessary to continuously adjust the parameters of the edge enhancement algorithm according to the number of iterations, so this type of method is more sensitive to parameter settings.

另一类盲解卷积方法为模糊核或清晰图像建立先验概率分布模型,通过求解最大后验问题来估计模糊核和清晰图像。最优化目标函数中包含了对清晰图像和模糊核的正则化约束,该正则化约束应有助于解决盲解卷积问题的欠定性、限定可行解的空间。如何选取合适的正则化约束是盲解卷积方法需要研究的关键。前期的工作主要利用关于图像梯度的先验知识,图像梯度表示图像相邻像素之间的关系,并不足以表示较大尺度的图像结构。近年来,一些盲复原方法利用了关于图像块的先验信息,相比于利用梯度先验的算法,利用图像块先验知识的盲复原方法的准确性和稳健性都有较大的改善。Another class of blind deconvolution methods builds a prior probability distribution model for the blur kernel or sharp image, and estimates the blur kernel and sharp image by solving the maximum a posteriori problem. The optimization objective function contains regularization constraints on sharp images and blur kernels, which should help solve the underdetermination of blind deconvolution problems and limit the space of feasible solutions. How to choose an appropriate regularization constraint is the key to the study of blind deconvolution methods. Previous work mainly utilizes prior knowledge about image gradients, which represent the relationship between adjacent pixels in an image and are not sufficient to represent larger-scale image structures. In recent years, some blind restoration methods utilize prior information about image blocks. Compared with algorithms that utilize gradient priors, blind restoration methods utilizing prior knowledge of image blocks have greater improvements in accuracy and robustness.

基于稀疏表示的方法则利用了图像在某组基(或字典)下具有稀疏表示形式,并将这种稀疏性作为正则化约束项加入目标函数中。基于稀疏表示的图像盲复原方法的关键是构造表示清晰图像的字典,使得图像块在字典下具有稀疏表示形式。稀疏表示有两个明显的问题:1)字典学习需要很大的计算量;2)忽视不同块之间的相关性。稀疏框架下K-SVD算法是最常用的自适应字典学习方法,这种字典学习通常需要一定规模的数据集来构建字典,利用数据集作为字典学习的样本。一方面,在这种全局字典的构建过程中,为了使各种不同类型的图像块在字典下均能稀疏表示,必须利用大量的样本进行字典学习,这导致字典学习效率低;另一方面,当字典学习的样本不能准确提供与待处理低分辨率图像相似的信息时,这种方法难以保证图像重建的效果。此外,稀疏表示中的各个数据向量在字典下的稀疏表示系数的求解是相互独立的,缺乏对重建图像全局结构约束的问题。The methods based on sparse representation take advantage of the sparse representation of images under a certain set of bases (or dictionaries), and add this sparseness as a regularization constraint to the objective function. The key to the blind image restoration method based on sparse representation is to construct a dictionary representing clear images, so that the image block has a sparse representation under the dictionary. There are two obvious problems with sparse representation: 1) dictionary learning requires a large amount of computation; 2) it ignores the correlation between different blocks. The K-SVD algorithm under the sparse framework is the most commonly used adaptive dictionary learning method. This kind of dictionary learning usually requires a certain scale of data set to construct a dictionary, and uses the data set as a sample for dictionary learning. On the one hand, in the construction of this global dictionary, in order to make various types of image patches sparsely represented under the dictionary, a large number of samples must be used for dictionary learning, which leads to low dictionary learning efficiency; on the other hand, When the samples of dictionary learning cannot accurately provide similar information to the low-resolution images to be processed, it is difficult for this method to guarantee the effect of image reconstruction. In addition, the solution of the sparse representation coefficients under the dictionary for each data vector in the sparse representation is independent of each other, which lacks the problem of constraints on the global structure of the reconstructed image.

尽管目前已经提出许多图像盲复原方法,然而绝大多数均假设低噪声水平的情况。图像盲复原是病态逆问题,在缺乏噪声的先验知识时,图像盲复原会放大噪声,导致无法准确估计模糊核。对于广泛研究的基于图像梯度稀疏先验的图像盲复原方法,图像噪声会破坏梯度分布的先验预测,从而导致模糊核的估计产生一定的偏差。此外,启发式边缘增强的盲复原模型通常会寻找显著边缘来估计模糊核,模糊图像中的噪声会引起边缘定位的误差,从而引起模糊核估计的偏差,导致图像盲复原的失败。Although many blind image restoration methods have been proposed, most of them assume low noise levels. Blind image restoration is an ill-conditioned inverse problem. In the absence of prior knowledge of noise, blind image restoration will amplify the noise, resulting in the inability to accurately estimate the blur kernel. For the widely studied blind image restoration method based on image gradient sparse prior, image noise will destroy the prior prediction of gradient distribution, resulting in a certain bias in the estimation of blur kernel. In addition, the blind restoration model of heuristic edge enhancement usually looks for salient edges to estimate the blur kernel, and the noise in the blurred image will cause the error of edge localization, which will cause the bias of blur kernel estimation and lead to the failure of image blind restoration.

发明内容SUMMARY OF THE INVENTION

本发明公开了一种基于组稀疏表示的图像盲复原方法,通过在特定字典下约束对各个组稀疏表示系数进行整体的稀疏性约束,实现各个组矩阵的秩最小化,迫使当前图像的边缘更接近清晰图像的边缘。组稀疏表示可以融合图像的局部稀疏与非局部自相似先验,特定字典下秩最小化可以更好地表示数据的全局结构特性。本发明公开的方法提高了对噪声的鲁棒性,能够处理受噪声干扰的模糊图像,并能够估计大尺度的模糊核,对大尺度模糊图像进行复原。The invention discloses an image blind restoration method based on group sparse representation. By constraining the overall sparsity constraint of each group sparse representation coefficient under a specific dictionary, the rank of each group matrix is minimized, and the edge of the current image is forced to be more near the edge of a sharp image. The group sparse representation can fuse the local sparse and non-local self-similar priors of the image, and the rank minimization under a specific dictionary can better represent the global structural properties of the data. The method disclosed in the invention improves the robustness to noise, can process blurred images disturbed by noise, and can estimate large-scale blur kernels to restore large-scale blurred images.

本发明采用的技术方案为一种基于组稀疏表示的图像盲复原方法,其特征在于,包含以下步骤:The technical solution adopted in the present invention is a method for blind restoration of images based on group sparse representation, which is characterized in that it comprises the following steps:

步骤1.读入模糊图像,构建图像金字塔;Step 1. Read in the blurred image and build an image pyramid;

读入模糊图像,设置模糊核的初始尺寸,对模糊图像逐层下采样构造图像金字塔,不同层级图像金字塔对应的模糊核尺寸不同,在构造图像金字塔时,如果当前层图像金字塔对应的模糊核尺寸小于3×3,即停止图像金字塔的构造。在估计过程中,粗一层图像金字塔上估计的清晰图像通过图像插值后作为细一层图像金字塔上清晰图像的初始估计。在图像金字塔的每一层上,建立的盲解卷积目标函数是相同的。以下步骤是图像金字塔每一层上盲解卷积目标函数的求解过程。Read in the blurred image, set the initial size of the blur kernel, and downsample the blurred image layer by layer to construct an image pyramid. The size of the blur kernel corresponding to the image pyramids at different levels is different. Less than 3×3, that stops the construction of the image pyramid. In the estimation process, the clear image estimated on the image pyramid of the coarse layer is used as the initial estimate of the clear image on the image pyramid of the thin layer after image interpolation. At each layer of the image pyramid, the established blind deconvolution objective function is the same. The following steps are the solution process of the blind deconvolution objective function on each layer of the image pyramid.

步骤2.初始化清晰图像;Step 2. Initialize clear image;

记循环迭代次数为k,且初始化设置为k=0,若当前层为图像金字塔的第一层,则将模糊图像y作为清晰图像初始估计

Figure BDA0002222144890000041
设置
Figure BDA0002222144890000042
否则粗一层金字塔估计的清晰图像的插值结果作为当前层清晰图像的初始估计
Figure BDA0002222144890000043
Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image
Figure BDA0002222144890000041
set up
Figure BDA0002222144890000042
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer
Figure BDA0002222144890000043

步骤3.筛选图像块,估计标记矩阵M;Step 3. Screen the image blocks and estimate the marker matrix M;

设置标记矩阵为M,M同时也为二值图像,若M中某一像素的取值为1,表明对应的图像块参与模糊核估计以及组稀疏表示约束。由于本方法仅将组稀疏性正则化约束限制在图像的边缘区域,导致复原图像

Figure BDA0002222144890000051
的背景平滑区域受到的约束较少,从而可能导致复原图像的平滑区域含有较多的噪声,为减小噪声对边缘估计造成的干扰,首先对复原图像
Figure BDA0002222144890000052
进行高斯滤波,然后对滤波后的图像估计边缘。Set the marker matrix to M, and M is also a binary image. If the value of a pixel in M is 1, it indicates that the corresponding image block participates in blur kernel estimation and group sparse representation constraints. Since this method only restricts the group sparsity regularization constraint to the edge region of the image, the restored image
Figure BDA0002222144890000051
The background smooth area is less constrained, which may cause the smooth area of the restored image to contain more noise. In order to reduce the interference of noise on edge estimation, first of all
Figure BDA0002222144890000052
Gaussian filtering is performed, and then edges are estimated on the filtered image.

步骤4.对模糊核进行估计;Step 4. Estimate the blur kernel;

固定当前图像的估计

Figure BDA0002222144890000053
用下式更新下一次迭代的模糊核
Figure BDA0002222144890000054
fix the estimate for the current image
Figure BDA0002222144890000053
Update the blur kernel for the next iteration with
Figure BDA0002222144890000054

Figure BDA0002222144890000055
Figure BDA0002222144890000055

式中,y表示模糊图像,

Figure BDA0002222144890000056
为图像的梯度算子,λh为正则化参数,⊙表示逐元素相乘,
Figure BDA0002222144890000057
表示傅里叶变换,
Figure BDA0002222144890000058
表示傅里叶逆变换,
Figure BDA0002222144890000059
表示傅里叶变化的复共轭。将复原图像
Figure BDA00022221448900000510
中梯度
Figure BDA00022221448900000511
小于一定阈值的像素点的梯度置为0。记梯度阈值为τ,模糊核大小为Nh,则τ的选取方式为:首先将图像梯度根据其方向分为四组,然后设置τ的取值以保证每一组都保留至少有
Figure BDA00022221448900000512
个像素点用于估计模糊核。由于随着迭代次数的增加,复原图像越来越清晰,为了使复原图像中更多的像素点逐步加入模糊核估计的过程中,在每一次迭代时将τ的值缩小为上一次迭代时的1.1倍。where y represents the blurred image,
Figure BDA0002222144890000056
is the gradient operator of the image, λ h is the regularization parameter, ⊙ means element-wise multiplication,
Figure BDA0002222144890000057
represents the Fourier transform,
Figure BDA0002222144890000058
represents the inverse Fourier transform,
Figure BDA0002222144890000059
Represents the complex conjugate of the Fourier transform. will restore the image
Figure BDA00022221448900000510
medium gradient
Figure BDA00022221448900000511
The gradient of pixels less than a certain threshold is set to 0. Denote the gradient threshold as τ and the size of the blur kernel as N h , then the selection method of τ is as follows: first, the image gradients are divided into four groups according to their directions, and then the value of τ is set to ensure that each group retains at least some
Figure BDA00022221448900000512
pixels are used to estimate the blur kernel. Since the restored image becomes clearer with the increase of the number of iterations, in order to gradually add more pixels in the restored image to the blur kernel estimation process, the value of τ is reduced to the value of the previous iteration at each iteration. 1.1 times.

步骤5.对清晰图像进行估计;Step 5. Estimate the clear image;

给定当前图像的估计

Figure BDA00022221448900000513
固定下一次迭代的模糊核估计
Figure BDA00022221448900000514
更新下一次迭代的图像
Figure BDA00022221448900000515
an estimate given the current image
Figure BDA00022221448900000513
Fixed blur kernel estimation for next iteration
Figure BDA00022221448900000514
Update the image for the next iteration
Figure BDA00022221448900000515

步骤5.1构建相似图像块组;Step 5.1 Construct similar image block groups;

设清晰图像及其降采样图像按列分别表示为

Figure BDA00022221448900000516
Figure BDA00022221448900000517
其中N为清晰图像的像素数目,a表示降采样因子,
Figure BDA0002222144890000061
为N维列向量,
Figure BDA0002222144890000062
为N/a2维列向量。从清晰图像X和降采样图像Xa中抽取的图像块分别表示为QjX和RiXa,其中
Figure BDA0002222144890000063
Figure BDA0002222144890000064
为抽取矩阵,分别用于从清晰图像和降采样图像中抽取第j个和第i个图像块,抽取的图像块尺寸为n。对于图像中的任意图像块QjX,在降采样图像Xa中搜索相似图像块RiXa。由于图像的不同尺度间广泛存在着图像块的相似性,即对于QjX,在降采样图像中的一定尺寸搜索窗口内利用图像块匹配方法寻找多个与其相似的图像块,设在Xa中搜索m-1个与QjX最相似的图像块,并按列表示为
Figure BDA0002222144890000065
QjX与这些在降采样图像中的非局部相似图像块聚合构成一个相似图像块组Pj,表示为:Let the clear image and its downsampled image be represented by columns as
Figure BDA00022221448900000516
and
Figure BDA00022221448900000517
where N is the number of pixels in the clear image, a is the downsampling factor,
Figure BDA0002222144890000061
is an N-dimensional column vector,
Figure BDA0002222144890000062
is an N/a 2 -dimensional column vector. The image patches extracted from the clear image X and the downsampled image X a are denoted as Q j X and R i X a , respectively, where
Figure BDA0002222144890000063
and
Figure BDA0002222144890000064
is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image, respectively, and the size of the extracted image block is n. For any image block Q j X in the image, a similar image block R i X a is searched in the down-sampled image X a . Due to the widespread similarity of image blocks between different scales of the image, that is, for Q j X, the image block matching method is used to find a number of similar image blocks in a certain size search window in the down-sampled image, set at X a Search for m-1 image patches that are most similar to Q j X in the
Figure BDA0002222144890000065
Q j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P j , which is expressed as:

Figure BDA0002222144890000066
Figure BDA0002222144890000066

其中,n为图像块尺寸,m为相似图像块的个数。Among them, n is the size of the image block, and m is the number of similar image blocks.

步骤5.2通过组稀疏表示重建参考图像zsStep 5.2 Reconstruct the reference image z s by the group sparse representation;

利用奇异值阈值算子对组矩阵Pj进行重建,其闭合解为Using the singular value threshold operator to reconstruct the group matrix P j , its closed solution is

Figure BDA0002222144890000067
Figure BDA0002222144890000067

其中,

Figure BDA00022221448900000618
是以β≥0为参数的∑j软阈值算子,定义为
Figure BDA00022221448900000619
in,
Figure BDA00022221448900000618
is the ∑ j soft threshold operator with β≥0 as the parameter, defined as
Figure BDA00022221448900000619

由上式知,β越大,重建组矩阵

Figure BDA0002222144890000068
的秩越小。用
Figure BDA0002222144890000069
近似
Figure BDA00022221448900000610
Figure BDA00022221448900000611
中的第一列即重建图像块
Figure BDA00022221448900000612
Dj为自适应字典。令
Figure BDA00022221448900000613
在交替迭代求解的过程中,zs作为下一次迭代更新待估计清晰图像
Figure BDA00022221448900000614
的参考图像。From the above formula, the larger the β, the reconstruction of the group matrix
Figure BDA0002222144890000068
the smaller the rank. use
Figure BDA0002222144890000069
approximate
Figure BDA00022221448900000610
Figure BDA00022221448900000611
The first column in the reconstructed image block
Figure BDA00022221448900000612
D j is an adaptive dictionary. make
Figure BDA00022221448900000613
In the process of alternate iterative solution, z s is used as the next iteration to update the clear image to be estimated
Figure BDA00022221448900000614
reference image.

步骤5.3求解

Figure BDA00022221448900000615
Step 5.3 Solve
Figure BDA00022221448900000615

在获得

Figure BDA00022221448900000616
之后,通过求解如下线性方程组更新对
Figure BDA00022221448900000617
的估计:in getting
Figure BDA00022221448900000616
After that, update the pair by solving the following system of linear equations
Figure BDA00022221448900000617
Estimate:

Figure BDA0002222144890000071
Figure BDA0002222144890000071

式中,|M|为M中非零元的个数,N为图像像素个数,

Figure BDA0002222144890000072
为y的一维列向量,
Figure BDA0002222144890000073
为点扩散函数
Figure BDA0002222144890000074
的矩阵形式,即模糊矩阵,
Figure BDA0002222144890000075
为梯度算子
Figure BDA0002222144890000076
的矩阵形式。由于M的作用,无法通过傅里叶变换在频域中快速求解,采取双共轭梯度法来求解如上线性方程组,获得
Figure BDA0002222144890000077
where |M| is the number of non-zero elements in M, N is the number of image pixels,
Figure BDA0002222144890000072
is a one-dimensional column vector of y,
Figure BDA0002222144890000073
is the point spread function
Figure BDA0002222144890000074
The matrix form of , the fuzzy matrix,
Figure BDA0002222144890000075
is the gradient operator
Figure BDA0002222144890000076
matrix form. Due to the effect of M, it cannot be quickly solved in the frequency domain through Fourier transform, and the double conjugate gradient method is used to solve the above linear equation system, and obtain
Figure BDA0002222144890000077

步骤6.判断收敛,获得模糊核的估计;Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel;

通过步骤3、步骤4和步骤5,进行对目标函数的一次迭代求解,获得对模糊核的估计

Figure BDA0002222144890000078
并对清晰图像的估计
Figure BDA0002222144890000079
进行更新,更新为
Figure BDA00022221448900000710
如果此时的迭代达到最大迭代次数或者迭代收敛,则停止迭代。否则,令k=k+1,k代表迭代次数,然后重复步骤3、步骤4和步骤5。Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel
Figure BDA0002222144890000078
and estimates for sharp images
Figure BDA0002222144890000079
to update, update to
Figure BDA00022221448900000710
If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped. Otherwise, let k=k+1, where k represents the number of iterations, and then repeat step 3, step 4, and step 5.

步骤7.模糊核估计之后,利用非盲复原算法对清晰图像进行估计。Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image.

通过步骤1至步骤6,获得对模糊核的估计结果

Figure BDA00022221448900000711
Figure BDA00022221448900000712
的基础上采用有效的非盲复原算法获得最终的清晰图像估计。非盲复原算法有全变分正则化方法、稀疏非盲复原方法和EPLL算法等。Through steps 1 to 6, the estimation results of the blur kernel are obtained
Figure BDA00022221448900000711
exist
Figure BDA00022221448900000712
On the basis of the effective non-blind restoration algorithm, the final clear image estimation is obtained. Non-blind restoration algorithms include total variation regularization method, sparse non-blind restoration method and EPLL algorithm.

优选的,所述对图像块进行筛选,排除平滑区域对模糊核估计干扰的具体实现方式为:通过边缘检测方法,对当前图像估计进行边缘估计,利用边缘像素对应的图像块估计模糊核和清晰图像。Preferably, the specific implementation method of screening the image blocks and eliminating the interference of smooth regions on the estimation of the blur kernel is: performing edge estimation on the current image estimation by an edge detection method, and using the image blocks corresponding to the edge pixels to estimate the blur kernel and sharpness. image.

优选的,所述图像块的尺寸n为5×5。Preferably, the size n of the image block is 5×5.

优选的,所述奇异值阈值β为0.2。Preferably, the singular value threshold β is 0.2.

优选的,所述相似图像块的个数m为19。Preferably, the number m of the similar image blocks is 19.

优选的,所述搜索窗口的大小为25×25。Preferably, the size of the search window is 25×25.

优选的,所述降采样因子a为4/3。Preferably, the downsampling factor a is 4/3.

优选的,所述循环的迭代次数为14,不设置迭代收敛阈值。Preferably, the number of iterations of the loop is 14, and no iteration convergence threshold is set.

优选的,所述模糊核的尺寸为51×51。Preferably, the size of the blur kernel is 51×51.

由于图像盲复原是严重的病态逆问题,在缺乏噪声的先验知识时,图像盲复原会放大噪声,导致无法准确估计模糊核。稀疏表示用尽可能少的字典元素的线性组合来表示各个图像块,且各个图像块在字典下的稀疏表示系数的求解是相互独立的,因此稀疏表示缺乏对数据在字典下表示系数的整体约束。由于图像具有自相似性,组稀疏表示可以从整体上约束组稀疏表示系数的稀疏性。本发明提出一种组稀疏表示的图像先验约束项,用于正则化图像盲复原问题的解。由于模糊的程度随着图像降采样减小,降采样图像的边缘与清晰图像具有更强的相似性,因此,利用跨尺度自相似性,本发明在降采样图像中搜索多个相似的图像块,构成相似块图像组,由于这些相似图像块合并的组矩阵应具有低秩性,利用组稀疏表示与低秩矩阵的关联,通过约束表示系数的稀疏性来实现组矩阵的低秩性,无需考虑组稀疏表示字典构建的问题。低秩约束能够避免盲复原模型受噪声的干扰,同时通过与降采样图像中跨尺度相似块的整体约束,迫使当前图像的边缘在迭代中更加清晰,使重建图像的边缘接近清晰图像的边缘。Since blind image restoration is a serious ill-conditioned inverse problem, in the absence of prior knowledge of noise, blind image restoration will amplify the noise, resulting in the inability to accurately estimate the blur kernel. The sparse representation uses the linear combination of as few dictionary elements as possible to represent each image block, and the solution of the sparse representation coefficients of each image block under the dictionary is independent of each other, so the sparse representation lacks the overall constraints on the data representation coefficients under the dictionary . Due to the self-similarity of images, the group sparse representation can constrain the sparsity of the group sparse representation coefficients as a whole. The present invention proposes an image prior constraint term of group sparse representation, which is used for the solution of regularized image blind restoration problem. Since the degree of blurring decreases with image downsampling, the edge of the downsampled image is more similar to the clear image. Therefore, the present invention searches for a plurality of similar image blocks in the downsampled image by using cross-scale self-similarity. , constitute a similar block image group. Since the group matrix of these similar image blocks merged should have low rank, the low rank of the group matrix is achieved by constraining the sparsity of the representation coefficient by using the association between the group sparse representation and the low rank matrix. Consider the problem of group sparse representation dictionary construction. The low-rank constraint can avoid the interference of the blind restoration model by noise, and at the same time, through the overall constraint with the cross-scale similar blocks in the down-sampled image, the edges of the current image are forced to be clearer in the iteration, so that the edges of the reconstructed image are close to the edges of the clear image.

附图说明Description of drawings

为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

图1为本发明实施例所提供的图像盲复原的示意图;1 is a schematic diagram of blind image restoration provided by an embodiment of the present invention;

图2为本发明实施例所提供的基于组稀疏表示的图像盲复原方法的流程图;2 is a flowchart of an image blind restoration method based on group sparse representation provided by an embodiment of the present invention;

图3为本发明实施例所提供的图像金字塔结构图;3 is an image pyramid structure diagram provided by an embodiment of the present invention;

图4为本发明实施例所提供的构造相似图像块组的示意图;4 is a schematic diagram of constructing similar image block groups according to an embodiment of the present invention;

图5为本发明实施例所提供的清晰图像与模糊图像以及模糊图像降采样图像的相似性示意图;5 is a schematic diagram of similarity between a clear image and a blurred image and a down-sampled image of the blurred image provided by an embodiment of the present invention;

图6为本发明实施例所提供的边缘图像块标记矩阵示例;6 is an example of an edge image block marking matrix provided by an embodiment of the present invention;

图7为本发明实施例所提供的在Koehler数据集上各种方法的平均PSNR比较。FIG. 7 is a comparison of the average PSNR of various methods on the Koehler dataset provided by an embodiment of the present invention.

具体实施方式Detailed ways

下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

从清晰图像退化为模糊图像的降质过程通常表示为如下所示的卷积模型:The degradation process from a clear image to a blurred image is usually expressed as a convolutional model as shown below:

y=h*x+n (1)y=h*x+n (1)

其中,y表示模糊图像,x表示清晰图像,*表示卷积操作,h为模糊核,n为噪声。这里,模糊核h和噪声n包含降质模型的全部信息。在卷积模型下,图像盲复原方法即研究如何从模糊图像y中同时估计出模糊核h和清晰图像x,如图1所示。由于将模糊过程建模为卷积的形式,因此盲去模糊问题也称为盲解卷积问题。Among them, y represents the blurred image, x represents the clear image, * represents the convolution operation, h is the blur kernel, and n is the noise. Here, the blur kernel h and noise n contain all the information of the degraded model. Under the convolution model, the blind image restoration method studies how to simultaneously estimate the blur kernel h and the clear image x from the blurred image y, as shown in Figure 1. Since the blurring process is modeled as a form of convolution, the blind deblurring problem is also known as the blind deconvolution problem.

图像盲解卷积是一个欠定问题,待求解的未知变量数目大于已知变量的数目,解不惟一。为了寻找准确解,有必要引入关于图像的先验知识,称为图像先验模型,表示为Ψ(x),将其作为正则化约束项加入到目标函数中,这个过程称为正则化。正则化约束为解决病态问题提供额外的附加信息,约束目标函数的解空间,正则化方法通常表示为如下的最优化问题:Image blind deconvolution is an underdetermined problem, the number of unknown variables to be solved is greater than the number of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce prior knowledge about the image, called the image prior model, denoted as Ψ(x), and add it to the objective function as a regularization constraint. This process is called regularization. Regularization constraints provide additional additional information for solving ill-conditioned problems and constrain the solution space of the objective function. Regularization methods are usually expressed as the following optimization problems:

Figure BDA0002222144890000101
Figure BDA0002222144890000101

式中,Ψx(x)和Ψh(h)分别为清晰图像x和模糊核h的先验约束,常数λx和λh为拉格朗日乘子,也称为正则化参数。第一项表示数据保真项,后两项表示正则化约束项,正则化参数λx和λh的作用是平衡目标函数中重建误差和先验约束的比重。where Ψ x (x) and Ψ h (h) are the prior constraints of the clear image x and the blur kernel h, respectively, and the constants λ x and λ h are Lagrange multipliers, also known as regularization parameters. The first term represents the data fidelity term, and the last two terms represent the regularization constraint term. The role of the regularization parameters λx and λh is to balance the proportion of reconstruction error and prior constraints in the objective function.

图像先验模型在图像盲解卷积问题中占据着非常重要的地位,如何设计有效的正则化约束项来描述图像的先验知识或附加信息是图像盲解卷积方法的关键问题。本发明从图像自身所具有的多尺度自相似性挖掘先验知识,通过构造正则化约束项将多尺度自相似结构中蕴含的附加信息有效地加入到重建图像中,将图像盲解卷积问题转换为寻找满足特定约束条件解的最优化问题。Image prior model occupies a very important position in the problem of blind image deconvolution. How to design effective regularization constraints to describe the prior knowledge or additional information of images is the key issue of blind image deconvolution methods. The invention mines prior knowledge from the multi-scale self-similarity of the image itself, and effectively adds the additional information contained in the multi-scale self-similar structure to the reconstructed image by constructing regularization constraints, so as to solve the problem of blind deconvolution of the image. Converted to an optimization problem that finds a solution that satisfies certain constraints.

图像中广泛存在着多尺度自相似结构,多尺度自相似性可以为图像盲复原提供必要的附加信息。不论是航拍飞机、舰船等显在的相似性,还是房屋道路边缘、山川河流脉络等潜在的相似性,这种自相似性具体表现为图像中所具有的相同尺度以及不同尺度的相似图像块。设清晰图像组成的一维列向量表示为

Figure BDA0002222144890000102
从中抽取出图像块并按列表示为QjX,其中
Figure BDA0002222144890000103
为抽取矩阵,通过图像块匹配方法在图像中搜索QjX的相似图像块并按列表示为
Figure BDA0002222144890000111
将这些非局部相似图像块聚合为一个相似图像组,记为
Figure BDA0002222144890000112
其中,m为相似图像块的个数。与稀疏表示类似,设计一个字典
Figure BDA0002222144890000113
这里t为字典大小,相似图像块组
Figure BDA0002222144890000114
的稀疏表示通过求解如下最小化问题:Multi-scale self-similar structures widely exist in images, and multi-scale self-similarity can provide necessary additional information for blind image restoration. Whether it is the apparent similarity of aerial photography planes, ships, etc., or the potential similarity of houses, road edges, mountains and rivers, etc., this self-similarity is embodied in the image with the same scale and different scales. Similar image blocks . Let the one-dimensional column vector composed of clear images be expressed as
Figure BDA0002222144890000102
Image patches are extracted from them and denoted by columns as Q j X, where
Figure BDA0002222144890000103
In order to extract the matrix, the similar image blocks of Q j X are searched in the image by the image block matching method and expressed in columns as
Figure BDA0002222144890000111
Aggregate these non-locally similar image patches into a similar image group, denoted as
Figure BDA0002222144890000112
Among them, m is the number of similar image blocks. Similar to sparse representation, design a dictionary
Figure BDA0002222144890000113
Here t is the dictionary size, similar image block group
Figure BDA0002222144890000114
The sparse representation of is solved by solving the following minimization problem:

Figure BDA0002222144890000115
Figure BDA0002222144890000115

式中,||·||F表示矩阵Frobenius范数,card(Aj)表示系数矩阵Aj中非零元素的个数,K表示对表示系数Aj的稀疏约束。式(3)表示在约束表示系数稀疏度的前提下最小化组稀疏表示误差。In the formula, ||·|| F represents the Frobenius norm of the matrix, card(A j ) represents the number of non-zero elements in the coefficient matrix A j , and K represents the sparse constraint on the represented coefficient A j . Equation (3) represents minimizing the group sparse representation error under the premise of constraining the sparsity of the representation coefficients.

将组矩阵Pj用字典Dj进行组稀疏表示,即

Figure BDA0002222144890000116
为组稀疏表示系数。通过选取合适的字典保证表示系数的稀疏性,这种稀疏性可以作为对图像的正则化约束加入到图像盲复原的目标函数中,目标函数可表示为:The group matrix P j is sparsely represented by the dictionary D j , that is,
Figure BDA0002222144890000116
Coefficients are sparsely represented for groups. By selecting a suitable dictionary to ensure the sparsity of the representation coefficients, this sparsity can be added to the objective function of image blind restoration as a regularization constraint on the image. The objective function can be expressed as:

Figure BDA0002222144890000117
Figure BDA0002222144890000117

式中,y为模糊图像,x为清晰图像,h为模糊核,

Figure BDA0002222144890000118
为图像的梯度算子,*表示卷积操作,||·||2表示向量l2范数,||·||F表示矩阵Frobenius范数,Pj表示图像块QjX与其相似图像块构成的组矩阵,Dj为自适应字典,Aj为组稀疏表示系数,λg、λs、λh为正则化参数。式(4)中第一项为数据保真项,第二项为组稀疏表示正则化约束项,第三项为梯度约束项,第四项为模糊核的正则化约束项。数据保真项的形式是由噪声的概率模型决定的,本发明将模糊图像的梯度图像中含有的噪声建模为常见的加性高斯白噪声,于是得到了如式(4)所示的由2范数平方约束的数据保真项。数据保真项可以定义在图像域中,也可以定义在图像的梯度域中,由于本发明通过二维快速傅里叶变换(FFT)来加快目标函数的求解速度,而梯度图像在图像边界处的取值为0,可以更好地满足FFT要求的周期性边界条件,因而本发明将数据保真项定义在图像的梯度域中,以减小复原图像在边界处的失真。对于模糊核,本发明采取简单的2范数约束以简化求解过程。where y is the blurred image, x is the clear image, h is the blur kernel,
Figure BDA0002222144890000118
is the gradient operator of the image, * represents the convolution operation, ||·|| 2 represents the vector l 2 norm, ||·|| F represents the matrix Frobenius norm, P j represents the image block Q j X and its similar image blocks The formed group matrix, D j is the adaptive dictionary, A j is the group sparse representation coefficient, λ g , λ s , λ h are regularization parameters. In formula (4), the first term is the data fidelity term, the second term is the group sparse representation regularization constraint term, the third term is the gradient constraint term, and the fourth term is the regularization constraint term of the fuzzy kernel. The form of the data fidelity term is determined by the probability model of noise. In the present invention, the noise contained in the gradient image of the blurred image is modeled as a common additive Gaussian white noise, so the equation (4) is obtained by 2-norm squared constraint data fidelity term. The data fidelity term can be defined in the image domain or in the gradient domain of the image, because the present invention uses two-dimensional fast Fourier transform (FFT) to speed up the solution speed of the objective function, and the gradient image is at the image boundary. The value of is 0, which can better meet the periodic boundary conditions required by FFT, so the present invention defines the data fidelity item in the gradient domain of the image to reduce the distortion of the restored image at the boundary. For the fuzzy kernel, the present invention adopts a simple 2-norm constraint to simplify the solution process.

本发明通过式(4)所示的最优化问题求解模糊核的估计,由于式(4)是非凸的,没有闭合解,本发明采取交替迭代的方式对式(4)所示的最优化问题进行求解,即固定当前对清晰图像的估计结果

Figure BDA0002222144890000121
更新对模糊核的估计结果
Figure BDA0002222144890000122
然后在固定
Figure BDA0002222144890000123
的基础上更新
Figure BDA0002222144890000124
如此循环迭代直到估计结果收敛或者迭代次数达到某一预设的阈值。The present invention solves the estimation of the fuzzy kernel through the optimization problem shown in Equation (4). Since Equation (4) is non-convex and has no closed solution, the present invention adopts an alternate iteration method to solve the optimization problem shown in Equation (4). Solve, that is, fix the current estimation result for the clear image
Figure BDA0002222144890000121
Update the estimation result of the blur kernel
Figure BDA0002222144890000122
then fixed
Figure BDA0002222144890000123
update based on
Figure BDA0002222144890000124
Iterates in this way until the estimation result converges or the number of iterations reaches a preset threshold.

依据图像自身的多尺度自相似性,本实施例公开了一种基于组稀疏表示的图像盲复原方法,以实现模糊图像的盲复原。参见图2,上述方法至少包括以下步骤:According to the multi-scale self-similarity of the image itself, this embodiment discloses an image blind restoration method based on group sparse representation, so as to realize the blind restoration of the blurred image. Referring to Figure 2, the above method at least includes the following steps:

步骤1.读入模糊图像,构建图像金字塔;Step 1. Read in the blurred image and build an image pyramid;

本发明采取图像盲解卷积常用的图像金字塔方式,由粗到细逐层估计清晰图像和模糊核,在估计过程中,粗一层图像金字塔上估计的清晰图像通过图像插值后作为细一层图像金字塔上清晰图像的初始估计,如图3所示。由于降采样操作降低了模糊图像的模糊程度,且降采样因子越大,模糊程度越小,因而图像金字塔底层的模糊图像的模糊程度越小,对其进行盲解卷积估计的清晰图像就越准确,因而若将该估计结果插值后作为细一层图像金字塔中对清晰图像的初始估计,则细一层图像金字塔上对清晰图像的初始估计更加接近待估计的真实清晰图像,从而可以加快细一层图像金字塔上的估计过程,且有助于提高细一层图像金字塔上估计结果的准确性。通过对模糊观测图像逐层下采样构造图像金字塔,不同层级图像金字塔对应的模糊核尺寸不同,在构造图像金字塔时,如果当前层图像金字塔对应的模糊核尺寸小于3×3,即停止图像金字塔的构造。在图像金字塔的每一层上,建立的盲解卷积目标函数是相同的。以下步骤是图像金字塔每一层上盲解卷积目标函数的求解过程。The invention adopts the image pyramid method commonly used in image blind deconvolution, and estimates the clear image and the blur kernel layer by layer from coarse to fine. The initial estimates of sharp images on the image pyramid are shown in Figure 3. Since the downsampling operation reduces the blurriness of the blurred image, and the larger the downsampling factor is, the smaller the blurriness is. Therefore, the blurriness of the blurred image at the bottom of the image pyramid is smaller, and the clearer image is estimated by blind deconvolution. Therefore, if the estimation result is interpolated as the initial estimation of the clear image in the image pyramid of the thin layer, the initial estimation of the clear image on the image pyramid of the thin layer is closer to the real clear image to be estimated, which can speed up the refinement process. The estimation process on one layer of image pyramids, and helps to improve the accuracy of estimation results on a thin layer of image pyramids. The image pyramid is constructed by downsampling the fuzzy observation image layer by layer. The size of the blur kernel corresponding to the image pyramid of different levels is different. When constructing the image pyramid, if the size of the blur kernel corresponding to the image pyramid of the current layer is less than 3×3, the image pyramid is stopped. structure. At each layer of the image pyramid, the established blind deconvolution objective function is the same. The following steps are the solution process of the blind deconvolution objective function on each layer of the image pyramid.

步骤2.初始化清晰图像;Step 2. Initialize clear image;

记循环迭代次数为k,且初始化设置为k=0,若当前层为图像金字塔的第一层,则将模糊图像y作为清晰图像初始估计

Figure BDA0002222144890000131
设置
Figure BDA0002222144890000132
否则粗一层金字塔估计的清晰图像的插值结果作为当前层清晰图像的初始估计
Figure BDA0002222144890000133
Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image
Figure BDA0002222144890000131
set up
Figure BDA0002222144890000132
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer
Figure BDA0002222144890000133

步骤3.筛选图像块,估计标记矩阵M;Step 3. Screen the image blocks and estimate the marker matrix M;

尽管图像中的多尺度自相似性广泛存在,但并不是所有的图像块都能为图像复原提供有效的附加信息,复原图像中不同区域的像素对模糊核估计的作用不尽相同,例如若某一图像区域的像素值为常数,则该区域经过模糊后的像素取值仍为同一常数,清晰图像与模糊图像在该区域完全相同,因此这一区域并不能为模糊核的估计提供有效的信息。因此,仅图像边缘对模糊核的估计起着关键的作用,且边缘广泛存在着多尺度自相似性,从而更容易搜索相似块。Although multi-scale self-similarity exists widely in images, not all image blocks can provide effective additional information for image restoration. Pixels in different regions of the restored image have different effects on blur kernel estimation. The pixel value of an image area is constant, then the blurred pixel value of this area is still the same constant, the clear image and the blurred image are exactly the same in this area, so this area cannot provide effective information for the estimation of the blur kernel . Therefore, only the image edges play a key role in the estimation of blur kernels, and multi-scale self-similarity widely exists in the edges, which makes it easier to search for similar blocks.

本发明提出三种筛选方案对图像块进行筛选,从而排除平滑区域对模糊核估计的干扰。第一种,计算图像块的方差,选取方差较大的图像块,它们对应图像中变化较为剧烈的边缘;第二种,通过边缘检测方法,对当前图像估计进行边缘估计,利用边缘像素对应的图像块估计模糊核;第三种,计算图像梯度,梯度是图像局部区域中灰度变化的度量,利用梯度大于一定阈值的像素对应的图像块估计模糊核。The present invention proposes three screening schemes to screen image blocks, so as to eliminate the interference of smooth regions on blur kernel estimation. The first is to calculate the variance of the image blocks, and select the image blocks with larger variances, which correspond to the edges in the image that change sharply; the second is to use the edge detection method to estimate the current image. The image block estimates the blur kernel; the third is to calculate the image gradient. The gradient is a measure of the gray level change in the local area of the image, and the image block corresponding to the pixel whose gradient is greater than a certain threshold is used to estimate the blur kernel.

设置标记矩阵为M,M为二值图像,如图5所示,若M中某一像素的取值为1,表明对应的图像块参与模糊核估计以及组稀疏表示约束。由于本发明仅将组稀疏性正则化约束限制在图像的边缘区域,导致复原图像

Figure BDA0002222144890000134
的背景平滑区域受到的约束较少,从而可能导致复原图像的平滑区域含有较多的噪声,为了减小噪声对边缘估计造成的干扰,本发明首先对复原图像
Figure BDA0002222144890000141
进行高斯滤波,然后对滤波后的图像估计边缘。Set the labeling matrix to M, where M is a binary image, as shown in Figure 5, if a pixel in M takes a value of 1, it indicates that the corresponding image block participates in blur kernel estimation and group sparse representation constraints. Since the present invention only restricts the group sparsity regularization constraint to the edge region of the image, the restored image
Figure BDA0002222144890000134
The background smooth area is less constrained, which may cause the smooth area of the restored image to contain more noise. In order to reduce the interference caused by noise to edge estimation, the present invention firstly adjusts
Figure BDA0002222144890000141
Gaussian filtering is performed, and then edges are estimated on the filtered image.

步骤4.对模糊核进行估计;Step 4. Estimate the blur kernel;

固定

Figure BDA0002222144890000142
更新
Figure BDA0002222144890000143
目标函数简化为:fixed
Figure BDA0002222144890000142
renew
Figure BDA0002222144890000143
The objective function simplifies to:

Figure BDA0002222144890000144
Figure BDA0002222144890000144

式中,⊙表示逐元素相乘。根据帕塞瓦尔定理可知,空域中图像的能量等价于傅里叶变换的频域能量,则空域中式(5)所示的最小化问题等效于频域中如下式的最小化问题:In the formula, ⊙ represents element-wise multiplication. According to Parseval's theorem, the energy of the image in the air domain is equivalent to the frequency domain energy of the Fourier transform, then the minimization problem shown in equation (5) in the air domain is equivalent to the minimization problem in the frequency domain as follows:

Figure BDA0002222144890000145
Figure BDA0002222144890000145

式(6)为关于

Figure BDA0002222144890000146
的二次函数,因此存在闭合解。令上式对
Figure BDA0002222144890000147
的导数为零,可得闭合解:Equation (6) is about
Figure BDA0002222144890000146
, so there is a closed solution. Let the above formula be correct
Figure BDA0002222144890000147
The derivative of is zero, the closed solution can be obtained:

Figure BDA0002222144890000148
Figure BDA0002222144890000148

式中,

Figure BDA0002222144890000149
表示傅里叶变换,
Figure BDA00022221448900001410
表示傅里叶逆变换,
Figure BDA00022221448900001411
表示傅里叶变化的复共轭。在式(7)中,本发明采取一种常用的方式排除图像平滑区域对模糊核估计的干扰,即将复原图像
Figure BDA00022221448900001412
中梯度
Figure BDA00022221448900001413
小于一定阈值的像素点的梯度置为0。记梯度阈值为τ,模糊核大小为Nh,则τ的选取方式为:首先将图像梯度根据其方向分为4组,然后设置τ的取值以保证每一组都保留至少有
Figure BDA00022221448900001414
个像素点用于估计模糊核。由于随着迭代次数的增加,复原图像越来越清晰,为了使复原图像中更多的像素点逐步加入到估计模糊核的过程中,在每一次迭代时将τ的值缩小为上一次迭代时的1.1倍。In the formula,
Figure BDA0002222144890000149
represents the Fourier transform,
Figure BDA00022221448900001410
represents the inverse Fourier transform,
Figure BDA00022221448900001411
Represents the complex conjugate of the Fourier transform. In the formula (7), the present invention adopts a common method to eliminate the interference of the image smooth area to the blur kernel estimation, that is, to restore the image
Figure BDA00022221448900001412
medium gradient
Figure BDA00022221448900001413
The gradient of pixels less than a certain threshold is set to 0. Denote the gradient threshold as τ and the size of the blur kernel as N h , then the selection method of τ is as follows: first, the image gradients are divided into 4 groups according to their directions, and then the value of τ is set to ensure that each group retains at least some
Figure BDA00022221448900001414
pixels are used to estimate the blur kernel. Since the restored image becomes clearer with the increase of the number of iterations, in order to gradually add more pixels in the restored image to the process of estimating the blur kernel, in each iteration, the value of τ is reduced to the value of the previous iteration. 1.1 times.

步骤5.对清晰图像进行估计;Step 5. Estimate the clear image;

给定

Figure BDA00022221448900001415
固定
Figure BDA00022221448900001416
更新
Figure BDA00022221448900001417
根据参与计算的图像块个数调整组稀疏表示正则化约束项的权重,式(4)所示的目标函数简化为:given
Figure BDA00022221448900001415
fixed
Figure BDA00022221448900001416
renew
Figure BDA00022221448900001417
According to the number of image blocks involved in the calculation, the weight of the regularization constraint term of the group sparse representation is adjusted, and the objective function shown in equation (4) is simplified as:

Figure BDA0002222144890000151
Figure BDA0002222144890000151

其中,|M|为M中非零元的个数,N为图像像素数目。设

Figure BDA0002222144890000152
为y的一维列向量,将式(8)转换为矩阵向量乘积的形式:Among them, |M| is the number of non-zero elements in M, and N is the number of image pixels. Assume
Figure BDA0002222144890000152
is a one-dimensional column vector of y, convert equation (8) into the form of matrix-vector product:

Figure BDA0002222144890000153
Figure BDA0002222144890000153

式中,

Figure BDA0002222144890000154
为梯度算子
Figure BDA0002222144890000155
的矩阵形式,
Figure BDA0002222144890000156
为点扩散函数
Figure BDA0002222144890000157
的矩阵形式,即模糊矩阵。In the formula,
Figure BDA0002222144890000154
is the gradient operator
Figure BDA0002222144890000155
in matrix form,
Figure BDA0002222144890000156
is the point spread function
Figure BDA0002222144890000157
The matrix form of , the fuzzy matrix.

步骤5.1构建相似图像块组;Step 5.1 Construct similar image block groups;

依据图像自身的多尺度自相似性,包括同尺度和跨尺度自相似性,本发明设计了一种组稀疏表示的正则化约束项,即

Figure BDA0002222144890000158
由于图像具有跨尺度自相似性,且降采样中的模糊程度小于当前层的模糊程度,其边缘更加清晰,与清晰图像具有更强的相似性。因此,本发明提出在降采样图像中搜索多个相似图像块从而构成相似图像块组,在整体上约束相似图像块组的稀疏性,迫使当前图像的边缘更接近清晰图像的边缘。According to the multi-scale self-similarity of the image itself, including the same-scale and cross-scale self-similarity, the present invention designs a regularization constraint term for group sparse representation, namely
Figure BDA0002222144890000158
Since the image has cross-scale self-similarity, and the blurring degree in downsampling is smaller than that of the current layer, its edges are sharper and have stronger similarity to the clear image. Therefore, the present invention proposes to search for a plurality of similar image blocks in the down-sampled image to form a similar image block group, constrain the sparsity of the similar image block group as a whole, and force the edge of the current image to be closer to the edge of the clear image.

本发明给出这一结论的一个不严格的证明。记二维坐标为ξ,清晰图像中的某一个图像块表示为f(ξ),模糊核为h(ξ),在模糊核h(ξ)的作用下,清晰图像块f(ξ)对应的模糊图像块表示为q(ξ),则有The present invention gives a loose proof of this conclusion. Denote the two-dimensional coordinates as ξ, a certain image block in the clear image is expressed as f(ξ), and the blur kernel is h(ξ). Under the action of the blur kernel h(ξ), the clear image block f(ξ) corresponds to The fuzzy image block is expressed as q(ξ), then there is

q(ξ)=h(ξ)*f(ξ) (10)q(ξ)=h(ξ)*f(ξ) (10)

由于多尺度自相似性普遍存在于清晰图像中,假设清晰图像中存在一个与f(ξ)相似的图像块,且其尺度为f(ξ)尺度的a倍(a>1),则该相似图像块可以表示为f(ξ/a)。在模糊核h(ξ)的作用下,清晰图像块f(ξ/a)对应的模糊图像块表示为r(ξ),则有Since multi-scale self-similarity generally exists in clear images, assuming that there is an image block similar to f(ξ) in the clear image, and its scale is a times the scale of f(ξ) (a>1), then the similarity An image block can be represented as f(ξ/a). Under the action of the blur kernel h(ξ), the blurred image block corresponding to the clear image block f(ξ/a) is expressed as r(ξ), then we have

r(ξ)=h(ξ)*f(ξ/a) (11)r(ξ)=h(ξ)*f(ξ/a) (11)

若将模糊图像降采样,其降采样因子为a,则模糊图像块r(ξ)对应的缩小后的图像块可以表示为If the blurred image is downsampled and its downsampling factor is a, the reduced image block corresponding to the blurred image block r(ξ) can be expressed as

ra(ξ)=r(aξ)=h(aξ)*f(ξ) (12)r a (ξ)=r(aξ)=h(aξ)*f(ξ) (12)

根据式(12)可知,图像块ra(ξ)可以认为是由清晰图像块f(ξ)与模糊核h(aξ)卷积的结果。由于h(aξ)的尺寸是h(ξ)尺寸的1/a倍,因此,相对于h(ξ),h(aξ)对图像造成的模糊程度更小。通过式(10)和式(12)的比较可知,ra(ξ)的模糊程度小于q(ξ),即与模糊图像块q(ξ)相比,模糊图像降采样图像中的图像块ra(ξ)与清晰图像块f(ξ)更相似,能够为f(ξ)的复原提供更准确的附加信息。图6直观说明了上述证明过程中各图像块的关系。According to formula (12), the image block ra (ξ) can be considered as the result of convolution of the clear image block f (ξ) and the blur kernel h(aξ). Since the size of h(aξ) is 1/a times the size of h(ξ), h(aξ) causes less blur to the image than h(ξ). By comparing equations (10) and (12), it can be known that the blur degree of ra (ξ) is less than q(ξ), that is, compared with the blurred image block q (ξ), the image block r in the down-sampled image of the blurred image a (ξ) is more similar to the clear image block f(ξ) and can provide more accurate additional information for the restoration of f(ξ). Fig. 6 intuitively illustrates the relationship of each image block in the above proof process.

由于相似图像块经常出现在邻近区域,因而在降采样图像中的一定尺寸搜索窗口内利用图像块匹配方法搜索相似图像块。设清晰图像及其降采样图像按列分别表示为

Figure BDA0002222144890000161
Figure BDA0002222144890000162
其中N为清晰图像的大小,a表示降采样因子。从清晰图像X和降采样图像Xa中抽取的图像块分别表示为QjX和RiXa,其中
Figure BDA0002222144890000163
Figure BDA0002222144890000164
为抽取矩阵,分别用于从清晰图像和降采样图像中抽取第j个和第i个图像块,抽取的图像块尺寸为n。对于图像中的任意图像块QjX,在降采样图像Xa中搜索其相似图像块RiXa。由于图像的不同尺度间广泛存在着图像块的相似性,即对于QjX,可以在降采样图像中寻找多个与其相似的图像块,设在Xa中搜索m-1个与QjX最相似的图像块,并按列表示为
Figure BDA0002222144890000165
QjX与这些在降采样图像中的非局部相似图像块聚合构成一个相似图像块组Pj,可表示为:Because similar image blocks often appear in adjacent areas, the image block matching method is used to search for similar image blocks within a certain size search window in the down-sampled image. Let the clear image and its downsampled image be represented by columns as
Figure BDA0002222144890000161
and
Figure BDA0002222144890000162
where N is the size of the clear image and a is the downsampling factor. The image patches extracted from the clear image X and the downsampled image X a are denoted as Q j X and R i X a , respectively, where
Figure BDA0002222144890000163
and
Figure BDA0002222144890000164
is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image, respectively, and the size of the extracted image block is n. For any image block Q j X in the image, search for its similar image block R i X a in the down-sampled image X a . Due to the widespread similarity of image blocks between different scales of the image, that is, for Q j X, multiple image blocks similar to it can be found in the down-sampled image, and it is assumed that m-1 in X a are searched for with Q j X the most similar image patches, and are represented in columns as
Figure BDA0002222144890000165
Q j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P j , which can be expressed as:

Figure BDA0002222144890000166
Figure BDA0002222144890000166

其中,n为图像块尺寸,m为相似图像块的个数。Among them, n is the size of the image block, and m is the number of similar image blocks.

图像块相似性的判据有多种度量准则,如欧氏距离、相关系数等,本发明采用欧氏距离作为图像块之间相似性的度量依据,搜索欧氏距离小于设定阈值Δd的相似图像块,而不是对于每一个图像块搜索固定数量的相似图像块。对于相似图像块的搜索,本发明采用一种图像块相似性判断的自适应阈值方法,对原始图像X进行插值移位,生成具有1/2亚像素位移的图像

Figure BDA0002222144890000171
对于每一个输入图像块QjX,在
Figure BDA0002222144890000172
中找到对应位置的图像块
Figure BDA0002222144890000173
阈值Δd的计算式为:There are various metric criteria for the similarity of image blocks, such as Euclidean distance, correlation coefficient, etc. The present invention uses Euclidean distance as the basis for measuring the similarity between image blocks, and searches for similarity whose Euclidean distance is less than the set threshold Δd. image patches instead of searching for a fixed number of similar patches for each patch. For the search of similar image blocks, the present invention adopts an adaptive threshold method for judging the similarity of image blocks, performs interpolation shift on the original image X, and generates an image with 1/2 sub-pixel displacement.
Figure BDA0002222144890000171
For each input image patch Q j X, in
Figure BDA0002222144890000172
Find the corresponding image block in
Figure BDA0002222144890000173
The calculation formula of the threshold Δd is:

Figure BDA0002222144890000174
Figure BDA0002222144890000174

其中,γ为控制系数。由式(14)可知,对于平滑图像块,阈值Δd较小,对于边缘图像块,阈值Δd较大;换句话说,图像块内容变化越剧烈,阈值Δd越大;反之,图像块内容变化越平坦,阈值Δd则越小,其目的是为了均衡平滑图像块和边缘图像块搜索的相似图像块个数。此外,本发明设置相似块搜索个数的下限Llow和上限Lhigh,则相似块个数Llow≤m≤Lhigh。如果搜索到的相似块个数小于Llow,那么在建立式(9)的方程组时不采用此输入图像块。如果搜索到的相似块个数大于Lhigh,那么仅选取前Lhigh个相似图像块。Among them, γ is the control coefficient. It can be seen from equation (14) that for smooth image blocks, the threshold Δd is smaller, and for edge image blocks, the threshold Δd is larger; in other words, the more severe the content of the image block changes, the larger the threshold Δd; otherwise, the more the content of the image block changes. Flat, the smaller the threshold Δd, the purpose is to balance the number of similar image blocks searched for smooth image blocks and edge image blocks. In addition, the present invention sets the lower limit L low and the upper limit L high of the number of similar blocks to be searched, so that the number of similar blocks L low ≤m≤L high . If the number of searched similar blocks is less than L low , this input image block is not used when establishing the equation system of equation (9). If the number of searched similar blocks is greater than L high , then only the first L high similar image blocks are selected.

由式(13)可知,Pj为相似图像块组,其中第一列为图像块QjX,为了在表达式中建立与图像X之间的关系,式(9)中将

Figure BDA0002222144890000175
写为
Figure BDA0002222144890000176
的形式,可表示为It can be seen from equation (13) that P j is a similar image block group, and the first column is the image block Q j X. In order to establish the relationship with the image X in the expression, in equation (9), the
Figure BDA0002222144890000175
written as
Figure BDA0002222144890000176
form, which can be expressed as

Figure BDA0002222144890000177
Figure BDA0002222144890000177

令其对X的导数为0,可得如下方程:Let its derivative with respect to X be 0, the following equation can be obtained:

Figure BDA0002222144890000181
Figure BDA0002222144890000181

其中,

Figure BDA0002222144890000182
in,
Figure BDA0002222144890000182

步骤5.2通过组稀疏表示重建参考图像zsStep 5.2 Reconstruct the reference image z s by the group sparse representation;

由于方程(16)等号右边未知的稀疏表示系数αj依赖于方程的解

Figure BDA0002222144890000183
因而该方程没有闭合解。本发明用
Figure BDA0002222144890000184
近似
Figure BDA0002222144890000185
求解字典Dj下的组稀疏表示系数,利用组稀疏表示对上一次迭代中估计的清晰图像
Figure BDA0002222144890000186
进行重建,作为更新
Figure BDA0002222144890000187
的参考图像。令
Figure BDA0002222144890000188
在交替迭代求解的过程中,zs可以视为下一次迭代更新待估计清晰图像
Figure BDA0002222144890000189
的参考图像,其中,zs是由当前对清晰图像的估计结果
Figure BDA00022221448900001810
及其降采样图像中的相似图像块通过组稀疏重建得到,由于降采样操作降低了图像的模糊程度,通过对相似图像块组矩阵进行整体的组稀疏表示,迫使重建图像的边缘更加清晰,zs的模糊程度更小,用模糊程度更小的zs作为参考图像,可使下一次迭代得到的对清晰图像的估计
Figure BDA00022221448900001811
比当前估计
Figure BDA00022221448900001812
更清晰,因此随着迭代次数的增加,清晰图像估计结果的模糊程度越来越小。这正是本发明利用跨尺度而非同尺度结构自相似性作为正则化约束的原因。若采用相同尺度结构自相似性作为正则化约束,则参考图像zs是依据
Figure BDA00022221448900001813
中的相似图像块重建得到的,其模糊程度与
Figure BDA00022221448900001814
相当,因此不能作为更清晰的参考图像,从而不能在迭代过程中为待估计的清晰图像提供有效的信息。Since the unknown sparse representation coefficient α j on the right-hand side of the equal sign of equation (16) depends on the solution of the equation
Figure BDA0002222144890000183
Therefore, the equation has no closed solution. for the present invention
Figure BDA0002222144890000184
approximate
Figure BDA0002222144890000185
Solve the group sparse representation coefficients under the dictionary D j , and use the group sparse representation for the clear image estimated in the previous iteration
Figure BDA0002222144890000186
Rebuild as an update
Figure BDA0002222144890000187
reference image. make
Figure BDA0002222144890000188
In the process of alternate iterative solution, z s can be regarded as the next iteration to update the clear image to be estimated
Figure BDA0002222144890000189
The reference image of , where z s is the result of the current estimation of the sharp image
Figure BDA00022221448900001810
and the similar image blocks in the down-sampled image are obtained through group sparse reconstruction. Since the down-sampling operation reduces the blurring degree of the image, the overall group sparse representation of the similar image block group matrix forces the edges of the reconstructed image to be clearer, z The blur degree of s is smaller, and using z s with a smaller blur degree as the reference image can make the estimation of the clear image obtained in the next iteration
Figure BDA00022221448900001811
than current estimates
Figure BDA00022221448900001812
Sharper, so as the number of iterations increases, the blurriness of the result of the sharp image estimation becomes less and less. This is why the present invention utilizes cross-scale rather than intra-scale structural self-similarity as a regularization constraint. If the same-scale structure self-similarity is used as the regularization constraint, the reference image z s is based on
Figure BDA00022221448900001813
It is reconstructed from similar image blocks in
Figure BDA00022221448900001814
Therefore, it cannot be used as a sharper reference image, and thus cannot provide effective information for the sharp image to be estimated in the iterative process.

下面将详细描述方程(16)的近似求解过程。对于当前图像

Figure BDA00022221448900001815
中的每一个图像块,在其下采样图像
Figure BDA00022221448900001816
中搜索相似图像块,构成组矩阵。由于这些相似图像块合并的组矩阵应是低秩矩阵,本发明利用组稀疏表示与低秩矩阵的关联,通过约束表示系数的稀疏性来实现组矩阵的低秩性,无需考虑组稀疏表示字典构建的问题。稀疏表示中的各个数据向量在字典下的稀疏表示系数的求解是相互独立的,缺乏对数据在字典下表示系数的整体约束,而特定字典下秩最小化可以更好地表示数据的全局结构特征,以解决稀疏表示缺乏对重建图像全局结构约束的问题,可以提高对噪声的鲁棒性,进而提高模糊核估计的准确性。The approximate solution process of equation (16) will be described in detail below. for the current image
Figure BDA00022221448900001815
for each image block in which downsamples the image
Figure BDA00022221448900001816
Search for similar image blocks in the group to form a group matrix. Since the group matrix for merging these similar image blocks should be a low-rank matrix, the present invention utilizes the association between the group sparse representation and the low-rank matrix, and realizes the low-rank property of the group matrix by constraining the sparsity of the representation coefficients, without considering the group sparse representation dictionary build problem. The solution of the sparse representation coefficients of each data vector in the sparse representation under the dictionary is independent of each other, lacking the overall constraints on the coefficients of the data representation under the dictionary, and the rank minimization under a specific dictionary can better represent the global structural features of the data , to solve the problem that the sparse representation lacks constraints on the global structure of the reconstructed image, which can improve the robustness to noise, thereby improving the accuracy of blur kernel estimation.

首先对组矩阵Pj进行奇异值分解:First perform singular value decomposition on the group matrix P j :

Figure BDA0002222144890000191
Figure BDA0002222144890000191

其中,∑j=diag(σj,1,…,σj,r)为奇异值对角矩阵,σj,i,i=1,…,r为矩阵Pj的奇异值,r=min(n,m),m和n表示组矩阵Pj的维度。uj,i和vj,i分别为Uj和Vj的列向量,

Figure BDA0002222144890000192
为秩1矩阵。Among them, ∑ j =diag(σ j,1 ,...,σ j,r ) is the singular value diagonal matrix, σ j,i ,i=1,...,r is the singular value of the matrix P j , r=min( n, m), m and n represent the dimensions of the group matrix P j . u j, i and v j, i are the column vectors of U j and V j , respectively,
Figure BDA0002222144890000192
is a rank 1 matrix.

Zha等提出了一种自适应字典构建方法,作为组稀疏表示与低秩矩阵关联的解释。将

Figure BDA0002222144890000193
按列排列为列向量
Figure BDA0002222144890000194
每一个相似图像块组Pj的自适应字典Dj由r个dj,i构成,即
Figure BDA0002222144890000195
其中,dj,i为字典Dj中的原子。将Pj按列排列为列向量
Figure BDA0002222144890000196
奇异值向量表示为
Figure BDA0002222144890000197
于是,式(17)可写为:Zha et al. propose an adaptive dictionary construction method as an explanation for the association of group sparse representations with low-rank matrices. Will
Figure BDA0002222144890000193
Arrange by column as a column vector
Figure BDA0002222144890000194
The adaptive dictionary D j of each similar image block group P j is composed of r d j, i , that is,
Figure BDA0002222144890000195
Among them, d j, i are atoms in the dictionary D j . Arrange P j column by column as a column vector
Figure BDA0002222144890000196
The singular value vector is represented as
Figure BDA0002222144890000197
So, equation (17) can be written as:

Pj=DjAj (18)P j =D j A j (18)

从式(18)可知,组矩阵Pj的秩函数等价于其奇异值向量Aj的l0范数,即rank(Pj)=||Aj||0,||·||0为向量l0范数。也就是说,组矩阵的低秩性等价于矩阵奇异值向量的稀疏性。这样,约束稀疏表示系数Aj的稀疏性,等价于约束组矩阵Pj的低秩性。由于l0范数是非凸函数,这里用l1范数对l0范数进行凸松弛,由于组矩阵Pj的核范数等价于其奇异值向量Aj的l1范数,即||Pj||*=||Aj||1,用矩阵的核范数对矩阵的秩函数进行凸松弛,来约束组矩阵Pj的低秩性。利用奇异值阈值算子求解组矩阵Pj核范数||Pj||*的最小化问题,其闭合解为It can be seen from equation (18) that the rank function of the group matrix P j is equivalent to the l 0 norm of its singular value vector A j , that is, rank(P j )=||A j || 0 , ||·|| 0 is the l 0 norm of the vector. That is, the low rank of the group matrix is equivalent to the sparsity of the matrix singular value vector. In this way, the constrained sparsity represents the sparsity of the coefficients A j , which is equivalent to the low rank of the constraint group matrix P j . Since the l 0 norm is a non-convex function, the l 1 norm is used to perform convex relaxation on the l 0 norm. Since the nuclear norm of the group matrix P j is equivalent to the l 1 norm of its singular value vector A j , that is | |P j || * =||A j || 1 , using the nuclear norm of the matrix to perform convex relaxation on the rank function of the matrix to constrain the low rank of the group matrix P j . Using the singular value threshold operator to solve the minimization problem of the group matrix P j kernel norm ||P j || * , its closed solution is

Figure BDA0002222144890000198
Figure BDA0002222144890000198

其中,

Figure BDA0002222144890000199
是以β≥0为参数的∑j软阈值算子,定义为in,
Figure BDA0002222144890000199
is the ∑ j soft threshold operator with β≥0 as the parameter, defined as

Figure BDA00022221448900002016
Figure BDA00022221448900002016

由式(20)可知,β越大,重建组矩阵

Figure BDA0002222144890000201
的秩越小。
Figure BDA0002222144890000202
中的第一列即重建图像块
Figure BDA0002222144890000203
由于降采样中图像块的边缘更加清晰,与清晰图像具有更强的相似性,通过约束相似图像块组成矩阵的秩,迫使当前图像的边缘更接近清晰图像的边缘。It can be seen from equation (20) that the larger the β is, the more the group matrix is reconstructed.
Figure BDA0002222144890000201
the smaller the rank.
Figure BDA0002222144890000202
The first column in the reconstructed image block
Figure BDA0002222144890000203
Since the edge of the image block in downsampling is clearer and has a stronger similarity with the clear image, by constraining the rank of the matrix composed of similar image blocks, the edge of the current image is forced to be closer to the edge of the clear image.

步骤5.3求解

Figure BDA0002222144890000204
Step 5.3 Solve
Figure BDA0002222144890000204

在获得近似的组稀疏重建块

Figure BDA0002222144890000205
之后,对上一次迭代中估计的清晰图像
Figure BDA0002222144890000206
进行重建,作为更新
Figure BDA0002222144890000207
的参考图像。将通过求解线性方程组更新对
Figure BDA0002222144890000208
的估计,此时方程(16)可以近似表示为Obtaining approximate group sparse reconstruction blocks
Figure BDA0002222144890000205
After that, for the sharp image estimated in the previous iteration
Figure BDA0002222144890000206
Rebuild as an update
Figure BDA0002222144890000207
reference image. will update the pair by solving a system of linear equations
Figure BDA0002222144890000208
, then equation (16) can be approximately expressed as

Figure BDA0002222144890000209
Figure BDA0002222144890000209

由于M的作用,无法通过傅里叶变换在频域中快速求解,本发明采取双共轭梯度法(Bi-Conjugate Gradient,BICG)来求解式(21),获得

Figure BDA00022221448900002010
Due to the effect of M, it cannot be quickly solved in the frequency domain by Fourier transform. The present invention adopts the Bi-Conjugate Gradient (BICG) method to solve the formula (21), and obtains
Figure BDA00022221448900002010

步骤6.判断收敛,获得模糊核的估计;Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel;

通过步骤3、步骤4和步骤5,进行对目标函数的一次迭代求解,获得对模糊核的估计

Figure BDA00022221448900002011
并对清晰图像的估计
Figure BDA00022221448900002012
进行更新,更新为
Figure BDA00022221448900002013
如果此时的迭代达到最大迭代次数或者迭代收敛,则停止迭代。否则,令k=k+1,然后重复步骤3、步骤4和步骤5,直至迭代达到最大迭代次数或者迭代收敛。Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel
Figure BDA00022221448900002011
and estimates for sharp images
Figure BDA00022221448900002012
to update, update to
Figure BDA00022221448900002013
If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped. Otherwise, let k=k+1, and then repeat step 3, step 4 and step 5 until the iteration reaches the maximum number of iterations or the iteration converges.

步骤7.模糊核估计之后,利用非盲复原算法对清晰图像进行估计。Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image.

模糊核估计对应于求解式(4)所示的最优化问题,获得对模糊核的估计结果

Figure BDA00022221448900002014
Figure BDA00022221448900002015
的基础上采用有效的非盲复原算法获得最终的清晰图像估计。非盲复原算法有全变分正则化方法、稀疏非盲复原方法和EPLL算法等。Fuzzy kernel estimation corresponds to solving the optimization problem shown in equation (4), and obtains the estimation result of the fuzzy kernel
Figure BDA00022221448900002014
exist
Figure BDA00022221448900002015
On the basis of the effective non-blind restoration algorithm, the final clear image estimation is obtained. Non-blind restoration algorithms include total variation regularization method, sparse non-blind restoration method and EPLL algorithm.

优选的,本发明设置图像块大小n=5×5,奇异值阈值β=0.2,正则化参数λs=0.0008、λg=0.002和λh=0.0003N,相似图像块的个数m=19,搜索窗口的尺寸为25×25。降采样因子a越大,模糊图像降采样图像中的图像块越清晰,但同时不同尺度图像之间的相似图像块的个数越少,因此需要综合考虑设置降采样因子的取值,本发明设置金字塔之间的缩放因子为4/3。由于盲复原不知道模糊核的真实大小,在复原时需要通过其他方式估计模糊核的尺寸,或者预先设定模糊核的大小。若估计或预先设定的模糊核尺寸过大,则不容易求解小尺寸的模糊核;若估计或预设的模糊核尺寸较小,则无法求解出完整的模糊核。本发明设置模糊核的尺寸为51×51。模拟和真实数据上实验表明绝大多数模糊图像的模糊核尺寸不大于51×51,且当真实模糊核较小时,本发明在设定模糊核尺寸为51×51的情况仍然可以获得准确的估计结果。本发明设置循环的迭代次数为14,不设置迭代收敛阈值。Preferably, the present invention sets the image block size n=5×5, the singular value threshold β=0.2, the regularization parameters λ s =0.0008, λ g =0.002 and λ h =0.0003N, the number of similar image blocks m=19 , the size of the search window is 25×25. The larger the downsampling factor a, the clearer the image blocks in the downsampling image of the blurred image, but at the same time the number of similar image blocks between images of different scales is smaller, so it is necessary to comprehensively consider setting the value of the downsampling factor. Set the scaling factor between pyramids to 4/3. Since blind restoration does not know the real size of the blur kernel, it is necessary to estimate the size of the blur kernel in other ways during restoration, or pre-set the size of the blur kernel. If the estimated or preset size of the blur kernel is too large, it is not easy to solve the small size of the blur kernel; if the estimated or preset size of the blur kernel is small, the complete blur kernel cannot be solved. The present invention sets the size of the blur kernel to be 51×51. Experiments on simulation and real data show that the blur kernel size of most blurred images is not greater than 51×51, and when the real blur kernel is small, the present invention can still obtain accurate estimation when the blur kernel size is set to 51×51 result. In the present invention, the iteration number of the loop is set to 14, and the iteration convergence threshold is not set.

本发明在Koehler数据集上验证所公开的方法,Koehler数据集包括4幅图像,每幅图像有12种模糊核类型(其中,后5个为大尺寸模糊核),共产生48幅模糊图像。对于大尺寸模糊核,其初始尺寸设置为151×151。通过比较每一幅去模糊的结果与沿着相机运动轨迹捕获的199个未模糊图像的峰值信噪比(Peak signal-to-noise ratio,PSNR),并记录最大的PSNR作为定量评估的指标。复原图像与真值图像之间的PSNR越大,表明复原图像与真值图像越接近。Pan等和Yan等提出的方法是目前普遍认可的两种性能最优的盲去模糊方法,其中,Pan等在核估计中引入基于图像块统计量的暗通道先验,而Yan等提出了基于图像块亮通道先验的图像正则项。在Koehler数据集上,图7定量比较了在每一幅图像上本发明公开的方法与这两种方法的平均PSNR,可以看出,本发明公开的方法在所有图像的平均PSNR上都优于Yan等的方法,在后两幅图像的平均PSNR上优于Pan等的方法。The present invention verifies the disclosed method on the Koehler data set. The Koehler data set includes 4 images, each image has 12 types of blur kernels (the last 5 are large-sized blur kernels), and 48 blurred images are generated in total. For large size blur kernel, its initial size is set to 151×151. By comparing the peak signal-to-noise ratio (PSNR) of each deblurred result with 199 unblurred images captured along the camera motion trajectory, and recording the maximum PSNR as an indicator for quantitative evaluation. The larger the PSNR between the restored image and the ground-truth image, the closer the restored image is to the ground-truth image. The methods proposed by Pan et al. and Yan et al. are currently two widely recognized blind deblurring methods with the best performance. Among them, Pan et al. introduced a dark channel prior based on image block statistics in the kernel estimation, while Yan et al. Image regularization term for the patch bright channel prior. On the Koehler dataset, Figure 7 quantitatively compares the average PSNR of the method disclosed in the present invention and these two methods on each image. It can be seen that the method disclosed in the present invention is superior to the average PSNR of all images. The method of Yan et al. outperformed the method of Pan et al. on the average PSNR of the latter two images.

对所公开的实施例的上述说明,使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的,本发明中所定义的一般原理可以在不脱离本发明的精神或范围的情况下,在其它实施例中实现。因此,本发明将不会被限制于本发明所示的这些实施例,而是要符合与本发明所公开的原理和新颖特点相一致的最宽的范围。The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of this invention. Thus, the present invention is not intended to be limited to the embodiments of the present invention shown, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (6)

1.一种基于组稀疏表示的图像盲复原方法,其特征在于,包含以下步骤:1. an image blind restoration method based on group sparse representation, is characterized in that, comprises the following steps: 步骤1.读入模糊图像,构建图像金字塔;Step 1. Read in the blurred image and build an image pyramid; 读入模糊图像,设置模糊核的初始尺寸,对模糊图像逐层下采样构造图像金字塔,不同层级图像金字塔对应的模糊核尺寸不同,在构造图像金字塔时,如果当前层图像金字塔对应的模糊核尺寸小于3×3,即停止图像金字塔的构造;Read in the blurred image, set the initial size of the blur kernel, and downsample the blurred image layer by layer to construct an image pyramid. The size of the blur kernel corresponding to the image pyramids at different levels is different. Less than 3 × 3, that is, stop the construction of the image pyramid; 步骤2.初始化清晰图像;Step 2. Initialize clear image; 记循环迭代次数为k,且初始化设置为k=0,若当前层为图像金字塔的第一层,则将模糊图像y作为清晰图像初始估计
Figure FDA0003539560340000011
设置
Figure FDA0003539560340000012
否则粗一层金字塔估计的清晰图像的插值结果作为当前层清晰图像的初始估计
Figure FDA0003539560340000013
Note that the number of loop iterations is k, and the initialization is set to k=0. If the current layer is the first layer of the image pyramid, the blurred image y is used as the initial estimate of the clear image
Figure FDA0003539560340000011
set up
Figure FDA0003539560340000012
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimate of the clear image of the current layer
Figure FDA0003539560340000013
步骤3.筛选图像块,估计标记矩阵M;Step 3. Screen the image blocks and estimate the marker matrix M; 设置标记矩阵为M,M同时也为二值图像,若M中某一像素的取值为1,表明对应的图像块参与模糊核估计以及组稀疏表示约束;对复原图像
Figure FDA0003539560340000014
进行高斯滤波,然后对滤波后的图像估计边缘;
Set the marker matrix to M, and M is also a binary image. If the value of a pixel in M is 1, it indicates that the corresponding image block participates in the blur kernel estimation and group sparse representation constraints; for the restored image
Figure FDA0003539560340000014
Perform Gaussian filtering, and then estimate the edge of the filtered image;
步骤4.对模糊核进行估计;Step 4. Estimate the blur kernel; 固定当前图像的估计
Figure FDA0003539560340000015
用下式更新下一次迭代的模糊核
Figure FDA0003539560340000016
fix the estimate for the current image
Figure FDA0003539560340000015
Update the blur kernel for the next iteration with
Figure FDA0003539560340000016
Figure FDA0003539560340000017
Figure FDA0003539560340000017
式中,y表示模糊图像,
Figure FDA0003539560340000018
为图像的梯度算子,λh为正则化参数,⊙表示逐元素相乘,
Figure FDA0003539560340000019
表示傅里叶变换,
Figure FDA00035395603400000110
表示傅里叶逆变换,
Figure FDA00035395603400000111
表示傅里叶变化的复共轭;将复原图像
Figure FDA00035395603400000112
中梯度
Figure FDA00035395603400000113
小于一定阈值的像素点的梯度置为0;记梯度阈值为
Figure FDA00035395603400000114
模糊核大小为Nh,则
Figure FDA00035395603400000115
的选取方式为:首先将图像梯度根据其方向分为四组,然后设置
Figure FDA00035395603400000116
的取值以保证每一组都保留至少有
Figure FDA0003539560340000021
个像素点用于估计模糊核;由于随着迭代次数的增加,复原图像越来越清晰,为了使复原图像中更多的像素点逐步加入模糊核估计的过程中,在每一次迭代时将
Figure FDA0003539560340000022
的值缩小为上一次迭代时的1.1倍;
where y represents the blurred image,
Figure FDA0003539560340000018
is the gradient operator of the image, λ h is the regularization parameter, ⊙ means element-wise multiplication,
Figure FDA0003539560340000019
represents the Fourier transform,
Figure FDA00035395603400000110
represents the inverse Fourier transform,
Figure FDA00035395603400000111
Represents the complex conjugate of the Fourier transform; will restore the image
Figure FDA00035395603400000112
medium gradient
Figure FDA00035395603400000113
The gradient of pixels less than a certain threshold is set to 0; the gradient threshold is recorded as
Figure FDA00035395603400000114
The size of the blur kernel is N h , then
Figure FDA00035395603400000115
The selection method is: first divide the image gradients into four groups according to their directions, and then set
Figure FDA00035395603400000116
value to ensure that each group retains at least
Figure FDA0003539560340000021
pixels are used to estimate the blur kernel; as the number of iterations increases, the restored image becomes clearer and clearer, in order to gradually add more pixels in the restored image to the process of blur kernel estimation, in each iteration,
Figure FDA0003539560340000022
The value of is reduced to 1.1 times of the previous iteration;
步骤5.对清晰图像进行估计;Step 5. Estimate the clear image; 给定当前图像的估计
Figure FDA0003539560340000023
固定下一次迭代的模糊核估计
Figure FDA0003539560340000024
更新下一次迭代的图像
Figure FDA0003539560340000025
an estimate given the current image
Figure FDA0003539560340000023
Fixed blur kernel estimation for next iteration
Figure FDA0003539560340000024
Update the image for the next iteration
Figure FDA0003539560340000025
步骤6.判断收敛,获得模糊核的估计;Step 6. Judging convergence and obtaining an estimate of the fuzzy kernel; 通过步骤3、步骤4和步骤5,进行对目标函数的一次迭代求解,获得对模糊核的估计
Figure FDA0003539560340000026
并对清晰图像的估计
Figure FDA0003539560340000027
进行更新,更新为
Figure FDA0003539560340000028
如果此时的迭代达到最大迭代次数或者迭代收敛,则停止迭代;否则,令k=k+1,k代表迭代次数,然后重复步骤3、步骤4和步骤5;
Through steps 3, 4 and 5, an iterative solution of the objective function is performed to obtain an estimate of the fuzzy kernel
Figure FDA0003539560340000026
and estimates for sharp images
Figure FDA0003539560340000027
to update, update to
Figure FDA0003539560340000028
If the iteration at this time reaches the maximum number of iterations or the iteration converges, the iteration is stopped; otherwise, let k=k+1, k represents the number of iterations, and then repeat steps 3, 4 and 5;
步骤7.模糊核估计之后,利用非盲复原算法对清晰图像进行估计;Step 7. After the blur kernel is estimated, use the non-blind restoration algorithm to estimate the clear image; 通过步骤1至步骤6,获得对模糊核的估计结果
Figure FDA0003539560340000029
Figure FDA00035395603400000210
的基础上采用有效的非盲复原算法获得最终的清晰图像估计;非盲复原算法有全变分正则化方法、稀疏非盲复原方法和EPLL算法。
Through steps 1 to 6, the estimation results of the blur kernel are obtained
Figure FDA0003539560340000029
exist
Figure FDA00035395603400000210
On the basis of , an effective non-blind restoration algorithm is used to obtain the final clear image estimation; non-blind restoration algorithms include total variational regularization method, sparse non-blind restoration method and EPLL algorithm.
2.根据权利要求1所述的一种基于组稀疏表示的图像盲复原方法,其特征在于,2. a kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, 步骤5.1构建相似图像块组;Step 5.1 Construct similar image block groups; 设清晰图像及其降采样图像按列分别表示为
Figure FDA00035395603400000211
Figure FDA00035395603400000212
其中N为清晰图像的像素数目,a表示降采样因子,
Figure FDA00035395603400000213
为N维列向量,
Figure FDA00035395603400000214
为N/a2维列向量;从清晰图像X和降采样图像Xa中抽取的图像块分别表示为QjX和RiXa,其中
Figure FDA00035395603400000215
Figure FDA00035395603400000216
为抽取矩阵,分别用于从清晰图像和降采样图像中抽取第j个和第i个图像块,抽取的图像块尺寸为n;对于图像中的任意图像块QjX,在降采样图像Xa中搜索相似图像块RiXa;由于图像的不同尺度间广泛存在着图像块的相似性,即对于QjX,在降采样图像中的一定尺寸搜索窗口内利用图像块匹配方法寻找多个与其相似的图像块,设在Xa中搜索m-1个与QjX最相似的图像块,并按列表示为
Figure FDA0003539560340000031
QjX与这些在降采样图像中的非局部相似图像块聚合构成一个相似图像块组Pj,表示为:
Let the clear image and its downsampled image be represented by columns as
Figure FDA00035395603400000211
and
Figure FDA00035395603400000212
where N is the number of pixels in the clear image, a is the downsampling factor,
Figure FDA00035395603400000213
is an N-dimensional column vector,
Figure FDA00035395603400000214
is an N/a 2 -dimensional column vector; the image blocks extracted from the clear image X and the down-sampled image X a are denoted as Q j X and R i X a , respectively, where
Figure FDA00035395603400000215
and
Figure FDA00035395603400000216
is the extraction matrix, which is used to extract the j-th and i-th image blocks from the clear image and the down-sampled image respectively, and the size of the extracted image block is n; for any image block Q j X in the image, in the down-sampled image X Search for similar image blocks R i X a in a ; because the similarity of image blocks widely exists between different scales of the image, that is, for Q j X, the image block matching method is used to search for multiple image blocks in a certain size search window in the down-sampled image. There are image blocks similar to it, suppose m-1 image blocks that are most similar to Q j X are searched in X a , and they are represented by columns as
Figure FDA0003539560340000031
Q j X is aggregated with these non-local similar image blocks in the down-sampled image to form a similar image block group P j , which is expressed as:
Figure FDA0003539560340000032
Figure FDA0003539560340000032
其中,n为图像块尺寸,m为相似图像块的个数;Among them, n is the size of the image block, and m is the number of similar image blocks; 步骤5.2通过组稀疏表示重建参考图像zsStep 5.2 Reconstruct the reference image z s by the group sparse representation; 利用奇异值阈值算子对组矩阵Pj进行重建,其闭合解为Using the singular value threshold operator to reconstruct the group matrix P j , its closed solution is
Figure FDA0003539560340000033
Figure FDA0003539560340000033
其中,Sβ(∑j)是以β>0为参数的∑j软阈值算子,定义为Among them, S β (∑ j ) is the ∑ j soft threshold operator with β>0 as the parameter, which is defined as Sβ(∑j)=soft(∑j;β)=max(∑j-β;0)S β (∑ j )=soft(∑ j ; β)=max(∑ j −β; 0) 由上式知,β越大,重建组矩阵
Figure FDA0003539560340000034
的秩越小;用
Figure FDA0003539560340000035
近似
Figure FDA0003539560340000036
中的第一列即重建图像块
Figure FDA0003539560340000037
Dj为自适应字典;令
Figure FDA0003539560340000038
在交替迭代求解的过程中,zs作为下一次迭代更新待估计清晰图像
Figure FDA0003539560340000039
的参考图像;
From the above formula, the larger the β is, the reconstruction of the group matrix
Figure FDA0003539560340000034
The smaller the rank is; the
Figure FDA0003539560340000035
approximate
Figure FDA0003539560340000036
The first column in the reconstructed image block
Figure FDA0003539560340000037
D j is an adaptive dictionary; let
Figure FDA0003539560340000038
In the process of alternate iterative solution, z s is used as the next iteration to update the clear image to be estimated
Figure FDA0003539560340000039
the reference image;
步骤5.3求解
Figure FDA00035395603400000310
Step 5.3 Solve
Figure FDA00035395603400000310
在获得
Figure FDA00035395603400000311
之后,通过求解如下线性方程组更新对
Figure FDA00035395603400000312
的估计:
in getting
Figure FDA00035395603400000311
After that, update the pair by solving the following system of linear equations
Figure FDA00035395603400000312
Estimate:
Figure FDA00035395603400000313
Figure FDA00035395603400000313
式中,|M|为M中非零元的个数,N为图像像素个数,
Figure FDA00035395603400000314
为y的一维列向量,
Figure FDA00035395603400000315
为点扩散函数
Figure FDA00035395603400000316
的矩阵形式,即模糊矩阵,
Figure FDA00035395603400000317
为梯度算子
Figure FDA0003539560340000041
的矩阵形式;由于M的作用,无法通过傅里叶变换在频域中快速求解,采取双共轭梯度法来求解如上线性方程组,获得
Figure FDA0003539560340000042
where |M| is the number of non-zero elements in M, N is the number of image pixels,
Figure FDA00035395603400000314
is a one-dimensional column vector of y,
Figure FDA00035395603400000315
is the point spread function
Figure FDA00035395603400000316
The matrix form of , the fuzzy matrix,
Figure FDA00035395603400000317
is the gradient operator
Figure FDA0003539560340000041
The matrix form of ; due to the effect of M, it cannot be quickly solved in the frequency domain by Fourier transform, and the biconjugate gradient method is used to solve the above linear equation system, and obtain
Figure FDA0003539560340000042
3.根据权利要求1所述的一种基于组稀疏表示的图像盲复原方法,其特征在于,3. a kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, 所述图像块的尺寸n为5×5。The size n of the image block is 5×5. 4.根据权利要求2所述的一种基于组稀疏表示的图像盲复原方法,其特征在于,4. a kind of image blind restoration method based on group sparse representation according to claim 2, is characterized in that, 所述降采样因子a为4/3。The downsampling factor a is 4/3. 5.根据权利要求1所述的一种基于组稀疏表示的图像盲复原方法,其特征在于,5. A kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, 所述循环的迭代次数为14,不设置迭代收敛阈值。The iteration number of the loop is 14, and no iteration convergence threshold is set. 6.根据权利要求1所述的一种基于组稀疏表示的图像盲复原方法,其特征在于,6. A kind of image blind restoration method based on group sparse representation according to claim 1, is characterized in that, 所述模糊核的尺寸为51×51。The size of the blur kernel is 51×51.
CN201910938167.5A 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation Active CN110675347B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910938167.5A CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910938167.5A CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Publications (2)

Publication Number Publication Date
CN110675347A CN110675347A (en) 2020-01-10
CN110675347B true CN110675347B (en) 2022-05-06

Family

ID=69080294

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910938167.5A Active CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Country Status (1)

Country Link
CN (1) CN110675347B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12216277B2 (en) 2021-10-14 2025-02-04 Samsung Electronics Co., Ltd. Optical element for deconvolution

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112365420B (en) * 2020-11-12 2024-03-15 重庆邮电大学 Fuzzy image blind restoration method based on non-local window gradient
US11721001B2 (en) 2021-02-16 2023-08-08 Samsung Electronics Co., Ltd. Multiple point spread function based image reconstruction for a camera behind a display
US11722796B2 (en) 2021-02-26 2023-08-08 Samsung Electronics Co., Ltd. Self-regularizing inverse filter for image deblurring
CN113487491B (en) * 2021-05-26 2024-04-26 辽宁工程技术大学 Image restoration method based on sparsity and non-local mean self-similarity
CN113535996B (en) * 2021-05-27 2023-08-04 中国人民解放军火箭军工程大学 A road image data set preparation method and device based on aerial images
CN113379647B (en) * 2021-07-08 2024-07-02 湘潭大学 Multi-feature image restoration method for optimizing PSF estimation
WO2023050026A1 (en) * 2021-09-28 2023-04-06 华为技术有限公司 Image processing method and related device
CN114418877B (en) * 2022-01-12 2023-09-01 新乡学院 Image non-blind deblurring method and system based on gradient amplitude similarity
CN114418883B (en) * 2022-01-18 2024-03-29 北京工业大学 Blind image deblurring method based on depth priori
CN115249028B (en) * 2022-05-13 2023-06-23 哈尔滨工业大学 Blind deconvolution wireless communication signal reconstruction method based on sparse regularization constraint
CN118134804B (en) * 2024-02-02 2024-08-06 湖北师范大学 Image sharpening processing system based on ambiguity recognition

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875349A (en) * 2016-12-30 2017-06-20 无锡高新兴智能交通技术有限公司 The computational methods and blind image restoring method of fuzzy core in blind image restoring method
CN107730466A (en) * 2017-10-11 2018-02-23 北京工业大学 The method for blindly restoring image represented based on structure self-similarity and low-rank matrix
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 A Blind Restoration Method for Motion Blurred Images Combining L0 Norm and Spatial Scale Information
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Blur kernel estimation method based on image and blur kernel mixed constraints

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150030252A1 (en) * 2011-12-16 2015-01-29 The Research Foundation For The State University Of New York Methods of recognizing activity in video
CN105957025A (en) * 2016-04-21 2016-09-21 天津大学 Inconsistent image blind restoration method based on sparse representation
KR102614012B1 (en) * 2016-11-08 2023-12-14 한화비전 주식회사 Aapparatus of processing image and method of providing image thereof
CN108564544B (en) * 2018-04-11 2023-04-28 南京邮电大学 Combined sparse optimization method for image blind deblurring based on edge perception

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875349A (en) * 2016-12-30 2017-06-20 无锡高新兴智能交通技术有限公司 The computational methods and blind image restoring method of fuzzy core in blind image restoring method
CN107730466A (en) * 2017-10-11 2018-02-23 北京工业大学 The method for blindly restoring image represented based on structure self-similarity and low-rank matrix
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 A Blind Restoration Method for Motion Blurred Images Combining L0 Norm and Spatial Scale Information
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Blur kernel estimation method based on image and blur kernel mixed constraints

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Blur kernel estimation using sparse representation and cross-scale self-similarity;Jing Yu 等;《Multimedia Tools and Applications》;20190129;第18549-18570页 *
基于组稀疏残差约束的自适应强噪声图像复原算法;高红霞 等;《华南理工大学学报(自然科学版)》;20180831;第46卷(第8期);第11-18页 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12216277B2 (en) 2021-10-14 2025-02-04 Samsung Electronics Co., Ltd. Optical element for deconvolution

Also Published As

Publication number Publication date
CN110675347A (en) 2020-01-10

Similar Documents

Publication Publication Date Title
CN110675347B (en) Image blind restoration method based on group sparse representation
Sroubek et al. Robust multichannel blind deconvolution via fast alternating minimization
CN111709895B (en) Image blind deblurring method and system based on attention mechanism
US9692939B2 (en) Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence
Michaeli et al. Blind deblurring using internal patch recurrence
US9262815B2 (en) Algorithm for minimizing latent sharp image cost function and point spread function cost function with a spatial mask in a regularization term
CN110796616B (en) Turbulence degradation image recovery method based on norm constraint and self-adaptive weighted gradient
CN113962908B (en) Method and system for point-by-point correction and restoration of degraded images with aero-optic effect and large field of view
Dimlo et al. An Improved Blind Deconvolution for Restoration of Blurred Images Using Ringing Removal Processing
Javaran et al. Blind motion image deblurring using an effective blur kernel prior
CN114170087A (en) Cross-scale low-rank constraint-based image blind super-resolution method
Cao et al. Single image motion deblurring with reduced ringing effects using variational Bayesian estimation
Hongbo et al. Single image fast deblurring algorithm based on hyper‐Laplacian model
CN113379647A (en) Multi-feature image restoration method for optimizing PSF estimation
Qin et al. Kernel learning for blind image recovery from motion blur
CN110648291B (en) Unmanned aerial vehicle motion blurred image restoration method based on deep learning
Dawood et al. Probability weighted moments regularization based blind image De-blurring
Chen et al. Blind restoration for nonuniform aerial images using nonlocal Retinex model and shearlet-based higher-order regularization
Liu et al. Hybrid regularized blur kernel estimation for single-image blind deconvolution
Ghasemi-Falavarjani et al. Particle filter based multi-frame image super resolution
Gunturk et al. Fundamentals of image restoration
Yu et al. Blur kernel estimation using sparse representation and cross-scale self-similarity
Qiao et al. Joint blind super-resolution and shadow removing
Wen et al. Patch-wise blind image deblurring via Michelson channel prior
Li et al. Depth super-resolution using joint adaptive weighted least squares and patching gradient

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant