WO2018227882A1 - 基于先验约束和离群值抑制的图像去模糊方法 - Google Patents

基于先验约束和离群值抑制的图像去模糊方法 Download PDF

Info

Publication number
WO2018227882A1
WO2018227882A1 PCT/CN2017/111996 CN2017111996W WO2018227882A1 WO 2018227882 A1 WO2018227882 A1 WO 2018227882A1 CN 2017111996 W CN2017111996 W CN 2017111996W WO 2018227882 A1 WO2018227882 A1 WO 2018227882A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
equation
fuzzy
blurred
blurred image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/111996
Other languages
English (en)
French (fr)
Inventor
李革
张毅伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to US16/336,855 priority Critical patent/US11263728B2/en
Publication of WO2018227882A1 publication Critical patent/WO2018227882A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]

Definitions

  • the present invention relates to digital image processing techniques, and more particularly to an image deblurring method based on a priori constraint and outlier suppression.
  • Deblurring techniques are the subject of extensive research in the field of image and video processing. Blurring caused by camera shake seriously affects the image quality and visual perception of the image in a certain sense. As an extremely important branch of image preprocessing, the improvement of deblurring technology directly affects the performance of other computer vision algorithms, such as foreground segmentation, object detection, behavior analysis, etc. It also affects the encoding performance of images. Therefore, the development of high performance deblurring algorithms plays an important role.
  • the convolution model can usually be used to explain the cause of the blur, and the process of camera shake can be mapped to the Point Spread Function (PSF).
  • PSF Point Spread Function
  • the fuzzy kernel is unknown, the clear image is restored. This problem is an ill-posed problem. Therefore, it is usually necessary to estimate the fuzzy kernel first, and then use the evaluated fuzzy kernel to perform the convolution operation to obtain the restored image.
  • MAP-based EM algorithms include MAP-based EM algorithms; the original MAP x , k (where x represents a sharp image, and k represents a fuzzy kernel) algorithm in many cases will use a blurred image as a no-blur explanation, which makes Evaluating the failure of the iterative process of the image and the fuzzy kernel; the subsequent MAP k (k represents the fuzzy kernel) algorithm is an improvement of MAP x,k , which solves the problem of non-fuzzy interpretation, which first estimates the fuzzy kernel, and then Image restoration using non-blind deconvolution.
  • all of the above algorithms have problems of insufficient or inappropriate a priori constraints. At the same time, the problem of the outliers of the evaluated fuzzy kernels is not well solved. This subtle difference is likely to cause the deblurring process. failure.
  • the existing deblurring algorithms have the following disadvantages: (1) the prior assumption is not accurate; (2) the prior constraint is not suitable; (3) the outliers existing in the fuzzy kernel are not well Suppression, because the process of camera shake is continuous, which determines the continuity of the fuzzy kernel trajectory. Therefore, the outliers existing in the fuzzy kernel must cause the failure of the deconvolution process.
  • the present invention proposes an image deblurring method based on a priori constraint and outlier suppression, which solves the inaccuracy of the priori assumptions in the existing algorithms, the prior constraint is not suitable, and The problem of outliers in the fuzzy kernel is solved.
  • the present invention can significantly improve the level of restoration of the blurred image.
  • the principle of the invention is to propose a deblurring algorithm based on a priori information constraint and outlier suppression, which aims to solve the inaccuracy of a priori assumptions in prior algorithms, the a priori constraints are not suitable, and the existence of fuzzy kernels
  • image deblurring is realized based on a priori constraint and outlier suppression.
  • the significant structure in the blurred image is obtained by using L0 norm constraint and repeated tail prior information.
  • the fuzzy kernel is evaluated by the L0 norm constraint. Then, the outlier suppression is performed on the evaluated fuzzy kernel.
  • the final reconstructed image is obtained by the non-blind deconvolution algorithm.
  • the method of the invention can effectively improve the restoration of the blurred image by solving the problem that the prior hypothesis existing in the existing algorithm is inaccurate, the prior constraint is not suitable, and the outlier value exists in the fuzzy kernel.
  • I represents a blurred image
  • k represents a fuzzy kernel
  • ⁇ represents noise (we assume that its distribution is Gaussian noise);
  • Equation 2 is the saliency structure of the blurred image (not the image to be restored), which is used to assist in evaluating the fuzzy kernel k;
  • the first term of Equation 2 can be regarded as a loss function (although this will cause the value of Equation 2 to become larger, However, the present invention is to evaluate the optimization equation to a minimum value S, so it does not affect the optimization process); the second term of Equation 2 is to simulate the heavy tail effect with a super Laplace;
  • Equation 3 the updated formula is as shown in Equation 3:
  • M is the binary calibration of the texture in the saliency structure S of the blurred image
  • (1-M) is the binary calibration of the smooth region in S
  • the latter two terms of the formula 3 ie the third and last term
  • the constraints of the details, the last one is the constraint on smoothing
  • Equation 4 x represents the position of the pixel point, y represents the pixel point centered on the pixel point, the window size is in the range of N h , and r(x) represents the extent to which the pixel point at the position x belongs to the texture portion;
  • the texture in S can be divided initially.
  • the value of r(x) is (0,1), the larger r(x) is, the larger the possible row of x belongs to the texture part.
  • formula 4 limits the mutation.
  • the appearance of texture because when the blur kernel size is larger than the blur image detail size, the image restoration fails, so it is necessary to limit the abrupt texture).
  • M is obtained by a Heaviside step function, where ⁇ r represents a threshold value of the degree r(x) of the pixel point belonging to the texture portion, which is used to distinguish the texture region and the smooth region in the salient structure S, in the present invention
  • ⁇ r represents a threshold value of the degree r(x) of the pixel point belonging to the texture portion, which is used to distinguish the texture region and the smooth region in the salient structure S, in the present invention
  • Equation 3 To solve Equation 3, introduce two substitution variables u and w to selectively replace And use the iterative method to update S, the variant of Equation 3 is as follows:
  • Representing the Fourier transform Represents the inverse Fourier transform.
  • the present invention estimates the fuzzy kernel using the gradient information and the saliency structure, and iteratively updates the nucleus of the nucleus by using Equations 14 and 15, as shown in FIG. 6 and FIG. 8 and FIG. 9(c).
  • the present invention utilizes the evaluated fuzzy image saliency structure S to estimate the fuzzy kernel.
  • the optimization equation is as follows:
  • Equation 12 The variant of Equation 12 is as follows:
  • the unambiguous deconvolution technique is used to recover the blurred image.
  • the Richardson-Lucy algorithm is used to implement non-blind back convolution, see Document 1 (Perrone, Daniele, and Paolo Favaro. "Total variation blind deconvolution: The devil is in the details.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014).
  • the invention proposes an image deblurring method based on a priori constraint and outlier suppression, and obtains a salient structure in a blurred image by using L0 norm constraint and repeated tail prior information; on the basis of the salient structure, L0 norm constraint
  • the fuzzy kernel is evaluated; the outlier suppression is performed on the evaluated fuzzy kernel; and the final restored image is obtained by the non-blind deconvolution algorithm.
  • the invention can solve the problem that the prior hypothesis existing in the existing algorithm is inaccurate, the prior constraint is not suitable, and the outlier value exists in the fuzzy kernel. By solving the above problems, the invention can significantly improve the recovery level of the blurred image.
  • Figure 2 is a flow chart of the method of the present invention
  • k n represents the fuzzy kernel estimated by the minimum size image
  • k 0 represents the finally obtained evaluated fuzzy kernel
  • FIG. 3 is a texture diagram of a saliency structure of a blurred image in an embodiment of the present invention.
  • FIG. 4 is a diagram showing an example of the saliency structure of a blurred image in the embodiment of the present invention.
  • 5 is a saliency structure of a blurred image of different sizes in an embodiment of the present invention.
  • (a) to (d) are blurred images of different sizes.
  • FIG. 6 is a fuzzy core obtained by iteratively updating in an embodiment of the present invention.
  • FIG. 7 is an image after the blurred image is restored in the embodiment of the present invention.
  • Figure 8 is a comparison diagram of the magnified image and its restored image in the embodiment of the present invention.
  • (a) is a raw blurred image; (b) is a significant structure; and (c) is a blurred restored image.
  • FIG. 2 The flow chart of the deblurring method based on a priori constraint and outlier suppression proposed by the present invention is shown in FIG. 2.
  • k n represents the fuzzy kernel obtained by the minimum size image evaluation
  • k 0 represents the finally obtained evaluation fuzzy kernel.
  • the method of the present invention continuously updates the fuzzy kernel by means of multiple sampling, and obtains the final restored image by using a non-busy deconvolution algorithm.
  • Table 1 is the description of the parameter names and their corresponding parameters used in the following steps.
  • Step 1 Fuzzy model selection: The present invention adopts the formula 1 model, and assumes that the noise obeys the Gaussian distribution, and obtains the optimization equation of Equation 2;
  • the present invention uses a convolution model to fit the blurring process of a clear image, as in Equation 1:
  • I represents a blurred image
  • k represents a fuzzy kernel
  • ⁇ represents noise (we assume that its distribution is Gaussian noise).
  • S is the salient structure of the blurred image
  • Step 2 Fuzzy image saliency structure assessment:
  • FIG. 3 is a texture diagram of the fuzzy image saliency structure in the embodiment; wherein (a) is the original blurred image; (b) is the original fuzzy image gradient distribution; (c) is the gradient histogram distribution; (d) is the value distribution of the original image r(x).
  • the texture calibration of the significant structure obtained by using Equations 4 and 5 is as follows:
  • M is the binary calibration of the texture in the saliency structure S of the blurred image
  • (1-M) is the binary calibration of the smooth region in S
  • Equation 4 x represents the position of the pixel point, y represents the pixel point centered on the pixel point, the window size is in the range of N h , and r(x) represents the extent to which the pixel point at the position x belongs to the texture portion;
  • the texture in S can be divided initially.
  • the value of r(x) is (0,1), the larger r(x) is, the larger the possible row of x belongs to the texture part.
  • formula 4 limits the mutation.
  • the appearance of texture because when the blur kernel size is larger than the blur image detail size, the image restoration fails, so it is necessary to limit the abrupt texture).
  • Equation 5 M is obtained by the Heaviside step function, where ⁇ r represents the threshold of r, which is used to distinguish the texture region and the smooth region in the salient structure S.
  • ⁇ r represents the threshold of r, which is used to distinguish the texture region and the smooth region in the salient structure S.
  • the fuzzy image saliency structure is obtained by using the iterative update of Equations 8, 9, and 11, as shown in Figure 4;
  • Figure 5 is the saliency structure estimated under different sizes;
  • Equation 3 To solve Equation 3, we introduce two alternative variables u and w to selectively replace And use the iterative method to update S, the variant of Equation 3 is as follows:
  • Representing the Fourier transform Represents the inverse Fourier transform.
  • Step 3 Blurring the kernel estimate as follows:
  • the present invention estimates the fuzzy kernel using the gradient information and the saliency structure, and iteratively updates the nucleus of the nucleus by using Equations 14 and 15, as shown in FIG. 6 and FIG. 8 and FIG. 9(c).
  • the present invention utilizes the evaluated fuzzy image saliency structure S to perform fuzzy sum estimation.
  • the optimization equation is as follows:
  • Equation 12 The variant of Equation 12 is as follows:
  • the non-blind deconvolution specifically uses the literature 1 (Perrone, Daniele, and Paolo Favaro. "Total variation blind deconvolution: The devil is in the details." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. The algorithm proposed in 2014) is shown in Fig. 7, which is the restored image of the blurred image in Fig. 3; another example is Fig. 8, the left picture is the blurred image and its enlarged area, and the right picture is the restoration effect and the corresponding enlargement. region.
  • the fuzzy kernel is represented in the form of an energy map in the upper left corner of the right image.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

提供了一种图像去模糊方法，基于先验约束和离群值抑制，采用卷积模型来拟合清晰图像的模糊过程，再对模糊图像I进行复原，由此达到图像去模糊的目的；包括模糊图像显著性结构评估过程、模糊核估计及其离群值抑制过程、非忙反卷积模糊图像复原过程；利用L0范数约束和重尾先验信息获得模糊图像中的显著性结构；具体采用L0范数约束对模糊核进行评估；对评估的模糊核进行离群值抑制；采用非盲反卷积算法得到最终的复原图像。本发明能够解决现有算法中存在的先验假设不准确、先验约束不合适，以及模糊核中存在离群值的问题，能够明显提高模糊图像的复原水平。

Description

基于先验约束和离群值抑制的图像去模糊方法 技术领域
本发明涉及数字图像处理技术，尤其涉及一种基于先验约束和离群值抑制的图像去模糊方法。
背景技术
去模糊技术是图像和视频处理领域被广泛研究的主题。基于相机抖动造成的模糊在一定意义上严重影响图像的成像质量和视觉观感。作为图像预处理领域一个极其重要的分支，去模糊技术的提升直接影响其他计算机视觉算法的性能，如前景分割、物体检测、行为分析等；同时它也影响着图像的编码性能。因此，开发高性能的去模糊算法具有重要作用。
通常情况下可用卷积模型来解释模糊成因，相机抖动的过程可以映射为模糊核轨迹PSF(Point Spread Function)。在模糊核未知的情况下还原清晰图像，这一问题属于不适定(ill-posed)问题，所以，通常意义上需要先估计模糊核，再利用评估的模糊核进行返卷积操作得到复原图像。目前，常用算法包括基于MAP的EM算法；原始的MAPx,k(其中x表示清晰图像，k表示模糊核)算法很多情况下会将模糊图像作为非模糊解释(no-blur explanation)，这使得评估图像和模糊核依次迭代过程的失败；之后的MAPk(k表示模糊核)算法是MAPx,k的改进，它解决了非模糊解释的问题，这一算法首先估计出模糊核，之后再利用非盲返卷积(non-blind deconvolution)进行图像的复原。然而，以上算法都存在先验约束不足或不合适的问题，同时，评估的模糊核存在离群值这一问题也没有得到很好的解决，这一细微的差别很可能会造成去模糊过程的失败。
综上所述，现有的去模糊算法存在的缺点包括：(一)先验假设不准确；(二)先验约束不合适；(三)模糊核中存在的离群值未得到很好的抑制，这是由于相机抖动的过程是连续的，这就决定了模糊核轨迹的连续性，因此，模糊核中存在的离群值一定会造成返卷积过程的失败。
发明内容
为了克服上述现有技术的不足，本发明提出了一种基于先验约束和离群值抑制的图像去模糊方法，解决现有算法中存在的先验假设不准确、先验约束不合适，以及模糊核中存在离群值的问题，通过解决以上这些问题，本发明能够明显提高模糊图像的复原水平。
本发明的原理是：提出一种基于先验信息约束和离群值抑制的去模糊算法，旨在解决现有算法中存在的先验假设不准确、先验约束不合适，以及模糊核中存在离群值的问题。具体是在MAPk算法思想的基础上，基于先验约束和离群值抑制，实现图像去模糊；首先，利用L0范数约束以及重尾先验信息获得模糊图像中的显著性结构；其次，在显著性结构的基础上，采用L0范数约束对模糊核进行评估；然后，对评估的模糊核进行离群值抑制；最后，采用非盲反卷积算法得到最终的复原图像。本发明方法通过解决现有算法中存在的先验假设不准确、先验约束不合适，以及模糊核中存在离群值的问题，能够有效地提高模糊图像的复原。
本发明提供的技术方案是：
一种基于先验约束和离群值抑制的图像去模糊方法，采用卷积模型来拟合清晰图像的模糊过程，包括模糊图像显著性结构评估、模糊核估计及其离群值抑制、非忙反卷积模糊图像复原过程；
1)模糊图像显著结构评估过程，具体包括如下步骤：
11)采用式1的卷积模型来拟合清晰图像的模糊过程：
Figure PCTCN2017111996-appb-000001
其中，I表示模糊图像，k表示模糊核，η表示噪声(我们假设其分布为高斯噪声)；
采用重尾效应的先验约束作为模糊图像显著型结构梯度的分布情况，见式2：
Figure PCTCN2017111996-appb-000002
其中，S为模糊图像的显著性结构(并不是待还原的图像)，用来辅助评估模糊核k；式2的第一项可以看成是损失函数(虽然这样会造成式2值变大，但本发明要评估的是使优化方程达到最小的值S，所以不影响优化过程)；式2的第二项是用超拉普拉斯来模拟重尾效应；
12)评估模糊图像显著性结构：
引入L0范数对模糊图像的显著性结构S的纹理进行约束，同时利用L2范数限制S中平滑区域的噪声，更新后的公式如式3：
Figure PCTCN2017111996-appb-000003
其中，M是对模糊图像显著性结构S中纹理的二值标定，(1-M)是对S中平滑区域的二值标定；式3的后两项(即第三项和最后一项)用来进行纹理约束，其中第三项对大尺寸 细节进的约束，最后一项是对平滑的约束；
我们用式4和式5对M进行定义：
Figure PCTCN2017111996-appb-000004
M＝H(r-τr)      (式5)
式4中，x表示像素点的位置，y表示以像素点为中心，窗口大小为Nh范围内的像素点，r(x)表示位置x处的像素点属于纹理部分的程度；利用式4可以对S中的纹理做初步的划分，r(x)的取值在(0，1)，r(x)越大，表示x属于纹理部分的可能行越大；同时式4又限制了突变纹理的出现(因为当模糊核尺寸大于模糊图像细节尺寸时，图像复原失败，因此需要限制突变纹理)。式5中M用Heaviside阶跃函数来求取，其中τr表示像素点属于纹理部分的程度r(x)的阈值，它用来区分显著性结构S中的纹理区域和平滑区域，本发明中我们采用直方图均衡的方法来获取τr。
13)求解模糊图像显著性结构，具体如下：
为求解式3，引入两个替代变量u和w来选择性替代
Figure PCTCN2017111996-appb-000005
并用迭代的方法来更新S，式3的变种如下：
Figure PCTCN2017111996-appb-000006
我们用交替更新的方法获得每一次迭代S，u和w的解；
变量u的求解：
Figure PCTCN2017111996-appb-000007
Figure PCTCN2017111996-appb-000008
变量w的求解：
Figure PCTCN2017111996-appb-000009
我们采用相对全偏差RTV(Relative Total Variation)的方法来求解式9；
模糊图像显著性结构变量S的求解如下：
Figure PCTCN2017111996-appb-000010
基于帕斯瓦尔定理，将式10经过傅里叶变换之后求得S：
Figure PCTCN2017111996-appb-000011
其中，
Figure PCTCN2017111996-appb-000012
表示傅里叶变换，
Figure PCTCN2017111996-appb-000013
表示傅里叶逆变换。
2)模糊核估计及其离群值抑制过程，具体如下：
本发明采用梯度信息和显著性结构来估计模糊核，利用式14和15迭代更新得到模糊核轨迹，如图6所示和图8右图和图9(c)所示。
具体地，本发明利用评估的模糊图像显著性结构S来进行模糊核的估计，我们采用L0范数对模糊核中的离群值进行限制，优化方程如下：
Figure PCTCN2017111996-appb-000014
类似地，我们引入替换变量v进行迭代更新，式12的变种如下：
Figure PCTCN2017111996-appb-000015
对于以上两个变量(v和k)求解如下：
Figure PCTCN2017111996-appb-000016
Figure PCTCN2017111996-appb-000017
3)模糊图像复原过程，具体如下：
利用估计出的模糊核，采用非盲反卷积技术对模糊图像进行复原操作。
本发明实施方案中采用Richardson-Lucy算法来实现非盲返卷积，参见文献1(Perrone,Daniele,and Paolo Favaro."Total variation blind deconvolution:The devil is in the details."Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014)。
与现有技术相比，本发明的有益效果是：
本发明提出了一种基于先验约束和离群值抑制的图像去模糊方法，利用L0范数约束以及重尾先验信息获得模糊图像中的显著性结构；在显著性结构的基础上，采用L0范数约束 对模糊核进行评估；对评估的模糊核进行离群值抑制；采用非盲反卷积算法得到最终的复原图像。本发明能够解决现有算法中存在的先验假设不准确、先验约束不合适，以及模糊核中存在离群值的问题，通过解决以上这些问题，本发明能够明显提高模糊图像的复原水平。
附图说明
图1是本发明具体实施的流程示意图；
图2是本发明方法的流程框图；
其中，kn表示最小尺寸图像评估得到的模糊核，k0表示最终得到的评估模糊核。
图3是本发明实施例中模糊图像显著性结构的纹理图；
其中，(a)为原始模糊图像；(b)为原始模糊图像梯度分布情况；(c)为梯度直方图分布；(d)是用颜色信息表示的r值能量图、即原始图像r(x)的取值分布图。
图4是本发明实施例中模糊图像的显著性结构示例图。
图5是本发明实施例中不同尺寸下模糊图像的显著性结构；
其中，(a)～(d)为不同尺寸下模糊图像。
图6是本发明实施例迭代更新得到的模糊核。
图7是本发明实施例模糊图像复原后的图像。
图8是本发明实施例中模糊图像及其复原图像放大的对比图。
图9是本发明实施例中模糊图像、显著性结构、复原图的对比；
其中，图(a)为原始模糊图像；图(b)为显著性结构；图(c)为模糊复原图像。
具体实施方式
下面结合附图，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
本发明提出的基于先验约束和离群值抑制的去模糊方法流程图如图2所示，图中，kn表示最小尺寸图像评估得到的模糊核，k0表示最终得到的评估模糊核。本发明方法利用多次采样的方式不断迭代更新模糊核，并利用非忙反卷积算法获得最终的复原图象。
本发明方法具体步骤如下：
表1是以下步骤中采用的参数名称及其相应参数含义说明
表1 参数列表
Figure PCTCN2017111996-appb-000018
Figure PCTCN2017111996-appb-000019
步骤1.模糊模型选择：本发明采用式1模型，并假设噪声服从高斯分布，得到如式2的优化方程；
本发明采用卷积模型来拟合清晰图像的模糊过程，如式1：
Figure PCTCN2017111996-appb-000020
其中，I表示模糊图像，k表示模糊核，η表示噪声(我们假设其分布为高斯噪声)。
采用重尾效应的先验约束作为模糊图像显著型结构梯度的分布情况，见式2：
Figure PCTCN2017111996-appb-000021
其中，S为模糊图像的显著性结构；
步骤2.模糊图像显著性结构评估：
首先，利用式4和式5得到显著性结构的纹理标定，如图3(d)所示；图3是实施例中模糊图像显著性结构的纹理图；其中，(a)为原始模糊图像；(b)为原始模糊图像梯度分布情况；(c)为梯度直方图分布；(d)为原始图像r(x)的取值分布。利用式4和式5得到显著性结构的纹理标定具体如下：
我们引入L0范数对模糊图像的显著性结构S的纹理进行约束，同时利用L2范数限制S中平滑区域的噪声，更新后的公式如式3：
Figure PCTCN2017111996-appb-000022
其中，M是对模糊图像显著性结构S中纹理的二值标定，(1-M)是对S中平滑区域的二值标定；我们用式4和式5对M进行定义：
Figure PCTCN2017111996-appb-000023
M＝H(r-τr)  (式5)
式4中，x表示像素点的位置，y表示以像素点为中心，窗口大小为Nh范围内的像素点，r(x)表示位置x处的像素点属于纹理部分的程度；利用式4可以对S中的纹理做初步的划分，r(x)的取值在(0，1)，r(x)越大，表示x属于纹理部分的可能行越大；同时式4又限制了突变纹理的出现(因为当模糊核尺寸大于模糊图像细节尺寸时，图像复原失败，因此需要限制突变纹理)。式5中M用Heaviside阶跃函数来求取，其中τr表示r的阈值，它用来区分显著性结构S中的纹理区域和平滑区域，本发明中我们采用直方图均衡的方法来获取τr。
其次，利用式8，9，11迭代更新得到模糊图像显著性结构，如图4所示；图5为不同尺寸下估计得到的显著性结构；具体如下：
为求解式3，我们引入两个替代变量u和w来选择性替代
Figure PCTCN2017111996-appb-000024
并用迭代的方法来更新S，式3的变种如下:
Figure PCTCN2017111996-appb-000025
我们用交替更新的方法获得每一次迭代S，u和w的解；
变量u的求解：
Figure PCTCN2017111996-appb-000026
Figure PCTCN2017111996-appb-000027
变量w的求解：
Figure PCTCN2017111996-appb-000028
我们采用相对全偏差RTV(Relative Total Variation)的方法来求解式9；
变量S的求解：
Figure PCTCN2017111996-appb-000029
基于帕斯瓦尔定理，我们将式10经过傅里叶变换之后求得S：
Figure PCTCN2017111996-appb-000030
其中，
Figure PCTCN2017111996-appb-000031
表示傅里叶变换，
Figure PCTCN2017111996-appb-000032
表示傅里叶逆变换。
步骤3.模糊核估计，具体如下：
本发明采用梯度信息和显著性结构来估计模糊核，利用式14和15迭代更新得到模糊核轨迹，如图6所示和图8右图和图9(c)所示。
具体地，本发明利用评估的模糊图像显著性结构S来进行模糊和的估计，我们采用L0范数对模糊核中的离群值进行限制，优化方程如下：
Figure PCTCN2017111996-appb-000033
类似地，我们引入替换变量v进行迭代更新，式12的变种如下：
Figure PCTCN2017111996-appb-000034
对于以上两个变量求解如下：
Figure PCTCN2017111996-appb-000035
Figure PCTCN2017111996-appb-000036
步骤4.非盲反卷积，具体如下：
此处可采用任何现有的非盲反卷积算法。
以上步骤可表示为如下算法流程：
Figure PCTCN2017111996-appb-000037
Figure PCTCN2017111996-appb-000038
本发明实施方案中，非盲反卷积具体采用文献1(Perrone,Daniele,and Paolo Favaro."Total variation blind deconvolution: The devil is in the details."Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014)中提出的算法，如图7所示，其为图3中模糊图像的复原图像；另以图8为例，左图为模糊图像及其放大区域，右图为复原效果及对应的放大区域。其中模糊核以能量图的形式表示在右图左上角。
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (8)

  1. 一种图像去模糊方法，基于先验约束和离群值抑制，采用卷积模型来拟合清晰图像的模糊过程，再对模糊图像I进行复原，由此达到图像去模糊的目的；包括模糊图像显著性结构评估过程、模糊核估计及其离群值抑制过程、非忙反卷积模糊图像复原过程；
    1)模糊图像I的显著结构评估过程，包括如下步骤：
    11)采用重尾效应的先验约束作为模糊图像显著型结构梯度的分布情况，如式2：
    Figure PCTCN2017111996-appb-100001
    其中，S为模糊图像的显著性结构(并不是待还原的图像)，用来辅助评估模糊核k；式2的第一项可看成是损失函数；式2的第二项是用超拉普拉斯来模拟重尾效应；
    12)评估模糊图像显著性结构：
    引入L0范数对模糊图像的显著性结构S的纹理进行约束，同时利用L2范数限制S中平滑区域的噪声，更新后如式3：
    Figure PCTCN2017111996-appb-100002
    其中，M是对模糊图像显著性结构S中纹理的二值标定，(1-M)是对S中平滑区域的二值标定；式3中第三项对大尺寸细节进的约束，最后一项是对平滑的约束；
    13)求解模糊图像显著性结构，具体如下：
    为求解式3，引入两个替代变量u和w来选择性替代
    Figure PCTCN2017111996-appb-100003
    式3的变种为式6：
    Figure PCTCN2017111996-appb-100004
    用交替更新的方法获得每一次迭代S、u和w的解；再经过傅里叶变换之后求得模糊图像显著性结构S并更新；
    2)模糊核估计及其离群值抑制过程：采用梯度信息和显著性结构S来估计模糊核k，通过迭代更新估计得到模糊核k的轨迹；
    3)模糊图像复原过程：利用估计出的模糊核，采用非盲反卷积方法对模糊图像进行复原操作。
  2. 如权利要求1所述图像去模糊方法，其特征是，步骤12)用式4和式5对模糊图像显著性结构S中纹理的二值标定M进行定义：
    Figure PCTCN2017111996-appb-100005
    M＝H(r-τr)            (式5)
    式4中，x表示像素点的位置；y表示以像素点为中心，窗口大小为Nh范围内的像素点；r(x)表示位置x处的像素点属于纹理部分的程度；利用式4对S中的纹理做初步的划分，r(x)的取值在(0，1)，r(x)越大，表示x属于纹理部分的可能行越大；同时式4限制了突变纹理的出现；式5中M用Heaviside阶跃函数来求取，其中τr表示r的阈值，用来区分显著性结构S中的纹理区域和平滑区域。
  3. 如权利要求2所述图像去模糊方法，其特征是，所述τr采用直方图均衡的方法来获取。
  4. 如权利要求1所述图像去模糊方法，其特征是，步骤13)用迭代的方法来更新S，所述变量u的求解如下：
    Figure PCTCN2017111996-appb-100006
    Figure PCTCN2017111996-appb-100007
    所述变量w的求解如下：
    Figure PCTCN2017111996-appb-100008
    再采用相对全偏差RTV方法来求解式9；
    所述变量S的求解如下：
    Figure PCTCN2017111996-appb-100009
    基于帕斯瓦尔定理，将式10经过傅里叶变换之后求得S：
    Figure PCTCN2017111996-appb-100010
    式11中，
    Figure PCTCN2017111996-appb-100011
    表示傅里叶变换，
    Figure PCTCN2017111996-appb-100012
    表示傅里叶逆变换。
  5. 如权利要求1所述图像去模糊方法，其特征是，步骤2)具体利用评估的模糊图像显著性结构S来进行模糊核k的估计，采用L0范数对模糊核中的离群值进行限制，优化方程如式12：
    Figure PCTCN2017111996-appb-100013
    引入替换变量v进行迭代更新，式12的变种如式13：
    Figure PCTCN2017111996-appb-100014
    再通过式14～15对两个变量v、k求解：
    Figure PCTCN2017111996-appb-100015
    Figure PCTCN2017111996-appb-100016
    其中，
    Figure PCTCN2017111996-appb-100017
    表示傅里叶变换，
    Figure PCTCN2017111996-appb-100018
    表示傅里叶逆变换。
  6. 如权利要求1所述图像去模糊方法，其特征是，步骤3)采用Richardson-Lucy算法来实现所述非盲反卷积方法。
  7. 如权利要求1所述图像去模糊方法，其特征是，采用式1的卷积模型来拟合清晰图像的模糊过程：
    Figure PCTCN2017111996-appb-100019
    其中，I表示模糊图像，k表示模糊核，η表示噪声。
  8. 如权利要求7所述图像去模糊方法，其特征是，所述噪声η的分布为高斯分布。
PCT/CN2017/111996 2017-06-15 2017-11-21 基于先验约束和离群值抑制的图像去模糊方法 Ceased WO2018227882A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/336,855 US11263728B2 (en) 2017-06-15 2017-11-21 Priori constraint and outlier suppression based image deblurring method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710452806.8A CN107292842B (zh) 2017-06-15 2017-06-15 基于先验约束和离群值抑制的图像去模糊方法
CN201710452806.8 2017-06-15

Publications (1)

Publication Number Publication Date
WO2018227882A1 true WO2018227882A1 (zh) 2018-12-20

Family

ID=60097972

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/111996 Ceased WO2018227882A1 (zh) 2017-06-15 2017-11-21 基于先验约束和离群值抑制的图像去模糊方法

Country Status (3)

Country Link
US (1) US11263728B2 (zh)
CN (1) CN107292842B (zh)
WO (1) WO2018227882A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2021124385A1 (zh) * 2019-12-16 2021-06-24
CN113538374A (zh) * 2021-07-15 2021-10-22 中国科学院上海技术物理研究所 一种面向高速运动物体的红外图像模糊校正方法
CN114418883A (zh) * 2022-01-18 2022-04-29 北京工业大学 一种基于深度先验的盲图像去模糊方法
CN114418892A (zh) * 2022-01-21 2022-04-29 合肥工业大学 一种结合鲁棒显著边缘选择与稀疏性的图像去模糊方法

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292842B (zh) * 2017-06-15 2020-08-07 北京大学深圳研究生院 基于先验约束和离群值抑制的图像去模糊方法
US11354784B2 (en) * 2020-03-02 2022-06-07 GM Global Technology Operations LLC Systems and methods for training a non-blind image deblurring module
CN113177890B (zh) * 2021-04-27 2023-04-18 深圳市慧鲤科技有限公司 图像处理方法及装置、电子设备和存储介质
CN115690144B (zh) * 2021-07-21 2026-04-10 西安大医集团股份有限公司 图像处理方法及装置、放疗系统
CN114140337B (zh) * 2021-10-11 2025-02-11 长江大学 基于模糊因子预估变分的超声测井图像盲复原方法及系统
CN114092416B (zh) * 2021-11-04 2024-12-20 上海市特种设备监督检验技术研究院 一种dr模糊图像盲反卷积复原方法及系统
CN114418891B (zh) * 2022-01-21 2025-04-08 中国科学院长春光学精密机械与物理研究所 一种基于局部二值模式的图像盲去模糊算法
CN114897734B (zh) * 2022-05-18 2024-05-28 北京化工大学 一种基于梯度方向先验的被测目标图像复原方法
CN114820773B (zh) * 2022-06-26 2022-09-27 山东济宁运河煤矿有限责任公司 基于计算机视觉的筒仓运输车辆车厢位置检测方法
CN116823643A (zh) * 2023-06-02 2023-09-29 湖南涉外经济学院 一种可预去除模糊度的图像处理系统
CN117036196A (zh) * 2023-08-21 2023-11-10 深圳大学 一种基于图像梯度内积先验的盲去模糊算法
CN117994166B (zh) * 2024-04-03 2024-07-19 中国科学院光电技术研究所 一种基于非盲反卷积及Codeformer的衍射图像增强方法
CN120339121B (zh) * 2025-06-19 2025-08-15 长春理工大学 一种对脉冲噪声鲁棒的盲图像去模糊方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080025627A1 (en) * 2006-07-28 2008-01-31 Massachusetts Institute Of Technology Removing camera shake from a single photograph
CN105894463A (zh) * 2016-03-24 2016-08-24 重庆信科设计有限公司 一种区域信息分离的全变分图像盲去模糊方法
CN106709877A (zh) * 2016-11-11 2017-05-24 天津大学 一种基于多参数正则优化模型的图像去模糊方法
CN107292842A (zh) * 2017-06-15 2017-10-24 北京大学深圳研究生院 基于先验约束和离群值抑制的图像去模糊方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8184926B2 (en) * 2007-02-28 2012-05-22 Microsoft Corporation Image deblurring with blurred/noisy image pairs
US8390704B2 (en) * 2009-10-16 2013-03-05 Eastman Kodak Company Image deblurring using a spatial image prior
CN102867289B (zh) * 2012-08-20 2015-06-24 清华大学 互补模糊图像采集系统及利用该系统的模糊图像恢复方法
US9076205B2 (en) * 2012-11-19 2015-07-07 Adobe Systems Incorporated Edge direction and curve based image de-blurring
US8897588B2 (en) * 2012-11-20 2014-11-25 Adobe Systems Incorporated Data-driven edge-based image de-blurring
WO2015017194A1 (en) * 2013-07-29 2015-02-05 Nikon Corporation Multiple phase method for image deconvolution
US9779491B2 (en) * 2014-08-15 2017-10-03 Nikon Corporation Algorithm and device for image processing
CN105493140B (zh) * 2015-05-15 2018-02-02 北京大学深圳研究生院 图像去模糊方法及系统
CN105976332B (zh) * 2016-05-03 2019-03-01 北京大学深圳研究生院 基于图像中亮条纹信息的图像去模糊方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080025627A1 (en) * 2006-07-28 2008-01-31 Massachusetts Institute Of Technology Removing camera shake from a single photograph
CN105894463A (zh) * 2016-03-24 2016-08-24 重庆信科设计有限公司 一种区域信息分离的全变分图像盲去模糊方法
CN106709877A (zh) * 2016-11-11 2017-05-24 天津大学 一种基于多参数正则优化模型的图像去模糊方法
CN107292842A (zh) * 2017-06-15 2017-10-24 北京大学深圳研究生院 基于先验约束和离群值抑制的图像去模糊方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2021124385A1 (zh) * 2019-12-16 2021-06-24
WO2021124385A1 (ja) * 2019-12-16 2021-06-24 三菱電機株式会社 画像処理装置および画像処理方法
JP7209867B2 (ja) 2019-12-16 2023-01-20 三菱電機株式会社 画像処理装置および画像処理方法
CN113538374A (zh) * 2021-07-15 2021-10-22 中国科学院上海技术物理研究所 一种面向高速运动物体的红外图像模糊校正方法
CN113538374B (zh) * 2021-07-15 2023-11-28 中国科学院上海技术物理研究所 一种面向高速运动物体的红外图像模糊校正方法
CN114418883A (zh) * 2022-01-18 2022-04-29 北京工业大学 一种基于深度先验的盲图像去模糊方法
CN114418883B (zh) * 2022-01-18 2024-03-29 北京工业大学 一种基于深度先验的盲图像去模糊方法
CN114418892A (zh) * 2022-01-21 2022-04-29 合肥工业大学 一种结合鲁棒显著边缘选择与稀疏性的图像去模糊方法

Also Published As

Publication number Publication date
CN107292842B (zh) 2020-08-07
CN107292842A (zh) 2017-10-24
US11263728B2 (en) 2022-03-01
US20210287345A1 (en) 2021-09-16

Similar Documents

Publication Publication Date Title
WO2018227882A1 (zh) 基于先验约束和离群值抑制的图像去模糊方法
CN108537756B (zh) 基于图像融合的单幅图像去雾方法
CN108230264B (zh) 一种基于ResNet神经网络的单幅图像去雾方法
CN105657402B (zh) 一种深度图恢复方法
CN110517197B (zh) 基于参数模型的单幅图像去模糊方法
CN105608679B (zh) 一种融合结构张量与非局域全变分的图像去噪方法
CN103413277B (zh) 基于l0稀疏先验的盲相机抖动去模糊方法
WO2022233252A1 (zh) 图像处理方法、装置、计算机设备和存储介质
CN102208100A (zh) 基于Split Bregman 迭代的全变差正则化图像盲复原方法
CN110796616B (zh) 基于范数约束和自适应加权梯度的湍流退化图像恢复方法
CN114897741A (zh) 基于深度残差傅里叶变换的图像盲去模糊方法
CN108564544A (zh) 基于边缘感知的图像盲去模糊组合稀疏优化方法
CN112508923B (zh) 弱小目标检测方法
Dai et al. Fully connected guided image filtering
CN105184766B (zh) 一种频域边界能量模型的水平集图像分割方法
CN110349099A (zh) 一种复杂场景视频阴影检测与消除方法
CN105678715A (zh) 一种基于核回归全变分的去除图像噪声的方法
CN115689960A (zh) 一种夜间场景下照度自适应的红外与可见光图像融合方法
CN107248143B (zh) 一种基于图像分割的深度图像修复方法
CN106683129B (zh) 一种基于双边核回归的相对约减纹理分解方法及其装置
CN102682437A (zh) 一种基于总变分正则约束的图像解卷积方法
CN110706180A (zh) 一种极暗图像视觉质量提升方法、系统、设备及介质
CN102629369A (zh) 一种基于照度表面建模的单幅彩色图像阴影去除方法
CN106023097A (zh) 一种基于迭代法的流场图像预处理算法
CN120298237A (zh) 一种基于加性高斯扩散模型的图像降噪增强方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17913713

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17913713

Country of ref document: EP

Kind code of ref document: A1