CN111724307A - An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium - Google Patents
An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium Download PDFInfo
- Publication number
- CN111724307A CN111724307A CN202010568221.4A CN202010568221A CN111724307A CN 111724307 A CN111724307 A CN 111724307A CN 202010568221 A CN202010568221 A CN 202010568221A CN 111724307 A CN111724307 A CN 111724307A
- Authority
- CN
- China
- Prior art keywords
- image
- rank
- low
- resolution
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 143
- 230000008569 process Effects 0.000 claims abstract description 17
- 238000005065 mining Methods 0.000 claims abstract description 3
- 230000006870 function Effects 0.000 claims description 41
- 238000011156 evaluation Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 8
- 230000015556 catabolic process Effects 0.000 claims description 6
- 238000006731 degradation reaction Methods 0.000 claims description 6
- 239000000654 additive Substances 0.000 claims description 4
- 230000000996 additive effect Effects 0.000 claims description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 description 21
- 238000004422 calculation algorithm Methods 0.000 description 20
- 230000000694 effects Effects 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 9
- 210000004072 lung Anatomy 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000003384 imaging method Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000011160 research Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000000052 comparative effect Effects 0.000 description 4
- 239000002420 orchard Substances 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013441 quality evaluation Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 210000004185 liver Anatomy 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000002285 radioactive effect Effects 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
本发明提供一种基于最大后验概率和非局部低秩先验的图像超分辨重建方法,终端及可读存储介质,采用连续图像序列作为数据输入,利用单幅图像内与连续图像间的相似性作为先验知识,结合图像块局部分组方式将相似块进行块匹配,挖掘图像像素级的空间结构关系;以最大后验概率框架建模,使用高斯分布和吉布斯分布拟合模型参数,提升模型泛化能力;采用低秩截断的方式抑制噪声干扰;采用非局部低秩约束正则化图像重建过程,再利用单幅图像内的局部信息以及连续图像间的局信息,提升目标图像质量。在每次迭代中交替优化模型中的参数,提高模型的鲁棒性,避免局部收敛。最后加权平均重建的图像块,得出目标高分辨图像。
The present invention provides an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, a terminal and a readable storage medium, using a continuous image sequence as data input, and using the similarity within a single image and between continuous images Using the property as prior knowledge, combining the local grouping method of image blocks to match similar blocks, mining the spatial structure relationship at the pixel level of the image; modeling with the maximum posterior probability framework, using Gaussian distribution and Gibbs distribution to fit model parameters, Improve the generalization ability of the model; use low-rank truncation to suppress noise interference; use non-local low-rank constraints to regularize the image reconstruction process, and then use local information in a single image and local information between consecutive images to improve the quality of the target image. Alternately optimize the parameters in the model in each iteration to improve the robustness of the model and avoid local convergence. Finally, the reconstructed image blocks are weighted and averaged to obtain the target high-resolution image.
Description
技术领域technical field
本发明涉及图像处理技术领域,尤其涉及一种基于最大后验概率和非局部低秩先验的图像超分辨重建方法,终端及可读存储介质。The present invention relates to the technical field of image processing, and in particular, to an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, a terminal and a readable storage medium.
背景技术Background technique
视觉是人类从外界获取信息的主要途径之一,而大部分基于视觉的应用性能依赖于图像的质量。高分辨率(High Resolution,HR)图像分辨率高,包含丰富的图像细节和更多的图像信息,因此在医学领域、视频监控领域、遥感领域等实际应用中具有重要价值。尽管在这些实际应用领域中,图像成像技术已经趋于成熟,但受成像设备、成像环境、人为干扰等因素的相互制约,大部分图像的分辨率很低。例如在医学领域,医学图像成像技术受医学成像设备、放射性元素危害度以及人体生理健康等因素的相互制约,许多医学图像成像分辨率很低,低分辨率(Low Resolution,LR)图像无法有效辅助病灶组织分类检测任务,因此,图像超分辨技术应运而生。超分辨率(Super Resolution,SR)技术是一种图像分析和处理技术,将观测到的低分辨率单幅图像或图像序列作为输入,生成高分辨率的单幅图像或图像序列。基于插值的方法、基于学习的方法、基于重建的方法是目前流行的3类图像超分辨方法。Vision is one of the main ways for humans to obtain information from the outside world, and the performance of most vision-based applications depends on the quality of images. High Resolution (HR) images have high resolution and contain rich image details and more image information, so they are of great value in practical applications such as medicine, video surveillance, and remote sensing. Although image imaging technology has matured in these practical application fields, the resolution of most images is very low due to the mutual constraints of imaging equipment, imaging environment, human interference and other factors. For example, in the field of medicine, medical image imaging technology is constrained by factors such as medical imaging equipment, radioactive element hazards, and human physiological health. Many medical image imaging resolutions are very low, and low resolution (LR) images cannot effectively assist The task of classification and detection of lesion tissue, therefore, image super-resolution technology came into being. Super Resolution (SR) technology is an image analysis and processing technology that takes the observed low-resolution single image or image sequence as input to generate a high-resolution single image or image sequence. Interpolation-based methods, learning-based methods, and reconstruction-based methods are currently three types of popular image super-resolution methods.
基于插值的方法是一种较早提出且相对简单的算法。首先是计算低分辨率图像与目标高分辨率图像之间的配准关系,再根据插值公式,利用邻域内已知像素值得到待插值点像素值,从而得到目标高分辨率图像。常见的有双线性插值(Bilinear Interpolation)、双三次插值(Bicubic Interpolation)和最邻近插值(Nearest NeighborInterpolation)。基于插值的方法实现简单、计算复杂度低,因此具有很好的实时性。但在插值过程中未考虑图像的各向异性,不能有效地保留图像高频信息,导致放大后图像的轮廓和纹理比较模糊,容易出现块状外观,产生伪边缘,图像质量较差。插值算法难于处理图像中的模糊现象、图像引入噪声等问题,也无法添加先验信息,因此方法的适应性也较差。The interpolation-based method is an earlier and relatively simple algorithm. Firstly, the registration relationship between the low-resolution image and the target high-resolution image is calculated, and then according to the interpolation formula, the pixel value of the point to be interpolated is obtained by using the known pixel value in the neighborhood, so as to obtain the target high-resolution image. Common ones are Bilinear Interpolation, Bicubic Interpolation and Nearest Neighbor Interpolation. The method based on interpolation is simple in implementation and low in computational complexity, so it has good real-time performance. However, the anisotropy of the image is not considered in the interpolation process, and the high-frequency information of the image cannot be effectively preserved, resulting in blurred contours and textures of the enlarged image, prone to blocky appearance, false edges, and poor image quality. The interpolation algorithm is difficult to deal with the blur phenomenon in the image and the noise introduced by the image, and cannot add prior information, so the adaptability of the method is also poor.
在现有技术中,基于学习的方法逐渐成为近年较为流行的一类技术。算法的基本思想是从训练样本集中学习低分辨率图像与高分辨率图像之间的映射关系,从而对未知低分辨率图像进行预测,达到提高分辨率的目的。例如,利用局部嵌入流行学习的思想,基于邻域嵌入(Neighbor Embedding)的学习策略,基于样例学习(Example Learning)的算法,结合反卷积的神经网络算法等,但此类方法对外部训练数据集有较大的依赖性,模型增量性差。最近对图像统计数据的研究表明,图像块可以由其过完备字典的元素进行稀疏线性表示,因此基于稀疏表示(Sparse Representation)的学习方法得到应用,但该算法需要大量的训练数据集,且部分数据集需要人工标注,并且内部字典通常不足以包含复杂纹理信息而进行良好地重建。综上可见,学习模型对于图像超分辨的效果至关重要,但目前的模型还无法有效地结合图像重建所需的全部先验知识,并且基于学习的方法运行时间较长,很难满足实时性要求,因此更适合于离线的图像预处理。In the prior art, learning-based methods have gradually become a popular type of technology in recent years. The basic idea of the algorithm is to learn the mapping relationship between low-resolution images and high-resolution images from the training sample set, so as to predict the unknown low-resolution images and improve the resolution. For example, using the idea of local embedding for popular learning, learning strategies based on Neighbor Embedding, algorithms based on Example Learning, neural network algorithms combined with deconvolution, etc., but such methods are not suitable for external training. The data set has a large dependency, and the model increment is poor. Recent research on image statistics shows that image patches can be sparsely linearly represented by the elements of their overcomplete dictionary, so the learning method based on sparse representation (Sparse Representation) has been applied, but this algorithm requires a large number of training data sets, and some Datasets require manual annotation, and internal dictionaries are often insufficient to contain complex texture information for good reconstruction. To sum up, it can be seen that the learning model is very important for the effect of image super-resolution, but the current model cannot effectively combine all the prior knowledge required for image reconstruction, and the learning-based method runs for a long time, which is difficult to meet the real-time performance. requirements, so it is more suitable for offline image preprocessing.
基于重建的方法目的是重建在降质过程中丢失的高频信号。假设输入的低分辨率图像共有n帧,则基于重建的超分辨问题的数学模型可以表示为:Reconstruction-based methods aim to reconstruct high-frequency signals lost during degradation. Assuming that the input low-resolution image has a total of n frames, the mathematical model of the reconstruction-based super-resolution problem can be expressed as:
这里lk是由待重建的原始高分辨率图像H经过一系列的图像变换过程得到的低分辨率图像。是大气模糊算子,Mk是运动变换算子,指成像模糊算子。D为降采样算子,Nk代表在成像过程中引入的加性噪声。已知输入lk,则基于重建的方法目标是寻找真实高分辨率图像H的最优估计 Here lk is a low-resolution image obtained from the original high-resolution image H to be reconstructed through a series of image transformation processes. is the atmospheric blur operator, M k is the motion transformation operator, Refers to the imaging blur operator. D is the down-sampling operator, and N k represents the additive noise introduced in the imaging process. Given the input l k , the goal of the reconstruction-based method is to find the optimal estimate of the real high-resolution image H
频域法是被最早提出的基于重建的超分辨方法之一,由Tsai和Huang在1984年提出。分别对低分辨率图像和目标高分辨图像做离散傅里叶变换和连续傅里叶变换,并根据傅里叶变换的性质,在频域中建立起二者之间的线性关系。Rhee和Kang等人以离散余弦变换代替频域的离散傅里叶变换,降低了存储要求和代价。虽然频域法的理论简单,在推导和计算上都有一定的优势,但难于处理噪声,并且难以添加先验信息。另外由于频域与空域存在复杂的变换关系,只能处理全局整体运动的情况,难于处理具有局部运动的情况。The frequency domain method is one of the earliest reconstruction-based super-resolution methods proposed by Tsai and Huang in 1984. The discrete Fourier transform and the continuous Fourier transform are performed on the low-resolution image and the target high-resolution image respectively, and the linear relationship between the two is established in the frequency domain according to the properties of the Fourier transform. Rhee and Kang et al. replaced the discrete Fourier transform in the frequency domain with discrete cosine transform, which reduced storage requirements and costs. Although the theory of frequency domain method is simple and has certain advantages in derivation and calculation, it is difficult to deal with noise and add prior information. In addition, due to the complex transformation relationship between the frequency domain and the space domain, it can only deal with the situation of global overall motion, and it is difficult to deal with the situation with local motion.
现有技术中还有采用迭代反向投影法,采用迭代反向投影法(Iterative BackProjection,IBP)是由Irani和Peleg提出。该方法是将退化模型生成的低分辨率图像与输入的低分辨率图像二者的差值反向投影到高分辨率图像上,不断迭代使误差收敛,从而得到目标高分辨率图像。IBP方法直观,易于理解,但IBP是逆问题,它的病态性将导致解不唯一。凸集投影(Projection onto convex sets,POCS)法是一种采用迭代的超分辨重建方法。在POCS方法中,可以加入先验信息对结果的影响,如对目标图像峰值像素的约束等。POCS方法形式比较灵活,能够比较方便地添加先验信息。但方法的计算复杂度高,要求多次迭代及投影,收敛速度比较慢,算法稳定性不高。There is also an iterative backprojection method in the prior art, and the iterative backprojection (IBP) method was proposed by Irani and Peleg. The method is to back-project the difference between the low-resolution image generated by the degradation model and the input low-resolution image onto the high-resolution image, and iterate continuously to make the error converge to obtain the target high-resolution image. The IBP method is intuitive and easy to understand, but IBP is an inverse problem, and its ill-posedness will lead to non-unique solutions. Projection onto convex sets (POCS) is an iterative super-resolution reconstruction method. In the POCS method, the influence of prior information on the results can be added, such as constraints on the peak pixels of the target image. The POCS method is more flexible in form and can easily add prior information. However, the computational complexity of the method is high, requiring multiple iterations and projections, the convergence speed is relatively slow, and the algorithm stability is not high.
发明内容SUMMARY OF THE INVENTION
为了克服上述现有技术中的不足,本发明提供一种基于最大后验概率和非局部低秩先验的图像超分辨重建方法,方法包括:In order to overcome the above-mentioned deficiencies in the prior art, the present invention provides an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, the method comprising:
步骤一、采用连续图像序列作为数据输入,利用单幅图像内与连续图像间的相似性作为先验知识,结合图像块局部分组方式将相似块进行块匹配,挖掘图像像素级的空间结构关系;Step 1: Using a continuous image sequence as data input, using the similarity between a single image and the continuous image as prior knowledge, and combining the local grouping of image blocks to perform block matching on similar blocks, and mining the spatial structure relationship at the pixel level of the image;
步骤二、以最大后验概率框架建模,使用高斯分布和吉布斯分布拟合模型参数,提升模型泛化能力;
步骤三、通过相似块奇异值的最大后验估计出待求块的奇异值,采用低秩截断的方式抑制噪声干扰;Step 3, estimating the singular value of the block to be determined by the maximum a posteriori of the singular value of the similar blocks, and suppressing the noise interference by means of low-rank truncation;
步骤四、采用非局部低秩约束正则化图像重建过程,再利用单幅图像内的局部信息以及连续图像间的局信息,提升目标图像质量。Step 4: Use the non-local low-rank constraint to regularize the image reconstruction process, and then use the local information in a single image and the local information between consecutive images to improve the quality of the target image.
进一步需要说明的是,步骤四之后还包括:It should be further noted that, after
采用峰值信噪比、结构相似性以及特征相似度评价重建后的图像质量。The reconstructed image quality was evaluated by peak signal-to-noise ratio, structural similarity and feature similarity.
进一步需要说明的是,步骤一还包括:It should be further noted that step 1 also includes:
连续图像序列为Y{yk},k=1,...,10;连续图像序列由一系列低分辨率图像组成,以序列中间的图像y作为基准图像进行超分辨重建,其他图像协同辅助;The continuous image sequence is Y{y k }, k=1,...,10; the continuous image sequence consists of a series of low-resolution images, and the image y in the middle of the sequence is used as the reference image for super-resolution reconstruction, and other images are cooperatively assisted ;
已知图像退化模型为:The known image degradation model is:
y=DBkx+nk (1)y=DB k x+n k (1)
式中,x是待重建的高分辨率图像,y是输入的低分辨率图像,是由原始图像双三次插值放大得到;In the formula, x is the high-resolution image to be reconstructed, and y is the input low-resolution image, which is obtained by bicubic interpolation of the original image;
D为下采样算子,Bk为模糊算子。假设nk是均值为0、方差为的加性高斯白噪声;D is the down-sampling operator, and B k is the fuzzy operator. Suppose n k is zero mean and variance is The additive white Gaussian noise;
在步骤二中,超分辨的目的是从低分辨率基准图像y重建出高分辨率图像x;由MAP模型表述为下面的目标函数:In
公式(2)中用高斯函数来拟合,In formula (2), a Gaussian function is used to fit,
假设像素值与满足吉布斯概率密度函数的相邻像素有关,则用吉布斯函数拟合,Assuming that the pixel value is related to the adjacent pixels that satisfy the Gibbs probability density function, the Gibbs function is used to fit,
将式(3)和式(4)代入式(2),整理得:Substituting equations (3) and (4) into equation (2), we get:
式(5)即为待求的目标函数。Equation (5) is the objective function to be sought.
进一步需要说明的是,步骤四还包括:It should be further noted that
使用非局部低秩先验,并将非局部低秩先验,表示为最大后验概率估计;Use a non-local low-rank prior, and express the non-local low-rank prior as a maximum posterior probability estimate;
假设表示图像块xj的一系列相似块,其中j为图像块索引,相似块是以j为中心形成的大小为的图像块;Assumption Represents a series of similar blocks of the image block x j , where j is the image block index, and the size of the similar block is formed with j as the center the image block;
在相似块匹配时,选取整个序列图像中最相似的p块相似块,再进行低秩截断处理;When similar blocks are matched, select the most similar p-block similar blocks in the whole sequence image, and then perform low-rank truncation processing;
假设每组低秩块是相互独立的,低秩先验表示为:Assuming that each group of low-rank blocks is independent of each other, the low-rank prior is expressed as:
将式(6)代入式(5)得:Substitute equation (6) into equation (5) to get:
通常低秩矩阵常用核范式来解决,Usually low-rank matrices are usually solved in nuclear normal form,
其中,Lj为待求的的低秩块,||Lj||*为Lj的核范数,用以表示奇异值的和;使用迭代方向乘子,通过构造增强拉格朗日方程,求解式(8);Among them, L j is the desired The low-rank block of , ||L j || * is the nuclear norm of L j , which is used to represent the sum of singular values; using the iterative direction multiplier, by constructing the enhanced Lagrangian equation, solve Equation (8);
式中,Uj是拉格朗日乘子,μ是一个常量参数。求解式(9)可以分解成两个子问题:where U j is the Lagrange multiplier and μ is a constant parameter. Solving equation (9) can be decomposed into two sub-problems:
其中,对x求解,可由式(10)直接求得:Among them, the solution of x can be directly obtained by formula (10):
采用MAP方法求解低秩块Lj,用的奇异值估计出Lj的奇异值,得出Lj;Using the MAP method to solve the low-rank block L j , use Estimate the singular value of L j from the singular value of , and obtain L j ;
公式(12)可由贝叶斯准则得,Equation (12) can be obtained from the Bayesian criterion,
假设扭曲度f表示高分辨率图像块奇异值与低分辨率图像块奇异值的扭曲度;公式(13)式中,第一部分是用均值为0,标准差是f的高斯函数来拟合,Suppose the distortion degree f represents the distortion degree of the singular value of the high-resolution image block and the singular value of the low-resolution image block; in formula (13), the first part is fitted with a Gaussian function with a mean value of 0 and a standard deviation of f,
P(σi(Lj))用核密度估计来计算,其概率密度函数被认定为一系列核函数的和;奇异值的和即为一系列核函数的和,核函数由以σi(Lj)为中心的1×3邻域Ωi确定;P(σ i (L j )) is calculated by kernel density estimation, and its probability density function is regarded as the sum of a series of kernel functions; the sum of singular values is the sum of a series of kernel functions, and the kernel function is determined by σ i ( L j ) is determined by the 1×3 neighborhood Ω i of the center;
假设核函数均值符合均值标准差为hi的高斯分布,则奇异值的概率密度函数被定义为:Assuming that the kernel function mean fits the mean If the standard deviation is a Gaussian distribution of hi , the probability density function of singular values is defined as:
将式(14)和式(15)中的第i个索引带入式(13)得,Substituting the i-th index in equations (14) and (15) into equation (13), we get,
令式(16)导数为0,解得:Let the derivative of Eq. (16) be 0, the solution is:
再平均所得出的全部MAP估计:Averaging all the resulting MAP estimates:
最后得出低秩图像相似块Lj的估计,Finally, the estimation of the low-rank image similarity block L j is obtained,
增强拉格朗日乘子Uj可通过公式(20)更新,The enhanced Lagrange multiplier U j can be updated by formula (20),
进一步需要说明的是,峰值信噪比的评价方式包括:It should be further noted that the evaluation methods of peak signal-to-noise ratio include:
峰值信噪比计算公式如下所示:The formula for calculating the peak signal-to-noise ratio is as follows:
其中,MSE是待评价图像与参考图像之间的均方误差;G为图像灰度级数;PSNR值越大,待评价图像与参考图像之间的差异越小,图像质量越高。Among them, MSE is the mean square error between the image to be evaluated and the reference image; G is the gray level of the image; the larger the PSNR value, the smaller the difference between the image to be evaluated and the reference image, and the higher the image quality.
进一步需要说明的是,结构相似性的评价方式包括:It should be further noted that the evaluation methods of structural similarity include:
结构相似性计算公式如下所示:The formula for calculating structural similarity is as follows:
其中,μx和μy分别是待评价图像与参考图像的灰度平均值,σx和σy表示标准差,C1=(k1G)2、C2=(k2G)2作为常量来维持数值稳定性,在式中k1=0.01,k2=0.03,G为图像灰度级数。Among them, μ x and μ y are the grayscale average values of the image to be evaluated and the reference image, respectively, σ x and σ y represent the standard deviation, C 1 =(k 1 G) 2 , C 2 =(k 2 G) 2 as Constant to maintain numerical stability, in the formula k 1 =0.01, k 2 =0.03, G is the image gray scale.
进一步需要说明的是,特征相似度的评价方式包括:It should be further noted that the evaluation methods of feature similarity include:
特征相似度计算公式如下所示:The formula for calculating feature similarity is as follows:
其中,SL(x)=SPC(x)·SG(x),SPC(x)和SG(x)分别是图像间的PC和GM的值;PCm(x)是最大的PC值,用于加权每个点对两幅图像整体相似性的贡献;x是给定像素点的位置,Ω为图像的全部空域。Among them, S L (x)=S PC (x) · S G (x), S PC (x) and S G (x) are the values of PC and GM between images, respectively; PC m (x) is the largest The PC value is used to weight the contribution of each point to the overall similarity of the two images; x is the position of a given pixel, and Ω is the full airspace of the image.
本发明还提供一种实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的设备,包括:The present invention also provides a device for realizing an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, including:
存储器,用于存储计算机程序及基于最大后验概率和非局部低秩先验的图像超分辨重建方法;a memory for storing a computer program and an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior;
处理器,用于执行所述计算机程序及基于最大后验概率和非局部低秩先验的图像超分辨重建方法,以实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的步骤。a processor for executing the computer program and the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior, so as to realize the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior A step of.
本发明还提供一种具有基于最大后验概率和非局部低秩先验的图像超分辨重建方法的可读存储介质,可读存储介质上存储有计算机程序,计算机程序被处理器执行以实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的步骤。The present invention also provides a readable storage medium with an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, where a computer program is stored on the readable storage medium, and the computer program is executed by a processor to realize Steps of an image super-resolution reconstruction method with maximum posterior probability and non-local low-rank priors.
从以上技术方案可以看出,本发明具有以下优点:As can be seen from the above technical solutions, the present invention has the following advantages:
本发明基于最大后验概率重建方法,将MAP框架应用于图像超分辨重建技术中,并结合图像的自相似性和非局部低秩先验建立一个充分利用图像蕴含信息的超分辨重建模型;本发明的方法以连续图像序列作为输入,采用相似块分组技术将单幅基准图像及前后的图像预处理为高维张量形式,方便图像块相似性比较,达到快速块匹配的目的,提高了计算速度。在构建的MAP模型中加入基于图像块的非局部低秩先验,充分利用图像细节,避免狭小特征丢失。通过相似块的奇异值估计待求块的奇异值,选取最相似的图像块并进行低秩截断处理,从而抑制噪声等微小因素的干扰,提高重建图像的质量。同时,在每次迭代中交替优化模型中的参数,可提高模型的鲁棒性,避免局部收敛。最后加权平均重建的图像块,得出目标高分辨图像。Based on the maximum a posteriori probability reconstruction method, the present invention applies the MAP framework to the image super-resolution reconstruction technology, and combines the self-similarity of the image and the non-local low-rank prior to establish a super-resolution reconstruction model that fully utilizes the information contained in the image; The method of the invention takes a continuous image sequence as an input, and adopts a similar block grouping technology to preprocess a single reference image and the images before and after it into a high-dimensional tensor form, which facilitates the similarity comparison of image blocks, achieves the purpose of fast block matching, and improves the computational efficiency. speed. A non-local low-rank prior based on image patches is added to the constructed MAP model, which makes full use of image details and avoids the loss of narrow features. The singular value of the block to be determined is estimated by the singular value of the similar block, and the most similar image block is selected and processed by low-rank truncation, so as to suppress the interference of small factors such as noise and improve the quality of the reconstructed image. At the same time, the parameters in the model are optimized alternately in each iteration, which can improve the robustness of the model and avoid local convergence. Finally, the reconstructed image blocks are weighted and averaged to obtain the target high-resolution image.
附图说明Description of drawings
为了更清楚地说明本发明的技术方案,下面将对描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions of the present invention more clearly, the accompanying drawings required in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, which are not relevant to ordinary skills in the art. As far as personnel are concerned, other drawings can also be obtained from these drawings on the premise of no creative work.
图1为图像超分辨重建方法流程图;Fig. 1 is the flow chart of the image super-resolution reconstruction method;
图2为图像梯度分布示意图;Fig. 2 is a schematic diagram of image gradient distribution;
图3为迭代次数对PSNR、SSIM、FSIM的影响示意图;Figure 3 is a schematic diagram of the influence of the number of iterations on PSNR, SSIM, and FSIM;
图4为不同正则项的实验对比图;Figure 4 is an experimental comparison diagram of different regular terms;
图5为四种方法与本发明方法的对比医学图像实例图;Fig. 5 is the comparative medical image example diagram of four kinds of methods and the method of the present invention;
图6为四种方法与本发明方法的对比自然图像实例图;Fig. 6 is the contrast natural image example diagram of four kinds of methods and the method of the present invention;
图7为四种方法与本发明方法的对比自然图像实例图。FIG. 7 is an example diagram of a natural image comparing the four methods and the method of the present invention.
具体实施方式Detailed ways
本领域普通技术人员可以意识到,结合本发明中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the hardware and software In the above description, the components and steps of each example have been generally described according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.
附图中所示的方框图仅仅是功能实体,不一定必须与物理上独立的实体相对应。即,可以采用软件形式来实现这些功能实体,或在一个或多个硬件模块或集成电路中实现这些功能实体,或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。The block diagrams shown in the figures are merely functional entities and do not necessarily necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices entity.
在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另外,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接,也可以是电的,机械的或其它的形式连接。In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
此外,所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中,提供许多具体细节从而给出对本发明的实施例的充分理解。然而,本领域技术人员将意识到,可以实践本发明的技术方案而没有特定细节中的一个或更多,或者可以采用其它的方法、组元、装置、步骤等。在其它情况下,不详细示出或描述公知方法、装置、实现或者操作以避免模糊本发明的各方面。Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the present invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the present invention.
本发明提供一种基于最大后验概率和非局部低秩先验的图像超分辨重建方法,过程如图1所示。采用连续图像序列作为数据输入,利用单幅图像内与连续图像间的相似性作为先验知识,提升相似图像块匹配度,消除图像细节丢失现象。然后,以最大后验概率框架建模,使用高斯分布和吉布斯分布拟合模型参数,提升模型泛化能力。通过相似块的奇异值估计待求块的奇异值,采用低秩截断抑制重建过程中引入的噪声。最后,利用图像的非局部自相似性和低秩性质,以非局部低秩约束正则化图像重建过程,添加图像的局部和全局信息提升重建效果。本发明的方法有效地提升了重建的图像质量,与已有算法相比,在峰值信噪比、结构相似性等性能指标上均取得了更好的效果。The present invention provides an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, and the process is shown in FIG. 1 . The continuous image sequence is used as the data input, and the similarity between the single image and the continuous image is used as the prior knowledge to improve the matching degree of similar image blocks and eliminate the loss of image details. Then, the model is modeled with the maximum posterior probability framework, and the model parameters are fitted with Gaussian distribution and Gibbs distribution to improve the generalization ability of the model. The singular value of the block to be determined is estimated by the singular value of the similar block, and the low-rank truncation is used to suppress the noise introduced in the reconstruction process. Finally, using the non-local self-similarity and low-rank properties of the image, the image reconstruction process is regularized with non-local low-rank constraints, and the local and global information of the image is added to improve the reconstruction effect. The method of the present invention effectively improves the quality of the reconstructed image, and achieves better results in performance indicators such as peak signal-to-noise ratio and structural similarity compared with the existing algorithms.
其中,最大后验概率(Maximum a posterior,MAP)方法是一种基于统计概率的算法框架,是目前实际应用和科学研究中运用最多的一类方法。算法基本思想源于条件概率,将已知LR图像序列作为观测结果,对未知的HR图像进行估计。在MAP框架中,正则项(Regularization term)对于控制重建图像的质量起到关键作用。MAP方法比较灵活,尤其在MAP框架的正则项部分,可以加入对具体问题的具体约束。因此一个有效的正则项是保证MAP框架性能的关键。例如,常用的正则项包括二范数形式的Tikhonov正则项、一范数形式的全变差(Total Variation,TV)正则项以及双边全变差(Bilateral TV,BTV)正则项,还有更复杂的Student-t正则项。MAP有完整的理论框架、灵活的空间域模型以及强大的先验知识包含性,本发明涉及的最大后验概率法具有较好的适应性、灵活性和鲁棒性,能够产生优异的重建结果,是一种有效的超分辨重建方法。Among them, the maximum a posteriori (Maximum a posterior, MAP) method is an algorithm framework based on statistical probability, which is the most widely used method in practical application and scientific research. The basic idea of the algorithm is derived from conditional probability, which takes the known LR image sequence as the observation result and estimates the unknown HR image. In the MAP framework, the regularization term plays a key role in controlling the quality of the reconstructed image. The MAP method is more flexible, especially in the regular term part of the MAP framework, which can add specific constraints to specific problems. Therefore, an effective regular term is the key to ensure the performance of the MAP framework. For example, commonly used regularization terms include Tikhonov regularization terms in two-norm form, Total Variation (TV) regularization terms in one-norm form, and Bilateral TV (BTV) regularization terms in one-norm form, and there are more complex regularization terms. The Student-t regular term. MAP has a complete theoretical framework, a flexible spatial domain model and strong prior knowledge inclusion. The maximum a posteriori probability method involved in the present invention has better adaptability, flexibility and robustness, and can produce excellent reconstruction results. , is an effective super-resolution reconstruction method.
本发明涉及的MAP方法是基于统计概率的算法框架,其基本思想来源于条件概率,通过概率最大得出高分辨率图像H的最优估计进而重建出目标高分辨图像。加入图像相似性和低秩性能有效地提升重建图像质量,针对低分辨率图像序列Y{yk},k=1,...,10的最大后验概率模型,以及利用自然图像的非局部自相似性,采用非局部低秩正则项(NLR)对图像重建过程正则化。The MAP method involved in the present invention is an algorithm framework based on statistical probability. Then the high-resolution image of the target is reconstructed. The addition of image similarity and low-rank performance can effectively improve the quality of reconstructed images, the maximum posterior probability model for low-resolution image sequences Y{y k }, k=1,...,10, and the non-local use of natural images. For self-similarity, a non-local low-rank regularizer (NLR) is used to regularize the image reconstruction process.
本发明中,Y{yk}由一系列低分辨率图像组成,以序列中间的图像y作为基准图像进行超分辨重建,其他图像协同辅助。已知图像退化模型为:In the present invention, Y{y k } is composed of a series of low-resolution images, and the image y in the middle of the sequence is used as a reference image for super-resolution reconstruction, and other images are cooperatively assisted. The known image degradation model is:
y=DBkx+nk (1)y=DB k x+n k (1)
式中,x是待重建的高分辨率图像,y是输入的低分辨率图像,是由原始图像双三次插值放大得到。In the formula, x is the high-resolution image to be reconstructed, and y is the input low-resolution image, which is obtained by bicubic interpolation of the original image.
D为下采样算子,Bk为模糊算子。假设nk是均值为0、方差为的加性高斯白噪声。超分辨的目的是从低分辨率基准图像y重建出高分辨率图像x。该问题可由MAP模型表述为下面的目标函数:D is the down-sampling operator, and B k is the fuzzy operator. Suppose n k is zero mean and variance is additive white Gaussian noise. The purpose of super-resolution is to reconstruct a high-resolution image x from a low-resolution reference image y. The problem can be formulated by the MAP model as the following objective function:
其中,自然图像的梯度分布存在重尾现象,如图2所示的城市自然图像(City)的梯度分布符合重尾分布,通过对医学断层扫描图像(CT)梯度研究,在医学图像中也存在重尾现象。因此上式中的第一项可以用高斯函数来拟合,Among them, the gradient distribution of natural images has a heavy-tailed phenomenon. As shown in Figure 2, the gradient distribution of the urban natural image (City) conforms to the heavy-tailed distribution. Through the study of the gradient of medical tomography (CT) images, it also exists in medical images. Heavy tail phenomenon. Therefore, the first term in the above formula can be fitted by a Gaussian function,
假设像素值仅与满足吉布斯概率密度函数的相邻像素有关,则第二项可以用吉布斯函数拟合,Assuming that pixel values are only related to adjacent pixels that satisfy the Gibbs probability density function, the second term can be fitted with the Gibbs function,
将式(3)和式(4)代入式(2),整理得:Substituting equations (3) and (4) into equation (2), we get:
式(5)即为待求的目标函数。Equation (5) is the objective function to be sought.
本发明涉及的非局部低秩正则化及过程为:基于图像块的非局部低秩先验在图像去噪和图像超分辨等领域得到了较好的应用。本发明在上发明工作的基础上加入了非局部低秩先验,并将其表示为最大后验概率估计。假设表示图像块xj的一系列相似块,其中j为图像块索引,相似块是以j为中心形成的大小为的图像块。在相似块匹配时,选取整个序列图像中最相似的p块相似块,再进行低秩截断处理。假设每组低秩块是相互独立的,因此低秩先验表示为:The non-local low-rank regularization and process involved in the present invention are as follows: the image block-based non-local low-rank prior has been well applied in the fields of image denoising and image super-resolution. The present invention adds a non-local low-rank prior on the basis of the above invention, and expresses it as a maximum a posteriori probability estimate. Assumption Represents a series of similar blocks of the image block x j , where j is the image block index, and the size of the similar block is formed with j as the center image block. When the similar blocks are matched, the most similar p-block similar blocks in the whole sequence image are selected, and then low-rank truncation processing is performed. It is assumed that each group of low-rank blocks is independent of each other, so the low-rank prior is expressed as:
将式(6)代入式(5)得:Substitute equation (6) into equation (5) to get:
通常低秩矩阵常用核范式来解决,Usually low-rank matrices are usually solved in nuclear normal form,
其中,Lj为待求的的低秩块,||Lj||*为Lj的核范数,用以表示奇异值的和。Among them, L j is the desired The low-rank block of , ||L j || * is the nuclear norm of L j , which is used to represent the sum of singular values.
本发明使用迭代方向乘子,通过构造增强拉格朗日方程,求解式(8)。The invention uses the iterative direction multiplier to solve the formula (8) by constructing the enhanced Lagrangian equation.
式中,Uj是拉格朗日乘子,μ是一个常量参数。求解式(9)可以分解成两个子问题:where U j is the Lagrange multiplier and μ is a constant parameter. Solving equation (9) can be decomposed into two sub-problems:
其中,对x求解,可由式(10)直接求得:Among them, the solution of x can be directly obtained by formula (10):
本发明采用MAP方法求解低秩块Lj,用的奇异值估计出Lj的奇异值,从而得出Lj。The present invention adopts the MAP method to solve the low-rank block L j , using The singular values of L j are estimated to obtain L j .
公式(12)可由贝叶斯准则得,Equation (12) can be obtained from the Bayesian criterion,
假设扭曲度f表示高分辨率图像块奇异值与低分辨率图像块奇异值的扭曲度。上式中,第一部分可以看作是用均值为0,标准差是f的高斯函数来拟合,It is assumed that the distortion degree f represents the distortion degree of the singular value of the high-resolution image block and the singular value of the low-resolution image block. In the above formula, the first part can be regarded as fitting with a Gaussian function with a mean of 0 and a standard deviation of f,
另外,P(σi(Lj))可以用核密度估计来计算,其概率密度函数被认定为一系列核函数的和,因此奇异值的和即为一系列核函数的和,核函数由以σi(Lj)为中心的1×3邻域Ωi确定。在本发明中,假设核函数均值符合均值标准差为hi的高斯分布,则奇异值的概率密度函数被定义为:In addition, P(σ i (L j )) can be calculated by kernel density estimation, and its probability density function is identified as the sum of a series of kernel functions, so the sum of singular values is the sum of a series of kernel functions, and the kernel function is given by The 1×3 neighborhood Ω i centered on σ i (L j ) is determined. In the present invention, it is assumed that the mean value of the kernel function conforms to the mean value If the standard deviation is a Gaussian distribution of hi , the probability density function of singular values is defined as:
将式(14)和式(15)中的第i个索引带入式(13)得,Substituting the i-th index in equations (14) and (15) into equation (13), we get,
令式(16)导数为0,解得:Let the derivative of Eq. (16) be 0, the solution is:
再平均所得出的全部MAP估计:Averaging all the resulting MAP estimates:
最后得出低秩图像相似块Lj的估计,Finally, the estimation of the low-rank image similarity block L j is obtained,
这里,增强拉格朗日乘子Uj可通过公式(20)更新,Here, the enhanced Lagrangian multiplier U j can be updated by Equation (20),
为了评价上述基于最大后验概率和非局部低秩先验的图像超分辨重建方法,本发明设置三个评价指标,其中第一个指标是峰值信噪比;In order to evaluate the above-mentioned image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, the present invention sets three evaluation indexes, wherein the first index is peak signal-to-noise ratio;
峰值信噪比(Peak Signal to Noise Ratio,PSNR)是一种评价图像与参考图像类似度的客观标准,衡量经过处理后的图像品质,广泛应用于图像质量评价。其计算公式如下所示:Peak Signal to Noise Ratio (PSNR) is an objective standard for evaluating the similarity between an image and a reference image. It measures the image quality after processing and is widely used in image quality evaluation. Its calculation formula is as follows:
其中,MSE是待评价图像与参考图像之间的均方误差。G为图像灰度级数。PSNR值越大,待评价图像与参考图像之间的差异越小,图像质量越高。但PSNR值只是表示图像质量评价的客观标准,并没有将人体视觉因素考虑在内,即便PSNR值很高,但实际图像质量与期望的图像质量还可能存在较大误差,所以评价图像质量需要综合分析各种评价指标。Among them, MSE is the mean square error between the image to be evaluated and the reference image. G is the gray scale of the image. The larger the PSNR value, the smaller the difference between the image to be evaluated and the reference image, and the higher the image quality. However, the PSNR value is only an objective standard for image quality evaluation, and does not take human visual factors into account. Even if the PSNR value is high, there may be a large error between the actual image quality and the expected image quality, so the evaluation of image quality needs to be comprehensive. Analyze various evaluation indicators.
另一个评价指标是:结构相似性,结构相似性(Structural Similarity,SSIM)是一种基于退化的图像质量评价方法,通过比较待评价图像与参考图像之间的结构相似度来判断图像质量。在一幅图像中的每一个像素点对周围像素都有强依赖性,这些依赖在视觉感知上会携带目标图像结构的重要信息。其计算公式如下所示:Another evaluation index is: structural similarity. Structural Similarity (SSIM) is a degradation-based image quality evaluation method, which judges the image quality by comparing the structural similarity between the image to be evaluated and the reference image. Each pixel in an image has strong dependencies on surrounding pixels, and these dependencies carry important information about the target image structure in visual perception. Its calculation formula is as follows:
其中,μx和μy分别是待评价图像与参考图像的灰度平均值,σx和σy表示标准差,C1=(k1G)2、C2=(k2G)2作为常量来维持数值稳定性,在式中k1=0.01,k2=0.03,G为图像灰度级数。SSIM是由与结构信息相关的亮度和对比度来定义的,平均灰度值作为亮度测量的估计,标准差作为对比度测量的估计。另外,因为SSIM是对称度量,所以可将其视为用于比较任意两个信号的相似性度量。信号可以是离散的或连续的,并且可以存在于任意维度的空间中。Among them, μ x and μ y are the grayscale average values of the image to be evaluated and the reference image, respectively, σ x and σ y represent the standard deviation, C 1 =(k 1 G) 2 , C 2 =(k 2 G) 2 as Constant to maintain numerical stability, in the formula k 1 =0.01, k 2 =0.03, G is the image gray scale. SSIM is defined by the brightness and contrast associated with structural information, with the mean gray value as an estimate of the brightness measure and the standard deviation as an estimate of the contrast measure. In addition, because SSIM is a symmetric metric, it can be regarded as a similarity metric for comparing any two signals. Signals can be discrete or continuous, and can exist in any dimension of space.
另一个评价指标是:特征相似度。Another evaluation indicator is: feature similarity.
特征相似度(Feature Similarity,FSIM)是对SSIM一种比较成功的变种。其中,FSIM中的相位一致性(Phase Congruency,PC)用来度量局部结构重要性,考虑到PC具有对比度不变性,而对比度又影响人眼视觉系统对图像质量的感知,所以在FSIM中采用图像梯度幅度(Gradient Magnitude,GM)作为二级特征。将FSIM划分成PC和GM两部分,其计算公式如下所示:Feature Similarity (FSIM) is a relatively successful variant of SSIM. Among them, Phase Congruency (PC) in FSIM is used to measure the importance of local structure. Considering that PC has contrast invariance, and contrast affects the perception of image quality by the human visual system, the image quality is used in FSIM. Gradient Magnitude (GM) is used as a secondary feature. The FSIM is divided into two parts, PC and GM, and the calculation formula is as follows:
其中,SL(x)=SPC(x)·SG(x),SPC(x)和SG(x)分别是图像间的PC和GM的值。PCm(x)是最大的PC值,用于加权每个点对两幅图像整体相似性的贡献。x是给定像素点的位置,Ω为图像的全部空域。FSIM主要以相位相似度和图像梯度相似度来度量局部结构的重要性,在评价质量分数阶段,将相位相似度作为权值,增大了与人眼视觉感知的相关性,取得了良好的质量评估效果。Among them, S L (x)=S PC (x) · S G (x), S PC (x) and S G (x) are the values of PC and GM between images, respectively. PC m (x) is the maximum PC value used to weight the contribution of each point to the overall similarity of the two images. x is the position of a given pixel, and Ω is the entire airspace of the image. FSIM mainly uses phase similarity and image gradient similarity to measure the importance of local structure. In the evaluation quality score stage, the phase similarity is used as a weight, which increases the correlation with human visual perception and achieves good quality. Evaluate the effect.
本发明还对上述方法进行验证与分析。本发明的实验图像选用医学图像集、自然图像集以及视频分帧图像集等各类型图像5组,每组图像序列10幅,选取其中任意1幅图像作为重建图像。其中,每幅图像大小是128×128,重建的HR图像大小是256×256。实验医学图像数据由相关医院提供。测试的自然图像数据集中,城市数据集来源于纽约大学理工学院等开发的视频数据库,该数据库包含使用H.264编码的10个视频,分辨率从QCIF到4CIF不等,量化参数在28到44之间,帧速率在3.75到30帧每秒,本发明实验只选取其中的10帧作为实验数据。小果园数据集源于光流标准实验数据集,选取其作为自然图像来测试本发明方法的重建性能。实验用MATLAB 2014(b)实现本发明中的图像超分辨重建算法。实验的硬件设施为Intel(R)Xeon(R)E5-2643 v4@3.40GHz CPU,NVIDIA GeForce GTX 1080M GPU,256GB内存,操作系统是ubuntu 14.04。The present invention also verifies and analyzes the above method. The experimental images of the present invention are selected from 5 groups of various types of images such as medical image sets, natural image sets and video framed image sets, each group of 10 image sequences, and any one image is selected as the reconstructed image. Among them, the size of each image is 128×128, and the size of the reconstructed HR image is 256×256. The experimental medical image data were provided by the relevant hospitals. In the natural image dataset tested, the urban dataset comes from the video database developed by New York University Institute of Technology, etc. The database contains 10 videos encoded with H.264, the resolution ranges from QCIF to 4CIF, and the quantization parameters range from 28 to 44. The frame rate is between 3.75 and 30 frames per second, and only 10 frames are selected as experimental data in the experiment of the present invention. The small orchard data set is derived from the optical flow standard experimental data set, which is selected as a natural image to test the reconstruction performance of the method of the present invention. The experiment uses MATLAB 2014(b) to realize the image super-resolution reconstruction algorithm in the present invention. The hardware facilities of the experiment are Intel(R) Xeon(R) E5-2643 v4@3.40GHz CPU, NVIDIA GeForce GTX 1080M GPU, 256GB memory, and the operating system is ubuntu 14.04.
实验中所使用的图像均是标准连续图像序列,迭代次数设为12。通过峰值信噪比、结构相似度和特征相似度评价实验结果。The images used in the experiments are standard continuous image sequences, and the number of iterations is set to 12. The experimental results were evaluated by peak signal-to-noise ratio, structural similarity and feature similarity.
在本发明提出的基于MAP和非局部低秩先验的实验模型中,通过不同类型对比实验,优化实验参数设置。研究表明图像块的大小不仅会影响实验的运行速度,还会影响图像配准的精度和重建图像的质量。实验分别设置块大小为3×3、5×5、7×7、9×9、11×11,迭代次数均设置为12次,实验的PSNR、SSIM、FSIM表1所示。In the experimental model based on MAP and non-local low-rank prior proposed in the present invention, the experimental parameter settings are optimized through different types of comparative experiments. The research shows that the size of the image patch not only affects the running speed of the experiment, but also affects the accuracy of image registration and the quality of the reconstructed image. In the experiment, the block size is set to 3×3, 5×5, 7×7, 9×9, and 11×11 respectively, and the number of iterations is set to 12 times. The PSNR, SSIM, and FSIM of the experiment are shown in Table 1.
表1图像块大小对FSIM、SSIM的影响Table 1 Influence of image block size on FSIM and SSIM
通过3×3、5×5的图像块的数值统计指标明显高于其余组实验结果。图像块设置为7×7、9×9、11×11相比于5×5,PSNR值的数值统计指标差别较小,但算法运行效率会随着图像分块大小的增加而增加。表1的实验结果,11×11的图像块仅在SSIM中性能较佳;其中,3×3与5×5的图像块在FSIM三项比较中相差最大仅为0。0020,但在SSIM比较中,5×5明显高于3×3,且三项差值为0.0330、0.0214、0.0160。综上所述,本实验中图像块大小选取5×5为最佳参数设置,且运行时间也较3×3相差很小,相对于7×7得到较大降低。从客观评价指标及下发明重建图像视觉效果可以看出,对图像进行分块处理能够有效提升重建图像的质量。The numerical statistics of image blocks of 3×3 and 5×5 are significantly higher than the experimental results of other groups. When the image blocks are set to 7×7, 9×9, 11×11, compared with 5×5, the difference in the numerical statistical indicators of PSNR value is smaller, but the operating efficiency of the algorithm will increase with the increase of image block size. According to the experimental results in Table 1, the 11×11 image block has better performance only in SSIM; among them, the difference between the 3×3 and 5×5 image blocks is only 0.0020 in the three comparisons of FSIM, but in the comparison of SSIM Among them, 5×5 was significantly higher than 3×3, and the three-term differences were 0.0330, 0.0214, and 0.0160. To sum up, in this experiment, the image block size of 5×5 is selected as the optimal parameter setting, and the running time is also very small compared with 3×3, which is greatly reduced compared with 7×7. From the objective evaluation index and the visual effect of the reconstructed image of the invention, it can be seen that the block processing of the image can effectively improve the quality of the reconstructed image.
其中,实验中正则项迭代次数设置为12次,迭代次数不同也会对实验结果产生不同影响。如图3所示,采用医学肺部图像(CT)和自然图像(City)作实验数据,对比不同迭代次数对PSNR、SSIM、FSIM产生的影响,其中箭头指向迭代次数设置为12次的数据点。随着迭代次数的增加,PSNR、FSIM、SSIM值趋于平稳,但从PSNR的曲线可以看出,实验结果在12次后,两组实验在各项指标上均会有较小的浮动,相较于FSIM、SSIM的变化曲线相对平缓,PSNR的曲线有较为明显的下降趋势,这是因为基于重建的超分辨率方法中,图像求解是典型的不适定反问题,在迭代求解过程中会引入像噪声等其他微小因素的干扰。因此,本发明实验迭代次数设置为12次避免求解病态问题时带来的干扰,同时在求解相似块过程中利用低秩截断能够有效地抑制噪声等微小因素对实验的扰动。Among them, the number of iterations of the regular term in the experiment is set to 12 times, and the different number of iterations will have different effects on the experimental results. As shown in Figure 3, the medical lung image (CT) and natural image (City) are used as experimental data to compare the effects of different iteration times on PSNR, SSIM, and FSIM, where the arrow points to the data point where the iteration number is set to 12 times. . With the increase of the number of iterations, the values of PSNR, FSIM and SSIM tend to be stable, but it can be seen from the PSNR curve that after 12 experiments, the two groups of experiments will have small fluctuations in various indicators. Compared with FSIM and SSIM, the change curve is relatively flat, and the PSNR curve has a relatively obvious downward trend. This is because in the reconstruction-based super-resolution method, the image solution is a typical ill-posed inverse problem, which will be introduced in the iterative solution process. Interference from other minor factors like noise. Therefore, the number of experimental iterations in the present invention is set to 12 to avoid interference when solving ill-conditioned problems, and at the same time, the use of low-rank truncation in the process of solving similar blocks can effectively suppress the disturbance of small factors such as noise to the experiment.
本发明对最大后验先验概率项的约束性研究。不同正则项在基于重建的方法上作用不同,本发明在关于二范数形式的Tikhonov正则项、一范数形式的全变差正则项(L1TV)、二范数形式的全变差正则项(L2TV)、基于低秩与全变分的正则项(LRTV)与本发明非局部低秩正则项(MAP_NLR)的研究中,分别以迭代次数为6、8、10、12、14情况下进行测试(为了简便表达,以迭代次数等于12为例)。The present invention is a restrictive study of the maximum a posteriori prior probability term. Different regular terms have different functions in the reconstruction-based method. The present invention is concerned with the Tikhonov regular term in the form of two norm, the regular term of total variation in the form of one norm (L1TV), and the regular term of total variation in the form of two norm ( In the research of L2TV), the regularization term based on low rank and total variation (LRTV) and the non-local low-rank regularization term (MAP_NLR) of the present invention, the tests were carried out with the number of iterations being 6, 8, 10, 12, and 14, respectively. (For simplicity of expression, the number of iterations is equal to 12 as an example).
通过研究Tikhonov正则项、一范数TV正则项、二范数TV正则项、LRTV正则项与MAP_NLR正则项的对比实验,使用图像超分辨重建过程中的基准图像进行测试。经过实验,本发明选取了最具代表性的例子:若是Tikhonov正则项,则正则项系数其取值为0.01;若是L1TV正则项,取值为0.005;若是L2TV,则取值0.05;若是LRTV正则项,均取值为0.01;MAP_NLR正则项,以0.1为测试值。选取各自最佳实验结果,进行对比实验,观察与本发明正则项的峰值信噪比,如表2所示:By studying the comparison experiments of Tikhonov regular term, one-norm TV regular term, two-norm TV regular term, LRTV regular term and MAP_NLR regular term, the benchmark images in the process of image super-resolution reconstruction are used for testing. After experiments, the present invention selects the most representative example: if it is a Tikhonov regular term, the regular term coefficient takes a value of 0.01; if it is an L1TV regular term, it takes a value of 0.005; if it is an L2TV, it takes a value of 0.05; if it is an LRTV regular term term, the average value is 0.01; MAP_NLR regular term, with 0.1 as the test value. Select the respective best experimental results, carry out comparative experiments, and observe the peak signal-to-noise ratio with the regular term of the present invention, as shown in Table 2:
表2 PSNR对比(迭代次数等于12)Table 2 PSNR comparison (the number of iterations is equal to 12)
由表2实验结果可以看出MAP_NLR与LRTV在肺部图像上的PSNR结果要优于L1TV、L2TV、Tikhonov的PSNR结果。并且在城市自然图像上MAP_NLR的PSNR值优于其余四项正则项的PSNR值,这说明低秩先验的存在是本发明模型效果较好的前提;其中,在两幅测试图像的PSNR值比较中,MAP_NLR均优于LRTV,可以看出非局部低秩先验的存在是本发明模型的核心要素。From the experimental results in Table 2, it can be seen that the PSNR results of MAP_NLR and LRTV on lung images are better than the PSNR results of L1TV, L2TV, and Tikhonov. And the PSNR value of MAP_NLR on the urban natural image is better than the PSNR value of the other four regular terms, which shows that the existence of low-rank prior is the premise for the better effect of the model of the present invention; among them, the PSNR value of the two test images is compared. Among them, MAP_NLR is better than LRTV, it can be seen that the existence of non-local low-rank prior is the core element of the model of the present invention.
图4是选择肺部医学图像和城市自然图像作测试图像产生的实际结果,虽然总体图像差别不大,但通过局部展示,可以看出区别更多体现在图像细节上。依据共同选取出的图像区域,其中MAP_NLR、LRTV、Tikhonov的细节描述能力明显高于L1TV、L2TV。相对于LRTV与Tikhonov,MAP_NLR纹理清晰,没有产生较重的模糊边缘现象,这从图像视觉方面证明了相似块分组存在的必要性以及采取的低秩截断方法抑制噪声的有效性。另外Tikhonov处理的图像虽然PSNR相对较低,但其细节还原能力反而不差,优于L1TV和L2TV。Figure 4 shows the actual results of selecting lung medical images and urban natural images as test images. Although the overall images are not very different, through local display, it can be seen that the differences are more reflected in the image details. According to the image regions selected jointly, the detail description ability of MAP_NLR, LRTV and Tikhonov is obviously higher than that of L1TV and L2TV. Compared with LRTV and Tikhonov, the texture of MAP_NLR is clear, and there is no heavy blurred edge phenomenon, which proves the necessity of grouping similar blocks and the effectiveness of the low-rank truncation method to suppress noise from the image visual aspect. In addition, although the PSNR of the image processed by Tikhonov is relatively low, its detail restoration ability is not bad, which is better than L1TV and L2TV.
本发明将模型与最近邻插值(Nearest Neighbor Interpolation)、双三次插值(Bicubic)、基于低秩和全变分正则化的方法(LRTV)以及基于二范数二次项解析算法(L2-L2)的图像超分辨处理结果进行对比,其中,在对比实验时,参数统一,模型进行实验所使用的图像,均为连续图像序列中的相同图像,迭代次数设为12。实验的硬件设施均相同且为Intel(R)Xeon(R)E5-2643 v4@3.40GHz CPU,NVIDIA GeForce GTX 1080M GPU,256GB内存,操作系统是ubuntu 14.04。实验结果通过分别计算相应PSNR、SSIM以及FSIM来直观比较。The present invention combines the model with the nearest neighbor interpolation (Nearest Neighbor Interpolation), bicubic interpolation (Bicubic), method based on low rank and total variation regularization (LRTV) and based on two norm quadratic term parsing algorithm (L2-L2) The results of image super-resolution processing are compared with each other, in which, in the comparison experiment, the parameters are unified, the images used by the model for the experiment are the same images in the continuous image sequence, and the number of iterations is set to 12. The hardware facilities of the experiments are all the same and are Intel(R) Xeon(R) E5-2643 v4@3.40GHz CPU, NVIDIA GeForce GTX 1080M GPU, 256GB memory, and the operating system is ubuntu 14.04. The experimental results are visually compared by calculating the corresponding PSNR, SSIM and FSIM respectively.
在本发明完整数据集下,本发明方法与此4种方法对比计算PSNR、SSIM和FSIM的结果如表2所示。Under the complete data set of the present invention, the results of calculating PSNR, SSIM and FSIM between the method of the present invention and the four methods are shown in Table 2.
PSNR是客观评价图像质量的重要指标之一,其值的大小可以在很大程度上判定算法超分辨重建效果的有效性。而SSIM和FSIM可以对PSNR形成补充参照。从表3可以看出,在本发明使用的图像数据集下,作对比实验使用的4种方法中,L2-L2的PSNR所计算的值相对较高,LRTV的SSIM、FSIM所计算的值相对较高,证明图像的低秩性质可以很好地重构图像细节。PSNR is one of the important indicators to objectively evaluate the image quality, and its value can determine the effectiveness of the super-resolution reconstruction effect of the algorithm to a large extent. SSIM and FSIM can form a supplementary reference to PSNR. As can be seen from Table 3, under the image data set used in the present invention, among the four methods used in the comparative experiments, the calculated value of PSNR of L2-L2 is relatively high, and the calculated value of SSIM and FSIM of LRTV is relatively high. It is higher, which proves that the low-rank nature of the image can reconstruct the image details well.
除对比全部数据集的实验结果外,本发明还分别单独列出肺部医学图像、城市自然图像、小果园自然图像的PSNR、SSIM、FSIM。在图像大小均为128×128的肝部图像、肺部图像医学数据集,城市图像、果园图像和花朵图像自然数据集的PSNR、SSIM、FSIM的对比结果如表3所示。其中,不同实验方法对应的图像数据集的实验结果指标值由上到下依次为PSNR、SSIM、FSIM。In addition to comparing the experimental results of all data sets, the present invention also separately lists PSNR, SSIM, and FSIM of lung medical images, urban natural images, and small orchard natural images. Table 3 shows the comparison results of PSNR, SSIM and FSIM in the liver image, lung image medical dataset, city image, orchard image and flower image natural dataset with image size of 128×128. Among them, the experimental results index values of the image datasets corresponding to different experimental methods are PSNR, SSIM, FSIM from top to bottom.
如图表3所示,使用不同算法进行图像超分辨获得的图像如图5、图6和图7所示,可以看出,本发明方法在全部图像的PSNR平均值差了上一名0.17dB,对比肝部影像数据,相较于LRTV三项指标差值分别为1.91dB、0.0270、0.0227,但在肺部图像、城市图像、小果园图像的PSNR、SSIM、FSIM以及全部图像的SSIM、FSIM平均值在内,本发明方法有着一定优势。如图5、图6和图7所示,在对图像进行局部对比时,本发明方法无论是对于肺部图像的图像重建还是自然图像的超分辨重建都具有较大的优势,能够更好地保持图像细节,不会产生模糊伪影和锯齿效应。通过对比重建图像视觉效果和图像质量的指标,可以看出本发明方法均优于其他四种方法。As shown in Fig. 3, the images obtained by using different algorithms for image super-resolution are shown in Fig. 5, Fig. 6 and Fig. 7. It can be seen that the average PSNR of the method of the present invention is 0.17dB worse than the previous one. Compared with the liver image data, the differences of the three indicators compared with LRTV were 1.91dB, 0.0270, and 0.0227, respectively, but the average PSNR, SSIM, FSIM of the lung image, city image, small orchard image, and SSIM and FSIM of all images were averaged. Including the value, the method of the present invention has certain advantages. As shown in Fig. 5, Fig. 6 and Fig. 7, when comparing images locally, the method of the present invention has great advantages in both image reconstruction of lung images and super-resolution reconstruction of natural images, and can better Image details are preserved without blurring artifacts and aliasing effects. By comparing the indicators of the visual effect and image quality of the reconstructed image, it can be seen that the method of the present invention is superior to the other four methods.
表3不同算法图像超分辨的PSNR、SSIM和FSIM比较Table 3 Comparison of PSNR, SSIM and FSIM for image super-resolution with different algorithms
图5是从实验数据集选取的一幅肺部图像,在包括本发明在内的5种重建模型中得到的结果图,原始图像位于最左侧,Ground Truth位于图像最右侧,其余是其中超分辨方法的排列,验证方法中最后一种方法(位于Ground Truth图像左侧)是本发明方法。其中,将最顶端的重建图像进行局部展示,在视觉效果上可以看出本发明方法与Ground Truth最为接近。依据上述图像,可以看出LRTV与本发明方法最为稳定,图像波动不大,但最近邻法与双三次算法处于一种不稳定态,尤其最近邻法获得的结果锯齿效应明显。L2L2方法结果图像的局部展示表现了明显的模糊效应,而在本发明方法和LRTV中几乎没有,说明L2L2无法有效地保留图像细节。LRTV与本发明方法最为接近,但容易模糊图像纹理,造成图像边缘模糊。从最底端的误差结果图也可以看出,本发明方法与原图差异较小,重建图像质量高的同时也极其稳定。Fig. 5 is a lung image selected from the experimental data set, and the results obtained in the five reconstruction models including the present invention, the original image is located at the far left, the Ground Truth is located at the far right of the image, and the rest are among The arrangement of super-resolution methods, the last method in the verification method (located on the left side of the Ground Truth image) is the method of the present invention. Among them, the topmost reconstructed image is displayed locally, and it can be seen from the visual effect that the method of the present invention is the closest to Ground Truth. According to the above images, it can be seen that LRTV and the method of the present invention are the most stable, and the image fluctuation is not large, but the nearest neighbor method and the bicubic algorithm are in an unstable state, especially the results obtained by the nearest neighbor method have obvious sawtooth effect. The local display of the result image of the L2L2 method shows obvious blurring effect, which is almost absent in the method of the present invention and LRTV, indicating that the L2L2 cannot effectively preserve the image details. LRTV is the closest to the method of the present invention, but it is easy to blur the image texture, resulting in blurred image edges. It can also be seen from the error result graph at the bottom that the difference between the method of the present invention and the original image is small, and the reconstructed image is of high quality and extremely stable.
自然图像包含大量的细节信息,有效地重建出图像高频信息对图像质量尤为重要。从图6和图7可以看出,最近邻法和双三次法对图像的高频信息无法有效地还原,进而影响重建图像质量,造成边缘纹理模糊和块状伪影。观察自然图像的局部展示,可以看出LRTV会出现对边缘纹理粗化的状况,说明在实现细节保留上,低秩正则化与全变分正则化有叠加效应,造成图像细节无法有效地还原。L2L2方法在图像高频细节部分产生颗粒状伪影,不能有效地保存平滑区域的图像信息,从而造成在纹理边缘处产生模糊效应,不能抑制噪声干扰,重建图像清晰度较差。相比以上四种方法,本发明方法可以更好地保留图像细节,在图像平滑区域的重建过程中能够有效抑制噪声等微小因素的干扰,进而提升重建结果图像质量。Natural images contain a lot of detailed information, and it is very important to reconstruct the high-frequency information of the image effectively for the image quality. It can be seen from Figure 6 and Figure 7 that the nearest neighbor method and the bicubic method cannot effectively restore the high-frequency information of the image, which in turn affects the quality of the reconstructed image, resulting in blurred edge texture and block artifacts. Observing the local display of natural images, it can be seen that LRTV will coarsen the edge texture, indicating that low-rank regularization and total variational regularization have a superposition effect in the realization of detail preservation, resulting in the inability to effectively restore image details. The L2L2 method produces granular artifacts in the high-frequency details of the image, which cannot effectively preserve the image information in smooth areas, resulting in blurring effects at the edges of the texture, unable to suppress noise interference, and poor definition of the reconstructed image. Compared with the above four methods, the method of the present invention can better preserve the image details, and can effectively suppress the interference of small factors such as noise during the reconstruction process of the smooth area of the image, thereby improving the image quality of the reconstruction result.
基于本发明的方案,对最大后验框架进行了改进,使用非局部低秩先验作为正则化项,进行图像超分辨重建。采用连续图像序列作为数据输入,利用图像的非局部自相似性作为先验知识,结合图像块局部分组技术将相似块进行块匹配,充分挖掘图像像素级的空间结构关系。其次,在本发明的MAP框架中,通过相似块奇异值的最大后验估计出待求块的奇异值,采用低秩截断的方法抑制噪声等微小因素的干扰。最后,采用非局部低秩约束正则化图像重建过程,充分利用单幅图像内、连续图像间的局部和全局信息,提升目标高分辨率图像质量。本发明模型采用多个评价指标:峰值信噪比、结构相似性等,并表现出优势。实验结果表明,本发明超分辨模型在图像超分辨重建上具有良好的鲁棒性。但是,在低秩截断过程中,一些狭小细节被丢失了。研究发现,残差在保持细节方面具有很好的效果,如何将残差加入模型,进一步保持图像细节是下一步工作重点。Based on the scheme of the present invention, the maximum a posteriori framework is improved, and a non-local low-rank prior is used as a regularization term to perform image super-resolution reconstruction. The continuous image sequence is used as the data input, the non-local self-similarity of the image is used as the prior knowledge, and the similar blocks are matched by the local grouping technology of image blocks, and the spatial structure relationship at the pixel level of the image is fully exploited. Secondly, in the MAP framework of the present invention, the singular value of the block to be sought is estimated by the maximum a posteriori of the singular value of the similar blocks, and the low-rank truncation method is used to suppress the interference of small factors such as noise. Finally, the non-local low-rank constraint regularization image reconstruction process is adopted to make full use of the local and global information within a single image and between consecutive images to improve the quality of the target high-resolution image. The model of the present invention adopts multiple evaluation indicators: peak signal-to-noise ratio, structural similarity, etc., and shows advantages. The experimental results show that the super-resolution model of the present invention has good robustness in image super-resolution reconstruction. However, in the low-rank truncation process, some small details are lost. The research found that the residual has a good effect in preserving the details. How to add the residual to the model to further preserve the image details is the focus of the next step.
基于上述方法本发明还提供一种实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的设备,包括:Based on the above method, the present invention also provides a device for realizing an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, including:
存储器,用于存储计算机程序及基于最大后验概率和非局部低秩先验的图像超分辨重建方法;a memory for storing a computer program and an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior;
处理器,用于执行所述计算机程序及基于最大后验概率和非局部低秩先验的图像超分辨重建方法,以实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的步骤。a processor for executing the computer program and the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior, so as to realize the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior A step of.
基于上述方法本发明还提供一种具有基于最大后验概率和非局部低秩先验的图像超分辨重建方法的可读存储介质,可读存储介质上存储有计算机程序,计算机程序被处理器执行以实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的步骤。Based on the above method, the present invention also provides a readable storage medium with an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low-rank prior, where a computer program is stored on the readable storage medium, and the computer program is executed by a processor To implement the steps of image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior.
实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的设备是结合本发明中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。The device for realizing the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior is the unit and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention, and can use electronic hardware, computer software or two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.
通过以上的实施方式的描述,本领域的技术人员易于理解,这里描述的实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的设备可以通过软件实现,也可以通过软件结合必要的硬件的方式来实现。因此,根据实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法公开实施方式的技术方案可以以软件产品的形式体现出来,该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM,U盘,移动硬盘等)中或网络上,包括若干指令以使得一台计算设备(可以是个人计算机、服务器、移动终端、或者网络设备等)执行根据本公开实施方式的索引方法。From the description of the above embodiments, those skilled in the art can easily understand that the device for realizing the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior described here can be realized by software, or can be combined by software necessary hardware way to achieve. Therefore, the technical solutions according to the disclosed embodiments for realizing the image super-resolution reconstruction method based on the maximum a posteriori probability and the non-local low-rank prior can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network, including several instructions to cause a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the implementation according to the present disclosure index method.
所属技术领域的技术人员能够理解,实现基于最大后验概率和非局部低秩先验的图像超分辨重建方法的各个方面可以实现为系统、方法或程序产品。因此,本公开的各个方面可以具体实现为以下形式,即:完全的硬件实施方式、完全的软件实施方式(包括固件、微代码等),或硬件和软件方面结合的实施方式,这里可以统称为“电路”、“模块”或“系统”。Those skilled in the art can understand that various aspects of implementing an image super-resolution reconstruction method based on a maximum a posteriori probability and a non-local low-rank prior can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein as implementations "circuit", "module" or "system".
可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码,所述程序设计语言包括面向对象的程序设计语言—诸如Java、C++等,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中,远程计算设备可以通过任意种类的网络,包括局域网(LAN)或广域网(WAN),连接到用户计算设备,或者,可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural Programming Language - such as the "C" language or similar programming language. The program code may execute entirely on the user computing device, partly on the user device, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using an Internet service provider business via an Internet connection).
对所公开的实施例的上述说明,使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的,本发明中所定义的一般原理可以在不脱离本发明的精神或范围的情况下,在其它实施例中实现。因此,本发明将不会被限制于本发明所示的这些实施例,而是要符合与本发明所公开的原理和新颖特点相一致的最宽的范围。The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of this invention. Thus, the present invention is not intended to be limited to the embodiments of the present invention shown, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010568221.4A CN111724307B (en) | 2020-06-19 | 2020-06-19 | An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010568221.4A CN111724307B (en) | 2020-06-19 | 2020-06-19 | An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111724307A true CN111724307A (en) | 2020-09-29 |
CN111724307B CN111724307B (en) | 2021-08-20 |
Family
ID=72568203
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010568221.4A Active CN111724307B (en) | 2020-06-19 | 2020-06-19 | An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111724307B (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113204869A (en) * | 2021-04-28 | 2021-08-03 | 桂林电子科技大学 | Phase unwrapping method based on rank information filtering |
CN113627484A (en) * | 2021-07-09 | 2021-11-09 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
CN114170080A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Image super-resolution reconstruction system based on non-convex low-rank tensor approximation |
CN114170081A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | 3D medical image super-resolution method based on non-local low-rank tensor decomposition |
CN114545330A (en) * | 2022-02-16 | 2022-05-27 | 红相股份有限公司 | A method for locating equipment fault sound source based on sparse imaging technology |
CN114820387A (en) * | 2022-05-27 | 2022-07-29 | 山东财经大学 | Image recovery method and terminal based on probability induced kernel norm minimization |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
-
2020
- 2020-06-19 CN CN202010568221.4A patent/CN111724307B/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
Non-Patent Citations (4)
Title |
---|
HE YU,YAP KIMHUI,CHEN LI,CHAU LAPPUI: "Blind Super-Resolution Image Reconstruction using a", 《2006 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING》 * |
LEI ZHU,CHI-WING FU,MICHAEL S BROWN,PHENG-ANN HENG: "A Non-local Low-Rank Framework for", 《IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION,》 * |
耿凤欢: "基于光流估计的肺部4D_CT图像超分辨重建", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
许静: "基于MAP技术图像超分辨率重建的研究", 《中国优秀硕士学位论文全文数据库信息科技辑 2009年第02期》 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113204869A (en) * | 2021-04-28 | 2021-08-03 | 桂林电子科技大学 | Phase unwrapping method based on rank information filtering |
CN113204869B (en) * | 2021-04-28 | 2023-06-23 | 桂林电子科技大学 | A Phase Unwrapping Method Based on Rank Information Filtering |
CN113627484A (en) * | 2021-07-09 | 2021-11-09 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
CN113627484B (en) * | 2021-07-09 | 2022-11-18 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
CN114170080A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Image super-resolution reconstruction system based on non-convex low-rank tensor approximation |
CN114170081A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | 3D medical image super-resolution method based on non-local low-rank tensor decomposition |
CN114170081B (en) * | 2021-11-26 | 2024-08-02 | 中国科学院沈阳自动化研究所 | Three-dimensional medical image super-resolution method based on non-local low-rank tensor decomposition |
CN114545330A (en) * | 2022-02-16 | 2022-05-27 | 红相股份有限公司 | A method for locating equipment fault sound source based on sparse imaging technology |
CN114545330B (en) * | 2022-02-16 | 2025-04-15 | 红相股份有限公司 | A method for locating equipment fault sound sources based on sparse imaging technology |
CN114820387A (en) * | 2022-05-27 | 2022-07-29 | 山东财经大学 | Image recovery method and terminal based on probability induced kernel norm minimization |
Also Published As
Publication number | Publication date |
---|---|
CN111724307B (en) | 2021-08-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111724307B (en) | An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium | |
Zhang et al. | Learning multiple linear mappings for efficient single image super-resolution | |
Guo et al. | An efficient SVD-based method for image denoising | |
Jiang et al. | Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation | |
Trinh et al. | Novel example-based method for super-resolution and denoising of medical images | |
CN106952228B (en) | A single image super-resolution reconstruction method based on image non-local self-similarity | |
CN101976435B (en) | Combination learning super-resolution method based on dual constraint | |
Chen et al. | Single image super-resolution via adaptive transform-based nonlocal self-similarity modeling and learning-based gradient regularization | |
Yang et al. | Deep learning in medical image super resolution: a review | |
CN103136734B (en) | The suppressing method of edge Halo effect during a kind of convex set projection super-resolution image reconstruction | |
CN107067367A (en) | A kind of Image Super-resolution Reconstruction processing method | |
Yancheng et al. | RED-MAM: A residual encoder-decoder network based on multi-attention fusion for ultrasound image denoising | |
Wang et al. | Video super-resolution via residual learning | |
Liu et al. | Video frame interpolation via optical flow estimation with image inpainting | |
CN115496663A (en) | Video super-resolution reconstruction method based on fusion network within D3D convolutional group | |
CN106600533B (en) | Single image super resolution ratio reconstruction method | |
Akyol et al. | Super-resolution reconstruction of faces by enhanced global models of shape and texture | |
Xie et al. | Super-resolution of Pneumocystis carinii pneumonia CT via self-attention GAN | |
Mikaeli et al. | Single-image super-resolution via patch-based and group-based local smoothness modeling | |
Liu et al. | Image super-resolution via hierarchical and collaborative sparse representation | |
Ma et al. | Despeckling sar images with log-yeo-johnson transformation and conditional diffusion models | |
Wen et al. | The power of complementary regularizers: Image recovery via transform learning and low-rank modeling | |
Hui et al. | Eigentransformation-based face super-resolution in the wavelet domain | |
Baraha et al. | Speckle removal using dictionary learning and PnP-based fast iterative shrinkage threshold algorithm | |
Cheng et al. | Classification-based video super-resolution using artificial neural networks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |