CN107169925A - The method for reconstructing of stepless zooming super-resolution image - Google Patents
The method for reconstructing of stepless zooming super-resolution image Download PDFInfo
- Publication number
- CN107169925A CN107169925A CN201710267753.2A CN201710267753A CN107169925A CN 107169925 A CN107169925 A CN 107169925A CN 201710267753 A CN201710267753 A CN 201710267753A CN 107169925 A CN107169925 A CN 107169925A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- low
- super
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000009467 reduction Effects 0.000 claims description 12
- 238000001914 filtration Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000015556 catabolic process Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 4
- 238000013144 data compression Methods 0.000 claims description 4
- 238000006731 degradation reaction Methods 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000006837 decompression Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 6
- 238000013507 mapping Methods 0.000 abstract description 5
- 230000000052 comparative effect Effects 0.000 abstract description 3
- 238000002474 experimental method Methods 0.000 abstract description 3
- 230000008859 change Effects 0.000 abstract description 2
- 230000009466 transformation Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012634 optical imaging Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009394 selective breeding Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Editing Of Facsimile Originals (AREA)
- Image Processing (AREA)
Abstract
本发明涉及一种无级缩放超分辨率图像的重建方法,包括:字典训练过程、图像重建过程、尺度变化过程三个处理步骤。本发明提出建立图像库训练字典的方法,将图像集离线训练字典库,得到高分辨率图像与低分辨率图像的映射关系,优化图像重建步骤,有效减少方法运行时间,具有自适应性和高效性。本发明中方法中的阈值是基于大量对比试验后进行优化固定,在处过程中不需要进行改变,在不影响图像重建效果的情况下,有效的提高了图像重建效率。本发明获得的超分辨率重建图像可以进行任意大小的尺度变换。在工程应用中可以满足不同的情况下的需求,有较好的适用性。
The invention relates to a reconstruction method of a steplessly zoomed super-resolution image, comprising three processing steps: a dictionary training process, an image reconstruction process, and a scale change process. The invention proposes a method for establishing an image database training dictionary, trains the dictionary database off-line with the image set, obtains the mapping relationship between high-resolution images and low-resolution images, optimizes the image reconstruction steps, effectively reduces the running time of the method, and is self-adaptive and efficient sex. The threshold value in the method of the present invention is optimized and fixed based on a large number of comparative experiments, and does not need to be changed during the process, effectively improving the image reconstruction efficiency without affecting the image reconstruction effect. The super-resolution reconstructed image obtained by the present invention can be subjected to scale transformation of any size. In engineering applications, it can meet the needs of different situations and has good applicability.
Description
技术领域technical field
本发明属于图像重建技术领域,具体涉及一种无级缩放超分辨率图像的重建方法。The invention belongs to the technical field of image reconstruction, and in particular relates to a reconstruction method of a steplessly zoomed super-resolution image.
背景技术Background technique
当前,图像分辨率会受到成像系统、拍摄环境等因素的影响,得到的图像分辨率往往无法满足人们的需求。然而,目前光学成像系统及其探测器又受限于技术水平及加工复杂度,无法从硬件角度改善系统成像质量。因此,图像超分辨率重建技术的出现,打破了图像分辨率的限制。通过软件处理的方式从一幅或者多幅退化的低分辨率图像中,计算出一幅高分辨率图像,获得更加丰富的细节信息。目前超分辨率重建技术广泛应用于图像处理及光学成像领域,得到了人们的重视。At present, the image resolution will be affected by factors such as the imaging system and the shooting environment, and the obtained image resolution often cannot meet people's needs. However, the current optical imaging system and its detector are limited by the technical level and processing complexity, and it is impossible to improve the imaging quality of the system from the perspective of hardware. Therefore, the emergence of image super-resolution reconstruction technology breaks the limitation of image resolution. Through software processing, a high-resolution image is calculated from one or more degraded low-resolution images to obtain richer detail information. At present, super-resolution reconstruction technology is widely used in the fields of image processing and optical imaging, and has attracted people's attention.
目前,基于学习的稀疏表示图像超分辨率重建算法可以有效提高图像的分辨率,并且对图像进行放大倍率为3的尺度放大。然而,在某些特定情况下,经过该方法得到的高分辨率图像达不到所需的尺度要求。At present, the learning-based sparse representation image super-resolution reconstruction algorithm can effectively improve the resolution of the image, and the scale of the image is enlarged with a magnification of 3. However, in some specific cases, the high-resolution images obtained by this method cannot meet the required scale requirements.
因此,如何提供一种图像重建方法使得获得的超分辨率重建图像可以进行任意尺度的变换,以达到相关工业要求变得至关重要。Therefore, how to provide an image reconstruction method so that the obtained super-resolution reconstructed image can be transformed at any scale to meet the relevant industrial requirements becomes very important.
发明内容Contents of the invention
为了解决现有技术中存在的上述问题,本发明提供了一种无级缩放超分辨率图像的重建方法,包括如下步骤:In order to solve the above-mentioned problems existing in the prior art, the present invention provides a kind of reconstruction method of stepless scaling super-resolution image, comprising the following steps:
步骤1,对高分辨率样本图像按照退化模型进行模糊处理和N倍下采样,获得低分辨率样本图像;Step 1, blurring and downsampling the high-resolution sample image according to the degradation model to obtain a low-resolution sample image;
步骤2,采用K-SVD方法,利用所述低分辨率样本图像进行字典训练,得到高分辨字典和低分辨率字典;Step 2, using the K-SVD method to perform dictionary training using the low-resolution sample image to obtain a high-resolution dictionary and a low-resolution dictionary;
步骤3,对待重建的低分辨率图像进行预处理得到若干个低分辨率图像块;Step 3, preprocessing the low-resolution image to be reconstructed to obtain several low-resolution image blocks;
步骤4,利用所述低分辨率样本图像计算固定阈值;Step 4, using the low-resolution sample image to calculate a fixed threshold;
步骤5,判断每一个所述低分辨率图像块的像素值是否小于所述固定阈值;若是,则判定该所述低分辨率图像块为低信息量图像块;若否,则判定该所述低分辨率图像块为高信息量图像块;Step 5, judging whether the pixel value of each of the low-resolution image blocks is smaller than the fixed threshold; if so, then judging that the low-resolution image block is a low-information image block; Low-resolution image blocks are high-information image blocks;
步骤6,将所述高信息量图像块采用基于字典学习与稀疏表示算法重建得到高信息量重建子区域;Step 6: Reconstruct the high-information image block using a dictionary learning and sparse representation algorithm to obtain a high-information reconstruction sub-region;
步骤7,将所述低信息量图像块采用插值算法进行重建得到低信息量重建子区域;Step 7, reconstructing the low-information image block using an interpolation algorithm to obtain a low-information reconstruction sub-region;
步骤8,将所述高信息量子区域与所述低信息量重建子区域进行图像拼接得到超分辨率重建图像;Step 8, performing image stitching on the high-information quantum region and the low-information reconstruction sub-region to obtain a super-resolution reconstruction image;
步骤9,采用插值算法将所述超分辨率重建图像进行M倍插值得到N*M(N=2,3,4;M>0)倍的超分辨率重建图像。Step 9: Perform M-fold interpolation on the super-resolution reconstructed image using an interpolation algorithm to obtain an N*M (N=2, 3, 4; M>0)-fold super-resolution reconstructed image.
在本发明的一个实施例中,所述步骤2包括如下步骤:In one embodiment of the present invention, said step 2 includes the following steps:
(21)通过特征提取方法提取所述低分辨率样本图像的图像特征,得到空间目标的高分辨率特征信息和低分辨率特征信息。(21) Extracting image features of the low-resolution sample image by a feature extraction method to obtain high-resolution feature information and low-resolution feature information of the space object.
(22)利用所述K-SVD方法,对所述高分辨率特征信息和所述低分辨率特征信息进行联合训练,得到所述高分辨率字典和所述低分辨率字典。(22) Using the K-SVD method, perform joint training on the high-resolution feature information and the low-resolution feature information to obtain the high-resolution dictionary and the low-resolution dictionary.
在本发明的一个实施例中,所述步骤(22)包括如下步骤:In one embodiment of the present invention, described step (22) comprises the following steps:
(221)利用稀疏K-SVD方法训练低分辨率字典;(221) Utilize sparse K-SVD method to train low-resolution dictionary;
(222)计算高分辨率字典。(222) Compute a high-resolution dictionary.
在本发明的一个实施例中,所述步骤3包括如下步骤:In one embodiment of the present invention, said step 3 includes the following steps:
(31)对所述待重建的低分辨率图像去噪声得到第一图像;(31) Denoising the low-resolution image to be reconstructed to obtain a first image;
(32)将所述第一图像去模糊得到第二图像;(32) Deblurring the first image to obtain a second image;
(33)将所述第二图像按照固定长宽进行分割、保存处理,形成所述低分辨率图像块。(33) Segment and save the second image according to a fixed length and width to form the low-resolution image block.
在本发明的一个实施例中,所述步骤4包括如下步骤:In one embodiment of the present invention, said step 4 includes the following steps:
(41)将所述低分辨率样本图像进行区块分割得到低分辨率样本图像块;(41) performing block segmentation on the low-resolution sample image to obtain a low-resolution sample image block;
(42)利用边缘提取算法提取所述低分辨率样本图像块的边缘信息,统计各所述低分辨率样本图像块的像素值和所有所述低分辨率样本图像块的像素值分布情况;(42) Using an edge extraction algorithm to extract edge information of the low-resolution sample image blocks, counting the pixel values of each of the low-resolution sample image blocks and the distribution of pixel values of all the low-resolution sample image blocks;
(43)根据所述各低分辨率样本图像块的像素值和所有所述低分辨率样本图像块的像素值分布情况选取X(X>1)个候选阈值;(43) Selecting X (X>1) candidate thresholds according to the pixel values of each of the low-resolution sample image blocks and the distribution of pixel values of all the low-resolution sample image blocks;
(44)对每个所述候选阈值利用基于学习的稀疏表示图像超分辨率重建算法进行计算形成多个高分辨率候选图像;(44) Utilizing a learning-based sparse representation image super-resolution reconstruction algorithm for each of the candidate thresholds to form a plurality of high-resolution candidate images;
(45)根据所述基于学习的稀疏表示图像超分辨率重建算法的运算时间及对应形成的所述高分辨率候选图像的分辨率,从X个候选阈值中选择所述信息量阈值。(45) Select the information amount threshold from X candidate thresholds according to the calculation time of the learning-based sparse representation image super-resolution reconstruction algorithm and the resolution of the correspondingly formed high-resolution candidate image.
在本发明的一个实施例中,所述边缘提取算法为Canny算子边缘检测算法。In one embodiment of the present invention, the edge extraction algorithm is a Canny operator edge detection algorithm.
在本发明的一个实施例中,所述步骤6包括如下步骤:In one embodiment of the present invention, described step 6 comprises the following steps:
(61)针对所述高信息量图像块进行图像滤波处理,并进行高频特征提取获得高频信息;(61) performing image filtering processing on the high-information image block, and performing high-frequency feature extraction to obtain high-frequency information;
(62)利用所述高频信息对所述高信息量图像块进行KPCA降维,实现高维数据压缩;(62) Using the high-frequency information to perform KPCA dimensionality reduction on the high-information image block to realize high-dimensional data compression;
(63)采用OMP算法对降维压缩后的数据进行重建得到所述高信息量重建子区域。(63) Using the OMP algorithm to reconstruct the data after dimensionality reduction and compression to obtain the reconstruction sub-region with high information content.
在本发明的一个实施例中,所述图像滤波操作采用二维滤波算子滤波器组,所述滤波器组为f={f1,f2,f3,f4},其中,In one embodiment of the present invention, the image filtering operation uses a filter bank of two-dimensional filtering operators, and the filter bank is f={f 1 , f 2 , f 3 , f 4 }, wherein,
f1=[1,-1], f2=f1 T f 1 =[1,-1], f 2 =f 1 T
f3=LOG, f3=f3 T f 3 =LOG, f 3 =f 3 T
T为矩阵转置操作,LOG为5×5的二维滤波算子。T is a matrix transpose operation, and LOG is a 5×5 two-dimensional filter operator.
在本发明的一个实施例中,所述步骤(63)包括如下步骤:In one embodiment of the present invention, described step (63) comprises the following steps:
(631)根据所述OMP算法计算稀疏表示系数β,其中, (631) Calculate the sparse representation coefficient β according to the OMP algorithm, wherein,
其中,y为降维解压后的数据,T0为给定的稀疏度,βi为矩阵β中的子元素;Among them, y is the data after dimension reduction and decompression, T 0 is a given sparsity, and β i is a sub-element in matrix β;
(632)将所述稀疏表示系数β与所述高分辨率字典相乘,得到所述高信息量重建子区域。(632) Multiply the sparse representation coefficient β by the high-resolution dictionary to obtain the high-information reconstruction sub-region.
在本发明的一个实施例中,所述插值算法为双三次插值算法。In one embodiment of the present invention, the interpolation algorithm is bicubic interpolation algorithm.
本发明实施例具有如下有益效果,The embodiments of the present invention have the following beneficial effects,
1、本发明实施例提出建立图像库训练字典的方法,将图像集离线训练字典库,得到高分辨率图像与低分辨率图像的映射关系,优化图像重建步骤,有效减少方法运行时间,具有自适应性和高效性。1. The embodiment of the present invention proposes a method for establishing a training dictionary in an image library, trains the dictionary library offline with an image set, obtains the mapping relationship between a high-resolution image and a low-resolution image, optimizes the image reconstruction steps, effectively reduces the running time of the method, and has automatic adaptability and efficiency.
2、本发明获得的超分辨率重建图像可以进行任意大小的尺度变换。在工程应用中可以满足不同的情况下的需求,有较好的适用性。2. The super-resolution reconstructed image obtained by the present invention can be scaled to any size. In engineering applications, it can meet the needs of different situations and has good applicability.
3、本发明中方法中的阈值是基于大量对比试验后进行优化固定,在处过程中不需要进行改变,在不影响图像重建效果的情况下,有效的提高了图像重建效率。3. The threshold value in the method of the present invention is optimized and fixed based on a large number of comparative experiments, and does not need to be changed during the process, and the image reconstruction efficiency is effectively improved without affecting the image reconstruction effect.
4、依据超分辨率重建后的图像再进行缩放,由于算法重构了高频信息,对于缩放效果会更好。4. Scaling is performed based on the super-resolution reconstructed image. Since the algorithm reconstructs high-frequency information, the zoom effect will be better.
附图说明Description of drawings
图1为本发明实施例提供的一种无级缩放超分辨率图像的重建方法流程示意图;FIG. 1 is a schematic flowchart of a method for reconstructing a steplessly zoomed super-resolution image provided by an embodiment of the present invention;
图2为本发明实施例提供的一种字典训练原理示意图;Fig. 2 is a schematic diagram of a dictionary training principle provided by an embodiment of the present invention;
图3为本发明实施例提供的一种无级缩放超分辨率图像的重建方法原理示意图。FIG. 3 is a schematic diagram of a method for reconstructing a steplessly zoomed super-resolution image provided by an embodiment of the present invention.
具体实施方式detailed description
下面结合具体实施例对本发明做进一步详细的描述,但本发明的实施方式不限于此。The present invention will be described in further detail below in conjunction with specific examples, but the embodiments of the present invention are not limited thereto.
实施例一Embodiment one
请参见图1、图2和图3,图1为本发明实施例提供的一种无级缩放超分辨率图像的重建方法流程示意图;图2为本发明实施例提供的一种字典训练原理示意图;图3为本发明实施例提供的一种无级缩放超分辨率图像的重建方法原理示意图。该无级缩放超分辨率图像的重建方法,包括如下步骤:Please refer to Fig. 1, Fig. 2 and Fig. 3, Fig. 1 is a schematic flow chart of a method for reconstructing a stepless zoom super-resolution image provided by an embodiment of the present invention; Fig. 2 is a schematic diagram of a dictionary training principle provided by an embodiment of the present invention ; FIG. 3 is a schematic diagram of the principle of a method for reconstructing a steplessly zoomed super-resolution image provided by an embodiment of the present invention. The reconstruction method of the steplessly zoomed super-resolution image comprises the following steps:
步骤1,对高分辨率样本图像按照退化模型进行模糊处理和N倍下采样,获得低分辨率样本图像;Step 1, blurring and downsampling the high-resolution sample image according to the degradation model to obtain a low-resolution sample image;
步骤2,采用K-SVD方法,利用所述低分辨率样本图像进行字典训练,得到高分辨字典和低分辨率字典;Step 2, using the K-SVD method to perform dictionary training using the low-resolution sample image to obtain a high-resolution dictionary and a low-resolution dictionary;
其中,步骤1和步骤2建立图像库训练字典的方法,将图像集离线训练字典库,得到高分辨率图像与低分辨率图像的映射关系,优化图像重建步骤,有效减少方法运行时间,具有自适应性和高效性。Among them, step 1 and step 2 establish the method of image library training dictionary, the image set is trained offline to obtain the mapping relationship between high-resolution images and low-resolution images, optimize the image reconstruction steps, effectively reduce the running time of the method, and have automatic adaptability and efficiency.
步骤3,对待重建的低分辨率图像进行预处理得到若干个低分辨率图像块;Step 3, preprocessing the low-resolution image to be reconstructed to obtain several low-resolution image blocks;
步骤4,利用所述低分辨率样本图像计算固定阈值;Step 4, using the low-resolution sample image to calculate a fixed threshold;
步骤5,判断每一个所述低分辨率图像块的像素值是否小于所述固定阈值;若是,则判定该所述低分辨率图像块为低信息量图像块;若否,则判定该所述低分辨率图像块为高信息量图像块;Step 5, judging whether the pixel value of each of the low-resolution image blocks is smaller than the fixed threshold; if so, then judging that the low-resolution image block is a low-information image block; Low-resolution image blocks are high-information image blocks;
其中,步骤5在保证重建质量的同时,显著提高了重建速度,适合对时间性能要求较高的应用。Among them, step 5 significantly improves the reconstruction speed while ensuring the reconstruction quality, which is suitable for applications with high time performance requirements.
步骤6,将所述高信息量图像块采用基于字典学习与稀疏表示算法重建得到高信息量重建子区域;Step 6: Reconstruct the high-information image block using a dictionary learning and sparse representation algorithm to obtain a high-information reconstruction sub-region;
其中,基于学习的稀疏表示重建方法避免了近邻数目的人为选择,重建效果较好。Among them, the learning-based sparse representation reconstruction method avoids the artificial selection of the number of neighbors, and the reconstruction effect is better.
步骤7,将所述低信息量图像块采用插值算法进行重建得到低信息量重建子区域;Step 7, reconstructing the low-information image block using an interpolation algorithm to obtain a low-information reconstruction sub-region;
步骤8,将所述高信息量子区域与所述低信息量重建子区域进行图像拼接得到超分辨率重建图像;Step 8, performing image stitching on the high-information quantum region and the low-information reconstruction sub-region to obtain a super-resolution reconstruction image;
步骤9,采用插值算法将所述超分辨率重建图像进行M倍插值得到N*M(N=2,3,4;M>0)倍的超分辨率重建图像。Step 9: Perform M-fold interpolation on the super-resolution reconstructed image using an interpolation algorithm to obtain an N*M (N=2, 3, 4; M>0)-fold super-resolution reconstructed image.
具体地,所述步骤2包括如下步骤:Specifically, the step 2 includes the following steps:
(21)通过特征提取方法提取所述低分辨率样本图像的图像特征,得到空间目标的高分辨率特征信息和低分辨率特征信息。(21) Extracting image features of the low-resolution sample image by a feature extraction method to obtain high-resolution feature information and low-resolution feature information of the space object.
(22)利用所述K-SVD方法,对所述高分辨率特征信息和所述低分辨率特征信息进行联合训练,得到所述高分辨率字典和所述低分辨率字典。(22) Using the K-SVD method, perform joint training on the high-resolution feature information and the low-resolution feature information to obtain the high-resolution dictionary and the low-resolution dictionary.
其中,所述步骤(22)包括如下步骤:Wherein, described step (22) comprises the steps:
(221)利用稀疏K-SVD方法训练低分辨率字典;(221) Utilize sparse K-SVD method to train low-resolution dictionary;
(222)计算高分辨率字典。(222) Compute a high-resolution dictionary.
具体地,所述步骤3包括如下步骤:Specifically, the step 3 includes the following steps:
(31)对所述待重建的低分辨率图像去噪声得到第一图像;(31) Denoising the low-resolution image to be reconstructed to obtain a first image;
(32)将所述第一图像去模糊得到第二图像;(32) Deblurring the first image to obtain a second image;
(33)将所述第二图像按照固定长宽进行分割、保存处理,形成所述低分辨率图像块。(33) Segment and save the second image according to a fixed length and width to form the low-resolution image block.
具体地,所述步骤4包括如下步骤:Specifically, said step 4 includes the following steps:
(41)将所述低分辨率样本图像进行区块分割得到低分辨率样本图像块;(41) performing block segmentation on the low-resolution sample image to obtain a low-resolution sample image block;
(42)利用边缘提取算法提取所述低分辨率样本图像块的边缘信息,统计各所述低分辨率样本图像块的像素值和所有所述低分辨率样本图像块的像素值分布情况;(42) Using an edge extraction algorithm to extract edge information of the low-resolution sample image blocks, counting the pixel values of each of the low-resolution sample image blocks and the distribution of pixel values of all the low-resolution sample image blocks;
(43)根据所述各低分辨率样本图像块的像素值和所有所述低分辨率样本图像块的像素值分布情况选取X(X>1)个候选阈值;(43) Selecting X (X>1) candidate thresholds according to the pixel values of each of the low-resolution sample image blocks and the distribution of pixel values of all the low-resolution sample image blocks;
(44)对每个所述候选阈值利用基于学习的稀疏表示图像超分辨率重建算法进行计算形成多个高分辨率候选图像;(44) Utilizing a learning-based sparse representation image super-resolution reconstruction algorithm for each of the candidate thresholds to form a plurality of high-resolution candidate images;
(45)根据所述基于学习的稀疏表示图像超分辨率重建算法的运算时间及对应形成的所述高分辨率候选图像的分辨率,从X个候选阈值中选择所述信息量阈值。(45) Select the information amount threshold from X candidate thresholds according to the calculation time of the learning-based sparse representation image super-resolution reconstruction algorithm and the resolution of the correspondingly formed high-resolution candidate image.
优选地,所述边缘提取算法为Canny算子边缘检测算法;Preferably, the edge extraction algorithm is a Canny operator edge detection algorithm;
再者,所述步骤6包括如下步骤:Furthermore, said step 6 includes the steps of:
(61)针对所述高信息量图像块进行图像滤波处理,并进行高频特征提取获得高频信息;(61) performing image filtering processing on the high-information image block, and performing high-frequency feature extraction to obtain high-frequency information;
(62)利用所述高频信息对所述高信息量图像块进行KPCA降维,实现高维数据压缩;(62) Using the high-frequency information to perform KPCA dimensionality reduction on the high-information image block to realize high-dimensional data compression;
(63)采用OMP算法对降维压缩后的数据进行重建得到所述高信息量重建子区域。(63) Using the OMP algorithm to reconstruct the data after dimensionality reduction and compression to obtain the reconstruction sub-region with high information content.
进一步的,所述图像滤波操作采用二维滤波算子滤波器组,所述滤波器组为f={f1,f2,f3,f4},其中,Further, the image filtering operation adopts a two-dimensional filtering operator filter bank, and the filter bank is f={f 1 , f 2 , f 3 , f 4 }, wherein,
f1=[1,-1], f2=f1 T f 1 =[1,-1], f 2 =f 1 T
f3=LOG, f3=f3 T f 3 =LOG, f 3 =f 3 T
T为矩阵转置操作,LOG为5×5的二维滤波算子。T is a matrix transpose operation, and LOG is a 5×5 two-dimensional filter operator.
进一步的,所述步骤(63)包括如下步骤:Further, said step (63) includes the following steps:
(631)根据所述OMP算法计算稀疏表示系数β,其中, (631) Calculate the sparse representation coefficient β according to the OMP algorithm, wherein,
其中,y为降维解压后的数据,T0为给定的稀疏度,βi为矩阵β中的子元素。Among them, y is the data after dimension reduction and decompression, T 0 is the given sparsity, and β i is the sub-element in the matrix β.
(632)将所述稀疏表示系数β与所述高分辨率字典相乘,得到所述高信息量重建子区域。(632) Multiply the sparse representation coefficient β by the high-resolution dictionary to obtain the high-information reconstruction sub-region.
优选地,所述的插值算法为双三次插值算法。还可以采用例如线性插值、最邻近元法、双线性内插法等。Preferably, the interpolation algorithm is bicubic interpolation algorithm. It is also possible to employ, for example, linear interpolation, nearest neighbor method, bilinear interpolation method, and the like.
本实施例具有如下优点:This embodiment has the following advantages:
1、本发明实施例提出建立图像库训练字典的方法,将图像集离线训练字典库,得到高分辨率图像与低分辨率图像的映射关系,优化图像重建步骤,有效减少方法运行时间,具有自适应性和高效性。1. The embodiment of the present invention proposes a method for establishing a training dictionary in an image library, trains the dictionary library offline with an image set, obtains the mapping relationship between a high-resolution image and a low-resolution image, optimizes the image reconstruction steps, effectively reduces the running time of the method, and has automatic adaptability and efficiency.
2、本发明获得的超分辨率重建图像可以进行任意大小的尺度变换。在工程应用中可以满足不同的情况下的需求,有较好的适用性。2. The super-resolution reconstructed image obtained by the present invention can be scaled to any size. In engineering applications, it can meet the needs of different situations and has good applicability.
3、本发明中方法中的阈值是基于大量对比试验后进行优化固定,在处过程中不需要进行改变,在不影响图像重建效果的情况下,有效的提高了图像重建效率。3. The threshold value in the method of the present invention is optimized and fixed based on a large number of comparative experiments, and does not need to be changed during the process, and the image reconstruction efficiency is effectively improved without affecting the image reconstruction effect.
实施例二Embodiment two
在上述实施例一的基础上,本实施例提供另一种无级缩放超分辨率图像的重建方法,包含字典训练过程、图像重建过程、尺度变化过程三个处理步骤。具体包括如下步骤:On the basis of the first embodiment above, this embodiment provides another stepless zoom super-resolution image reconstruction method, which includes three processing steps: a dictionary training process, an image reconstruction process, and a scale change process. Specifically include the following steps:
S1:字典重建过程。S1: Dictionary reconstruction process.
S11:利用大量高分辨率样本图像,将高分辨率图像按照修正后的退化模型进行模糊和N倍下采样,得到相应的低分辨率样本图像。S11: Using a large number of high-resolution sample images, the high-resolution images are blurred and down-sampled by N times according to the corrected degradation model to obtain corresponding low-resolution sample images.
S12:对步骤1获得的低分辨率样本图像通过特征提取方法提取图像特征,得到空间目标的高低分辨率特征信息即Xs和Ys。S12: Extract image features from the low-resolution sample image obtained in step 1 through a feature extraction method, and obtain high and low-resolution feature information of the space object, namely X s and Y s .
S13:利用K-SVD方法,对高低分辨率特征信息进行联合训练,得到高低分辨率字典。S13: Using the K-SVD method, perform joint training on the high and low resolution feature information to obtain a high and low resolution dictionary.
S13a:训练低分辨率字典。基字典Φ选择过完备DCT字典,利用稀疏K-SVD方法求解:S13a: Training a low-resolution dictionary. The base dictionary Φ selects an over-complete DCT dictionary, and uses the sparse K-SVD method to solve:
则低分辨率字典Dl=ΦW,W是一个原子表示矩阵。与解析字典模型相比,双稀疏字典模型通过对W的修改提供了自适应性。Then the low-resolution dictionary D l =ΦW, W is an atom representation matrix. Compared with the parsing dictionary model, the double-sparse dictionary model provides adaptability through the modification of W.
S13b:计算高分辨率字典。假设高分辨率—低分辨率图像块在高分辨率—低分辨率字典对下具有相同的稀疏表示系数A,则可以通过最小化以下公式中的逼近误差来计算高分辨率字典Dh:S13b: Calculate a high-resolution dictionary. Assuming that the high-resolution-low-resolution image blocks have the same sparse representation coefficient A under the high-resolution-low-resolution dictionary pair, the high-resolution dictionary D h can be calculated by minimizing the approximation error in the following formula:
使用伪逆求解:Solve using the pseudoinverse:
Dh=XsA+=XsAT(AAT)-1 (3)D h =X s A + =X s A T (AA T ) -1 (3)
其中,上标“+”表示伪逆。Among them, the superscript "+" indicates the pseudo-inverse.
S2图像重建过程。S2 image reconstruction process.
S21:对待重建的低分辨率图像进行预处理,其中主要包括图像去噪、图像去模糊和样本分块操作。其处理步骤为:S21: Perform preprocessing on the low-resolution image to be reconstructed, which mainly includes image denoising, image deblurring, and sample block operations. Its processing steps are:
S21a:对待重建的低分辨率图像去噪声;S21a: Denoising the low-resolution image to be reconstructed;
S21b:对步骤S21a得到的图像去模糊;S21b: deblurring the image obtained in step S21a;
S21c:对步骤S21b得到的图像进行区块划分,将整幅图像按照固定长宽进行分割、保存;S21c: dividing the image obtained in step S21b into blocks, dividing and saving the entire image according to a fixed length and width;
S22:利用步骤S11中获得的低分辨率样本图像计算固定阈值,具体处理步骤如下:S22: Using the low-resolution sample image obtained in step S11 to calculate a fixed threshold, the specific processing steps are as follows:
S22a:将步骤S11中得到的大量低分辨率样本图像进行区块分割获得图像块,图像块的长宽同步骤S21c;S22a: segment a large number of low-resolution sample images obtained in step S11 into blocks to obtain image blocks, and the length and width of the image blocks are the same as step S21c;
S22b:利用边缘提取算法提取图像块边缘信息,统计各图像块的信息量及所有图像块的信息量分布情况;S22b: Using the edge extraction algorithm to extract the edge information of the image block, counting the amount of information of each image block and the distribution of the amount of information of all image blocks;
S22c:选取图像块中信息量最高的,获取该图像块的像素值为F1,取f=F1/4,则f*40%<=阈值<=f*60%,取该范围内若干代表阈值,例如,额可以取如下代表阈值:f*40%,f*45%,f*50%,f*55%,f*60%,计算每个阈值点对应的基于学习的稀疏表示图像超分辨率重建算法的重建时间和重建后图像的分辨率,可以根据主观评价和PSNR来判断。然后根据使用者需求从若干代表阈值中确定固定阈值,如使用者需求更注重时间,那就调高比例,时间就快;相反使用者如果注重重建效果,那就调低比例,带来的就是重建效果较好但时间较长。例如,经计算后f*50%最符合使用者需求,则取固定阈值=f*50%。S22c: Select the image block with the highest amount of information, obtain the pixel value of the image block as F1, take f=F1/4, then f*40%<=threshold<=f*60%, take several representative thresholds within this range For example, the amount can be taken as the following representative thresholds: f*40%, f*45%, f*50%, f*55%, f*60%, and calculate the sparse representation image super-resolution based on learning corresponding to each threshold point The reconstruction time of the high-rate reconstruction algorithm and the resolution of the reconstructed image can be judged according to the subjective evaluation and PSNR. Then determine the fixed threshold from several representative thresholds according to the user's needs. If the user needs to pay more attention to time, then increase the ratio, and the time will be faster; on the contrary, if the user pays attention to the reconstruction effect, then lower the ratio, which will bring Reconstruction works better but takes longer. For example, after calculation, f*50% best meets the needs of users, and then the fixed threshold=f*50%.
S22d:输入步骤S21获得的待重建的低分辨率图像块,采用边缘提取算法提取图像块边缘信息,当该图像块信息量不超过步骤S22c确定的固定阈值时该图像块为低信息量图像块;否则,为高信息量图像块。优选地,所述边缘提取算法为Canny算子边缘检测算法。S22d: Input the low-resolution image block to be reconstructed obtained in step S21, and use the edge extraction algorithm to extract the edge information of the image block. When the information amount of the image block does not exceed the fixed threshold determined in step S22c, the image block is a low-information image block ; Otherwise, it is a high-information image block. Preferably, the edge extraction algorithm is a Canny operator edge detection algorithm.
S23:针对高信息量图像块进行图像滤波操作,进行高频特征提取,采用二维滤波算子滤波器组,所用滤波器组为f={f1,f2,f3,f4},其由四个不同的滤波器组成,分别为:S23: Perform image filtering operations on high-information image blocks, and extract high-frequency features, using a two-dimensional filter operator filter bank, the filter bank used is f={f 1 , f 2 , f 3 , f 4 }, It consists of four different filters, namely:
f1=[1,-1], f2=f1 T f 1 =[1,-1], f 2 =f 1 T
(4)(4)
f3=LOG, f3=f3 T f 3 =LOG, f 3 =f 3 T
(5)(5)
其中上角标T表示矩阵转置操作,LOG代表一种5×5的二维滤波算子。经过高频特征提取操作后获得图像块高频信息,以xl表示。The superscript T represents the matrix transpose operation, and LOG represents a 5×5 two-dimensional filter operator. After the high-frequency feature extraction operation, the high-frequency information of the image block is obtained, represented by x l .
S24:对步骤S23获得的图像块进行KPCA降维,实现高维数据压缩。降维步骤如下:S24: Perform KPCA dimensionality reduction on the image blocks obtained in step S23 to achieve high-dimensional data compression. The dimensionality reduction steps are as follows:
S24a:将高维数据集合表示为X={x1,x2,x3,…,xM},xi∈RD,KPCA方法经过非线性映射函数x→Φ(x)∈F,其中F是特征空间,这样便能将每个数据x映射到一个高维特征空间。S24a: Express the high-dimensional data set as X={x 1 ,x 2 ,x 3 ,…,x M }, x i ∈ R D , the KPCA method goes through a nonlinear mapping function x→Φ(x)∈F, where F is the feature space, so that each data x can be mapped to a high-dimensional feature space.
S24b:核函数通过Φ将进行一个点x到F的对应操作,并且由此获得的F数据满足中心化的条件,即:S24b: The kernel function will perform a corresponding operation from a point x to F through Φ, and the obtained F data meets the centralization condition, namely:
则特征空间F中的协方差矩阵为:Then the covariance matrix in the feature space F is:
S24c:求c的特征值λ≥0以及特征向量S24c: Find the eigenvalue λ≥0 and eigenvector of c
V∈F\{0},Cv=λv (8)V∈F\{0}, Cv=λv (8)
则有then there is
(Φ(xv)·Cv)=λ(Φ(xv)·v) (9)(Φ(x v )·Cv)=λ(Φ(x v )·v) (9)
考虑到所有的特征向量可表示为Φ(x1),Φ(x2),…,Φ(xM)的线性组合,即:Considering that all eigenvectors can be expressed as a linear combination of Φ(x 1 ), Φ(x 2 ),…,Φ(x M ), namely:
则有:Then there are:
式中,v=1,2,3,…,M,定义M×M维矩阵Kμv In the formula, v=1,2,3,...,M, define M×M dimensional matrix K μv
Kμv:=(Φ(xμ)·Φ(xv)) (12)K μv :=(Φ(x μ )·Φ(x v )) (12)
S24d:求解上式得到特征值及特征向量,对于数据集合在特征向量空间Vk的投影可以写成:S24d: Solve the above formula to get the eigenvalues and eigenvectors, and the projection of the data set in the eigenvector space V k can be written as:
那么,数据被投影到协方差矩阵的特征向量Vk上,投影结果(也就是低维数据的表示y)可以表示为:Then, the data is projected onto the eigenvector V k of the covariance matrix, and the projection result (that is, the representation y of low-dimensional data) can be expressed as:
S25:对经过步骤S24降维压缩后的数据进行基于学习的稀疏表示图像超分辨率重建,具体步骤如下:S25: Carry out learning-based sparse representation image super-resolution reconstruction on the data after step S24 dimensionality reduction and compression, the specific steps are as follows:
S25a:采用OMP算法对步骤S24d获得的低维数据y在低分辨率字典Dl下的稀疏表示系数β,即求解如下方程:S25a: using the OMP algorithm for the sparse representation coefficient β of the low-dimensional data y obtained in step S24d under the low-resolution dictionary D1, that is, solving the following equation:
其中T0为给定的稀疏度,βi为矩阵β中的子元素。where T 0 is a given sparsity, and β i is a sub-element in matrix β.
S25b:将求得的稀疏表示系数β与高分辨率字典Dh相乘,得到重建的高信息量重建子区域,即:S25b: Multiply the obtained sparse representation coefficient β by the high-resolution dictionary D h to obtain the reconstructed high-information reconstruction sub-region, namely:
X=DhβX=D h β
其中X为求得的超分辨率子区域。Where X is the obtained super-resolution sub-region.
S26:将步骤22d获得的低信息量图像块采用双三次插值算法进行重建得到低信息量重建子区域;S26: Reconstruct the low-information image block obtained in step 22d using a bicubic interpolation algorithm to obtain a low-information reconstruction sub-region;
S27:将高信息量子区域与所述低信息量重建子区域进行图像拼接得到超分辨率重建图像;S27: Perform image stitching on the high-information quantum region and the low-information reconstruction sub-region to obtain a super-resolution reconstructed image;
S28:采用双三次插值算法将所述超分辨率重建图像进行M倍插值得到N*M(N=2,3,4;M>0)倍的超分辨率重建图像。S28: Perform M times interpolation on the super-resolution reconstructed image by using a bicubic interpolation algorithm to obtain an N*M (N=2, 3, 4; M>0) times super-resolution reconstructed image.
以上内容是结合具体的优选实施方式对本发明所作的进一步详细说明,不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干简单推演或或替换,都应当视为属于本发明的保护范围。The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be considered as belonging to the protection scope of the present invention.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710267753.2A CN107169925B (en) | 2017-04-21 | 2017-04-21 | A Reconstruction Method for Stepless Scaling Super-resolution Images |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710267753.2A CN107169925B (en) | 2017-04-21 | 2017-04-21 | A Reconstruction Method for Stepless Scaling Super-resolution Images |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107169925A true CN107169925A (en) | 2017-09-15 |
CN107169925B CN107169925B (en) | 2019-10-22 |
Family
ID=59812837
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710267753.2A Active CN107169925B (en) | 2017-04-21 | 2017-04-21 | A Reconstruction Method for Stepless Scaling Super-resolution Images |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107169925B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109064394A (en) * | 2018-06-11 | 2018-12-21 | 西安电子科技大学 | A kind of image super-resolution rebuilding method based on convolutional neural networks |
CN111986078A (en) * | 2019-05-21 | 2020-11-24 | 四川大学 | A Method for Fusion and Reconstruction of Multi-scale Core CT Images Based on Guidance Data |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104463801A (en) * | 2014-12-04 | 2015-03-25 | 河西学院 | Multi-sensing-information fusion method based on self-adaptation dictionary learning |
WO2015180055A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Super-resolution image reconstruction method and apparatus based on classified dictionary database |
CN105427243A (en) * | 2015-10-29 | 2016-03-23 | 中国科学院自动化研究所 | Video super-resolution reconstruction method based on adaptive interpolation kernel learning |
US9466132B2 (en) * | 2014-12-08 | 2016-10-11 | General Electric Company | Systems and methods for motion mitigation determinations |
CN106408550A (en) * | 2016-09-22 | 2017-02-15 | 天津工业大学 | Improved self-adaptive multi-dictionary learning image super-resolution reconstruction method |
-
2017
- 2017-04-21 CN CN201710267753.2A patent/CN107169925B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015180055A1 (en) * | 2014-05-28 | 2015-12-03 | 北京大学深圳研究生院 | Super-resolution image reconstruction method and apparatus based on classified dictionary database |
CN104463801A (en) * | 2014-12-04 | 2015-03-25 | 河西学院 | Multi-sensing-information fusion method based on self-adaptation dictionary learning |
US9466132B2 (en) * | 2014-12-08 | 2016-10-11 | General Electric Company | Systems and methods for motion mitigation determinations |
CN105427243A (en) * | 2015-10-29 | 2016-03-23 | 中国科学院自动化研究所 | Video super-resolution reconstruction method based on adaptive interpolation kernel learning |
CN106408550A (en) * | 2016-09-22 | 2017-02-15 | 天津工业大学 | Improved self-adaptive multi-dictionary learning image super-resolution reconstruction method |
Non-Patent Citations (1)
Title |
---|
何阳 等: "稀疏阈值的超分辨率图像重建", 《中国光学》 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109064394A (en) * | 2018-06-11 | 2018-12-21 | 西安电子科技大学 | A kind of image super-resolution rebuilding method based on convolutional neural networks |
CN111986078A (en) * | 2019-05-21 | 2020-11-24 | 四川大学 | A Method for Fusion and Reconstruction of Multi-scale Core CT Images Based on Guidance Data |
CN111986078B (en) * | 2019-05-21 | 2023-02-10 | 四川大学 | A Fusion and Reconstruction Method of Multi-scale Core CT Image Based on Guidance Data |
Also Published As
Publication number | Publication date |
---|---|
CN107169925B (en) | 2019-10-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Li et al. | Face hallucination based on sparse local-pixel structure | |
Yan et al. | Single image superresolution based on gradient profile sharpness | |
Kang et al. | Learning-based joint super-resolution and deblocking for a highly compressed image | |
Yang et al. | Consistent coding scheme for single-image super-resolution via independent dictionaries | |
Zhang et al. | Collaborative representation cascade for single-image super-resolution | |
CN107133915A (en) | A kind of image super-resolution reconstructing method based on study | |
Liu et al. | Multi-morphology image super-resolution via sparse representation | |
Yang et al. | Single-image super-resolution by subdictionary coding and kernel regression | |
Ren et al. | Learning image profile enhancement and denoising statistics priors for single-image super-resolution | |
CN107169925B (en) | A Reconstruction Method for Stepless Scaling Super-resolution Images | |
Liu et al. | Single image super-resolution using a deep encoder–decoder symmetrical network with iterative back projection | |
Zhai et al. | Bayesian error concealment with DCT pyramid for images | |
CN107133916A (en) | Image-scaling method | |
Jiang et al. | Face hallucination and recognition in social network services | |
Ogawa et al. | Missing intensity interpolation using a kernel PCA-based POCS algorithm and its applications | |
Han et al. | Cluster-based image super-resolution via jointly low-rank and sparse representation | |
Hu et al. | DIRformer: A Novel Image Restoration Approach Based on U-shaped Transformer and Diffusion Models | |
CN107093170A (en) | A kind of real-time super-resolution rate method for reconstructing | |
Wang et al. | Resolution-invariant coding for continuous image super-resolution | |
Oh et al. | Super-resolution texture synthesis using stochastic PAR/NL model | |
Pei et al. | Super-resolution of face images using weighted elastic net constrained sparse representation | |
Hung et al. | Real-time interpolation using bilateral filter for image zoom or video up-scaling/transcoding | |
Zhu et al. | A novel image super-resolution reconstruction method based on sparse representation using classified dictionaries | |
Xiao et al. | Single image super-resolution in compressed domain based on field of expert prior | |
Zhang et al. | Multi-example feature-constrained back-projection method for image super-resolution |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |