CN107093170B - Real-time super-resolution reconstruction method - Google Patents
Real-time super-resolution reconstruction method Download PDFInfo
- Publication number
- CN107093170B CN107093170B CN201710267750.9A CN201710267750A CN107093170B CN 107093170 B CN107093170 B CN 107093170B CN 201710267750 A CN201710267750 A CN 201710267750A CN 107093170 B CN107093170 B CN 107093170B
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- dictionary
- reconstructed
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012545 processing Methods 0.000 claims abstract description 23
- 230000000903 blocking effect Effects 0.000 claims abstract description 8
- 238000000605 extraction Methods 0.000 claims description 14
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000012549 training Methods 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 230000015556 catabolic process Effects 0.000 claims description 4
- 238000006731 degradation reaction Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000008901 benefit Effects 0.000 abstract description 6
- 238000003384 imaging method Methods 0.000 abstract description 6
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The invention relates to a real-time super-resolution reconstruction method, which comprises the steps of processing a first resolution image to form a second resolution image, and determining a resolution dictionary according to the first resolution image and the second resolution image; the image to be reconstructed is subjected to blocking processing to form a plurality of image blocks to be reconstructed; setting an information quantity threshold and judging whether the information quantity of the image block to be reconstructed is larger than the information quantity threshold or not; and if so, carrying out image reconstruction on the image block to be reconstructed according to the resolution dictionary to form a first sub-region, and carrying out image splicing on the first sub-region to obtain a super-resolution reconstruction image. The invention realizes the real-time resolution reconstruction of the image by the learning-based super-resolution reconstruction algorithm without changing the hardware structure of the imaging system, and has the advantages of low cost and high economic benefit.
Description
Technical Field
The invention relates to the field of image videos, in particular to a real-time super-resolution reconstruction method.
Background
Image super-resolution reconstruction techniques use a set of low-quality, low-resolution images (or motion sequences) to produce a single high-quality, high-resolution image. The image super-resolution reconstruction application field is wide, and the method has important application prospects in the aspects of military affairs, medicine, public safety, computer vision and the like. In the field of computer vision, image super-resolution reconstruction technology makes it possible to convert an image from a detection level (detective level) to a recognition level (recognitionlevel) or further to a fine resolution level (identification level). The image super-resolution reconstruction technology can improve the recognition capability and the recognition accuracy of the image. The image super-resolution reconstruction technology can realize the concentration analysis of the target object, so that an image with higher spatial resolution of the region of interest can be acquired without directly adopting the configuration of a high-spatial resolution image with huge data volume.
With the development of computer technology, the demand of people on high-resolution images in the fields of military, satellite remote sensing imaging, medicine and the like is more and more urgent, however, the resolution of the acquired images is low due to the fact that the traditional equipment is influenced by various factors such as system blur, atmospheric motion, noise, imaging environment and the like in the imaging process, and the specific demand is difficult to meet. If the conventional means of increasing the resolution by hardware equipment is adopted, a series of new problems of increased equipment volume, high manufacturing cost, difficult processing and the like are caused. Therefore, it is a new way to improve the image resolution by software technology.
The current super-resolution reconstruction algorithm mainly comprises an interpolation-based reconstruction algorithm, a model-based reconstruction algorithm and a learning-based reconstruction algorithm, and the learning-based reconstruction algorithm has better effect in the aspect of reconstructed image quality and can remarkably improve the image resolution. Although the learning-based super-resolution reconstruction algorithm has a good effect, the reconstruction efficiency is low because a large amount of prior information needs to be trained, and the purpose of real-time super-resolution reconstruction cannot be realized.
Disclosure of Invention
Therefore, in order to solve the technical defects and shortcomings in the prior art, the invention provides a real-time super-resolution reconstruction method.
Specifically, an embodiment of the present invention provides a real-time super-resolution reconstruction algorithm, which includes:
step 1, processing according to a first resolution image to form a second resolution image, and determining a resolution dictionary according to the first resolution image and the second resolution image;
step 2, carrying out blocking processing on an image to be reconstructed to form a plurality of image blocks to be reconstructed;
step 3, setting an information quantity threshold and judging whether the information quantity of the image block to be reconstructed is larger than the information quantity threshold; if so, carrying out image reconstruction on the image block to be reconstructed according to the resolution dictionary to form a first sub-area;
and 4, carrying out image splicing on the first sub-region to obtain a super-resolution reconstruction image.
In one embodiment of the present invention, forming a second resolution image according to a first resolution image process includes:
and carrying out blurring processing and N-time down-sampling processing on the first resolution image according to a degradation model to form the second resolution image.
In one embodiment of the invention, determining a resolution dictionary from the first resolution image and the second resolution image comprises:
extracting image features according to a feature extraction algorithm to obtain first-resolution image feature information and second-resolution image feature information;
and training the first-resolution image characteristic information and the second-resolution image characteristic information according to a K-SVD algorithm to obtain a first-resolution dictionary and a second-resolution dictionary.
In one embodiment of the present invention, obtaining a first resolution dictionary and a second resolution dictionary comprises:
calculating the first resolution dictionary by minimizing an approximation error: :
Dh＝XsA+＝XsAT(AAT)-1；
wherein it is assumed that the sparse representation coefficients of the first resolution dictionary and the second resolution dictionary are the same and are both A, DhFor said first resolution dictionary, XsRepresenting first resolution image feature information;
solving the second resolution dictionary by a sparse K-SVD algorithm:
Dl＝ΦW
wherein D islAnd phi is the second resolution dictionary, phi is a base dictionary, and W is an atom representation matrix. In an embodiment of the present invention, before step 2, further comprising:
and denoising and deblurring the image to be reconstructed.
In one embodiment of the invention, setting the information amount threshold comprises:
blocking the second resolution image to form a second resolution image block, and extracting the edge information quantity of the second resolution image block by using an edge extraction algorithm;
selecting X (X >1) pieces of edge information quantity as a candidate information quantity threshold according to the distribution condition of the edge information quantity of the second resolution image block;
and selecting one of the X candidate information quantity thresholds as the information quantity threshold according to the reconstruction time of the super-resolution algorithm and the resolution of the image formed by corresponding reconstruction.
In an embodiment of the present invention, before performing image reconstruction on the image block to be reconstructed according to the resolution dictionary, the method further includes:
and carrying out filtering operation on the image block to be reconstructed, extracting high-frequency characteristics, and acquiring high-frequency information of the image block to be reconstructed so as to complete dimension reduction processing.
In an embodiment of the present invention, image reconstructing the image block to be reconstructed according to the resolution dictionary to form a first sub-region includes:
solving sparse representation coefficients of the image block to be reconstructed under the second resolution dictionary through an OMP algorithm;
multiplying the sparse representation coefficient with the first resolution dictionary to form a first sub-region.
In one embodiment of the present invention, the solution equation for the sparse representation coefficients is:
wherein β is a sparse representation coefficient, T0For a given sparsity, βiAs sub-elements in matrix β
In one embodiment of the present invention, step 3 further comprises:
if not, carrying out image reconstruction on the image block to be reconstructed according to a bicubic interpolation algorithm to form a second sub-region;
accordingly, step 4 may comprise:
and carrying out image splicing on the first sub-region and the second sub-region to obtain a super-resolution reconstruction image.
Based on this, the invention has the following advantages:
the method realizes the real-time resolution reconstruction of the image based on the learning super-resolution reconstruction algorithm, has low cost and does not need to change the hardware structure of the imaging system.
The invention processes the learning dictionary, effectively reduces the number of dictionary atoms, and simultaneously can linearly represent all information of the initialization dictionary, thereby improving the training efficiency of the dictionary.
According to the invention, the image to be reconstructed is subjected to block management, the information quantity of the image block is judged by a reconstruction strategy of setting an information quantity threshold, and an image reconstruction method is selected according to the size of the information quantity of the image, so that the calculation efficiency of the algorithm is greatly improved.
Other aspects and features of the present invention will become apparent from the following detailed description, which proceeds with reference to the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention, for which reference should be made to the appended claims. It should be further understood that the drawings are not necessarily drawn to scale and that, unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein.
Drawings
The following detailed description of embodiments of the invention will be made with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a real-time super-resolution reconstruction method according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Example one
Referring to fig. 1, fig. 1 is a schematic diagram of a real-time super-resolution reconstruction method according to an embodiment of the present invention. The method comprises the following steps:
step 1, processing according to a first resolution image to form a second resolution image, and determining a resolution dictionary according to the first resolution image and the second resolution image;
step 2, carrying out blocking processing on an image to be reconstructed to form a plurality of image blocks to be reconstructed;
step 3, setting an information quantity threshold and judging whether the information quantity of the image block to be reconstructed is larger than the information quantity threshold; if so, carrying out image reconstruction on the image block to be reconstructed according to the resolution dictionary to form a first sub-area;
and 4, carrying out image splicing on the first sub-region to obtain a super-resolution reconstruction image.
Wherein, for step 1, it may include:
and carrying out blurring processing and N-time down-sampling processing on the first resolution image according to a degradation model to form the second resolution image.
Wherein, for step 1, it may further include:
a1) extracting image features according to a feature extraction algorithm to obtain first-resolution image feature information and second-resolution image feature information;
a2) and training the first-resolution image characteristic information and the second-resolution image characteristic information according to a K-SVD algorithm to obtain a first-resolution dictionary and a second-resolution dictionary.
Further, the obtaining the first resolution dictionary and the second resolution dictionary in the step a2) may further include:
a21) calculating the first resolution dictionary D by minimizing the approximation errorh：
Dh＝XsA+＝XsAT(AAT)-1；
Wherein it is assumed that the sparse representation coefficients of the first resolution dictionary and the second resolution dictionary are the same and are both A, DhFor said first resolution dictionary, XsRepresenting first resolution image feature information;
a22) solving the second resolution dictionary through a sparse K-SVD algorithm:
Dl＝ΦW
wherein D islFor the second resolution dictionary, Φ is a base dictionary, W is an atom representation matrix
Before step 2, the method may further include:
and denoising and deblurring the image to be reconstructed.
Wherein, for step 3, it may include:
b1) blocking the second resolution image to form a second resolution image block, and extracting the edge information quantity of the second resolution image block by using an edge extraction algorithm;
b2) selecting X (X >1) pieces of edge information quantity as a candidate information quantity threshold according to the distribution condition of the edge information quantity of the second resolution image block;
b3) and selecting one of the X candidate information quantity thresholds as the information quantity threshold according to the reconstruction time of the super-resolution algorithm and the resolution of the image formed by corresponding reconstruction.
Before the image reconstruction is performed on the image block to be reconstructed according to the resolution dictionary in the step 3, the method further includes:
and carrying out filtering operation on the image block to be reconstructed, extracting high-frequency characteristics, and acquiring high-frequency information of the image block to be reconstructed so as to complete dimension reduction processing.
Further, for step 3, the method may further include:
c1) solving the sparse representation coefficient of the image block to be reconstructed under the second resolution dictionary through an OMP algorithm;
c2) and multiplying the sparse representation coefficient by the first resolution dictionary to form a first sub-region.
Further, c1) in step 3, the method may further include:
the solution equation of the sparse representation coefficient is as follows:
wherein β is a sparse representation coefficient, T0For a given sparsity, βiAre sub-elements in the matrix β.
Further, for step 3, the method may further include:
if not, carrying out image reconstruction on the image block to be reconstructed according to a bicubic interpolation algorithm to form a second sub-region;
accordingly, step 4 may comprise:
and carrying out image splicing on the first sub-region and the second sub-region to obtain a super-resolution reconstruction image.
The beneficial effects of the invention are as follows:
1. the embodiment of the invention provides a method for establishing an image library training dictionary, which is used for training the dictionary library off line for an image set to obtain the mapping relation between a high-resolution image and a low-resolution image, optimizing the image reconstruction step, effectively reducing the operation time of an algorithm and having self-adaptability and high efficiency.
2. The threshold value in the algorithm is optimized and fixed based on a large number of comparison tests, and does not need to be changed in the process, so that the image reconstruction efficiency is effectively improved under the condition of not influencing the image reconstruction effect.
3. The super-resolution reconstruction algorithm based on learning realizes the real-time resolution reconstruction of the image, so the hardware structure of an imaging system does not need to be changed, and the super-resolution reconstruction algorithm based on learning has the advantages of low cost and high economic benefit.
Example two
In this embodiment, a technical solution of the present invention is described in detail on the basis of the above embodiment. Specifically, the method comprises the following steps:
step 1: and carrying out fuzzy processing and N-time down-sampling processing on the high-resolution image according to the corrected degradation model by using a large number of samples of the high-resolution image (namely, the first-resolution image) to obtain corresponding samples of the low-resolution image (namely, the second-resolution image).
Step 2: for the low resolution image obtained in step 1, passing the featuresExtracting image features by an extraction algorithm to obtain high-resolution feature information X of the space targets(i.e., first resolution characteristic information) and low resolution characteristic information Ys(i.e., second resolution characteristic information).
And step 3: performing joint training on the characteristic information by using a K-SVD algorithm to obtain a high-resolution dictionary Dh(i.e., first resolution dictionary) and a low resolution dictionary Dl(i.e., the second resolution dictionary).
Step 3 a): a low resolution dictionary is trained. Selecting an over-complete DCT (discrete cosine transformation) dictionary from the base dictionary phi, and solving by using a sparse K-SVD (singular value decomposition) algorithm:
then the low resolution dictionary Dl＝ΦW。
Step 3 b): a high resolution dictionary is computed. Assuming a high resolution image block, a low resolution image block having the same sparse representation coefficient A under the corresponding high resolution dictionary, low resolution dictionary, the high resolution dictionary D can be calculated by minimizing the approximation error in the following formulah：
Solving using a pseudo-inverse:
Dh＝XsA+＝XsAT(AAT)-1(3)
where the superscript "+" indicates the pseudo-inverse.
And 4, step 4: and preprocessing the low-resolution image to be reconstructed, wherein the preprocessing mainly comprises image denoising, image deblurring and sample blocking operation. The processing steps are as follows:
step 4 a): denoising the low-resolution image obtained in the step 1;
step 4 b): deblurring the low-resolution image obtained in the step 4a) to obtain a low-resolution image to be reconstructed;
step 4 c): dividing blocks of a low-resolution image to be reconstructed, and dividing and storing the whole image according to the fixed length and width;
and 5: the method comprises the following specific processing steps of carrying out block management on a low-resolution image block to be reconstructed, judging the information content of the image block to be reconstructed by a reconstruction strategy with a set information content threshold, and selecting a method for reconstructing an image to be reconstructed according to the size of the image information content:
step 5 a): carrying out block segmentation on a large number of low-resolution image samples obtained in the step (1), wherein the length and the width of a low-resolution image block are the same as those in the step (4 c); extracting edge information of the low-resolution image blocks by using an edge extraction algorithm, and counting the information quantity of each low-resolution image block and the information quantity distribution condition of all the image blocks; selecting a plurality of representative information quantity values as candidate threshold values according to the information quantity distribution, comparing the influences of different candidate threshold values on the super-resolution algorithm reconstruction quality and speed through a contrast test, and determining the most appropriate candidate threshold value as the information quantity threshold value according to the project requirements.
Step 5 b): inputting a low-resolution image block to be reconstructed, extracting edge information of the image block by adopting an edge extraction algorithm, and reconstructing the image block by adopting a bicubic interpolation algorithm when the information amount of the image block to be reconstructed does not exceed the information amount threshold determined in the step 5 a); otherwise, the super-resolution reconstruction algorithm is adopted for image reconstruction. The strategy ensures the reconstruction quality of image details and greatly improves the calculation speed of the algorithm.
For step 5a), the following embodiments may also be used to set the information amount threshold:
s1: carrying out block segmentation on a large number of low-resolution image samples to obtain image blocks;
s2: extracting edge information of the low-resolution image blocks by using an edge extraction algorithm, and counting the information quantity of each low-resolution image block and the information quantity distribution condition of all the low-resolution image blocks;
s3: selecting the value with the highest information content in the low-resolution image block, obtaining the pixel value of the low-resolution image block as F1, and taking F as F1/4, then F as 40% <asthreshold value as F60%, and taking several representative threshold values within the range, for example, the following representative threshold values can be taken: f 40%, f 45%, f 50%, f 55%, f 60%, calculating the reconstruction time of the learning-based sparse representation image super-resolution reconstruction algorithm and the resolution of the reconstructed image corresponding to each threshold point, and judging according to subjective evaluation and PSNR. Then, determining a fixed threshold value from a plurality of representative threshold values according to the user requirements as an information quantity threshold value, if the user requirements pay more attention to time, increasing the proportion, and shortening the time; on the contrary, if the user focuses on the reconstruction effect, the ratio is adjusted to be low, which results in better reconstruction effect but longer time. For example, if f is calculated to be the most suitable for the user, the threshold value of the amount of information is taken as f is 50%.
S4: inputting a low-resolution image block to be reconstructed, extracting edge information of the image block by adopting an edge extraction algorithm, and when the information amount of the image block does not exceed the information amount threshold determined in the step S3, determining the image block as a low-information-amount image block; otherwise, it is a high information content image block. Preferably, the edge extraction algorithm is a Canny operator edge detection algorithm
Step 6: performing image filtering operation on an image block to be reconstructed, performing high-frequency feature extraction, and adopting a2 x 2 filter bank, wherein the filter bank is f ═ { f1,f2,f3,f4It consists of four different filters, resolved as:
f1＝[1,-1],f2＝f1 T(4)
f3＝LOG,f3＝f3 T(5)
where the superscript T denotes the matrix transpose operation and LOG represents a 5 x 5 two-dimensional filter operator. Obtaining image block high-frequency information x after high-frequency feature extraction operationlAnd (4) showing.
And 7: and performing KPCA (kernel principal component analysis) dimension reduction on the image block adopting the super-resolution reconstruction algorithm to realize high-dimensional data compression. The dimensionality reduction steps are as follows:
step 7 a): representing a high-dimensional data set as X ═ X1,x2,x3,…,xM},xi∈RDThe KPCA algorithm is through a non-linear mapping function
x → Φ (x) e F, where F is the feature space, so that each data x can be mapped to a high-dimensional feature space.
Step 7 b): the kernel function will perform a corresponding operation of one point x to F by Φ, and the F data obtained thereby satisfies the condition of centralization, that is:
wherein phi is a mapping matrix, and can realize the mapping from points x to F, xμIs a sample image feature.
The covariance matrix C in the feature space F is:
wherein M is the total number of samples.
Step 7 c): c is obtained by finding the eigenvalue lambda is not less than 0 and the eigenvector V, V is 1,2, …, M
V∈F\{0},Cv＝λv (8)
Then there is
(Φ(xv)·Cv)＝λ(Φ(xv)·v) (9)
Consider that all feature vectors can be represented as Φ (x)1),Φ(x2),…,Φ(xM) The linear combination of (a):
then there are:
where v is 1,2,3, …, M, defining a M × M dimensional matrix Kμv
Kμv:＝(Φ(xμ)·Φ(xv))(12)
Step 7 d): solving the above formula to obtain the eigenvalue and the eigenvector,in feature vector space V for a data setkThe projection of (a) can be written as:
then, the data is projected to the eigenvector V of the covariance matrixkAbove, the projection result (i.e. the representation y of the low-dimensional data) can be expressed as:
and 8: performing super-resolution reconstruction on image block data subjected to dimension reduction compression, specifically comprising the following steps of:
step 8 a): solving the low-resolution dictionary D of the sub-region y to be reconstructed by adopting an OMP algorithmlThe following sparseness represents coefficients β, i.e., the following equation is solved:
wherein T is0For a given sparsity, βiAre sub-elements in the matrix β.
Step 8b) of comparing the obtained sparse representation coefficient β with the high resolution dictionary DhThe multiplication results in a reconstructed super-resolution reconstruction sub-region (i.e. the first sub-region), i.e.:
X＝Dhβ (16)
wherein X is the obtained super-resolution reconstruction sub-region.
And step 9: and (5) carrying out image splicing on the super-resolution reconstruction sub-region obtained in the step (8) and the reconstruction sub-region formed by reconstruction of the bicubic interpolation algorithm in the step (5) to obtain a super-resolution reconstruction image.
In summary, the present invention is explained based on the real-time super-resolution ratio reconstruction method by applying specific examples, and the above description of the embodiments is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention, and the scope of the present invention should be subject to the appended claims.
Claims (9)
1. A real-time super-resolution reconstruction method is characterized by comprising the following steps:
step 1, processing according to a first resolution image to form a second resolution image, and determining a resolution dictionary according to the first resolution image and the second resolution image;
step 2, carrying out blocking processing on an image to be reconstructed to form a plurality of image blocks to be reconstructed;
step 3, setting an information quantity threshold and judging whether the information quantity of the image block to be reconstructed is larger than the information quantity threshold; if so, carrying out image reconstruction on the image block to be reconstructed according to the resolution dictionary to form a first sub-area; wherein, setting the information quantity threshold value comprises:
blocking the second resolution image to form a second resolution image block, and extracting the edge information quantity of the second resolution image block by using an edge extraction algorithm;
selecting X (X >1) pieces of edge information quantity as a candidate information quantity threshold according to the distribution condition of the edge information quantity of the second resolution image block; the method comprises the following steps:
selecting the value with the highest information content in the second resolution image block, acquiring the pixel value of the second resolution image block as F1, and taking F as F1/4, wherein the range of the candidate information content threshold is F40% < ═ F as 60%;
selecting X (X >1) threshold points from the range of the candidate information quantity threshold value;
calculating the reconstruction time of a super-resolution reconstruction algorithm of the learning-based sparse representation image corresponding to each threshold point and the resolution of the reconstructed image;
selecting one of the X candidate information quantity thresholds as the information quantity threshold according to the reconstruction time and the resolution of the reconstructed image;
and 4, carrying out image splicing on the first sub-region to obtain a super-resolution reconstruction image.
2. The method of claim 1, wherein forming the second resolution image from the first resolution image processing comprises:
and carrying out blurring processing and N-time down-sampling processing on the first resolution image according to a degradation model to form the second resolution image.
3. The method of claim 1, wherein determining a resolution dictionary from the first resolution image and the second resolution image comprises:
extracting image features according to a feature extraction algorithm to obtain first-resolution image feature information and second-resolution image feature information;
and training the first-resolution image characteristic information and the second-resolution image characteristic information according to a K-SVD algorithm to obtain a first-resolution dictionary and a second-resolution dictionary.
4. The method of claim 3, wherein obtaining the first resolution dictionary and the second resolution dictionary comprises:
calculating the first resolution dictionary D by minimizing approximation errorh：
Dh＝XsA+＝XsAT(AAT)-1；
Wherein it is assumed that the sparse representation coefficients of the first resolution dictionary and the second resolution dictionary are the same and are both A, DhFor said first resolution dictionary, XsRepresenting first resolution image feature information;
solving the second resolution dictionary by a sparse K-SVD algorithm:
Dl＝ΦW
wherein D islFor the second resolution dictionary, Φ is the base dictionary and W is the originalThe sub-representation matrix.
5. The method of claim 1, prior to step 2, further comprising:
and denoising and deblurring the image to be reconstructed.
6. The method according to claim 1, wherein before performing image reconstruction on the image block to be reconstructed according to the resolution dictionary, the method further comprises:
and carrying out filtering operation on the image block to be reconstructed, extracting high-frequency characteristics, and acquiring high-frequency information of the image block to be reconstructed so as to complete dimension reduction processing.
7. The method according to claim 1, wherein image reconstructing the image block to be reconstructed according to the resolution dictionary to form a first sub-region comprises:
solving sparse representation coefficients of the image block to be reconstructed under the second resolution dictionary through an OMP algorithm;
multiplying the sparse representation coefficient with the first resolution dictionary to form a first sub-region.
8. The method of claim 7, wherein the solution equation for the sparse representation coefficients is:
wherein β is a sparse representation coefficient, T0For a given sparsity, βiAre sub-elements in the matrix β.
9. The method of claim 1, wherein step 3 further comprises:
if not, carrying out image reconstruction on the image block to be reconstructed according to a bicubic interpolation algorithm to form a second sub-region;
accordingly, step 4 comprises:
and carrying out image splicing on the first sub-region and the second sub-region to obtain a super-resolution reconstruction image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710267750.9A CN107093170B (en) | 2017-04-21 | 2017-04-21 | Real-time super-resolution reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710267750.9A CN107093170B (en) | 2017-04-21 | 2017-04-21 | Real-time super-resolution reconstruction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107093170A CN107093170A (en) | 2017-08-25 |
| CN107093170B true CN107093170B (en) | 2020-05-12 |
Family
ID=59637013
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710267750.9A Active CN107093170B (en) | 2017-04-21 | 2017-04-21 | Real-time super-resolution reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107093170B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722875A (en) * | 2012-05-29 | 2012-10-10 | 杭州电子科技大学 | Visual-attention-based variable quality ultra-resolution image reconstruction method |
| CN103903236A (en) * | 2014-03-10 | 2014-07-02 | 北京信息科技大学 | Method and device for reconstructing super-resolution facial image |
| CN105118025A (en) * | 2015-08-12 | 2015-12-02 | 西安电子科技大学 | Fast image super resolution method based on soft threshold coding |
| CN105678728A (en) * | 2016-01-19 | 2016-06-15 | 西安电子科技大学 | High-efficiency super-resolution imaging device and method with regional management |
| CN105844590A (en) * | 2016-03-23 | 2016-08-10 | 武汉理工大学 | Image super-resolution reconstruction method and system based on sparse representation |
-
2017
- 2017-04-21 CN CN201710267750.9A patent/CN107093170B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722875A (en) * | 2012-05-29 | 2012-10-10 | 杭州电子科技大学 | Visual-attention-based variable quality ultra-resolution image reconstruction method |
| CN103903236A (en) * | 2014-03-10 | 2014-07-02 | 北京信息科技大学 | Method and device for reconstructing super-resolution facial image |
| CN105118025A (en) * | 2015-08-12 | 2015-12-02 | 西安电子科技大学 | Fast image super resolution method based on soft threshold coding |
| CN105678728A (en) * | 2016-01-19 | 2016-06-15 | 西安电子科技大学 | High-efficiency super-resolution imaging device and method with regional management |
| CN105844590A (en) * | 2016-03-23 | 2016-08-10 | 武汉理工大学 | Image super-resolution reconstruction method and system based on sparse representation |
Non-Patent Citations (1)
| Title |
|---|
| Super-resolution Reconstruction via Multiple Sparse Dictionary Coding;Xuan Zhu 等;《Proceedings of the IEEE International Conference on Information and Automation》;20160831;第Ⅱ节，第Ⅲ节，图1-3 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107093170A (en) | 2017-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Learning multiple linear mappings for efficient single image super-resolution | |
| CN106952228B (en) | Super-resolution reconstruction method of single image based on image non-local self-similarity | |
| Pan et al. | Super-resolution based on compressive sensing and structural self-similarity for remote sensing images | |
| Li et al. | No-reference quality assessment of deblurred images based on natural scene statistics | |
| CN104933678B (en) | A kind of image super-resolution rebuilding method based on image pixel intensities | |
| Kato et al. | Multi-frame image super resolution based on sparse coding | |
| Zhao et al. | Single image super-resolution based on deep learning features and dictionary model | |
| CN108932699B (en) | Three-dimensional matching harmonic filtering image denoising method based on transform domain | |
| Li et al. | Perceptual quality evaluation for image defocus deblurring | |
| Kamble et al. | Performance evaluation of wavelet, ridgelet, curvelet and contourlet transforms based techniques for digital image denoising | |
| Liu et al. | An efficient residual learning neural network for hyperspectral image superresolution | |
| CN108460723B (en) | Bilateral total variation image super-resolution reconstruction method based on neighborhood similarity | |
| Liang et al. | Image deblurring by exploring in-depth properties of transformer | |
| Zhou et al. | A phase congruency based patch evaluator for complexity reduction in multi-dictionary based single-image super-resolution | |
| Chen et al. | Face super resolution based on parent patch prior for VLQ scenarios | |
| Yang et al. | Fast multisensor infrared image super-resolution scheme with multiple regression models | |
| Zin et al. | Local image denoising using RAISR | |
| CN110503606B (en) | Method for improving face definition | |
| CN107093170B (en) | Real-time super-resolution reconstruction method | |
| Lepcha et al. | An efficient medical image super resolution based on piecewise linear regression strategy using domain transform filtering | |
| Suryanarayana et al. | Deep Learned Singular Residual Network for Super Resolution Reconstruction. | |
| Solanki et al. | An efficient satellite image super resolution technique for shift-variant images using improved new edge directed interpolation | |
| CN113487491B (en) | Image restoration method based on sparsity and non-local mean self-similarity | |
| Zhang et al. | Hyperspectral and multispectral image fusion via superpixel-based weighted nuclear norm minimization | |
| Li et al. | Super-resolution using neighbourhood regression with local structure prior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |