CN111724307A - Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium - Google Patents
Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium Download PDFInfo
- Publication number
- CN111724307A CN111724307A CN202010568221.4A CN202010568221A CN111724307A CN 111724307 A CN111724307 A CN 111724307A CN 202010568221 A CN202010568221 A CN 202010568221A CN 111724307 A CN111724307 A CN 111724307A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- low
- rank
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 138
- 230000008569 process Effects 0.000 claims abstract description 17
- 238000012935 Averaging Methods 0.000 claims abstract description 5
- 230000006870 function Effects 0.000 claims description 35
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000011156 evaluation Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 7
- 238000006731 degradation reaction Methods 0.000 claims description 6
- 230000015556 catabolic process Effects 0.000 claims description 5
- 239000000654 additive Substances 0.000 claims description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 3
- 230000002401 inhibitory effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 description 21
- 238000004422 calculation algorithm Methods 0.000 description 20
- 230000000694 effects Effects 0.000 description 13
- 238000003384 imaging method Methods 0.000 description 8
- 210000004072 lung Anatomy 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 239000002420 orchard Substances 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- 238000013441 quality evaluation Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000004438 eyesight Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000000996 additive effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 210000004185 liver Anatomy 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000002285 radioactive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000003325 tomography Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides an image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, a terminal and a readable storage medium, wherein a continuous image sequence is used as data input, the similarity between a single image and a continuous image is used as prior knowledge, a local grouping mode of image blocks is combined to carry out block matching on similar blocks, and the spatial structure relation of image pixel levels is mined; modeling is carried out by using a maximum posterior probability frame, model parameters are fitted by using Gaussian distribution and Gibbs distribution, and the generalization capability of the model is improved; noise interference is suppressed by adopting a low-rank truncation mode; and a non-local low-rank constraint regularization image reconstruction process is adopted, and local information in a single image and local information between continuous images are utilized to improve the quality of a target image. Parameters in the model are optimized alternately in each iteration, the robustness of the model is improved, and local convergence is avoided. And finally, weighting and averaging the reconstructed image blocks to obtain a target high-resolution image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, a terminal and a readable storage medium.
Background
Vision is one of the main ways that humans obtain information from the outside world, and most vision-based application performance depends on the quality of the image. High Resolution (HR) images have High Resolution and contain abundant image details and more image information, so that the HR images have important value in the practical applications of the medical field, the video monitoring field, the remote sensing field and the like. Although the image imaging technology has become mature in these practical application fields, the resolution of most images is low due to the mutual restriction of the imaging equipment, the imaging environment, human interference and other factors. For example, in the medical field, the medical image imaging technology is mutually restricted by factors such as medical imaging equipment, radioactive element hazard degree, human physiological health and the like, and many medical image imaging technologies have Low imaging Resolution, and Low Resolution (LR) images cannot effectively assist lesion tissue classification detection tasks, so that the image super-Resolution technology is developed. Super Resolution (SR) is an image analysis and processing technique that takes an observed low-Resolution single image or image sequence as input to generate a high-Resolution single image or image sequence. The method based on interpolation, the method based on learning and the method based on reconstruction are popular 3-class image super-resolution methods at present.
The interpolation-based approach is an earlier proposed and relatively simple algorithm. Firstly, the registration relation between a low-resolution image and a target high-resolution image is calculated, and then a pixel value of a point to be interpolated is obtained by utilizing a known pixel value in a neighborhood according to an interpolation formula, so that the target high-resolution image is obtained. Bilinear Interpolation (Bilinear Interpolation), Bicubic Interpolation (Bicubic Interpolation), and Nearest neighbor Interpolation (Nearest neighbor Interpolation) are common. The interpolation-based method is simple to implement and low in calculation complexity, so that the method has good real-time performance. However, the anisotropy of the image is not considered in the interpolation process, and the high-frequency information of the image cannot be effectively reserved, so that the outline and the texture of the amplified image are fuzzy, blocky appearance is easy to appear, a false edge is generated, and the image quality is poor. The interpolation algorithm is difficult to process the problems of blurring in the image, noise introduced into the image and the like, and prior information cannot be added, so the method is poor in adaptability.
In the prior art, learning-based methods have become a popular class of techniques in recent years. The basic idea of the algorithm is to learn the mapping relation between the low-resolution images and the high-resolution images from a training sample set, so that the unknown low-resolution images are predicted, and the purpose of improving the resolution is achieved. For Example, the idea of local embedded popular Learning, the Learning strategy based on neighborhood Embedding (Neighbor Embedding), the algorithm based on sample Learning (Example Learning), the neural network algorithm combined with deconvolution, and the like are used, but such methods have a large dependence on external training data sets and are poor in model increment. Recent studies on image statistics show that image blocks can be sparsely and linearly represented by elements of their overcomplete dictionaries, so a learning method based on Sparse Representation (Sparse Representation) is applied, but the algorithm requires a large number of training data sets, and partial data sets require manual labeling, and an internal dictionary is usually not enough to contain complex texture information for good reconstruction. In conclusion, the learning model is crucial to the super-resolution effect of the image, but the current model cannot effectively combine all prior knowledge required by image reconstruction, and the learning-based method has long running time and is difficult to meet the real-time requirement, so that the method is more suitable for offline image preprocessing.
Reconstruction-based methods aim at reconstructing the high-frequency signals lost in the degradation process. Assuming that the input low resolution image has n frames in total, the mathematical model based on the reconstructed super resolution problem can be expressed as:
here lkIs a low resolution image obtained from an original high resolution image H to be reconstructed through a series of image transformation processes.
Is an atmospheric fuzzy operator, MkIs a motion-transformation operator, which is,
refers to the imaging blur operator. D is a down-sampling operator, NkRepresenting additive noise introduced during imaging. Known input lkThen the reconstruction-based approach aims to find the optimal estimate of the true high-resolution image H
The frequency domain method is one of the earliest proposed reconstruction-based super-resolution methods, proposed by Tsai and Huang in 1984. And respectively carrying out discrete Fourier transform and continuous Fourier transform on the low-resolution image and the target high-resolution image, and establishing a linear relation between the low-resolution image and the target high-resolution image in a frequency domain according to the property of the Fourier transform. Rhee and Kang et al replace the discrete fourier transform of the frequency domain with a discrete cosine transform, reducing storage requirements and costs. Although the frequency domain method is theoretically simple and has certain advantages in derivation and calculation, it is difficult to deal with noise and to add a priori information. In addition, because the frequency domain and the spatial domain have complex transformation relation, the situation of global overall motion can be only processed, and the situation of local motion is difficult to process.
In the prior art, an Iterative back projection method (IBP) is also adopted, and is proposed by Irani and Peleg. The method comprises the steps of reversely projecting the difference value between a low-resolution image generated by a degradation model and an input low-resolution image onto a high-resolution image, and continuously iterating to make the error converge, thereby obtaining a target high-resolution image. The IBP method is intuitive and easy to understand, but IBP is an inverse problem, and the ill-conditioned nature of IBP can cause the solution to be not unique. Projection Onto Convergence (POCS) is a super-resolution reconstruction method that employs iteration. In the POCS method, the influence of prior information on the result, such as the constraint on the peak pixel of the target image, can be added. The POCS method is flexible in form and can conveniently add prior information. However, the method has high computational complexity, requires multiple iterations and projections, and has slow convergence rate and low algorithm stability.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides an image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, which comprises the following steps:
the method comprises the following steps that firstly, a continuous image sequence is used as data input, the similarity between a single image and a continuous image is used as priori knowledge, block matching is carried out on similar blocks by combining a local grouping mode of image blocks, and the spatial structure relation of image pixel levels is mined;
modeling by using a maximum posterior probability frame, and fitting model parameters by using Gaussian distribution and Gibbs distribution to improve the generalization capability of the model;
estimating the singular value of the block to be solved through the maximum posterior of the singular value of the similar block, and inhibiting noise interference by adopting a low-rank truncation mode;
and step four, adopting a non-local low-rank constraint regularization image reconstruction process, and then utilizing local information in a single image and local information between continuous images to improve the quality of the target image.
It should be further noted that, after the step four, the method further includes:
and evaluating the quality of the reconstructed image by adopting the peak signal-to-noise ratio, the structural similarity and the characteristic similarity.
It should be further noted that the first step further includes:
the sequence of successive images is Y { Y }k1, 10; the continuous image sequence is composed of a series of low-resolution images, super-resolution reconstruction is carried out by taking an image y in the middle of the sequence as a reference image, and other images are cooperatively assisted;
the known image degradation model is:
y＝DBkx+nk(1)
in the formula, x is a high-resolution image to be reconstructed, y is an input low-resolution image and is obtained by carrying out bicubic interpolation and amplification on an original image;
d is a downsampling operator, BkIs a fuzzy operator. Suppose nkIs a mean of 0 and a variance of
Additive white gaussian noise of (1);
in step two, the purpose of super resolution is to reconstruct a high resolution image x from a low resolution reference image y; expressed by the MAP model as the following objective function:
fitting is performed by using a Gaussian function in the formula (2),
assuming that the pixel values are related to neighboring pixels that satisfy a gibbs probability density function, a gibbs function fit is used,
substituting the formula (3) and the formula (4) into the formula (2) to obtain:
equation (5) is the objective function to be solved.
It should be further noted that, step four further includes:
using a non-local low-rank prior, and expressing the non-local low-rank prior as a maximum posterior probability estimation;
suppose that
Representing image blocks xjWherein j is an image block index, and the similar blocks are formed with a size of j as a center
The image block of (1);
when the similar blocks are matched, selecting the most similar p similar blocks in the whole sequence image, and then performing low-rank truncation processing;
assuming that each set of low rank blocks is independent of each other, the low rank prior is expressed as:
substituting formula (6) for formula (5) to obtain:
typically the low rank matrix is solved with a kernel paradigm,
wherein L isjIs on demand
Low rank block, | Lj||*Is LjA kernel norm of (a) to represent the sum of singular values; solving the formula (8) by constructing an enhanced Lagrange equation by using an iteration direction multiplier;
in the formula of UjIs the lagrange multiplier and μ is a constant parameter. Solving equation (9) can be decomposed into two sub-problems:
wherein, solving for x can be directly solved by the formula (10):
solving low-rank block L by adopting MAP methodjBy using
Estimating L from the singular values ofjOfHeterodyning to obtain Lj；
Equation (12) can be derived from bayesian criteria,
assuming that the degree of torsion f represents the degree of torsion of the singular value of the high-resolution image block and the singular value of the low-resolution image block; in the equation (13), the first part is fitted with a gaussian function having a mean value of 0 and a standard deviation of f,
P(σi(Lj) Calculated using kernel density estimates, whose probability density function is assumed to be the sum of a series of kernel functions; the sum of singular values is the sum of a series of kernel functions defined by σi(Lj) Centered 1 × 3 neighborhood ΩiDetermining;
assuming that the mean of the kernel function is in line with the mean
Standard deviation of hiThe probability density function of the singular values is defined as:
the ith index in the formula (14) and the formula (15) is taken into the formula (13),
let the derivative of equation (16) be 0, solve for:
re-averaging the resulting total MAP estimates:
finally obtaining a low-rank image similar block LjIs estimated by the estimation of (a) a,
enhanced lagrange multiplier UjCan be updated by means of the formula (20),
it should be further noted that the evaluation method of the peak signal-to-noise ratio includes:
the peak signal-to-noise ratio calculation formula is as follows:
wherein MSE is the mean square error between the image to be evaluated and the reference image; g is the image gray level number; the larger the PSNR value is, the smaller the difference between the image to be evaluated and the reference image is, and the higher the image quality is.
Further, the evaluation method of the structural similarity includes:
the structural similarity calculation formula is as follows:
wherein, muxAnd muyRespectively the mean value of the gray levels, sigma, of the image to be evaluated and the reference imagexAnd σyDenotes the standard deviation, C1＝(k1G)2、C2＝(k2G)2As constants to maintain numerical stability, in which k is1＝0.01,k2G is 0.03, and G is the number of image gray levels.
Further, the evaluation method of the feature similarity includes:
the feature similarity calculation formula is as follows:
wherein S isL(x)＝SPC(x)·SG(x),SPC(x) And SG(x) Values of PC and GM between images, respectively; PC (personal computer)m(x) The PC value is the maximum PC value and is used for weighting the contribution of each point to the overall similarity of the two images; x is the position of a given pixel point and Ω is the entire airspace of the image.
The invention also provides a device for realizing the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior, which comprises the following steps:
a memory for storing a computer program and an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low rank prior;
a processor for executing the computer program and the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior to realize the steps of the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior.
The invention also provides a readable storage medium having an image super-resolution reconstruction method based on maximum a-posteriori probability and non-local low-rank prior, having stored thereon a computer program for execution by a processor for performing the steps of the image super-resolution reconstruction method based on maximum a-posteriori probability and non-local low-rank prior.
According to the technical scheme, the invention has the following advantages:
the invention is based on the maximum posterior probability reconstruction method, applies the MAP framework to the image super-resolution reconstruction technology, and combines the self-similarity and the non-local low-rank prior of the image to establish a super-resolution reconstruction model which fully utilizes the information contained in the image; the method of the invention takes a continuous image sequence as input, adopts a similar block grouping technology to preprocess a single reference image and images in front and back into a high-dimensional tensor form, facilitates the similarity comparison of image blocks, achieves the aim of rapid block matching and improves the calculation speed. And a non-local low-rank prior based on an image block is added into the constructed MAP model, so that the image details are fully utilized, and the loss of narrow features is avoided. And estimating the singular value of the block to be solved through the singular value of the similar block, selecting the most similar image block and performing low-rank truncation processing, thereby inhibiting the interference of noise and other small factors and improving the quality of a reconstructed image. Meanwhile, parameters in the model are optimized alternately in each iteration, so that the robustness of the model can be improved, and local convergence is avoided. And finally, weighting and averaging the reconstructed image blocks to obtain a target high-resolution image.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings used in the description will be briefly introduced, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart of a super-resolution reconstruction method of an image;
FIG. 2 is a schematic diagram of an image gradient profile;
FIG. 3 is a diagram illustrating the influence of iteration number on PSNR, SSIM, and FSIM;
FIG. 4 is a comparison graph of experiments with different regularization terms;
FIG. 5 is a diagram of an example of a medical image comparing four methods with the method of the present invention;
FIG. 6 is a diagram of four methods comparing examples of natural images with the method of the present invention;
FIG. 7 is a diagram of four methods and examples of natural images compared with the method of the present invention.
Detailed Description
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The invention provides an image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, and the process is shown in figure 1. And a continuous image sequence is used as data input, and the similarity between the single image and the continuous image is used as priori knowledge, so that the matching degree of similar image blocks is improved, and the phenomenon of image detail loss is eliminated. Then, modeling is carried out by a maximum posterior probability frame, and model parameters are fitted by using Gaussian distribution and Gibbs distribution, so that the generalization capability of the model is improved. And estimating the singular value of the block to be solved through the singular value of the similar block, and suppressing noise introduced in the reconstruction process by adopting low-rank truncation. And finally, utilizing the non-local self-similarity and low-rank property of the image, regularizing the image reconstruction process by using non-local low-rank constraint, and adding local and global information of the image to improve the reconstruction effect. The method effectively improves the quality of the reconstructed image, and obtains better effects on performance indexes such as peak signal-to-noise ratio, structural similarity and the like compared with the existing algorithm.
The Maximum A Posteriori (MAP) method is an algorithm framework based on statistical probability, and is the most applied method in actual application and scientific research at present. The basic idea of the algorithm is derived from conditional probability, and an unknown HR image is estimated by taking a known LR image sequence as an observation result. In the MAP framework, the Regularization term (Regularization term) plays a key role in controlling the quality of the reconstructed image. The MAP method is flexible, and particularly in the regular term part of the MAP framework, specific constraints on specific problems can be added. Therefore, a valid regularization term is key to guarantee the performance of the MAP framework. For example, commonly used regularization terms include the Tikhonov regularization term in two-norm form, the Total Variation (TV) regularization term in one-norm form, and the Bilateral Total Variation (BTV) regularization term, as well as the more complex Student-t regularization term. The MAP has a complete theoretical framework, a flexible spatial domain model and strong priori knowledge inclusiveness, and the maximum posterior probability method has good adaptability, flexibility and robustness, can generate excellent reconstruction results, and is an effective super-resolution reconstruction method.
The MAP method related to the invention is an algorithm framework based on statistical probability, whichThe basic idea is derived from conditional probabilities, by which the best estimate of the high-resolution image H is obtained
And then a target high-resolution image is reconstructed. Adding image similarity and low-rank performance to effectively improve the quality of reconstructed images and aiming at a low-resolution image sequence Y { Y k1, 10, and regularizing the image reconstruction process with a non-local low-rank regularization term (NLR) using non-local self-similarity of natural images.
In the present invention, Y { Y }kThe image y in the middle of the sequence is used as a reference image to carry out super-resolution reconstruction, and other images are cooperatively assisted. The known image degradation model is:
y＝DBkx+nk(1)
in the formula, x is a high-resolution image to be reconstructed, y is an input low-resolution image, and the input low-resolution image is obtained by bicubic interpolation and amplification of an original image.
D is a downsampling operator, BkIs a fuzzy operator. Suppose nkIs a mean of 0 and a variance of
White additive gaussian noise. The purpose of super resolution is to reconstruct a high resolution image x from a low resolution reference image y. The problem can be expressed by the MAP model as the following objective function:
the gradient distribution of the natural image has a heavy tail phenomenon, the gradient distribution of the City natural image (City) shown in fig. 2 conforms to the heavy tail distribution, and the heavy tail phenomenon also exists in the medical image through the gradient research of the medical tomography image (CT). The first term in the above equation can be fitted with a gaussian function,
assuming that the pixel values relate only to neighboring pixels that satisfy the gibbs probability density function, the second term may be fitted with the gibbs function,
substituting the formula (3) and the formula (4) into the formula (2) to obtain:
equation (5) is the objective function to be solved.
The invention relates to non-local low-rank regularization and a process thereof, which are as follows: the image block-based non-local low-rank prior is well applied to the fields of image denoising, image super-resolution and the like. The invention adds non-local low-rank prior on the basis of the work of the invention and expresses the non-local low-rank prior as the maximum posterior probability estimation. Suppose that
Representing image blocks xjWherein j is an image block index, and the similar blocks are formed with a size of j as a center
The image block of (1). And when the similar blocks are matched, selecting the most similar p similar blocks in the whole sequence image, and then performing low-rank truncation processing. Assuming that each set of low rank blocks is independent of each other, the low rank prior is therefore expressed as:
substituting formula (6) for formula (5) to obtain:
typically the low rank matrix is solved with a kernel paradigm,
wherein L isjIs on demand
Low rank block, | Lj||*Is LjIs used to represent the sum of the singular values.
The invention uses an iteration direction multiplier to solve the formula (8) by constructing an enhanced Lagrange equation.
In the formula of UjIs the lagrange multiplier and μ is a constant parameter. Solving equation (9) can be decomposed into two sub-problems:
wherein, solving for x can be directly solved by the formula (10):
the invention adopts the MAP method to solve the low-rank block LjBy using
Estimating L from the singular values ofjThereby obtaining Lj。
Equation (12) can be derived from bayesian criteria,
the degree of torsion f is assumed to represent the degree of torsion of the singular values of the high-resolution image block and the singular values of the low-resolution image block. In the above equation, the first part can be viewed as being fitted with a gaussian function with a mean of 0, a standard deviation of f,
in addition, P (σ)i(Lj) Can be computed using kernel density estimates whose probability density function is assumed to be the sum of a series of kernel functions, and thus the sum of singular values is the sum of a series of kernel functions consisting of a sum of σi(Lj) Centered 1 × 3 neighborhood ΩiAnd (4) determining. In the present invention, it is assumed that the mean of the kernel function conforms to the mean
Standard deviation of hiThe probability density function of the singular values is defined as:
the ith index in the formula (14) and the formula (15) is taken into the formula (13),
let the derivative of equation (16) be 0, solve for:
re-averaging the resulting total MAP estimates:
finally obtaining a low-rank image similar block LjIs estimated by the estimation of (a) a,
here, the Lagrangian multiplier U is enhancedjCan be updated by means of the formula (20),
in order to evaluate the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior, the invention is provided with three evaluation indexes, wherein the first index is a peak signal-to-noise ratio;
peak Signal to Noise Ratio (PSNR) is an objective standard for evaluating the similarity between an image and a reference image, measures the quality of the processed image, and is widely applied to image quality evaluation. The calculation formula is as follows:
wherein the MSE is the mean square error between the image to be evaluated and the reference image. G is the image gray scale number. The larger the PSNR value is, the smaller the difference between the image to be evaluated and the reference image is, and the higher the image quality is. However, the PSNR value is only an objective standard for representing image quality evaluation, and human visual factors are not taken into consideration, and even though the PSNR value is high, there may be a large error between the actual image quality and the expected image quality, so that various evaluation indexes are required to be comprehensively analyzed for evaluating the image quality.
Another evaluation index is: structural Similarity (SSIM), which is a degradation-based image quality evaluation method, determines image quality by comparing Structural similarities between an image to be evaluated and a reference image. Each pixel point in an image has strong dependencies on surrounding pixels, which carry important information about the structure of the target image in visual perception. The calculation formula is as follows:
wherein, muxAnd muyRespectively the mean value of the gray levels, sigma, of the image to be evaluated and the reference imagexAnd σyDenotes the standard deviation, C1＝(k1G)2、C2＝(k2G)2As constants to maintain numerical stability, in which k is1＝0.01,k2G is 0.03, and G is the number of image gray levels. SSIM is defined by the brightness and contrast associated with the structural information, with the mean gray value as an estimate of the brightness measure and the standard deviation as an estimate of the contrast measure. In addition, since SSIM is a symmetry metric, it can be considered as a similarity metric for comparing any two signals. The signals may be discrete or continuous and may exist in space of any dimension.
Another evaluation index is: feature similarity.
Feature Similarity (FSIM) is a more successful variant of SSIM. Phase Consistency (PC) in the FSIM is used to measure the importance of local structures, and considering that PC has contrast invariance, which affects the perception of human visual system to image quality, image Gradient Magnitude (GM) is used as a secondary feature in the FSIM. The FSIM is divided into two parts of PC and GM, and the calculation formula is as follows:
wherein S isL(x)＝SPC(x)·SG(x),SPC(x) And SG(x) Are the values of PC and GM between images, respectively. PC (personal computer)m(x) Is the largest PC value used to weight the contribution of each point to the overall similarity of the two images. x is the position of a given pixel point and Ω is the entire airspace of the image. The FSIM measures the importance of a local structure mainly by using the phase similarity and the image gradient similarity, and in the stage of evaluating the quality score, the phase similarity is used as a weight, so that the correlation with the visual perception of human eyes is increased, and a good quality evaluation effect is obtained.
The invention also verifies and analyzes the method. The experimental image of the invention selects 5 groups of images of various types such as a medical image set, a natural image set, a video framing image set and the like, each group of images has 10 image sequences, and any 1 image is selected as a reconstruction image. Where each image size is 128 × 128 and the reconstructed HR image size is 256 × 256. The experimental medical image data is provided by the relevant hospital. In the natural image dataset tested, the urban dataset was derived from a video database developed by the university institute of technology, new york, etc., which database contains 10 videos encoded using h.264, the resolution was varied from QCIF to 4CIF, the quantization parameter was between 28 and 44, and the frame rate was between 3.75 and 30 frames per second, only 10 of which were selected as experimental data in the experiment of the present invention. The small orchard data set is derived from an optical flow standard experiment data set and is selected as a natural image to test the reconstruction performance of the method. The MATLAB 2014(b) is used for realizing the image super-resolution reconstruction algorithm in the invention. The experimental hardware facilities were Intel (R) Xeon (R) E5-2643 v4@3.40GHz CPU, NVIDIA GeForce GTX 1080M GPU,256GB memory, and the operating system was ubuntu 14.04.
The images used in the experiment were all standard continuous image sequences with the number of iterations set to 12. And evaluating the experimental result through the peak signal-to-noise ratio, the structural similarity and the characteristic similarity.
In the experimental model based on the MAP and the non-local low-rank prior, the experimental parameter setting is optimized through different types of comparison experiments. Research shows that the size of the image block not only affects the running speed of the experiment, but also affects the precision of image registration and the quality of a reconstructed image. The block sizes of the experiments are respectively set to be 3 × 3, 5 × 5, 7 × 7, 9 × 9 and 11 × 11, the iteration times are all set to be 12 times, and the experimental PSNR, SSIM and FSIM are shown in table 1.
TABLE 1 Effect of image Block size on FSIM, SSIM
The numerical statistical indexes of the image blocks passing through 3 × 3 and 5 × 5 are obviously higher than those of the rest groups of experimental results. The image blocks are set to be 7 × 7, 9 × 9 and 11 × 11, compared with 5 × 5, the difference of the numerical statistical indicators of the PSNR values is small, but the algorithm operation efficiency increases with the increase of the image block size. The experimental results of table 1, the 11 × 11 image block only performed well in SSIM; wherein, the 3 × 3 and 5 × 5 image blocks have a maximum difference of only 0 in the FSIM three-term comparison. 0020, but in the SSIM comparison, 5 × 5 is significantly higher than 3 × 3, with three differences of 0.0330, 0.0214, 0.0160. In summary, in the experiment, the size of the image block is 5 × 5 as the best parameter setting, and the difference between the running time and the running time is smaller than that of 3 × 3, which is greatly reduced compared with 7 × 7. The quality of the reconstructed image can be effectively improved by blocking the image as seen from the objective evaluation index and the visual effect of the reconstructed image.
In the experiment, the iteration times of the regular term are set to be 12, and different influences on the experiment result can be caused by different iteration times. As shown in fig. 3, the impact of different iterations on PSNR, SSIM, FSIM was compared using medical lung images (CT) and nature images (City) as experimental data, where the arrows point to data points with 12 iterations. With the increase of iteration times, the PSNR, FSIM, SSIM values tend to be stable, but it can be seen from the PSNR curve that after 12 times of experiment results, both groups of experiments have small fluctuation in each index, and the PSNR curve has a relatively obvious downward trend compared with the FSIM, SSIM change curves, because in the reconstruction-based super-resolution method, image solving is a typical ill-defined inverse problem, and interference of other minor factors such as noise is introduced in the iterative solving process. Therefore, the experiment iteration times are set to be 12 times, so that the interference caused by solving the ill-conditioned problem is avoided, and meanwhile, the disturbance of noise and other tiny factors to the experiment can be effectively inhibited by using low-rank truncation in the process of solving the similar block.
The invention relates to constrained research on a maximum posterior prior probability item. The different regularization terms have different functions in a reconstruction-based method, and in the research on a two-norm Tikhonov regularization term, a one-norm total variation regularization term (L1TV), a two-norm total variation regularization term (L2TV), a low-rank and total variation-based regularization term (LRTV) and a non-local low-rank regularization term (MAP _ NLR) of the invention, the test is respectively carried out under the conditions that the iteration times are 6, 8, 10, 12 and 14 (for the sake of simple expression, the iteration times are equal to 12 as an example).
By researching the comparison experiment of the Tikhonov regular term, the first norm TV regular term, the second norm TV regular term, the LRTV regular term and the MAP _ NLR regular term, a reference image in the image super-resolution reconstruction process is used for testing. Through experiments, the invention selects the most representative example, if the regular term is Tikhonov, the value of the coefficient of the regular term is 0.01; if the value is the L1TV regular term, the value is 0.005; if L2TV, the value is 0.05; if the LRTV regular terms are adopted, the values are all 0.01; MAP _ NLR regular term, with 0.1 as the test value. The respective best experimental results are selected, a comparison experiment is carried out, and the peak signal-to-noise ratio of the regular term of the invention is observed, as shown in table 2:
TABLE 2 PSNR comparison (iteration number equal to 12)
From the experimental results in Table 2, it can be seen that the PSNR results of MAP _ NLR and LRTV on the lung images are better than those of L1TV, L2TV and Tikhonov. The PSNR value of the MAP _ NLR on the urban natural image is superior to that of the other four regular terms, which shows that the existence of low-rank prior is the premise of better model effect; in the PSNR value comparison of the two test images, MAP _ NLR is superior to LRTV, and the existence of non-local low-rank prior is a core element of the model.
Fig. 4 is a practical result of selecting a lung medical image and a city nature image as test images, and although the overall image is not very different, it can be seen that the difference is more reflected in the image details by local display. According to the commonly selected image areas, the detail description capability of MAP _ NLR, LRTV and Tikhonov is obviously higher than that of L1TV and L2 TV. Compared with LRTV and Tikhonov, MAP _ NLR has clear texture and does not generate a heavier fuzzy edge phenomenon, which proves the necessity of similar block grouping and the effectiveness of the adopted low-rank truncation method in suppressing noise from the aspect of image vision. In addition, although the image processed by Tikhonov has relatively low PSNR, the detail restoration capability is rather poor and better than that of L1TV and L2 TV.
The method compares a model with Nearest Neighbor Interpolation (Nearest Neighbor Interpolation), Bicubic (Bicubic), a low rank and total variation regularization based method (LRTV) and an image super-resolution processing result based on a two-norm quadratic term analysis algorithm (L2-L2), wherein in the comparison experiment, parameters are unified, images used by the model for the experiment are the same images in a continuous image sequence, and the iteration number is set as 12. The experimental hardware facilities were the same and were Intel (R) Xeon (R) E5-2643 v4@3.40GHz CPU, NVIDIA GeForce GTX 1080M GPU,256GB memory, and the operating system was ubuntu 14.04. The experimental results are visually compared by respectively calculating corresponding PSNR, SSIM and FSIM.
The results of the PSNR, SSIM and FSIM calculation comparing the method of the present invention with the 4 methods under the complete data set of the present invention are shown in Table 2.
The PSNR is one of important indexes for objectively evaluating the image quality, and the value of the PSNR can greatly judge the effectiveness of the super-resolution reconstruction effect of the algorithm. While SSIM and FSIM may form supplementary references to PSNR. As can be seen from Table 3, in the image data set used in the present invention, among the 4 methods used in the comparative experiment, the PSNR of L2-L2 has relatively high calculated values, and the SSIM and FSIM of LRTV have relatively high calculated values, which proves that the low-rank property of the image can reconstruct the image details well.
Besides the experimental results of comparing all data sets, the invention also lists PSNR, SSIM and FSIM of lung medical images, city natural images and small orchard natural images respectively and independently. The comparison results of PSNR, SSIM, and FSIM of the liver image and lung image medical data sets, city image, orchard image, and flower image natural data sets, with the image sizes of 128 × 128, are shown in table 3. The index values of the experimental results of the image data sets corresponding to different experimental methods are PSNR, SSIM and FSIM from top to bottom in sequence.
As shown in the graph 3, images obtained by performing image super-resolution by using different algorithms are shown in fig. 5, 6 and 7, and it can be seen that the PSNR average value of all the images differs by 0.17dB, compared with liver image data, the PSNR average value of all the images is 1.91dB, 0.0270 and 0.0227, but the PSNR average value, the SSIM average value and the FSIM average value of all the images are included in lung images, city images, orchard images and small orchard images, and the SSIM average value and the FSIM average value of all the images. As shown in fig. 5, fig. 6 and fig. 7, when the images are locally compared, the method of the present invention has great advantages in both the image reconstruction of the lung image and the super-resolution reconstruction of the natural image, and can better maintain the image details without generating blurring artifacts and aliasing effects. By comparing the indexes of visual effect and image quality of the reconstructed image, the method disclosed by the invention is superior to other four methods.
TABLE 3 PSNR, SSIM and FSIM comparison of different algorithm images for super resolution
Fig. 5 is a lung image selected from the experimental data set, the result graph obtained in 5 reconstruction models including the invention, the original image is located at the leftmost side, the group Truth is located at the rightmost side of the image, and the rest are the arrangement of super-resolution methods, and the last method (located at the left side of the group Truth image) in the verification method is the method of the invention. The topmost reconstructed image is locally displayed, and the method is closest to the Ground Truth in visual effect. According to the images, the LRTV and the method are most stable, the image fluctuation is not large, but the nearest neighbor method and the bicubic algorithm are in an unstable state, and particularly the result sawtooth effect obtained by the nearest neighbor method is obvious. The local display of the resulting image by the L2L2 method showed a significant blurring effect, which is almost absent in the present method and LRTV, indicating that L2L2 is not effective in preserving image details. LRTV is the closest to the method of the present invention, but tends to blur the image texture, causing blurring of the image edges. It can be seen from the error result diagram at the bottom end that the method of the invention has small difference with the original image, and the reconstructed image has high quality and is extremely stable.
Natural images contain a large amount of detail information, and the effective reconstruction of image high-frequency information is particularly important to image quality. As can be seen from fig. 6 and 7, the nearest neighbor method and the bicubic method cannot effectively restore the high-frequency information of the image, which further affects the quality of the reconstructed image, and causes edge texture blurring and blocking artifacts. By observing the local display of the natural image, the situation that the LRTV has coarsened edge textures can be seen, and the situation that the low-rank regularization and the total variation regularization have a superposition effect on the realization of detail preservation and the image details cannot be effectively restored is shown. The L2L2 method generates granular artifacts at high-frequency detail parts of an image, and cannot effectively store image information of a smooth area, so that a blurring effect is generated at texture edges, noise interference cannot be inhibited, and the definition of a reconstructed image is poor. Compared with the four methods, the method can better retain the image details, can effectively inhibit the interference of noise and other tiny factors in the reconstruction process of the image smooth area, and further improves the quality of the reconstructed result image.
Based on the scheme of the invention, a maximum posterior frame is improved, and the image super-resolution reconstruction is carried out by using a non-local low-rank prior as a regularization term. Continuous image sequences are adopted as data input, non-local self-similarity of images is used as priori knowledge, block matching is carried out on similar blocks by combining an image block local grouping technology, and the spatial structure relation of image pixel levels is fully excavated. Secondly, in the MAP framework of the invention, the singular value of the block to be solved is estimated through the maximum posterior of the singular value of the similar block, and the interference of noise and other small factors is inhibited by adopting a low-rank truncation method. And finally, a non-local low-rank constraint regularization image reconstruction process is adopted, local and global information in a single image and between continuous images is fully utilized, and the quality of the target high-resolution image is improved. The model of the invention adopts a plurality of evaluation indexes, such as peak signal-to-noise ratio, structural similarity and the like, and shows advantages. Experimental results show that the super-resolution model has good robustness in image super-resolution reconstruction. However, some narrow details are lost during low rank truncation. Researches show that the residual has a good effect on maintaining details, and how to add the residual into a model and further maintain image details is the next working focus.
Based on the method, the invention also provides equipment for realizing the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior, which comprises the following steps:
a memory for storing a computer program and an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low rank prior;
a processor for executing the computer program and the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior to realize the steps of the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior.
The invention further provides a readable storage medium with an image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior, wherein the readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the image super-resolution reconstruction method based on the maximum posterior probability and the non-local low-rank prior.
The apparatus for implementing the image super-resolution reconstruction method based on maximum a posteriori probability and non-local low rank prior is the unit and algorithm steps of the examples described in connection with the embodiments disclosed in the present invention, which can be implemented in electronic hardware, computer software, or a combination of both, and in the above description the components and steps of the examples have been generally described in terms of functions in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Through the above description of the embodiments, those skilled in the art can easily understand that the apparatus for implementing the image super-resolution reconstruction method based on maximum posterior probability and non-local low rank prior described herein can be implemented by software, and can also be implemented by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the disclosure for implementing the image super-resolution reconstruction method based on maximum a posteriori probability and non-local low rank prior may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the indexing method according to the embodiments of the disclosure.
Those skilled in the art will appreciate that various aspects of implementing a maximum a posteriori probability and non-local low rank prior based image super-resolution reconstruction method may be implemented as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. An image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior is characterized by comprising the following steps:
the method comprises the following steps that firstly, a continuous image sequence is used as data input, the similarity between a single image and a continuous image is used as priori knowledge, block matching is carried out on similar blocks by combining a local grouping mode of image blocks, and the spatial structure relation of image pixel levels is mined;
modeling by using a maximum posterior probability frame, and fitting model parameters by using Gaussian distribution and Gibbs distribution to improve the generalization capability of the model;
estimating the singular value of the block to be solved through the maximum posterior of the singular value of the similar block, and inhibiting noise interference by adopting a low-rank truncation mode;
and step four, adopting a non-local low-rank constraint regularization image reconstruction process, and then utilizing local information in a single image and local information between continuous images to improve the quality of the target image.
2. The image super-resolution reconstruction method according to claim 1,
the fourth step further comprises the following steps:
and evaluating the quality of the reconstructed image by adopting the peak signal-to-noise ratio, the structural similarity and the characteristic similarity.
3. The image super-resolution reconstruction method according to claim 1,
the first step further comprises the following steps:
the sequence of successive images is Y { Y }k1, 10; the continuous image sequence is composed of a series of low-resolution images, super-resolution reconstruction is carried out by taking an image y in the middle of the sequence as a reference image, and other images are cooperatively assisted;
the known image degradation model is:
y＝DBkx+nk(1)
in the formula, x is a high-resolution image to be reconstructed, y is an input low-resolution image and is obtained by carrying out bicubic interpolation and amplification on an original image;
d is a downsampling operator, BkIs a fuzzy operator; suppose nkIs a mean of 0 and a variance of
Additive white gaussian noise of (1);
in the second step, the first step is carried out,
the purpose of super resolution is to reconstruct a high resolution image x from a low resolution reference image y; expressed by the MAP model as the following objective function:
fitting is performed by using a Gaussian function in the formula (2),
assuming that the pixel values are related to neighboring pixels that satisfy a gibbs probability density function, a gibbs function fit is used,
substituting the formula (3) and the formula (4) into the formula (2) to obtain:
equation (5) is the objective function to be solved.
4. The image super-resolution reconstruction method according to claim 1,
the fourth step also comprises:
using a non-local low-rank prior, and expressing the non-local low-rank prior as a maximum posterior probability estimation;
suppose that
Representing image blocks xjWherein j is an image block index, and the similar blocks are formed with a size of j as a center
The image block of (1);
when the similar blocks are matched, selecting the most similar p similar blocks in the whole sequence image, and then performing low-rank truncation processing;
assuming that each set of low rank blocks is independent of each other, the low rank prior is expressed as:
substituting formula (6) for formula (5) to obtain:
typically the low rank matrix is solved with a kernel paradigm,
wherein L isjIs on demand
Low rank block, | Lj||*Is LjA kernel norm of (a) to represent the sum of singular values; solving the formula (8) by constructing an enhanced Lagrange equation by using an iteration direction multiplier;
in the formula of UjIs a lagrange multiplier, μ is a constant parameter; solving equation (9) can be decomposed into two sub-problems:
wherein, solving for x can be directly solved by the formula (10):
solving low-rank block L by adopting MAP methodjBy using
Estimating L from the singular values ofjTo obtain Lj；
Equation (12) can be derived from bayesian criteria,
assuming that the degree of torsion f represents the degree of torsion of the singular value of the high-resolution image block and the singular value of the low-resolution image block;
in the equation (13), the first part is fitted with a gaussian function having a mean value of 0 and a standard deviation of f,
P(σi(Lj) Calculated using kernel density estimates, whose probability density function is assumed to be the sum of a series of kernel functions;
the sum of singular values is the sum of a series of kernel functions defined by σi(Lj) Centered 1 × 3 neighborhood ΩiDetermining;
assuming that the mean of the kernel function is in line with the mean
Standard deviation of hiThe probability density function of the singular values is defined as:
the ith index in the formula (14) and the formula (15) is taken into the formula (13),
let the derivative of equation (16) be 0, solve for:
re-averaging the resulting total MAP estimates:
finally obtaining a low-rank image similar block LjIs estimated by the estimation of (a) a,
enhanced lagrange multiplier UjCan be updated by means of the formula (20),
5. the image super-resolution reconstruction method according to claim 2,
the evaluation mode of the peak signal-to-noise ratio comprises the following steps:
the peak signal-to-noise ratio calculation formula is as follows:
wherein MSE is the mean square error between the image to be evaluated and the reference image; g is the image gray level number;
the larger the PSNR value is, the smaller the difference between the image to be evaluated and the reference image is, and the higher the image quality is.
6. The image super-resolution reconstruction method according to claim 2,
the evaluation mode of the structural similarity comprises the following steps:
the structural similarity calculation formula is as follows:
wherein, muxAnd muyRespectively the mean value of the gray levels, sigma, of the image to be evaluated and the reference imagexAnd σyDenotes the standard deviation, C1＝(k1G)2、C2＝(k2G)2As constants to maintain numerical stability, in which k is1＝0.01,k2G is 0.03, and G is the number of image gray levels.
7. The image super-resolution reconstruction method according to claim 2,
the evaluation mode of the feature similarity comprises the following steps:
the feature similarity calculation formula is as follows:
wherein S isL(x)＝SPC(x)·SG(x),SPC(x) And SG(x) Values of PC and GM between images, respectively; PC (personal computer)m(x) The PC value is the maximum PC value and is used for weighting the contribution of each point to the overall similarity of the two images; x is the position of a given pixel point and Ω is the entire airspace of the image.
8. An apparatus for implementing a maximum a posteriori probability and non-local low rank prior based image super-resolution reconstruction method, comprising:
a memory for storing a computer program and an image super-resolution reconstruction method based on maximum a posteriori probability and non-local low rank prior;
processor for executing said computer program and the maximum a posteriori probability and non-local low rank prior based image super resolution reconstruction method for realizing the steps of the maximum a posteriori probability and non-local low rank prior based image super resolution reconstruction method according to any of the claims 1 to 7.
9. Readable storage medium having a maximum a posteriori probability and non-local low rank prior based image super resolution reconstruction method, characterized in that the readable storage medium has stored thereon a computer program which is executed by a processor for performing the steps of the maximum a posteriori probability and non-local low rank prior based image super resolution reconstruction method according to any of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010568221.4A CN111724307B (en) | 2020-06-19 | 2020-06-19 | Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010568221.4A CN111724307B (en) | 2020-06-19 | 2020-06-19 | Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111724307A true CN111724307A (en) | 2020-09-29 |
| CN111724307B CN111724307B (en) | 2021-08-20 |
Family
ID=72568203
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010568221.4A Active CN111724307B (en) | 2020-06-19 | 2020-06-19 | Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111724307B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204869A (en) * | 2021-04-28 | 2021-08-03 | 桂林电子科技大学 | Phase unwrapping method based on rank information filtering |
| CN113627484A (en) * | 2021-07-09 | 2021-11-09 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
| CN114170081A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Three-dimensional medical image super-resolution method based on non-local low-rank tensor decomposition |
| CN114170080A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Image super-resolution reconstruction system based on non-convex low-rank tensor approximation |
| CN114820387A (en) * | 2022-05-27 | 2022-07-29 | 山东财经大学 | Image recovery method and terminal based on probability induced kernel norm minimization |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
-
2020
- 2020-06-19 CN CN202010568221.4A patent/CN111724307B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025632A (en) * | 2017-04-13 | 2017-08-08 | 首都师范大学 | A kind of image super-resolution rebuilding method and system |
Non-Patent Citations (4)
| Title |
|---|
| HE YU,YAP KIMHUI,CHEN LI,CHAU LAPPUI: "Blind Super-Resolution Image Reconstruction using a", 《2006 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING》 * |
| LEI ZHU,CHI-WING FU,MICHAEL S BROWN,PHENG-ANN HENG: "A Non-local Low-Rank Framework for", 《IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION,》 * |
| 耿凤欢: "基于光流估计的肺部4D_CT图像超分辨重建", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 许静: "基于MAP技术图像超分辨率重建的研究", 《中国优秀硕士学位论文全文数据库信息科技辑 2009年第02期》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204869A (en) * | 2021-04-28 | 2021-08-03 | 桂林电子科技大学 | Phase unwrapping method based on rank information filtering |
| CN113204869B (en) * | 2021-04-28 | 2023-06-23 | 桂林电子科技大学 | Phase unwrapping method based on rank information filtering |
| CN113627484A (en) * | 2021-07-09 | 2021-11-09 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
| CN113627484B (en) * | 2021-07-09 | 2022-11-18 | 西安理工大学 | Hyperspectral image classification method based on local-non-local uncertainty analysis |
| CN114170081A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Three-dimensional medical image super-resolution method based on non-local low-rank tensor decomposition |
| CN114170080A (en) * | 2021-11-26 | 2022-03-11 | 中国科学院沈阳自动化研究所 | Image super-resolution reconstruction system based on non-convex low-rank tensor approximation |
| CN114170081B (en) * | 2021-11-26 | 2024-08-02 | 中国科学院沈阳自动化研究所 | Three-dimensional medical image super-resolution method based on non-local low-rank tensor decomposition |
| CN114820387A (en) * | 2022-05-27 | 2022-07-29 | 山东财经大学 | Image recovery method and terminal based on probability induced kernel norm minimization |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111724307B (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111724307B (en) | Image super-resolution reconstruction method based on maximum posterior probability and non-local low-rank prior, terminal and readable storage medium | |
| Trinh et al. | Novel example-based method for super-resolution and denoising of medical images | |
| US9652830B2 (en) | Method and apparatus for performing hierarchical super-resolution of an input image | |
| Chen et al. | Single image super-resolution via adaptive transform-based nonlocal self-similarity modeling and learning-based gradient regularization | |
| Cao et al. | Image Super-Resolution via Adaptive $\ell _ {p}(0< p< 1) $ Regularization and Sparse Representation | |
| CN102800071A (en) | Method for reconstructing super resolution of sequence image POCS | |
| CN106600533B (en) | Single image super resolution ratio reconstruction method | |
| Zhang et al. | MR image super-resolution reconstruction using sparse representation, nonlocal similarity and sparse derivative prior | |
| Akyol et al. | Super-resolution reconstruction of faces by enhanced global models of shape and texture | |
| Liu et al. | Video frame interpolation via optical flow estimation with image inpainting | |
| Li et al. | Space–time super-resolution with patch group cuts prior | |
| Liu et al. | Image super-resolution via hierarchical and collaborative sparse representation | |
| Zhai et al. | Image reconstruction from random samples with multiscale hybrid parametric and nonparametric modeling | |
| Wen et al. | The power of complementary regularizers: Image recovery via transform learning and low-rank modeling | |
| Diana Earshia et al. | A guided optimized recursive least square adaptive filtering based multi-variate dense fusion network model for image interpolation | |
| Cheng et al. | Classification-based video super-resolution using artificial neural networks | |
| Baraha et al. | Speckle removal using dictionary learning and pnp-based fast iterative shrinkage threshold algorithm | |
| Pei et al. | Face hallucination via gradient constrained sparse representation | |
| Han et al. | Cluster-based image super-resolution via jointly low-rank and sparse representation | |
| Tuli et al. | Structure preserving loss function for single image super resolution | |
| CN114757826A (en) | POCS image super-resolution reconstruction method based on multiple features | |
| Zhou et al. | Low-rank modeling and its applications in medical image analysis | |
| CN114219714A (en) | Super-resolution reconstruction method based on camera array depth estimation | |
| CN106780331B (en) | Novel super-resolution method based on neighborhood embedding | |
| Wang et al. | Video super-resolution using edge-based optical flow and intensity prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |