CN110097499B - Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression - Google Patents
Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression Download PDFInfo
- Publication number
- CN110097499B CN110097499B CN201910194123.6A CN201910194123A CN110097499B CN 110097499 B CN110097499 B CN 110097499B CN 201910194123 A CN201910194123 A CN 201910194123A CN 110097499 B CN110097499 B CN 110097499B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- super
- training
- training data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
- G06T3/4076—Super resolution, i.e. output image resolution higher than sensor resolution by iteratively correcting the provisional high resolution image using the original low-resolution image
Abstract
The invention belongs to the technical field of image processing, and discloses a single-frame image super-resolution reconstruction method and a system based on spectrum mixing kernel Gaussian process regression, wherein a high-resolution image is selected to form a high-resolution training image set, and downsampling and interpolation amplification operations are performed on elements of the high-resolution training image set to obtain an interpolation training image set; extracting features to form a training data set, and clustering the training data set to obtain a training data subset; training optimal super parameters of a Gaussian process regression model based on a spectrum mixing kernel for each training data subset; reading the low-resolution test image and constructing a low-resolution test data set; searching a cluster center closest to each feature in the low-resolution test data set in the cluster center to form a nearest neighbor training data subset; and carrying out Gaussian process regression to obtain a high-resolution feature set and outputting a high-resolution image. The invention ensures that the reconstructed image contains more high-frequency details, has clear texture structure and improves the reconstruction effect.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression.
Background
In real world production and life, the resolution of the acquired image may be low due to limited performance of the imaging system or limitations of imaging conditions, resulting in insufficient detail performance. There is therefore a strong real demand for obtaining high resolution images from low resolution images. The image super-resolution reconstruction method is to process the existing low-resolution image by a software method to obtain a corresponding high-resolution image so as to meet the actual needs. This technology plays an important role in the field of image processing and has received a great deal of attention.
Currently, image super-resolution reconstruction methods can be divided into three categories: interpolation-based methods, learning-based methods, and reconstruction-based methods.
Interpolation-based methods are the most common methods, including nearest neighbor interpolation, linear interpolation, and bicubic interpolation. The interpolation-based method is to input known pixels adjacent to unknown pixels in a high-resolution image grid to be reconstructed into a preset interpolation kernel function to estimate the unknown pixels, so that the purpose of improving the image resolution is achieved. The method has simple principle and small data volume, and realizes rapidness. However, in mathematical principle, the kernel function preset by the method cannot accurately reflect the real image texture structure, often results in lack of details of the reconstruction result, and may generate a jaggy effect at the image edge. A better refinement is to estimate the high resolution image using a kernel function that better embodies the image details.
The reconstruction-based method mainly comprises a non-uniform interpolation method, an iterative back projection method, a maximum posterior probability method, a convex set projection method and the like. The method takes a degradation model of an image as priori knowledge, comprises the processes of blurring, downsampling, noise superposition and the like, and seeks the minimization of errors between an initial reconstructed image and an input low-resolution image after the degradation model by adopting different constraint conditions, so that a final high-resolution reconstructed image is estimated. In mathematical principle, the reconstruction process of an image belongs to the solving process of an inverse problem, which is a typical ill-posed problem, and thus the existence and uniqueness of the solution cannot be guaranteed. Therefore, in the implementation process, the method has the problems that the iteration process is not easy to converge, the reconstructed image has one-to-many effect and the like. A better approach is to discard the simplistic degradation model, but to use a mathematical description that more closely approximates the actual degradation process.
The super-resolution reconstruction method based on the learning generally collects a large number of images and establishes a training set; then, the training set is learned to obtain the corresponding relation (or mapping model) between the high and low image blocks; then, the information (or pixel value) required by the high resolution image is predicted by inputting the low resolution image and combining the mapping model, and the high resolution image is reconstructed. In such methods, learning from a training set to obtain a mapping model is a key element, and similarity between image blocks and their features, or distance, is also a core for building a model. Existing learning-based methods mostly use mathematical tools such as euclidean distance, angular distance, etc. to measure similarity. However, these mathematical tools are too simple and direct, and cannot fully represent the logical relationship between image blocks, but may introduce additional information unrelated to the image blocks, resulting in insufficient learning ability of the mapping model, unreasonable distribution of reconstruction weights, and poor reconstruction effect due to fuzzy texture structure of the final reconstructed image. Therefore, in the super-resolution reconstruction method of the image based on learning, the key is to find a proper mathematical tool to characterize the similarity between the features, so as to accurately estimate the weight of each image block in the training set during reconstruction and search for an optimal weight value.
Several typical learning-based approaches exist, the shortcomings of which are represented by: yang et al in documents J.Yang, J.Wright, T.Huang and Y.Ma, "Image super-resolution via sparse representation," IEEE Trans.image Process., vol.19, no.11, pp.2861-2873, learn a dual dictionary by a joint optimization framework using sparse representation theory, and then super-resolution reconstruct an input low-resolution Image using the dictionary. The method adopts Euclidean distance to search the optimal feature combination in the low-resolution dictionary during reconstruction so as to obtain an optimal solution based on sparse representation. However, the actual reconstruction effect is of low image quality and there is edge blurring.
He et al learn dual spatial dictionaries using Beta procedures in documents l.he, h.qi, and r.zaretzki, "Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution," in proc.ieee conf. Comp.vis. Pattern recording, 2013, pp.345-352, to reconstruct high resolution image blocks. However, the method also sets the distance between the image features as Euclidean distance, and has the problems of edge distortion of the reconstructed image, fuzzy texture details and the like.
He et al in the literature H.He and W.C.Siu, "Single image super-resolution using Gaussian process regression", in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.,2011, pp.449-456, introduced Gaussian process regression into the field of image super-resolution reconstruction, and built Gaussian process regression models in local areas of images using self-similarity of natural images. However, the method uses a simple radial basis kernel as a kernel function, namely, a square index form of Euclidean distance, and has the problem that the description of the structural similarity between image blocks is insufficient, so that the image reconstruction effect is general and the detail presentation is insufficient.
Wang et al, in document H.Wang, X.Gao, K.Zhang, and J.Li, "Single-Image Super-Resolution Using Active-Sampling Gaussian Process Regression", IEEE transactions on Image processing, vol.25, no.2, february 2016, pp.935-947, uses active sampling to reduce the external training set and Gaussian process regression modeling of the high and low resolution features of the images in the training set to reconstruct high resolution images. However, the kernel function involved in the method is a linear kernel function, the essence of the kernel function is that dot products among vectors are calculated, the mathematical principle is too single, the capability is insufficient when the similarity of more complex image blocks is represented, and therefore the adaptability of a model is reduced, the image reconstruction effect is not ideal, and the texture structure is fuzzy.
In addition, a deep learning method has been introduced into the field of image super resolution in recent years. The method benefits from massive training data quantity, and a better image reconstruction effect is obtained. However, the huge data volume also brings problems in aspects of storage, management, processing, transmission and the like, such as the defects of overhigh cost of hardware equipment, overlong training time of a method and the like; in addition, in the mathematical principle, the method has the bottlenecks of opaque model, difficult parameter adjustment and the like. Therefore, the super-resolution reconstruction method of the deep learning image is limited in practical application and has not been widely used.
In summary, it can be seen that the model adopted by the existing method, or the mathematical description of the degradation process, is too simple to accurately reflect the texture structure of the image, so that the reconstruction effect is not ideal. The better method is to describe the image by referring to a Gaussian process regression model, so that the hidden nonlinear relation in the image block can be embodied, and the information in the image block can be discovered as much as possible to reduce false information. On the other hand, the Euclidean distance and angle distance equidistant functions commonly used in the existing method have single mathematical principle and poor characterizability, and cannot accurately reflect the logic relationship and the texture structure similarity between image blocks, so that the model learning capability is insufficient, the weight distribution is unreasonable during reconstruction, and the quality of the reconstructed image effect is poor. Therefore, a better method is to introduce a new kernel function (distance function) into the Gaussian process regression model to represent the distance between the image blocks and measure the similarity between the image blocks, so that the learning capacity of the model is integrally improved, and finally, the higher reconstructed image quality is obtained.
In short, the present invention will address the existing technical problems:
(1) The Gaussian process regression model is adopted to describe the image, so that the problems that in the existing method, mathematical description of a degradation process is too simple, a mapping model corresponding to high-low resolution features cannot accurately reflect the texture structure of the image, and therefore the reconstruction effect is not ideal are solved.
(2) The spectrum mixing kernel function is introduced as the kernel function of the Gaussian process, so that the problems that the existing method is poor in characterization, the similarity of logical relations and texture structures among image blocks cannot be accurately reflected, the model learning capability is insufficient, the weight distribution is unreasonable during reconstruction, the quality of the reconstructed image effect is poor and the like caused by using a simple distance function are solved.
When solving the existing technical problems, the invention has the technical difficulties that:
(1) The selection of the number K of training data subsets during learning is a difficulty that directly affects the learning effect of the training set. In the simulation experiment part of the invention, the change rule of the K value is shown by a curve (as shown in fig. 5), and the selection of the K value is described in detail.
(2) Solving the optimal super-parameter θ of the gaussian process regression model is a difficulty. The super parameter theta is a vector, and each dimension needs to be calculated to obtain an optimal value through an iterative algorithm. And step three, the solving process is described in detail.
Meaning of solving the technical problems:
(1) In terms of technical functions, the method introduces the spectrum mixing kernel function into the Gaussian process regression model, so that the super-resolution reconstructed image has clear structure and fine texture, is closer to a real image, has higher evaluation index, and integrally improves the algorithm performance.
(2) From the practical application aspect, the invention can be directly applied but is not limited to the following aspects: 1) Medical image processing: providing high-quality medical images, helping doctors to improve diagnosis efficiency; 2) Monitoring video: the definition of the monitoring video and the image is improved, and the viewing and the information extraction of related personnel are facilitated; 3) Television image: the resolution and visual effect of television pictures are improved, and the user experience is improved; 4) Remote sensing image: the resolution of the target or the region of interest in the satellite remote sensing image is improved, and the subsequent work such as target detection, extraction and classification can be conveniently unfolded. Therefore, the invention can be directly applied in the fields of medical treatment, public security, consumption, scientific research and the like, and can be popularized to more fields, so that the invention has a very wide application prospect.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression. The invention refers to a Gaussian process regression model to describe the image, so that the hidden nonlinear relation in the image block can be embodied, and the information in the image block can be discovered as much as possible to reduce false information. The invention introduces a new kernel function to embody the structure of the image blocks, describe the similarity between the image blocks, promote the learning ability of the regression model, accurately estimate the reconstruction weight of the image blocks and finally achieve higher reconstruction image quality.
The invention is realized in such a way that a single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression is realized, wherein the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression selects a high-resolution natural image in a training stage, performs downsampling on the high-resolution natural image by using a bicubic interpolation method to obtain a low-resolution training image, and performs interpolation amplification on the low-resolution training image to obtain a corresponding interpolation image; partitioning the high-resolution image and the interpolation image, extracting features, and forming a training data set; clustering the training data sets to obtain K training data subsets; and learning the K training data subsets by using a spectrum mixing kernel function to obtain the optimal super parameter theta of the Gaussian process regression model. The similarity between the image blocks can be more accurately represented by using the spectrum mixing kernel function as a distance function under the Gaussian process regression model framework, so that the reconstruction weight distribution is more reasonable.
In the test stage, selecting a low-resolution image as an input image; and performing bicubic interpolation amplification on the input image to obtain an interpolation test image, and performing blocking and feature extraction to obtain a low-resolution test data set.
Searching the K cluster centers of the training data set for the cluster center nearest to each feature in the test data set to form a nearest training data subset, and carrying out regression by utilizing the optimal super parameter theta obtained by previous learning to obtain a high-resolution image feature set; and superposing the high-resolution image features on the interpolation test image to reconstruct a high-resolution result image. The Gaussian process regression model based on the spectrum mixing kernel can enable the super-resolution reconstructed image to be clear in structure, fine in texture, closer to a real image and higher in evaluation index.
Further, the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression specifically comprises the following steps:
selecting a high-resolution image to form a high-resolution training image set; performing downsampling and interpolation amplification operation on the high-resolution training image set element to obtain an interpolation training image set; extracting features to form a training data set C= { H, L }, wherein H and L fractions respectively represent high-resolution training data sets and low-resolution training data sets;
clustering the training data set C to obtain a training data subset
Corresponding K clustering centers;
step three, for each training data subset C i Training out optimal super-parameter theta of Gaussian process regression model based on spectrum mixing kernel i ，θ i Is a column vector;
step four, reading a low-resolution test image and constructing a low-resolution test data set X;
searching a cluster center closest to each feature in the low-resolution test data set X from the K cluster centers to form a nearest neighbor training data subset;
step six, utilizing the theta obtained in the training in the step three i High processReturning the process to obtain a high-resolution feature set F;
and step seven, outputting a high-resolution image Y.
Further, the first step specifically includes:
selecting a plurality of color high-resolution natural images and converting the color high-resolution natural images from an RGB color space to a YCbCr color space; selecting a brightness image to form a high-resolution training image set;
selecting the current super-resolution reconstruction magnification as S=3, downsampling the high-resolution training image set by S times through bicubic interpolation, and amplifying the high-resolution training image set by S times through bicubic interpolation to obtain an interpolation training image set;
selecting the size of the image block as N multiplied by N, wherein N=7 is an odd number, and taking the image blocks from pixel to pixel according to the sequence from top to bottom and from left to right for the images corresponding to the high-resolution image training set and the interpolation training image set; then, the difference value of the center pixel of the high-resolution image block and the low-resolution image block is obtained to obtain a high-resolution training data set 
Vectorizing the low resolution image block to obtain a low resolution training data set +.>
Wherein p=100000, P is smaller than the maximum number of blocks of the fetched image block;
the second step specifically comprises:
Step 2, finding out the corresponding high-resolution data in the high-resolution training data set H to form { H } 1 ,H 2 ,…,H K }；
Step 3, combining { L ] 1 ,L 2 ,…,L K And { H } and 1 ,H 2 ,…,H K formation { C } of 1 ,C 2 ,…,C K }, item i C therein i ＝{L i ,H i And is referred to as a training data subset.
Further, the third step specifically includes:
step A, defining super parameter theta i ；
Training data subset C i ＝{L i ,H i Comprises Z data pairs with log likelihood functions of
Wherein θ is i Is a super parameter to be learned, (H) i )、(L i ) Respectively C i High and low resolution training data subset H of (b) i 、L i Element of (1) |i 1 Representing 1-norm, (. Cndot. T Representing a transpose operation, K y (l,l'|θ i ) Representing covariance matrix with the formula
Wherein k (l, l') comprises the super parameter θ i The expression is:
wherein the method comprises the steps of
And omega q,i 、∑ q,i Sum mu q,i Respectively represent the weight, variance and frequency parameter corresponding to the q accumulation parameter in the ith training data subset,/for>
The method is characterized in that the method comprises the steps of representing a noise standard deviation parameter, Q=15 is a set accumulation parameter, I·I represents an absolute value taking operation, I·I represents a vector modular value taking operation, and the expression of a function COS (·) is as follows
Wherein l p And l' p The p-th component of vectors l and l', respectively, where μ q,i (p) represents mu q,i The p-th component of (b), U represents the dimension of vector l; the function delta (. Cndot.) is
And step B, solving the super parameters by using an iteration method.
Further, the step B specifically includes:
step a, constructing partial derivatives
In the middle of
tr (·) represents the trace of the matrix. />
The element in the middle position (s, t) is
In the middle of
Represents L i The s-th vector l of (3) s S, t=1, 2, …, Z; />
The element in the middle position (s, t) is
∑ q,i (j) Representing sigma q,i J=1, 2, …, U;
the element in the middle position (s, t) is +.>
Mu in the middle q,i (j) Representation mu q,i The j-th component of (b).
The element in the middle position (s, t) is
Step b, setting the current step as the kth step, wherein k is more than or equal to 3,
the iteration parameter values are updated and,
in the middle of
d k-1 ＝(g k-1 ) T s k-2 ；
p＝6(f k-1 -f k )+3(d k-1 +d k )(v k-1 -v k-2 )；
q＝3(f k -f k-1 )-(2d k-1 +d k )(v k-1 -v k-2 )；
In the middle of
If f k <f opt Order in principle
f k →f opt And continuing the iterative process; f in opt Representing the minimum likelihood value, θ, in a previous iteration process opt Representing optimal parameters in a previous iteration process;
step c, setting termination conditions: if it is
Or when the number of the loop steps reaches 500, terminating the iterative process, and theta i ＝θ opt For optimal super parameters, τ=0.1 and γ=0.05 are set control coefficients;
step d, initializing iteration variables:
initializing superparameters
Wherein the method comprises the steps of
In the formula, std(s) represents taking standard deviation operation, max(s) represents taking the largest element in the matrix, and min(s) represents taking the smallest element in the matrix; wherein R is i For a low resolution data subset L i Is expressed as the scale matrix of
Middle column vector m 1 ,…,m Z From the following components
L i -mean(L i )＝[m 1 ,m 2 ,…m Z ]；
Obtaining, mean (·) represents an average value calculation;
the initial values of the iteration variables include
d 0 ＝d 1 ＝-(s 0 ) T s 0 ；
f opt ＝f 0 ；
Further, in the fourth step, specifically, the method includes:
reading a low-resolution color test image, amplifying the color test image by S times by using bicubic interpolation, converting the color test image from an RGB color space to a YCbCr color space, and respectively obtaining a brightness channel image, a blue channel image and a red channel image;
pixel-by-pixel blocking and vectorizing the luminance channel image in a top-to-bottom, left-to-right order to obtain a low resolution test dataset
M represents the maximum number of image blocks taken, test data x n Represents the nth vector in X;
the fifth step specifically comprises:
calculate the mth vector X in X m To the cluster center set v= (o) 1 ,o 2 ,…,o K ) Is a Euclidean distance to obtain a distance vector
E (x, x ') represents the Euclidean distance between the calculated vectors x and x';
for S m Ordering the elements in (m=1, 2 …, M) and finding the smallest element, marking the correspondingClass number a of cluster center of (2) m ∈[1,K]；
Performing operation on all vectors in X to obtain a marked vector
In the formula, the mth element a m For a corresponding low resolution nearest neighbor training data subset
Class number of->
Is x m Is a nearest neighbor training data subset +.>
Is->
A corresponding high resolution nearest neighbor training data subset; />
Corresponding optimal superparameter->
Is x m Is the nearest neighbor super parameter of (a); />
The sixth step specifically comprises:
using the formula
Calculating x m Corresponding output feature y m Obtaining a high-resolution feature set
Is of class number a m A low resolution nearest neighbor training data subset of (a); />
Is->
A corresponding high resolution nearest neighbor training data subset; wherein the method comprises the steps of
Middle l 1 ,l 2 …,l M Is that
All vectors in (a); computing spectral mixture kernel function->
Used->
The nearest neighbor super parameter obtained in the fifth step;
the seventh step specifically comprises:
from formula y m ＝y m +cen(x m ) Calculating to obtain high-resolution image pixel points, replacing corresponding pixel points in the brightness channel image to obtain a brightness reconstruction image, and enabling cen (·) to represent center-taking pixel operation;
and combining the brightness reconstruction image with the blue channel image and the red channel image to obtain a super-resolution reconstruction image under the YCbCr color space, and converting the super-resolution reconstruction image into the RGB color space to obtain a high-resolution reconstruction image Y.
The invention further aims to provide a medical image processing system for implementing the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression.
The invention further aims to provide a monitoring video processing system for implementing the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression.
The invention further aims to provide a television image processing system for implementing the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression.
The invention further aims to provide a remote sensing image processing system for implementing the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression.
In summary, the invention has the advantages and positive effects that:
firstly, aiming at the problems of insufficient information utilization of a training set and poor quality of a reconstructed image caused by low characterization of a distance function in the existing super-resolution reconstruction method, a spectrum mixing kernel function is introduced into a Gaussian regression model, and the corresponding relation of high-resolution and low-resolution features in the training set is learned. The spectral mixing kernel function can more accurately represent the structural similarity between the image blocks, so that weight distribution is more reasonable during reconstruction. FIG. 6 shows the relationship between the obtained weight distribution and the image rotation angle when different kernel functions are used. As can be seen by comparing several curves: compared with other kernel functions, the weight obtained by the spectrum mixing kernel is uniformly distributed when corresponding to different rotation angles, the numerical value is relatively flat, and the distribution is most reasonable. Table 2 shows the PSNR contrast for the reconstructed image obtained by the algorithm of the present invention and the algorithm based on the other kernel functions: the PSNR value of the present invention is highest, indicating that the spectral mixing kernel used can improve the performance of the reconstructed image. Fig. 4 is a Head diagram reconstructed image illustration. As can be seen from fig. 4, the spectrum mixing kernel reconstruction output image block in the invention contains more high-frequency information, and the reconstructed image is more lifelike;
Second, fig. 3 is a view showing the reconstruction effect of the Butterfly graph. FIG. 3 illustrates that compared with other prior image super-resolution algorithms, the invention has clear texture structure of the reconstructed image, rich high-frequency information and vivid visual effect. Table 1 shows the comparison of the reconstruction effect evaluation indexes of the algorithm and other algorithms, and the PSNR value of the algorithm is the highest, which shows that the objective quality evaluation is the best.
Third, in practical application, the present invention can be directly applied, but is not limited to the following aspects: 1) In terms of medical image processing as shown in fig. 7: providing high-quality medical images, helping doctors to improve diagnosis efficiency; 2) In terms of monitoring video as shown in fig. 8: the definition of the monitoring video and the image is improved, and the viewing and the information extraction of related personnel are facilitated; 3) As shown in fig. 9 in the television image: the resolution and visual effect of television pictures are improved, and the user experience is improved; 4) In terms of remote sensing image processing as shown in fig. 10: the resolution of the target or the region of interest in the satellite remote sensing image is improved, and the subsequent work such as target detection, extraction and classification can be conveniently unfolded. Therefore, the invention can be directly applied in the fields of medical treatment, public security, consumption, scientific research and the like, and can be popularized to more fields, so that the invention has a very wide application prospect. In addition, the invention has low cost of the required hardware platform, simple and clear code module and can be applied to independent and mobile image processing platforms.
Drawings
FIG. 1 is a general flow chart of a single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression, which is provided by the embodiment of the invention; the figure includes a training phase and a testing phase.
Fig. 2 is a flowchart of a single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression provided by an embodiment of the invention.
Fig. 3 is a schematic diagram showing the comparison of the reconstruction effect of the single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression and other methods provided by the embodiment of the invention.
Fig. 4 is a schematic diagram showing the comparison of the reconstruction effect of a single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression and an image super-resolution method based on other kernel functions according to an embodiment of the present invention.
Fig. 5 is a graph comparing the reconstruction effect of the single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression with the image super-resolution method based on other kernel functions with the relation curve of the parameter K.
Fig. 6 is a graph comparing a reconstruction weight and a rotation angle of a single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression with an image super-resolution method based on other kernel functions according to an embodiment of the present invention.
Fig. 7 is a diagram of a medical image processing system according to an embodiment of the present invention, which provides a single-frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression.
Fig. 8 is a diagram of a surveillance video processing system according to an embodiment of the present invention, where the method is used for reconstructing a super-resolution of a single-frame image based on regression of a spectral mixture kernel gaussian process.
Fig. 9 is a diagram of a television image processing system for providing a single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to an embodiment of the present invention.
Fig. 10 is a diagram of a remote sensing image processing system for providing a single frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Aiming at the problems that the prior art may have the problems of blurred details, insufficient high-frequency information, missing texture structures, low reconstruction quality, limited application range and the like of a super-resolution reconstructed image due to the fact that a distance function with poor performance is used, the invention discloses a single-frame image super-resolution reconstruction algorithm based on spectrum mixing kernel Gaussian process regression, which is used for reducing the details blur, increasing the high-frequency information of the image and improving the quality of the reconstructed image. Meanwhile, the algorithm software and hardware are low in cost and can also run on a middle-low level hardware platform; the independent and mobile image processing platforms are equally suitable. The spectrum mixing kernel can accurately reflect the similarity between image blocks with complex texture structures, and information in a training set can be reasonably utilized, so that more high-frequency detail information is recovered, and the reconstruction effect of the image is improved.
In order to solve the above problems, the present invention will be described in detail with reference to specific embodiments.
As shown in fig. 1, in the single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression in the embodiment of the invention, in a training stage, a high-resolution natural image is selected, downsampled by a bicubic interpolation method to obtain a low-resolution training image, and then interpolated and amplified to obtain a corresponding interpolated image; partitioning the high-resolution image and the interpolation image, extracting features, and forming a training data set; clustering the training data sets to obtain K training data subsets; the K training data subsets are learned by utilizing a spectrum mixing kernel function to obtain the optimal super parameter theta of the Gaussian process regression model; in the test stage, selecting a low-resolution image as an input image; performing bicubic interpolation amplification on an input image to obtain an interpolation test image, and performing block segmentation and feature extraction on the interpolation test image to obtain a low-resolution test data set; searching the K cluster centers of the training data set for the cluster center nearest to each feature in the test data set to form a nearest training data subset, and then carrying out regression by utilizing the optimal super parameter theta obtained by previous learning to obtain a high-resolution image feature set; these high resolution image features are then superimposed onto the interpolated test image to reconstruct a high resolution reconstructed image.
As shown in fig. 2, the single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression in the embodiment of the invention specifically comprises the following steps:
s101, selecting a high-resolution image to form a high-resolution training image set; performing downsampling and interpolation amplification operation on elements of the interpolation training image set to obtain the interpolation training image set; the extracted features form a training dataset.
S102, clustering the training data set to obtain a training data subset and K corresponding clustering centers.
S103, training out optimal super parameters of a Gaussian process regression model based on a spectrum mixing kernel for each training data subset.
And S104, reading the low-resolution test image and constructing a low-resolution test data set X.
S105, searching a cluster center closest to each feature in the test data set X from the K cluster centers to form a nearest neighbor training data subset.
S106, carrying out Gaussian process regression by utilizing the optimal super parameters obtained by training in the step S103 to obtain a high-resolution feature set F.
And S107, outputting a high-resolution reconstruction image Y.
In step S101, the extracted feature forms a training dataset c= { H, L }, where H and L represent high and low resolution training datasets, respectively. In step S102, the training data subset is 
In step S103, each training data subset is C i Optimum super parameter theta i Wherein θ is i Is a column vector.
In the embodiment of the present invention, the specific operations in step S101 include:
step 1.1, selecting a plurality of color high-resolution natural images and converting the color high-resolution natural images from an RGB color space to a YCbCr color space; and selecting the brightness image to form a high-resolution training image set.
And 1.2, selecting the current super-resolution reconstruction magnification to be S=3, downsampling the high-resolution training image set by S times through bicubic interpolation, and amplifying the high-resolution training image set by S times through bicubic interpolation to obtain an interpolation training image set.
Step 1.3, selecting an image block with the size of N multiplied by N (N=7 is an odd number), and taking the image blocks from pixel to pixel according to the sequence from top to bottom and from left to right for the images corresponding to the high-resolution image training set and the interpolation training image set; then, the difference value of the center pixel of the high-resolution image block and the low-resolution image block is obtained to obtain a high-resolution training data set
Vectorizing the low resolution image block to obtain a low resolution training data set +.>
Where p=100000 (P is smaller than the maximum number of blocks of the fetched image block).
The step S102 specifically includes:
step 2.1, setting the class number as K=35, and carrying out K-means clustering on the low-resolution training data set L to obtain { L } 1 ,L 2 ,…,L K The corresponding cluster center set v= (o) 1 ,o 2 ,…,o K )。
Step 2.2, find the corresponding high resolution data in the high resolution training data set H to form { H ] 1 ,H 2 ,…,H K }。
Step 2.3, combining { L ] 1 ,L 2 ,…,L K And { H } and 1 ,H 2 ,…,H K formation { C } of 1 ,C 2 ,…,C K }, item i C therein i ＝{L i ,H i And is referred to as a training data subset.
The step S103 specifically includes:
step 3.1, defining the super parameter θ i 。
Set training data subset C i ＝{L i ,H i Comprises Z data pairs with log likelihood functions of
Wherein θ is i Is a super parameter to be learned, (H) i )、(L i ) Respectively C i High and low resolution training data subset H of (b) i 、L i Element of (1) |i 1 Representing 1-norm, (. Cndot. T Representing a transpose operation, K y (l,l'|θ i ) Representing covariance matrix with formula
Wherein k (l, l') comprises the super parameter θ i The expression is:
wherein the method comprises the steps of
And ω therein q,i 、∑ q,i Sum mu q,i Respectively represent the weight, variance and frequency parameter corresponding to the q accumulation parameter in the ith training data subset,/for>
Represents a noise standard deviation parameter, Q=15 is a set accumulation parameter, and |·| represents absolute value operation, the expression of the vector modulus value is defined as the function COS (·) as follows
Wherein l p And l' p The p-th component of vectors l and l', respectively, where μ q,i (p) represents mu q,i The p-th component of (b), U represents the dimension of vector l; the definition of the function delta (.) is
And 3.2, solving the super parameters by using an iteration method.
In a preferred embodiment of the present invention, step 3.2 specifically comprises:
step 3.2.1 construction of the partial derivative
Wherein the method comprises the steps of
tr (·) represents the trace of the matrix. />
The element in the middle position (s, t) is
Wherein the method comprises the steps of
Represents L i The s-th vector l of (3) s S, t=1, 2, …, Z. />
The element in the middle position (s, t) is
Wherein is sigma q,i (j) Representing sigma q,i J=1, 2, …, U.
The element in the middle position (s, t) is
Wherein mu q,i (j) Representation mu q,i The j-th component of (b).
The element in the middle position (s, t) is
Step 3.2.2, setting the current step as the kth (k is more than or equal to 3),
updating iteration parameter values, wherein
Wherein the method comprises the steps of
d k-1 ＝(g k-1 ) T s k-2
p＝6(f k-1 -f k )+3(d k-1 +d k )(v k-1 -v k-2 )
q＝3(f k -f k-1 )-(2d k-1 +d k )(v k-1 -v k-2 )
Wherein the method comprises the steps of
If f k <f opt Order in principle
f k →f opt And continuing the iterative process; wherein f opt Representing the minimum likelihood value, θ, in a previous iteration process opt Representing the optimal parameters in the previous iteration process.
Step 3.2.3, setting termination conditions: if it is
Or when the number of the loop steps reaches 500, terminating the iterative process, and theta i ＝θ opt I.e. the optimal super-parameters, where τ=0.1 and γ=0.05 are the set control coefficients.
Step 3.2.4, initializing iteration variables:
initializing superparameters
Wherein the method comprises the steps of
Where std (·) represents taking the standard deviation operation, max (·) represents taking the largest element in the matrix, and min (·) represents taking the smallest element in the matrix. Wherein R is i For a low resolution data subset L i Is expressed as the scale matrix of
Wherein the direction of alignmentQuantity m 1 ,…,m Z From the following components
L i -mean(L i )＝[m 1 ,m 2 ,…m Z ]
The result, wherein mean (·) represents the averaging operation.
Initial values of other iteration variables include
d 0 ＝d 1 ＝-(s 0 ) T s 0
f opt ＝f 0
In the embodiment of the present invention, step S104 specifically includes:
and 4.1, reading a low-resolution color test image, amplifying the low-resolution color test image by S times by using bicubic interpolation, converting the low-resolution color test image from an RGB color space to a YCbCr color space, and respectively obtaining a brightness channel image, a blue channel image and a red channel image.
Step 4.2, for the luminance channel image from top to bottom,pixel-by-pixel blocking in a left-to-right order and vectorizing it to obtain a low resolution test dataset
Where M represents the maximum number of image blocks that can be taken, test data x n The nth vector in X is represented.
The step S105 specifically includes:
step 5.1, calculating the mth vector X in X m To the cluster center set v= (o) 1 ,o 2 ,…,o K ) Is the Euclidean distance of (2) to obtain its distance vector
Where E (x, x ') represents the Euclidean distance between the calculated vectors x and x'.
Step 5.2, for S m Sorting the elements in (m=1, 2 …, M) and finding out the smallest element, marking the class number a of the corresponding cluster center m ∈[1,K]。
Step 5.3, performing the operations in step 5.1 and step 5.2 on all vectors in X to obtain a marker vector
Wherein the mth element a m For a corresponding low resolution nearest neighbor training data subset
Class number of->
Is x m Is a nearest neighbor training data subset +.>
Is->
A corresponding high resolution nearest neighbor training data subset; />
Corresponding optimal superparameter->
Is x m Is a nearest neighbor super parameter of (c).
The step S106 specifically includes:
step 6.1, using the formula
Calculating x m Corresponding output feature y m Obtaining a high-resolution feature set
Wherein->
Is of class number a m A low resolution nearest neighbor training data subset of (a); />
Is->
A corresponding high resolution nearest neighbor training data subset; wherein the method comprises the steps of
Wherein l 1 ,l 2 …,l M Is that
Is included in the vector. Computing spectral mixture kernel function->
Used->
The nearest neighbor super parameter obtained in the step 5.3 is obtained.
The step S107 specifically includes:
step 7.1, from equation y m ＝y m +cen(x m ) And calculating to obtain high-resolution image pixel points, and replacing the corresponding pixel points in the brightness channel image to obtain a brightness reconstruction image, wherein cen (DEG) represents a center-taking pixel operation.
And 7.2, combining the brightness reconstruction image with the blue channel image and the red channel image to obtain a super-resolution reconstruction image under the YCbCr color space, and converting the super-resolution reconstruction image into the RGB color space to obtain a high-resolution reconstruction image Y.
The invention is further described in connection with simulation experiments.
(1) Simulation conditions
The experiment of the invention is carried out under the experimental environment that the CPU is Intel i5-52203.30GHz, the memory is 6G, the operating system is Windows 10, and the simulation platform is Matlab 2015 a.
In a simulation experiment, the method is compared and analyzed with the existing SCSR, BPJDL, SRGPR, AGPR and other methods; wherein the method comprises the steps of
SCSR corresponding references J.Yang, J.Wright, T.Huang and Y.Ma, "Image super-resolution via sparse representation," IEEE Trans. Image Process, vol.19, no.11, pp.2861-2873.
BPJDL is referred to by the corresponding references L.He, H.Qi, and R.Zaretzki, "Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution", in Proc.IEEE Conf.Comput.Vis.Pattern recording, 2013, pp.345-352.
SRGPR is referred to by the corresponding references H.He and W.C.Siu, "Single image super-resolution using Gaussian process regression", in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.,2011, pp.449-456.
AGPR is referred to by the corresponding reference H.Wang, X.Gao, K.Zhang and J.Li "Single-Image Super-Resolution Using Active-Sampling Gaussian Process Regression", IEEE transactions on Image processing, vol.25, no.2, february 2016, pp.935-947.
(2) Emulation content
Experiment one: the invention has better super-resolution reconstruction effect.
Simulation tests were performed on the Butterfly image using the present invention and the above-described 4 conventional methods, and the results are shown in fig. 3. Wherein fig. 3 (a) is the result of the super-resolution reconstruction of the SCSR; FIG. 3 (b) is the result of the BPDJL super-resolution reconstruction; FIG. 3 (c) is the result of SRGPR super resolution reconstruction; FIG. 3 (d) is the result of AGPR super-resolution reconstruction; FIG. 3 (e) is the result of the super-resolution reconstruction of the present invention; fig. 3 (f) is a true high resolution image. Each image has a rectangular area that is locally enlarged to facilitate viewing differences in reconstruction effects.
The simulation results of fig. 3 show that other methods cannot recover the complex texture of the butterfly wing pattern in the image, but fig. 3 (e) shows a better reconstruction effect. Compared with the prior art, the super-resolution result of the invention can better reconstruct a complex texture structure, the detail is better presented, and the visual effect of the reconstructed image is better than that of other four methods.
Table 1. The algorithm of the invention is compared with the other four algorithms to reconstruct the effect index (Butterfly image, magnification S is 3)
| SCSR | BPDJL | SRGPR | AGPR | The invention is that | |
| PSNR(dB) | 25.63 | 26.44 | 25.66 | 26.47 | 26.54 |
Table 1 shows the PSNR comparisons of the reconstructed images obtained by the present invention and the other four comparison algorithms. It can be seen that the PSNR of the reconstructed image of the present invention is highest, thereby objectively proving the reconstruction effect of the present invention.
Experiment II: the spectrum mixing kernel function of the invention is verified to help the reconstruction effect.
The kernel function in the invention is replaced by a radial base kernel and a linear kernel function respectively, other steps are unchanged, and the invention and the two methods are used for carrying out simulation test on the Head image, and the result is shown in figure 4. Wherein fig. 4 (a) is a reconstruction result using a radial radix kernel; fig. 4 (b) is a reconstruction result using a linear kernel. FIG. 4 (c) is the reconstruction result of the present method; fig. 4 (d) is a true high resolution image. Each image has a rectangular area that is locally enlarged to facilitate viewing differences in reconstruction effects.
As can be seen from fig. 4, the reconstruction effect of the spectrum mixing kernel function in the present invention is significantly better than that of other kernel functions, and the spots on the child's face in fig. 4 (c) are better than those of other methods.
Table 2. Reconstruction Effect index evaluation of the inventive algorithm and other kernel function based algorithm (Head diagram, magnification S is 3)
| Radial basis function | Linear core | Spectrum mixing core (invention) | |
| PSNR(dB) | 33.29 | 33.32 | 33.53 |
Table 2 shows the PSNR comparison of the present algorithm and the algorithm based on other kernel functions to obtain the reconstructed image. It can be seen that the spectral mixing kernel used in the present invention greatly helps to improve the PSNR of the reconstructed image, because the spectral mixing kernel can reasonably allocate weights during reconstruction, capturing more high-frequency information of the image block.
FIG. 5 is a graph comparing the reconstructed image quality based on the Kodak image set with the relationship curve of the parameter K by the other kernel function algorithm, wherein the abscissa is the K value, and the ordinate is the PSNR value. The parameter K represents the clustering class number, and if the K value is too small, redundant information in a single training data subset is too much, so that the quality of a reconstructed image is affected; if the K value is too large, the data in the training data subset is too small, so that the high-frequency information of the reconstructed image is insufficient, and the reconstruction effect is reduced. Experiments show that the optimal value of K in the invention is 35. Meanwhile, as can be seen from the graph, the influence of different kernel functions on the reconstruction effect is larger, the mathematical principles of the linear kernel and the radial kernel are simpler, the similarity between the image blocks cannot be accurately represented, and the spectrum mixing kernel function in the invention can accurately calculate the distance between the image blocks, so that the PSNR of the reconstructed image is higher, and experiments prove that the reconstruction effect of the kernel function adopted by the invention is better than that of the radial kernel or the linear kernel.
FIG. 6 is a graph comparing the rotation angle of the present invention with the rotation angle of other kernel algorithms and the reconstruction weight, wherein the graph is used for performing different angle rotations on the input image to obtain a training image, the abscissa is the rotation angle of the image block in the training set, and the ordinate is the reconstruction weight. The rotation of the image block does not change the texture structure of the image block, and the central pixels of the image block before and after the rotation are not changed, so that the influence of the change of the rotation angle on the reconstruction weight is smaller in theory, and the weight is closer to 1, the more reasonable the distribution of the image feature weights in the training set by the algorithm is, and the higher the accuracy of measuring the distance between the image features by the kernel function is. As can be seen from fig. 6, the weight change of the corresponding curve of the present invention is relatively stable, and the weight obtained by the present invention is closer to 1 than other algorithms no matter how the rotation angle changes, which indicates that the present invention is most reasonable for the distribution of the reconstruction weight, and the spectrum mixing kernel function of the present invention is more capable of characterizing the distance between the image features and capturing the structural similarity of the image blocks than other kernel functions.
The invention is further described below in connection with the application of a single frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression.
In the embodiment of the invention, fig. 7 is a block diagram of a medical image processing system of the embodiment of the invention, which provides a single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression. In the system in fig. 7, after the input low-resolution medical image is reconstructed by the invention, the output high-resolution image is clear, the focus position in the image is clear, the pathological change phenomenon is obvious, and the medical staff can judge the disease condition and improve the diagnosis accuracy.
Fig. 8 is a block diagram of a surveillance video processing system according to an embodiment of the present invention, which provides a single-frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression. The monitoring video is widely applied in the fields of road safety, criminal investigation monitoring and the like. However, due to factors such as poor imaging environment and limited imaging hardware, the imaging effect often cannot meet the requirements. In the system of fig. 8, the original image of the face collected in the monitoring device is blurred, the high-frequency information is insufficient, and the development of subsequent criminal investigation and other works cannot be satisfied; the invention takes the image collected in the monitoring equipment as the input image, and the image reconstructed by the invention is clear, and the details are increased, thereby being beneficial to the follow-up work such as face recognition and the like.
Fig. 9 is a block diagram of a television image processing system for providing a single-frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression according to an embodiment of the present invention. In the field of video entertainment, users have increasingly high requirements on the quality of television images. Because of the limitation of bandwidth, the image signal source obtained by the user receiving end often cannot meet the requirement on resolution, and the visual effect of the original image of the television shown in fig. 9 is poor and distortion occurs; the television image reconstructed by the method has high resolution, improves visual effect and optimizes user experience.
Fig. 10 is a block diagram of a remote sensing image processing system for providing a single frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to an embodiment of the present invention. The remote sensing image has wide application in the fields of military, scientific research, aerospace and the like; however, when the remote sensing equipment images, the problems of low resolution, fuzzy texture structure and the like of the original image occur under the influence of factors such as flight state, atmospheric refraction, topography fluctuation and the like; as shown in FIG. 10, the resolution of the specific area of the remote sensing image reconstructed by the method is improved, the texture structure is clear, and the subsequent work such as target detection, extraction and classification is convenient to develop.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (10)
1. A single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression is characterized in that the single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression is characterized in that in a training stage, a high-resolution natural image is selected, downsampling is conducted on the high-resolution natural image by a bicubic interpolation method to obtain a low-resolution training image, and interpolation amplification is conducted on the low-resolution training image to obtain a corresponding interpolation image; partitioning the high-resolution image and the interpolation image, extracting features, and forming a training data set; clustering the training data sets to obtain K training data subsets; the K training data subsets are learned by utilizing a spectrum mixing kernel function to obtain the optimal super parameter theta of the Gaussian process regression model;
in the test stage, selecting a low-resolution image as an input image; performing bicubic interpolation amplification on an input image to obtain an interpolation test image, and performing blocking and feature extraction to obtain a low-resolution test data set;
Searching the K cluster centers of the training data set for the cluster center nearest to each feature in the test data set to form a nearest training data subset, and carrying out regression by utilizing the optimal super parameter theta obtained by previous learning to obtain a high-resolution image feature set; and superposing the high-resolution image features on the interpolation test image to reconstruct a high-resolution result image.
2. The single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to claim 1, wherein the single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression specifically comprises the following steps:
selecting a high-resolution image to form a high-resolution training image set; performing downsampling and interpolation amplification operation on the high-resolution training image set element to obtain an interpolation training image set; extracting features to form a training data set C= { H, L }, wherein H and L respectively represent high-resolution and low-resolution training data sets;
clustering the training data set C to obtain a training data subset
Corresponding K clustering centers;
step three, for each training data subset C i Training out optimal super-parameter theta of Gaussian process regression model based on spectrum mixing kernel i ，θ i Is a column vector;
step four, reading a low-resolution test image and constructing a low-resolution test data set X;
searching a cluster center closest to each feature in the low-resolution test data set X from the K cluster centers to form a nearest neighbor training data subset;
step six, utilizing the theta obtained in the training in the step three i Carrying out Gaussian process regression to obtain a high-resolution feature set F;
and step seven, outputting a high-resolution image Y.
3. The method for reconstructing a single-frame image super-resolution based on spectral mixture kernel gaussian process regression according to claim 2, wherein the step one specifically comprises:
selecting a plurality of color high-resolution natural images and converting the color high-resolution natural images from an RGB color space to a YCbCr color space; selecting a brightness image to form a high-resolution training image set;
selecting the current super-resolution reconstruction magnification as S=3, downsampling the high-resolution training image set by S times through bicubic interpolation, and amplifying the high-resolution training image set by S times through bicubic interpolation to obtain an interpolation training image set;
selecting the size of the image block as N multiplied by N, wherein N=7 is an odd number, and taking the image blocks from pixel to pixel according to the sequence from top to bottom and from left to right for the images corresponding to the high-resolution image training set and the interpolation training image set; then, the difference value of the center pixel of the high-resolution image block and the low-resolution image block is obtained to obtain a high-resolution training data set 
Vectorizing low-resolution image blocks to obtain low-resolution trainingData set->
Wherein p=100000, P is smaller than the maximum number of blocks of the fetched image block;
the second step specifically comprises:
step 1, setting the class number as K=35, and carrying out K-means clustering on a low-resolution training data set L to obtain { L } 1 ，L， 2 ，...，L K The corresponding cluster center set v= (o) 1 ，o 2 ，...，o K )；
Step 2, finding out the corresponding high-resolution data in the high-resolution training data set H to form { H } 1 ，H 2 ，...，H K }；
Step 3, combining { L ] 1 ，L 2 ，...，L K And { H } and 1 ，H 2 ，...，H K formation { C } of 1 ，C 2 ，...，C K }, item i C therein i ＝{L i ，H i And is referred to as a training data subset.
4. The single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to claim 2, wherein the third step specifically comprises:
step A, defining super parameter theta i ；
Training data subset C i ＝{L i ，H i Comprises Z data pairs with log likelihood functions of
Wherein θ is i Is a super parameter to be learned, (H) i )、(L i ) Respectively C i High and low resolution training data subset H of (b) i 、L i Element of (1) |i 1 Representing 1-norm, (. Cndot. T Representing a transpose operation, K y (l，l′|θ i ) The covariance matrix is represented by a matrix of covariance,the formula is
Wherein k (l, l') comprises the super parameter θ i The expression is:
wherein the method comprises the steps of
And omega q，i ，∑ q，i Sum mu q，i Respectively represent the weight, variance and frequency parameter corresponding to the q accumulation parameter in the ith training data subset,/for >
The method is characterized in that the method comprises the steps of representing a noise standard deviation parameter, Q=15 is a set accumulation parameter, I·I represents an absolute value taking operation, I·I represents a vector modular value taking operation, and the expression of a function cos (·) is as follows
Wherein l p And l' p The p-th component of vectors l and l', respectively, where μ q，i (p) Representation mu q，i The p-th component of (b), U represents the dimension of vector l; the function delta (. Cndot.) is
And step B, solving the super parameters by using an iteration method.
5. The single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to claim 4, wherein step B specifically comprises:
step a, constructing partial derivatives
In the middle of
tr (·) represents the trace of the matrix; />
The element in the middle position (s, t) is
In the middle of
Represents L i The s-th vector l of (3) s S, t=1, 2,..z; />
The element in the middle position (s, t) is
∑ q，i (j) Representing sigma q，i Is selected from the group consisting of the i-th component, i=1, 2,. -%, U;
the element in the middle position (s, t) is
Mu in the middle q，i (j) Representation mu q，i The j-th component of (a);
the element in the middle position (s, t) is
Step b, setting the current step as the kth step, wherein k is more than or equal to 3,
the iteration parameter values are updated and,
in the middle of
d k-1 ＝(g k-1 ) T s k-2 ；
p＝6(f k-1 -f k )+3(d k-1 +d k )(v k-1 -v k-2 )；
q＝3(f k -f k-1 )-(2d k-1 +dk)(v k-1 -v k-2 )；
In the middle of
If f k ＜f opt Order in principle
f k →f opt And continuing the iterative process; f in opt Representing the minimum likelihood value, θ, in a previous iteration process opt Representing optimal parameters in a previous iteration process;
step c, setting termination conditions: if it is
Or when the number of the loop steps reaches 500, terminating the iterative process, and theta i ＝θ opt For optimal super parameters, τ=0.1 and γ=0.05 are set control coefficients;
step d, initializing iteration variables:
initializing superparameters
Wherein the method comprises the steps of
In the formula, std(s) represents taking standard deviation operation, max(s) represents taking the largest element in the matrix, and min(s) represents taking the smallest element in the matrix; wherein R is i For a low resolution data subset L i Is expressed as the scale matrix of
Middle column vector m 1 ，...，m Z From the following components
L i -mean(L i )＝[m 1 ，m 2 ，…m Z ]；
Obtaining, mean (·) represents an average value calculation;
the initial values of the iteration variables include
d 0 ＝d 1 ＝-(s 0 ) T s 0 ；
f opt ＝f 0 ；
6. The single-frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to claim 2, wherein in step four, the method specifically comprises:
reading a low-resolution color test image, amplifying the color test image by S times by using bicubic interpolation, converting the color test image from an RGB color space to a YCbCr color space, and respectively obtaining a brightness channel image, a blue channel image and a red channel image;
pixel-by-pixel blocking and vectorizing the luminance channel image in a top-to-bottom, left-to-right order to obtain a low resolution test dataset 
M represents the maximum number of image blocks taken, test data x n Represents the nth vector in X;
the fifth step specifically comprises:
calculate the mth vector X in X m To the cluster center set v= (o) 1 ，o 2 ，...，o K ) Is a Euclidean distance to obtain a distance vector
E (x, x ') represents the Euclidean distance between the calculated vectors x and x';
for S m The elements in (m=1, 2., M) are ordered and the smallest element is found, the class number a of the corresponding cluster center is marked out m ∈[1，K]；
Performing operation on all vectors in X to obtain a marked vector
In the formula, the mth element a m For a corresponding low resolution nearest neighbor training data subset
Class number of->
Is x m Is a nearest neighbor training data subset +.>
Is->
A corresponding high resolution nearest neighbor training data subset; />
Corresponding optimal superparameter->
Is x m Is the nearest neighbor super parameter of (a);
the sixth step specifically comprises:
using the formula
Calculating x m Corresponding output feature y m Obtaining a high-resolution feature set
Is of class number a m A low resolution nearest neighbor training data subset of (a); />
Is->
A corresponding high resolution nearest neighbor training data subset; wherein->
Middle l 1 ，l 2 ...，l M Is that
All vectors in (a); computing spectral mixture kernel function->
Used->
The nearest neighbor super parameter obtained in the fifth step;
the seventh step specifically comprises:
from formula y m ＝y m +cen(x m ) Calculating to obtain high-resolution image pixel points, replacing corresponding pixel points in the brightness channel image to obtain a brightness reconstruction image, and enabling cen (·) to represent center-taking pixel operation;
And combining the brightness reconstruction image with the blue channel image and the red channel image to obtain a super-resolution reconstruction image under the YCbCr color space, and converting the super-resolution reconstruction image into the RGB color space to obtain a high-resolution reconstruction image Y.
7. A medical image processing system implementing the single-frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression according to claim 1.
8. A surveillance video processing system implementing the single-frame image super-resolution reconstruction method based on spectral mixture kernel gaussian process regression of claim 1.
9. A television image processing system implementing the spectral mixture kernel gaussian process regression-based single frame image super resolution reconstruction method of claim 1.
10. A remote sensing image processing system for implementing the single frame image super-resolution reconstruction method based on spectrum mixing kernel gaussian process regression according to claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910194123.6A CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910194123.6A CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110097499A CN110097499A (en) | 2019-08-06 |
| CN110097499B true CN110097499B (en) | 2023-05-05 |
Family
ID=67443986
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910194123.6A Active CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110097499B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112330541A (en) * | 2020-11-11 | 2021-02-05 | 广州博冠信息科技有限公司 | Live video processing method and device, electronic equipment and storage medium |
| CN112581367B (en) * | 2020-12-01 | 2023-06-30 | 南京理工大学 | GPR image super-resolution reconstruction method based on random sample classification amplification |
| CN113435487B (en) * | 2021-06-16 | 2022-08-12 | 中国电子科技集团公司第五十四研究所 | Deep learning-oriented multi-scale sample generation method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780342A (en) * | 2016-12-28 | 2017-05-31 | 深圳市华星光电技术有限公司 | Single-frame image super-resolution reconstruction method and device based on the reconstruct of sparse domain |
| CN107993194B (en) * | 2017-11-30 | 2021-01-01 | 天津大学 | Super-resolution reconstruction method based on stationary wavelet transform |
| CN108257093B (en) * | 2018-01-18 | 2021-08-06 | 洛阳理工学院 | Single-frame image super-resolution method based on controllable kernel and Gaussian process regression |
-
2019
- 2019-03-14 CN CN201910194123.6A patent/CN110097499B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097499A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiang et al. | Noise robust face image super-resolution through smooth sparse representation | |
| Fang et al. | Spectral–spatial classification of hyperspectral images with a superpixel-based discriminative sparse model | |
| Li et al. | From beginner to master: A survey for deep learning-based single-image super-resolution | |
| CN110097499B (en) | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression | |
| Yu et al. | Can we see more? Joint frontalization and hallucination of unaligned tiny faces | |
| CN113177592B (en) | Image segmentation method and device, computer equipment and storage medium | |
| CN111695455B (en) | Low-resolution face recognition method based on coupling discrimination manifold alignment | |
| Liu et al. | Dual learning-based graph neural network for remote sensing image super-resolution | |
| Wei et al. | Joint learning sparsifying linear transformation for low-resolution image synthesis and recognition | |
| Liang et al. | Scale-invariant structure saliency selection for fast image fusion | |
| Zeng et al. | Self-attention learning network for face super-resolution | |
| Chen et al. | Registration of multimodal images with edge features and scale invariant PIIFD | |
| Liu et al. | Image super-resolution via hierarchical and collaborative sparse representation | |
| Tian et al. | SWDGAN: GAN-based sampling and whole image denoising network for compressed sensing image reconstruction | |
| Wang et al. | Joint depth map super-resolution method via deep hybrid-cross guidance filter | |
| Chen et al. | Face super resolution based on parent patch prior for VLQ scenarios | |
| Zhu et al. | Super-resolved image perceptual quality improvement via multifeature discriminators | |
| Zhang et al. | A parallel and serial denoising network | |
| Bai et al. | Survey of learning based single image super-resolution reconstruction technology | |
| Liu et al. | High-resolution facial expression image restoration via adaptive total variation regularization for classroom learning environment | |
| Chen et al. | Hierarchical factorization strategy for high-order tensor and application to data completion | |
| CN106709921B (en) | Color image segmentation method based on space Dirichlet mixed model | |
| Zhao et al. | Single image super-resolution via blind blurring estimation and anchored space mapping | |
| Sharma et al. | Facial Image Super-Resolution with CNN,“A Review” | |
| Gao et al. | Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |