CN110097499A - The single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process - Google Patents
The single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process Download PDFInfo
- Publication number
- CN110097499A CN110097499A CN201910194123.6A CN201910194123A CN110097499A CN 110097499 A CN110097499 A CN 110097499A CN 201910194123 A CN201910194123 A CN 201910194123A CN 110097499 A CN110097499 A CN 110097499A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- training
- super
- gaussian process
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 186
- 230000008569 process Effects 0.000 title claims abstract description 81
- 238000001228 spectrum Methods 0.000 title claims abstract description 67
- 238000012549 training Methods 0.000 claims abstract description 135
- 238000012360 testing method Methods 0.000 claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 27
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 claims abstract description 23
- 238000005070 sampling Methods 0.000 claims abstract description 13
- 239000000284 extract Substances 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 55
- 239000013598 vector Substances 0.000 claims description 34
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000012804 iterative process Methods 0.000 claims description 11
- 230000003321 amplification Effects 0.000 claims description 8
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 6
- 230000001186 cumulative effect Effects 0.000 claims description 5
- 229910010888 LiIn Inorganic materials 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 241000208340 Araliaceae Species 0.000 claims 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 claims 1
- 235000003140 Panax quinquefolius Nutrition 0.000 claims 1
- 235000008434 ginseng Nutrition 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 30
- 238000004422 calculation algorithm Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 10
- 238000009826 distribution Methods 0.000 description 8
- 238000002474 experimental method Methods 0.000 description 8
- 230000000007 visual effect Effects 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 108010072610 N-acetyl-gamma-glutamyl-phosphate reductase Proteins 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013480 data collection Methods 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000011840 criminal investigation Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000003412 degenerative effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 239000013604 expression vector Substances 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- YBJHBAHKTGYVGT-ZKWXMUAHSA-N (+)-Biotin Chemical compound N1C(=O)N[C@@H]2[C@H](CCCCC(=O)O)SC[C@@H]21 YBJHBAHKTGYVGT-ZKWXMUAHSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- FEPMHVLSLDOMQC-UHFFFAOYSA-N virginiamycin-S1 Natural products CC1OC(=O)C(C=2C=CC=CC=2)NC(=O)C2CC(=O)CCN2C(=O)C(CC=2C=CC=CC=2)N(C)C(=O)C2CCCN2C(=O)C(CC)NC(=O)C1NC(=O)C1=NC=CC=C1O FEPMHVLSLDOMQC-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to technical field of image processing, disclose a kind of single-frame image super-resolution reconstruction method and system returned based on spectrum mixed nucleus Gaussian process, it chooses high-definition picture and forms high-resolution training image collection, down-sampling is carried out to high-resolution training image collection element and interpolation magnification operation obtains interpolation training image collection；It extracts feature and forms training dataset, training dataset is clustered to obtain training data subset；Go out the optimal hyper parameter of the Gaussian process regression model based on spectrum mixed nucleus to each training data trained；It reads low resolution test image and constructs low resolution test data set；The cluster centre nearest with each feature in low resolution test data set is found in cluster centre, forms arest neighbors training data subset；Gaussian process recurrence is carried out, high-resolution features collection is obtained, exports high-definition picture.It include more high frequency details the invention enables reconstruction image, texture structure is clear, rebuilds effect and is improved.
Description
Technical field
The invention belongs to technical field of image processing more particularly to a kind of single frames returned based on spectrum mixed nucleus Gaussian process
Image super-resolution rebuilding method.
Background technique
In real production and living, due to the limitation of imaging system limited capacity or image-forming condition, acquired image
Resolution ratio may be lower, causes details performance insufficient, cannot meet the needs.Therefore high resolution graphics is obtained by low-resolution image
As there is very strong current demand.Image super-resolution rebuilding method is exactly by existing low-resolution image by software approach
Processing, obtains corresponding high-definition picture, to meet actual needs.This technology is occupied importantly in field of image processing
Position, receives significant attention.
Currently, image super-resolution rebuilding method can be divided into three classes: the method based on interpolation, the method based on study and
Method based on reconstruction.
Method based on interpolation is the most common method, including arest neighbors interpolation method, linear interpolation method and bi-cubic interpolation
Method.Method based on interpolation is to be input to the adjacent known pixels of unknown pixel in high-definition picture grid to be reconstructed in advance
In the interpolation kernel function of setting, unknown pixel is estimated, to achieve the purpose that improve image resolution ratio.Such methods principle letter
Single, data volume is small, realizes quick.But from mathematical principle, the preset kernel function of such methods cannot be accurately reflected very
Real image texture structure often leads to reconstructed results and lacks details, and sawtooth effect may occurs in image border.More preferably
Improved method be to estimate high-definition picture using can more embody the kernel function of image detail.
Method based on reconstruction mainly includes non-homogeneous interpolation method, iterative backprojection method, maximum a posteriori probability method, convex set
Sciagraphy etc..Such methods are using the model that degrades of image as priori knowledge, including the processes such as fuzzy, down-sampling, noise superposition,
By taking different constraint condition, seek initial reconstruction image low resolution image error with input after the model that degrades
It minimizes, to estimate final super-resolution reconstruction image.On mathematical principle, the reconstruction process of image belongs to inverse problem
Solution procedure, be typical ill-posed problem, thus cannot be guaranteed solution existence and uniqueness.Therefore, in the process of realization
In, such methods all have that iterative process is not easy to restrain, reconstruction image has the problems such as one-to-many.Better method is to give up excessively
Simply degrade model, and uses the mathematical description closer to the process that really degrades.
Image super-resolution rebuilding method based on study usually first collects a large amount of image, establishes training set；Then right
Training set is learnt, and the corresponding relationship (or mapping model) between high and low image block is obtained；Then, by input low resolution
Image combination mapping model predicts information required for high-definition picture (or pixel value), reconstructs high resolution graphics
Picture.In this type of method, learn to obtain mapping model to be key link by training set, and wherein between image block and its feature
Similitude, or be distance, and be the core for constructing model.The existing method based on study mostly uses Euclidean distance, angle
The mathematical tools such as distance measure similitude.But these mathematical tools are too simple and direct, cannot fully characterize image
Logical relation between block is possible to introducing and the incoherent additional information of image block instead, leads to the study energy of mapping model
Power is insufficient, rebuilds weight unreasonable distribution, causes final reconstruction image texture structure fuzzy, rebuilds ineffective.Therefore exist
In image super-resolution rebuilding method based on study, it is important to find suitable mathematical tool come it is similar between characteristic feature
Property, to accurately estimate weight of each image block when rebuilding in training set, seek an optimal weighting value.
Existing several typically based on the method for study, deficiency is shown: Yang et al. in document J.Yang,
J.Wright,T.Huang,and Y.Ma,“Image super-resolution via sparse representation,”
Pass through joint using sparse representation theory in IEEE Trans.Image Process., vol.19, no.11, pp.2861-2873
Optimization Framework learns antithesis dictionary out, and the dictionary is recycled to carry out Super-resolution Reconstruction to the low resolution image of input.This method exists
The optimal characteristics in low resolution dictionary are searched using Euclidean distance when reconstruction to combine, and are desirably to obtain based on the optimal of rarefaction representation
Solution.However, actual reconstruction effect image is of low quality, there are edge blurrys.
He et al. is in document L.He, H.Qi, and R.Zaretzki, " Beta process joint dictionary
learning for coupled feature spaces with application to single image super-
Resolution ", in Proc.IEEE Conf.Comput.Vis.Pattern Recognit., in 2013, pp.345-352
Learn dual spaces dictionary using Beta process, and then reconstructs full resolution pricture block.But this method is also by characteristics of image
The distance between be set as Euclidean distance, there is also reconstruction image edge distortion, the problems such as grain details obscure.
He et al. is in document H.He and W.C.Siu, " Single image super-resolution using
Gaussian process regression”,in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.,
In 2011, pp.449-456 Gaussian process recurrence is introduced into image super-resolution rebuilding field, and using natural image from
Similitude establishes Gaussian process regression model in image local area.However this method uses simple radial base core as core
Function, that is, square exponential form of Euclidean distance, there are what the structural similarity description between image block was not enough to ask
Topic, therefore image reconstruction effect is general, details presents insufficient.
Wang et al. is in document H.Wang, X.Gao, K.Zhang, and J.Li " Single-Image Super-
Resolution Using Active-Sampling Gaussian Process Regression”,IEEE
Master is utilized in transactions on image processing, vol.25, no.2, February 2016, pp.935-947
Dynamic sampling reduces external trainer collection, and differentiates feature to the height of image in training set and carry out Gaussian process regression modeling, into
And reconstruct high-definition picture.But the kernel function being related in this method is linear kernel function, essence is to calculate vector
Between dot product, mathematical principle is also excessively single, and seem scarce capacity when characterizing more complicated image block similarity, in turn
So that Model suitability declines, image reconstruction effect is undesirable, and texture structure is fuzzy.
In addition, deep learning method is also introduced into image super-resolution field in recent years.Such method has benefited from magnanimity
Amount of training data, obtain preferable image reconstruction effect.But huge data volume is also brought simultaneously in storage, pipe
The problem of reason, processing, transmission etc., such as the defects of hardware device cost is excessively high, the method training time is too long；Moreover, in number
It learns in principle, such methods there is also models bottlenecks such as opaque, difficulty of parameter tuning.Therefore, deep learning class image oversubscription
Resolution method for reconstructing is subject to many limitations in practical application, is not used widely still.
It in summary it can be seen, the mathematical description of model or degenerative process that existing method uses is too simple, cannot
The texture structure of accurate reflection image, so that it is undesirable to rebuild effect.Better method is reference Gaussian process regression model pair
Image is described, and can more embody the non-linear relation hidden in image block in this way, be excavated out in image block as much as possible
Information and reduce deceptive information.On the other hand, existing method generally uses Euclidean distance, the equidistant function of angular distance, it
Mathematical principle it is single, representational difference cannot accurately embody the logical relation and texture structure similitude between image block, lead
Model learning scarce capacity is caused, weight unreasonable distribution when reconstruction, reconstruction image effect quality is bad.Therefore better method is
The new kernel function (distance function) of one kind is introduced in Gaussian process regression model to characterize the distance between image block, measurement figure
As the similitude between block, and then the learning ability of whole lift scheme, finally obtain higher reconstructed image quality.
In brief, for existing technical problem, the present invention is incited somebody to action:
(1) image is described using Gaussian process regression model, solves degenerative process mathematical description in existing method
Too simple, the corresponding mapping model of high-low resolution feature cannot accurately reflect image texture structure, so that rebuilding effect not
The problems such as ideal.
(2) kernel function of the spectrum mixed kernel function as Gaussian process is introduced, solves to use simple distance in existing method
Function and caused by it is representational it is poor, cannot accurately embody logical relation and texture structure similitude, model between image block
Deficiency of learning ability, weight unreasonable distribution when rebuilding, the problems such as reconstruction image effect quality is bad.
The present invention solve the problems, such as existing technologies when, existing technical difficulty includes:
(1) selection of training data subset number K is a difficult point in learning process, it directly affects training set
Practise effect.In the emulation experiment part of " specification " of the invention, the changing rule (as shown in Figure 5) of K value is illustrated with curve, and
The selection of K value is described in detail.
(2) solution of the optimal hyper parameter θ of Gaussian process regression model is a difficult point.Hyper parameter θ is a vector, often
It is one-dimensional to require to find out optimal value by iterative algorithm.The step of " claims " three, is described in detail solution procedure.
Solve the meaning of above-mentioned technical problem:
(1) from technical functionality, present invention introduces spectrum mixed kernel functions into Gaussian process regression model, so that oversubscription
Resolution reconstruction image is clear in structure, texture is careful, and closer to true picture, evaluation index is higher, improves algorithm on the whole
Energy.
(2) in terms of practical application, the present invention can directly using but be not limited to following several respects: 1) at medical image
Reason: providing high-quality size medical image, and doctor is helped to improve diagnosis efficiency；2) monitor video: improve monitor video, image it is clear
Degree is conducive to related personnel's viewing and extracts information；3) television image: the resolution ratio and visual effect of television image are improved, is mentioned
User experience is risen；4) remote sensing images: improving target or the resolution ratio of area-of-interest in satellite remote sensing images, examines convenient for target
The expansion of the follow-up works such as survey, extraction, classification.As it can be seen that the present invention can be straight in fields such as medical treatment, public security, consumption and scientific researches
Use is scooped out, but also can be generalized to more areas, therefore the present invention has very big application prospect.
Summary of the invention
In view of the problems of the existing technology, the present invention provides a kind of single frames returned based on spectrum mixed nucleus Gaussian process
Image super-resolution rebuilding method.The present invention quotes Gaussian process regression model and image is described, and can more embody figure in this way
As in block hide non-linear relation, the information excavated out in image block as much as possible and reduce deceptive information.The present invention draws
Enter the new kernel function of one kind to embody the structure of image block, describes the similitude between image block, promote the study of regression model
Ability accurately estimates the reconstruction weight of image block, is finally reached higher reconstructed image quality.
The invention is realized in this way a kind of single-frame images super-resolution rebuilding returned based on spectrum mixed nucleus Gaussian process
Method, the single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process are chosen high in the training stage
Resolution ratio natural image, and carry out down-sampling to it using bi-cubic interpolation method and obtain low resolution training image, then to low point
Resolution training image carries out interpolation amplification and obtains corresponding interpolation image；Piecemeal is carried out simultaneously to high-definition picture and interpolation image
Feature is extracted, training dataset is formed；Training dataset is clustered, K training data subset is obtained；To this K trained number
Learn to obtain the optimal hyper parameter θ of Gaussian process regression model using spectrum mixed kernel function according to subset.Mould is returned in Gaussian process
It uses spectrum mixed kernel function as distance function under type frame, can more accurately characterize the similitude between image block, make weight
It is more reasonable to build weight distribution.
In test phase, low-resolution image is chosen as input picture；Bi-cubic interpolation amplification is carried out to input picture
Interpolation test image is obtained, and carries out piecemeal and feature extraction, obtains low resolution test data set.
It is found in the cluster nearest with each feature in test data set in K cluster centre of training dataset
The heart forms arest neighbors training data subset, recycles the optimal hyper parameter θ for previously learning to obtain to be returned, obtains high-resolution
Rate set of image characteristics；These high-definition picture features are added in interpolation test image, high-resolution result figure is reconstructed
Picture.It can make that super-resolution rebuilding picture structure is clear, texture is careful using the Gaussian process regression model based on spectrum mixed nucleus,
Closer to true picture, evaluation index is higher.
Further, the single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process is specifically included
Following steps:
Step 1 chooses high-definition picture and forms high-resolution training image collection；To high-resolution training image element of set
Element carries out down-sampling and interpolation magnification operation obtains interpolation training image collection；It extracts feature and forms training dataset C={ H, L },
Wherein H and L respectively indicates high-resolution and low-resolution training dataset；
Step 2 clusters training dataset C to obtain training data subsetAnd in corresponding K cluster
The heart；
Step 3, to each training data subset CiTrain the Gaussian process regression model based on spectrum mixed nucleus most
Excellent hyper parameter θi, θiFor column vector；
Step 4 reads low resolution test image and constructs low resolution test data set X；
Step 5 finds the cluster nearest with each feature in low resolution test data set X in K cluster centre
Center forms arest neighbors training data subset；
Step 6 utilizes the θ that training obtains in step 3iGaussian process recurrence is carried out, high-resolution features collection F is obtained；
Step 7 exports high-definition picture Y.
Further, step 1 specifically includes:
It chooses several high-resolution natural images and is transformed into YCbCr color space from RGB color；It chooses bright
It spends image and forms high-resolution training image collection；
Selecting current super-resolution rebuilding amplification factor is S=3, and high-resolution training image collection is utilized bi-cubic interpolation
S times of down-sampling recycles bi-cubic interpolation to amplify S times, obtains interpolation training image collection；
Selected digital image block size is N × N, and N=7 is odd number, to high-definition picture training set and interpolation training image collection
In corresponding image according to sequence from top to bottom, from left to right take image block pixel-by-pixel；Then high-resolution and low-resolution image block is sought
The difference of center pixel obtains high-resolution training datasetTo low-resolution image block vectorization, obtain low
Resolution ratio training datasetWherein P=100000, P are less than the maximum block number for the image block got；
Step 2 specifically includes:
Step 1, class number K=35 is set, K-means cluster is carried out to low resolution training dataset L, obtains { L1,
L2..., LKAnd corresponding cluster centre collection V=(o1, o2..., oK)；
Step 2, corresponding high-resolution data is found in high-resolution training dataset H forms { H1, H2..., HK}；
Step 3, merge { L1, L2..., LKAnd { H1, H2..., HKForm { C1, C2..., CK, wherein i-th Ci=
{Li, HiIt is known as training data subset.
Further, step 3 specifically includes:
Step A defines hyper parameter θi；
Training data subset Ci={ Li, HiContaining Z data pair, log-likelihood function is
Wherein θiFor hyper parameter to be learned, (Hi)、(Li) it is respectively CiIn high-resolution and low-resolution training data subset Hi、
LiElement, | |1Indicate 1- norm, ()TExpression takes transposition operation, Ky(l, l ' | θi) indicate that covariance matrix, formula are
Wherein k (l, l ') includes hyper parameter θi, expression formula are as follows:
WhereinAnd ωQ, i、∑Q, iAnd μQ, iRespectively indicate i-th of training data subset
In the corresponding weight of q-th of cumulative parameter, variance and frequency parameter,Indicate noise criteria difference parameter, Q=15 is the tired of setting
Add parameter, | | indicate the operation that takes absolute value, | | | | indicate the operation of amount of orientation modulus value, the expression formula of function COS () is
Wherein lpWith l 'pP-th of component of vector l and l ' is respectively indicated, wherein μQ, i (p)Indicate μQ, iIn p-th of component,
The dimension of U expression vector l；Function δ () is
Step B seeks hyper parameter using iterative method.
Further, step B is specifically included:
Step a constructs partial derivative
In formulaTr () indicates to seek the mark of matrix.The element of middle position (s, t) is
In formulaIndicate LiIn s-th of vector lsP-th of component, s, t=1,2 ..., Z；The member of middle position (s, t)
Element is
∑Q, i(j) ∑ is indicatedQ, iIn j-th of component, j=1,2 ..., U；The element of middle position (s, t) is
μ in formulaQ, i(j) μ is indicatedQ, iIn j-th of component.The element of middle position (s, t) is
Step b, if currently being walked for kth, k >=3,Iterative parameter value is updated,
In formula
dk-1=(gk-1)Tsk-2；
P=6 (fk-1-fk)+3(dk-1+dk)(vk-1-vk-2)；
Q=3 (fk-fk-1)-(2dk-1+dk)(vk-1-vk-2)；
In formula
If fk< fopt, then enablefk→fopt, and continue iterative process；F in formulaoptIteration mistake before expression
Minimum likelihood value in journey, θoptOptimized parameter before expression in iterative process；
Step c sets termination condition: if
Or circulation step number terminates iterative process, θ when reaching maximum set value 500i=θoptFor optimal hyper parameter, in formula
τ=0.1 and γ=0.05 are the control coefrficient of setting；
Step d initializes iteration variable:
Initialize hyper parameterWherein
Std () expression takes standard difference operation in formula, and max () expression takes greatest member in matrix, and min () expression takes
Least member in matrix；R in formulaiFor high-resolution data subset LiScale Matrixes, expression formula is
Column vector m in formula1..., mZBy
Li-mean(Li)=[m1, m2... mZ]；
It obtains, mean () expression takes mean operation；
The initial value of iteration variable includes
d0=d1=-(s0)Ts0；
v1=0；
fopt=f0；
Further, it is specifically included in step 4:
Low resolution colour chart picture is read, amplified S times using bi-cubic interpolation and is converted from RGB color
To YCbCr color space, and respectively obtain luminance channel image, blue channel image and red channel image；
To luminance channel image with from top to bottom, sequence from left to right takes block pixel-by-pixel and its vectorization is obtained low point
Resolution test data setM indicates the maximum image block number taken, test data xnIndicate n-th of vector in X；
Step 5 specifically includes:
Calculate m-th of vector x in XmTo cluster centre collection V=(o1, o2..., oK) Euclidean distance obtain distance vector
E (x, x ') indicates to calculate the Euclidean distance between vector x and x '；
To SmElement in (m=1,2..., M) sorts and finds out least member, marks the class of corresponding cluster centre
Number am∈ [1, K]；
Operation is executed to institute's directed quantity in X, obtains label vector
M-th of element a in formulamFor corresponding low resolution arest neighbors training data subsetClass number,For xmArest neighbors training data subset,It isCorresponding high-resolution arest neighbors training data
Collection；Corresponding optimal hyper parameterFor xmArest neighbors hyper parameter；
Step 6 specifically includes:
Utilize formula
Calculate xmCorresponding output feature ym, obtain high-resolution features collection Be class number be amIt is low
Resolution ratio arest neighbors training data subset；It isCorresponding high-resolution arest neighbors training data subset；Wherein
L in formula1, l2..., lMForMiddle whole vector；Calculate spectrum mixed kernel functionUsedIt is step 5
Obtained arest neighbors hyper parameter；
Step 7 specifically includes:
By formula ym=ym+cen(xm) high-definition picture pixel is calculated, and replace it is right in luminance channel image
The pixel answered obtains brightness reconstruction image, and cen () expression takes center pixel to operate；
Brightness reconstruction image is obtained in conjunction with blue channel image, red channel image super under YCbCr color space
Resolution reconstruction image is transformed under RGB color and obtains super-resolution reconstruction image Y.
Another object of the present invention is to provide the single frames figures returned described in a kind of implementation based on spectrum mixed nucleus Gaussian process
As the medical image processing system of super resolution ratio reconstruction method.
Another object of the present invention is to provide the single frames figures returned described in a kind of implementation based on spectrum mixed nucleus Gaussian process
As the monitor video processing system of super resolution ratio reconstruction method.
Another object of the present invention is to provide the single frames figures returned described in a kind of implementation based on spectrum mixed nucleus Gaussian process
As the TV image processing system of super resolution ratio reconstruction method.
Another object of the present invention is to provide the single frames figures returned described in a kind of implementation based on spectrum mixed nucleus Gaussian process
As the Remote Sensing Image Processing System of super resolution ratio reconstruction method.
In conclusion advantages of the present invention and good effect are as follows:
First, the present invention leads to training set because distance function is not representational high for existing super resolution ratio reconstruction method
Spectrum mixed kernel function is introduced into Gauss regression model, learns by the problem that use of information is insufficient, reconstructed image quality is bad
The corresponding relationship of high-low resolution feature in training set.Spectrum mixed kernel function can more accurately characterize the structure between image block
Similitude, so that weight distribution is more reasonable when rebuilding.Fig. 6 illustrate it is different using kernel function when, the weights distribution that acquires with
The relationship of image rotation angle.Through several curve comparisons it can be seen that compared with other kernel functions, spectrum mixed nucleus of the invention
Obtained weight is evenly distributed when corresponding to different rotary angle, and numerical value is relatively flat, and it is the most reasonable to distribute.Table 2 shows this
Invention algorithm and algorithm based on other kernel functions obtain the PSNR comparison of reconstruction image: PSNR value of the invention be it is highest,
Show using spectrum mixed kernel function can be improved the performance of reconstruction image.Fig. 4 is that Head figure reconstruction image is shown.It can by Fig. 4
It include more high-frequency informations to find out that the spectrum mixed kernel function in the present invention rebuilds output image block, reconstruction image is more life-like；
Second, Fig. 3 are that Butterfly figure rebuilds effect displaying.Fig. 3 illustrates the present invention and existing other Image Super-resolutions
Rate algorithm is compared, and reconstruction image texture structure is clear, and high-frequency information is abundant, and visual effect is true to nature.Table 1 illustrates calculation of the present invention
The reconstruction Indexes of Evaluation Effect of method and other algorithms compares, and PSNR value highest of the invention illustrates that its evaluating objective quality is best.
Third, in terms of practical application, the present invention can directly using but be not limited to following several respects: 1) exist as shown in Figure 7
In terms of medical image processing: high-quality size medical image being provided, doctor is helped to improve diagnosis efficiency；2) it is regarded as shown in Figure 8 in monitoring
Frequency aspect: improving the clarity of monitor video, image, is conducive to related personnel's viewing and extracts information；3) as shown in Figure 9 in electricity
In terms of visible image: improving the resolution ratio and visual effect of television image, the user experience is improved；4) as shown in Figure 10 in remote sensing figure
As processing aspect: improving target or the resolution ratio of area-of-interest in satellite remote sensing images, be convenient for target detection, extraction, classification
The expansion of equal follow-up works.As it can be seen that the present invention can be applied directly in fields such as medical treatment, public security, consumption and scientific researches, but also
It can be generalized to more areas, therefore the present invention has very big application prospect.In addition, hardware platform cost needed for the present invention
Cheap, code module is concisely clear, can apply in independence and mobile image processing platform.
Detailed description of the invention
Fig. 1 is the single-frame images super-resolution rebuilding provided in an embodiment of the present invention returned based on spectrum mixed nucleus Gaussian process
Method overview flow chart；It include training stage and test phase in figure.
Fig. 2 is the single-frame images super-resolution rebuilding provided in an embodiment of the present invention returned based on spectrum mixed nucleus Gaussian process
Method flow diagram.
Fig. 3 is the single-frame images super-resolution rebuilding provided in an embodiment of the present invention returned based on spectrum mixed nucleus Gaussian process
The reconstruction Contrast on effect schematic diagram of method and other methods.
Fig. 4 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The reconstruction Contrast on effect schematic diagram of method and the image super-resolution method based on other kernel functions.
Fig. 5 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The relation curve comparison diagram of the reconstruction effect and parameter K of method and the image super-resolution method based on other kernel functions.
Fig. 6 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The relation curve comparison diagram of the reconstruction weight and rotation angle of method and the image super-resolution method based on other kernel functions.
Fig. 7 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The medical image processing system figure of method.
Fig. 8 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The monitor video processing system figure of method.
Fig. 9 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The TV image processing system figure of method.
Figure 10 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding returned based on spectrum mixed nucleus Gaussian process
The Remote Sensing Image Processing System figure of method.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to embodiments, to the present invention
It is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not used to
Limit the present invention.
Being likely to occur for the prior art makes super-resolution rebuilding image detail because of the poor distance function of service performance
The problems such as fuzzy, high-frequency information deficiency, texture structure missing, reconstruction quality are low and application range is limited, the present invention is based on spectrums
The single-frame images super-resolution rebuilding algorithm that mixed nucleus Gaussian process returns, it is fuzzy to reduce details, increase image high-frequency information,
Improve the quality of reconstruction image.Meanwhile algorithm software and low in hardware cost, it can also be transported on low-level hardware platform in
Row；Independence and mobile image processing platform are equally applicable.The present invention, which composes mixed nucleus, can relatively accurately reflect there is complexity
Similitude between the image block of texture structure, the information in training set can be utilized rationally, to restore more high
Frequency detailed information improves the reconstruction effect of image.
To solve the above problems, below with reference to concrete scheme, the present invention is described in detail.
As shown in Figure 1, the single-frame images super-resolution rebuilding that the embodiment of the present invention is returned based on spectrum mixed nucleus Gaussian process
Method chooses high-resolution natural image in the training stage, and carries out down-sampling to it using bi-cubic interpolation method and obtain low point
Resolution training image, then interpolation amplification is carried out to low resolution training image and obtains corresponding interpolation image；To high resolution graphics
Picture and interpolation image carry out piecemeal and extract feature, form training dataset；Training dataset is clustered, K training is obtained
Data subset；This K training data subset is learnt to obtain the optimal super of Gaussian process regression model using spectrum mixed kernel function
Parameter θ；In test phase, a width low-resolution image is chosen as input picture；Bi-cubic interpolation is carried out to input picture to put
Interpolation test image is obtained greatly, and to its piecemeal and feature extraction, obtains low resolution test data set；In training dataset
The cluster centre nearest with each feature in test data set is found in K cluster centre, forms arest neighbors training data
Collection, is then returned using the optimal hyper parameter θ for previously learning to obtain, obtains high-definition picture feature set；Then by this
A little high-definition picture features are added in interpolation test image, reconstruct super-resolution reconstruction image.
As shown in Fig. 2, the single-frame images super-resolution rebuilding that the embodiment of the present invention is returned based on spectrum mixed nucleus Gaussian process
Method specifically includes the following steps:
S101 chooses high-definition picture and forms high-resolution training image collection；Down-sampling and interpolation are carried out to its element
Amplifying operation obtains interpolation training image collection；It extracts feature and forms training dataset.
S102 clusters training dataset to obtain training data subset and corresponding K cluster centre.
S103 goes out the optimal super of the Gaussian process regression model based on spectrum mixed nucleus to each training data trained
Parameter.
S104 reads low resolution test image and constructs low resolution test data set X.
S105 finds the cluster centre nearest with each feature in test data set X in K cluster centre, is formed
Arest neighbors training data subset.
S106 carries out Gaussian process recurrence using the optimal hyper parameter that training obtains in step S103, obtains high-resolution
Feature set F.
S107 exports super-resolution reconstruction image Y.
Wherein, in step S101, extracting feature and forming training dataset is C={ H, L }, wherein H and L respectively indicate it is high,
Low resolution training dataset.In step S102, training data subset isIn step S103, each training data
Subset is Ci, optimal hyper parameter θi, wherein θiFor column vector.
In embodiments of the present invention, step S101 concrete operations include:
Step 1.1, several high-resolution natural images are chosen and it is transformed into YCbCr face from RGB color
The colour space；It chooses luminance picture and forms high-resolution training image collection.
Step 1.2, selecting current super-resolution rebuilding amplification factor is S=3, high-resolution training image collection is utilized double
Cube S times of interpolation down-sampling recycles bi-cubic interpolation to amplify S times, obtains interpolation training image collection.
Step 1.3, selected digital image block size is N × N (N=7 is odd number), is instructed to high-definition picture training set and interpolation
Corresponding image takes image block according to sequence from top to bottom, from left to right pixel-by-pixel in white silk image set；Then high and low resolution is asked
The difference of rate image block center pixel, obtains high-resolution training datasetTo low-resolution image block vector
Change, obtains low resolution training datasetWherein P=100000 (the maximum block number that P is less than the image block got).
Step S102 is specifically included:
Step 2.1, class number K=35 is set, K-means cluster is carried out to low resolution training dataset L, obtains { L1,
L2..., LKAnd corresponding cluster centre collection V=(o1, o2..., oK)。
Step 2.2, corresponding high-resolution data is found in high-resolution training dataset H forms { H1, H2...,
HK}。
Step 2.3, merge { L1, L2..., LKAnd { H1, H2..., HKForm { C1, C2..., CK, wherein i-th Ci
={ Li, HiIt is known as training data subset.
Step S103 is specifically included:
Step 3.1, hyper parameter θ is definedi。
If training data subset Ci={ Li, HiContaining Z data pair, log-likelihood function is
Wherein θiFor hyper parameter to be learned, (Hi)、(Li) it is respectively CiIn high-resolution and low-resolution training data subset Hi、
LiElement, | |1Indicate 1- norm, ()TExpression takes transposition operation, Ky(l, l ' | θi) indicate that covariance matrix, formula are
Wherein k (l, l ') includes hyper parameter θi, expression formula are as follows:
WhereinAnd ω thereinQ, i、∑Q, iAnd μQ, iRespectively indicate i-th of trained number
According to the corresponding weight of q-th in subset cumulative parameter, variance and frequency parameter,Indicate noise criteria difference parameter, Q=15 is to set
Fixed cumulative parameter, | | indicate the operation that takes absolute value, | | | | indicate the operation of amount of orientation modulus value, function COS's () determines
Justice is
Wherein lpWith l 'pP-th of component of vector l and l ' is respectively indicated, wherein μQ, i (p)Indicate μQ, iIn p-th of component,
The dimension of U expression vector l；The definition of function δ () is
Step 3.2, hyper parameter is sought using iterative method.
In a preferred embodiment of the invention, step 3.2 specifically includes:
Step 3.2.1 constructs partial derivative
WhereinTr () indicates to seek the mark of matrix.The element of middle position (s, t) is
WhereinIndicate LiIn s-th of vector lsP-th of component, s, t=1,2 ..., Z.The member of middle position (s, t)
Element is
Wherein ∑Q, i (j)Indicate ∑Q, iIn j-th of component, j=1,2 ..., U.The element of middle position (s, t) is
Wherein μQ, i (j)Indicate μQ, iIn j-th of component.The element of middle position (s, t) is
Step 3.2.2, if being currently kth (k >=3) step,Iterative parameter value is updated, wherein
Wherein
dk-1=(gk-1)Tsk-2
P=6 (fk-1-fk)+3(dk-1+dk)(vk-1-vk-2)
Q=3 (fk-fk-1)-(2dk-1+dk)(vk-1-vk-2)
Wherein
If fk< fopt, then enablefk→fopt, and continue iterative process；Wherein foptIteration mistake before expression
Minimum likelihood value in journey, θoptOptimized parameter before expression in iterative process.
Step 3.2.3 sets termination condition: if
Or circulation step number terminates iterative process, θ when reaching maximum set value 500i=θoptAs optimal hyper parameter,
Middle τ=0.1 and γ=0.05 are the control coefrficient of setting.
Step 3.2.4 initializes iteration variable:
Initialize hyper parameterWherein
Wherein std () expression takes standard difference operation, and max () expression takes greatest member in matrix, and min () expression takes
Least member in matrix.Wherein RiFor high-resolution data subset LiScale Matrixes, expression formula is
Wherein column vector m1..., mZBy
Li-mean(Li)=[m1, m2... mZ]
It obtains, wherein mean () expression takes mean operation.
The initial value of other iteration variables includes
d0=d1=-(s0)Ts0
v1=0
fopt=f0
In embodiments of the present invention, step S104 is specifically included:
Step 4.1, low resolution colour chart picture is read, is amplified S times using bi-cubic interpolation and from RGB color
Space is transformed into YCbCr color space, and respectively obtains luminance channel image, blue channel image and red channel image.
Step 4.2, to luminance channel image with from top to bottom, sequence from left to right takes block and pixel-by-pixel by its vectorization
Obtain low resolution test data setWherein M indicates desirable maximum image block number, test data xnIt indicates in X
N-th of vector.
Step S105 is specifically included:
Step 5.1, m-th of vector x in X is calculatedmTo cluster centre collection V=(o1, o2..., oK) Euclidean distance obtain
Its distance vector
Wherein E (x, x ') indicates to calculate the Euclidean distance between vector x and x '.
Step 5.2, to SmElement in (m=1,2..., M) sorts and finds out least member, and it is corresponding poly- to mark its
The class number a at class centerm∈ [1, K].
Step 5.3, the operation in step 5.1 and step 5.2 is executed to institute's directed quantity in X, obtains label vector
Wherein m-th of element amFor corresponding low resolution arest neighbors training data subsetClass number, andFor xmArest neighbors training data subset,It isCorresponding high-resolution arest neighbors training data
Collection；Corresponding optimal hyper parameterFor xmArest neighbors hyper parameter.
Step S106 is specifically included:
Step 6.1, formula is utilized
Calculate xmCorresponding output feature ym, obtain high-resolution features collectionWhereinBe class number be am
Low resolution arest neighbors training data subset；It isCorresponding high-resolution arest neighbors training data subset；Wherein
Wherein l1, l2..., lMForMiddle whole vector.Calculate spectrum mixed kernel functionUsedIt is exactly step
Arest neighbors hyper parameter obtained in 5.3.
Step S107 is specifically included:
Step 7.1, by formula ym=ym+cen(xm) high-definition picture pixel is calculated, and replaced brightness
Corresponding pixel obtains brightness reconstruction image in channel image, and wherein cen () expression takes center pixel to operate.
Step 7.2, it is empty that brightness reconstruction image is obtained to YCbCr color in conjunction with blue channel image, red channel image
Between under super-resolution rebuilding image, be transformed under RGB color and obtain super-resolution reconstruction image Y.
Below with reference to emulation experiment, the invention will be further described.
(1) simulated conditions
It is Intel i5-5220 3.30GHz that experiment of the invention, which is in CPU, inside saves as 6G, operating system Windows
10, emulation platform be Matlab 2015a experimental situation under carry out.
In emulation experiment, by the method for the present invention and existing SCSR, BPJDL, SRGPR, the methods of AGPR are compared point
Analysis；Wherein
It is J.Yang, J.Wright, T.Huang, and Y.Ma, " Image super- that SCSR, which corresponds to bibliography,
resolution via sparse representation,”IEEE Trans.Image Process.,vol.19,no.11,
pp.2861-2873。
It is L.He, H.Qi, and R.Zaretzki, " Beta process joint that BPJDL, which corresponds to bibliography,
dictionary learning for coupled feature spaces with application to single
image super-resolution”,in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.,2013,
pp.345-352。
It is H.He and W.C.Siu, " Single image super-resolution that SRGPR, which corresponds to bibliography,
using Gaussian process regression”,in Proc.IEEE Conf.Comput.Vis.Pattern
Recognit.,2011,pp.449-456。
It is H.Wang, X.Gao, K.Zhang, and J.Li " Single-Image Super- that AGPR, which corresponds to bibliography,
Resolution Using Active-Sampling Gaussian Process Regression”,IEEE
transactions on image processing,vol.25,no.2,February 2016,pp.935-947。
(2) emulation content
Experiment one: the verifying present invention has preferable super-resolution rebuilding effect.
Emulation testing is carried out to Butterfly image with the present invention and above-mentioned existing 4 kinds of methods, as a result as shown in Figure 3.
Wherein, Fig. 3 (a) is the result of SCSR Super-resolution Reconstruction；Fig. 3 (b) is the result of BPDJL Super-resolution Reconstruction；Fig. 3 (c) is SRGPR
The result of Super-resolution Reconstruction；Fig. 3 (d) is the result of AGPR Super-resolution Reconstruction；Fig. 3 (e) is the result of Super-resolution Reconstruction of the present invention；
Fig. 3 (f) is true high-definition picture.Each image has the rectangular area of partial enlargement in order to observe the difference for rebuilding effect
Not.
The simulation result of Fig. 3, which illustrates other methods, cannot preferably recover the complicated line of butterfly's wing decorative pattern in image
Structure is managed, and Fig. 3 (e) then obviously has preferably reconstruction effect.Comparison is it is found that super-resolution result of the invention can be weighed preferably
Complicated texture structure is built out, details is presented preferably, and the visual effect of reconstruction image is better than other four kinds of methods.
1. inventive algorithm of table and other four kinds of algorithms rebuild effectiveness indicator comparison, and (Butterfly schemes, and amplification factor S is
3)
| SCSR | BPDJL | SRGPR | AGPR | The present invention | |
| PSNR(dB) | 25.63 | 26.44 | 25.66 | 26.47 | 26.54 |
Table 1 illustrates the present invention and other four kinds comparison algorithms obtain the PSNR comparison of reconstruction image.It can be seen that this hair
The PSNR of bright reconstruction image is highest, to objectively demonstrate reconstruction effect of the invention.
Experiment two: help of the spectrum mixed kernel function to effect is rebuild in the verifying present invention.
Kernel function in the present invention is replaced with into radial radix core and linear kernel function respectively, other steps are constant, with this
Invention and above two method carry out emulation testing to Head image, as a result as shown in Figure 4.Wherein, Fig. 4 (a) is to have used diameter
To the reconstructed results of radix core；Fig. 4 (b) is the reconstructed results for having used linear kernel.Fig. 4 (c) is the reconstructed results of this method；Figure
4 (d) be true high-definition picture.Each image has the rectangular area of partial enlargement in order to observe the difference for rebuilding effect
Not.
Fig. 4 can be seen that the reconstruction effect of the spectrum mixed kernel function in the present invention significantly better than other kernel functions, Fig. 4 (c)
The spot presentation of middle child's face is better than other methods, compares it is found that the spectrum mixed kernel function in the present invention has good property
Can, the effect for improving reconstruction can be helped.
The evaluation of the reconstruction effectiveness indicator of 2. inventive algorithm of table and the algorithm based on other kernel functions (Head figure, times magnification
3) number S is
| Radial base core | Linear kernel | It composes mixed nucleus (present invention) | |
| PSNR(dB) | 33.29 | 33.32 | 33.53 |
Table 2 illustrates inventive algorithm and the algorithm based on other kernel functions obtains the PSNR comparison of reconstruction image.It can be with
Find out, spectrum mixed kernel function used in the present invention has very great help to the promotion of reconstruction image PSNR, this is because spectrum is mixed
Synkaryon function can reasonably distribute weight when rebuilding, and capture the more high-frequency informations of image block.
Fig. 5 is the relationship of the present invention with other Kernels reconstructed image quality based on Kodak image set and parameter K
Curve comparison figure, abscissa are K value, and ordinate is PSNR value.Parameter K indicates cluster class number, and K value is too small, leads to single training
Redundancy is excessive in data subset, influences the quality of reconstruction image；Data are very few in the excessive then training data subset of K value, lead
It causes the high-frequency information of reconstruction image insufficient, reduces reconstruction effect.It is discovered by experiment that the best value of K is 35 in the present invention.
Simultaneously it can be seen from the figure that different kernel function counterweights build being affected for effect, the mathematics original of linear kernel and radial base core
Relatively simple, to be unable between accurate characterization image block similitude is managed, and the spectrum mixed kernel function in the present invention then can be more
The distance between image block is accurately calculated, so that the PSNR of reconstruction image is higher, experiments have shown that the core letter that the present invention uses
Number is rebuild effect and is better than using radial base core or linear kernel.
Fig. 6 is the relation curve comparison diagram of the present invention with other Kernels rotation angle and reconstruction weight, to input
Image does different angle and rotates to obtain training image, and abscissa is the angle of image block rotation in training set, and ordinate is to rebuild
Weight.The rotation of image block does not change its texture structure, while rotating the center pixel of front and back image block there is no variation,
So the change for theoretically rotating angle is smaller to the influence for rebuilding weight, and this weight then illustrates algorithm for instruction closer to 1
Practice and concentrate the distribution of characteristics of image weight more reasonable, the accuracy that kernel function measures distance between characteristics of image is higher.From Fig. 6
As can be seen that the weight variation of homologous thread of the present invention is relatively stable, and no matter rotate how angle changes, obtained by the present invention
Weight all than other algorithms closer to 1, this illustrates that the present invention is the most reasonable for the distribution for rebuilding weight, in the present invention
The distance between characteristics of image can more be characterized compared to other kernel functions by composing mixed kernel function, can more capture the structure phase of image block
Like property.
Below with reference to the application of the single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process to this
Invention is further described.
In embodiments of the present invention, Fig. 7 is that the embodiment of the present invention provides the single frames returned based on spectrum mixed nucleus Gaussian process
The medical image processing system block diagram of image super-resolution rebuilding method.In system in Fig. 7, the low resolution medicine figure of input
As after the present invention is rebuild, the full resolution pricture of output is clear, and the lesions position in image is clear, and lesion phenomenon is bright
It is aobvious, facilitate medical worker and judge the state of an illness, improves diagnosis rate.
Fig. 8 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The monitor video processing system block diagram of method.Monitor video is widely used in road safety, the fields such as criminal investigation monitoring.However due at
Factors, the imaging effects such as picture environment is poor, and imaging h ardware is limited tend not to reach requirement.In the system of fig. 8, in monitoring device
The original image of collected face is fuzzy, and high-frequency information is insufficient, is not able to satisfy the expansion of the work such as subsequent criminal investigation；The present invention will
Acquired image is as input picture in monitoring device, and the image clearly after the present invention is rebuild, detail increases, thus
Be conducive to the follow-up works such as face identification.
Fig. 9 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding side returned based on spectrum mixed nucleus Gaussian process
The TV image processing system block diagram of method.In video display entertainment field, user is higher and higher to the quality requirement of television image.Due to
The limitation of bandwidth, the image signal source that user receiving end obtains tend not to meet the requirement to resolution ratio, electricity as shown in Figure 9
It is poor depending on original image visual effect, it is distorted；And the television image high resolution after the present invention is rebuild, visual effect
Improve, optimizes the experience of user.
Figure 10 is that the embodiment of the present invention provides the single-frame images super-resolution rebuilding returned based on spectrum mixed nucleus Gaussian process
The Remote Sensing Image Processing System block diagram of method.Remote sensing images are in military affairs, scientific research, the fields such as space flight extensive application；However it is distant
It is influenced when sense equipment imaging by factors such as state of flight, atmospheric refraction, hypsographies, original image occurs that resolution ratio is low, texture
The problems such as structural fuzzy；As shown in Figure 10, the remote sensing images specific region resolution ratio after the present invention is rebuild improves, texture knot
Structure is clear, convenient for the expansion of the follow-up works such as target detection, extraction, classification.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Made any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.
Claims (10)
1. a kind of single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process, which is characterized in that described
It is natural to choose high-resolution in the training stage for single-frame image super-resolution reconstruction method based on spectrum mixed nucleus Gaussian process recurrence
Image, and carry out down-sampling to it using bi-cubic interpolation method and obtain low resolution training image, then low resolution training is schemed
Corresponding interpolation image is obtained as carrying out interpolation amplification；Piecemeal is carried out to high-definition picture and interpolation image and extracts feature,
Form training dataset；Training dataset is clustered, K training data subset is obtained；This K training data subset is utilized
Spectrum mixed kernel function learns to obtain the optimal hyper parameter θ of Gaussian process regression model；
In test phase, low-resolution image is chosen as input picture；Bi-cubic interpolation is carried out to input picture to amplify to obtain
Interpolation test image, and piecemeal and feature extraction are carried out, obtain low resolution test data set；
The cluster centre nearest with each feature in test data set, shape are found in K cluster centre of training dataset
At arest neighbors training data subset, recycles the optimal hyper parameter θ for previously learning to obtain to be returned, obtain high-definition picture
Feature set；These high-definition picture features are added in interpolation test image, high-resolution result images are reconstructed.
2. the single-frame image super-resolution reconstruction method returned as described in claim 1 based on spectrum mixed nucleus Gaussian process,
It is characterized in that, the single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process specifically includes following step
It is rapid:
Step 1 chooses high-definition picture and forms high-resolution training image collection；To high-resolution training image collection element into
Row down-sampling and interpolation magnification operation obtain interpolation training image collection；It extracts feature and forms training dataset C={ H, L }, wherein H
High-resolution and low-resolution training dataset is respectively indicated with L；
Step 2 clusters training dataset C to obtain training data subsetAnd corresponding K cluster centre；
Step 3, to each training data subset CiTrain the optimal super of the Gaussian process regression model based on spectrum mixed nucleus
Parameter θi, θiFor column vector；
Step 4 reads low resolution test image and constructs low resolution test data set X；
Step 5 is found in the cluster nearest with each feature in low resolution test data set X in K cluster centre
The heart forms arest neighbors training data subset；
Step 6 utilizes the θ that training obtains in step 3iGaussian process recurrence is carried out, high-resolution features collection F is obtained；
Step 7 exports high-definition picture Y.
3. the single-frame image super-resolution reconstruction method returned as claimed in claim 2 based on spectrum mixed nucleus Gaussian process,
It is characterized in that, step 1 specifically includes:
It chooses several high-resolution natural images and is transformed into YCbCr color space from RGB color；Choose luminance graph
As forming high-resolution training image collection；
Selecting current super-resolution rebuilding amplification factor is S=3, by high-resolution training image collection using adopting under bi-cubic interpolation
S times of sample recycles bi-cubic interpolation to amplify S times, obtains interpolation training image collection；
Selected digital image block size is N × N, and N=7 is odd number, to high-definition picture training set and interpolation training image concentration pair
The image answered takes image block according to sequence from top to bottom, from left to right pixel-by-pixel；Then high-resolution and low-resolution image block center is asked
The difference of pixel obtains high-resolution training datasetTo low-resolution image block vectorization, low resolution is obtained
Rate training datasetWherein P=100000, P are less than the maximum block number for the image block got；
Step 2 specifically includes:
Step 1, class number K=35 is set, K-means cluster is carried out to low resolution training dataset L, obtains { L1, L2...,
LKAnd corresponding cluster centre collection V=(o1, o2..., oK}；
Step 2, corresponding high-resolution data is found in high-resolution training dataset H forms { H1, H2..., HK}；
Step 3, merge { L1, L2..., LKAnd { H1, H2..., HKForm { C1, C2..., CK, wherein i-th Ci={ Li,
HiIt is known as training data subset.
4. the single-frame image super-resolution reconstruction method returned as claimed in claim 2 based on spectrum mixed nucleus Gaussian process,
It is characterized in that, step 3 specifically includes:
Step A defines hyper parameter θi；
Training data subset Ci={ Li, HiContaining Z data pair, log-likelihood function is
Wherein θiFor hyper parameter to be learned, (Hi)、(Li) it is respectively CiIn high-resolution and low-resolution training data subset Hi、Li's
Element, | |1Indicate 1- norm, ()TExpression takes transposition operation, Ky(l, l ' | θi) indicate that covariance matrix, formula are
Wherein k (l, l ') includes hyper parameter θi, expression formula are as follows:
WhereinAnd ωQ, i、∑Q, iAnd μQ, iIt respectively indicates in i-th of training data subset
The corresponding weight of the cumulative parameter of q, variance and frequency parameter,Indicate noise criteria difference parameter, Q=15 is the cumulative ginseng of setting
Amount, | | indicate the operation that takes absolute value, | | | | indicate the operation of amount of orientation modulus value, the expression formula of function COS () is
Wherein lpWith l 'pP-th of component of vector l and l ' is respectively indicated, wherein μQ, i (p)Indicate μQ, iIn p-th of component, U table
Show the dimension of vector l；Function δ () is
Step B seeks hyper parameter using iterative method.
5. the single-frame image super-resolution reconstruction method returned as claimed in claim 4 based on spectrum mixed nucleus Gaussian process,
It is characterized in that, step B is specifically included:
Step a constructs partial derivative
In formulaTr () indicates to seek the mark of matrix；The element of middle position (s, t) is
In formulaIndicate LiIn s-th of vector lsP-th of component, s, t=1,2 ..., Z；The element of middle position (s, t) is
∑Q, i (i)Indicate ∑Q, iIn j-th of component, j=1,2 ..., U；The element of middle position (s, t) is
μ in formulaQ, i (j)Indicate μQ, iIn j-th of component；The element of middle position (s, t) is
Step b, if currently being walked for kth, k >=3,Iterative parameter value is updated,
In formula
dk-1=(gk-1)Tsk-2；
P=6 (fk-1-fk)+3(dk-1+dk)(vk-1-vk-2)；
Q=3 (fk-fk-1)-(2dk-1+dk)(vk-1-vk-2)；
In formula
If fk< fopt, then enablefk→fopt, and continue iterative process；F in formulaoptBefore expression in iterative process
Minimum likelihood value, θoptOptimized parameter before expression in iterative process；
Step c sets termination condition: if
Or circulation step number terminates iterative process, θ when reaching maximum set value 500i=θoptFor optimal hyper parameter, τ in formula=
0.1 and γ=0.05 be setting control coefrficient；
Step d initializes iteration variable:
Initialize hyper parameterWherein
Std () expression takes standard difference operation in formula, and max () expression takes greatest member in matrix, and min () expression takes matrix
Middle least member；R in formulaiFor high-resolution data subset LiScale Matrixes, expression formula is
Column vector m in formula1..., mzBy
Li-mean(Li)=[m1, m2... mZ]；
It obtains, mean () expression takes mean operation；
The initial value of iteration variable includes
d0=d1=-(s0)Ts0；
fopt=f0；
6. the single-frame image super-resolution reconstruction method returned as claimed in claim 2 based on spectrum mixed nucleus Gaussian process,
It is characterized in that, is specifically included in step 4:
Low resolution colour chart picture is read, is amplified S times using bi-cubic interpolation and is transformed into from RGB color
YCbCr color space, and respectively obtain luminance channel image, blue channel image and red channel image；
To luminance channel image with from top to bottom, sequence from left to right takes block pixel-by-pixel and its vectorization is obtained low resolution
Test data setM indicates the maximum image block number taken, test data xnIndicate n-th of vector in X；
Step 5 specifically includes:
Calculate m-th of vector x in XmTo cluster centre collection V=(o1, o2..., oK) Euclidean distance obtain distance vector
E (x, x ') indicates to calculate the Euclidean distance between vector x and x '；
To SmElement in (m=1,2..., M) sorts and finds out least member, marks the class number a of corresponding cluster centrem∈
[1, K]；
Operation is executed to institute's directed quantity in X, obtains label vector
M-th of element a in formulamFor corresponding low resolution arest neighbors training data subsetClass number,
For xmArest neighbors training data subset,It isCorresponding high-resolution arest neighbors training data subset；It is corresponding most
Excellent hyper parameterFor xmArest neighbors hyper parameter；
Step 6 specifically includes:
Utilize formula
Calculate xmCorresponding output feature ym, obtain high-resolution features collection Be class number be amLow resolution
Rate arest neighbors training data subset；It isCorresponding high-resolution arest neighbors training data subset；Wherein
L in formula1, l2..., lMForMiddle whole vector；Calculate spectrum mixed kernel functionUsedIt is that step 5 obtains
Arest neighbors hyper parameter；
Step 7 specifically includes:
By formula ym=ym+cen(xm) high-definition picture pixel is calculated, and replace corresponding in luminance channel image
Pixel obtains brightness reconstruction image, and cen () expression takes center pixel to operate；
Brightness reconstruction image is obtained into the super-resolution under YCbCr color space in conjunction with blue channel image, red channel image
Rate reconstruction image is transformed under RGB color and obtains super-resolution reconstruction image Y.
7. a kind of single-frame image super-resolution reconstruction method for implementing to return described in claim 1 based on spectrum mixed nucleus Gaussian process
Medical image processing system.
8. a kind of single-frame image super-resolution reconstruction method for implementing to return described in claim 1 based on spectrum mixed nucleus Gaussian process
Monitor video processing system.
9. a kind of single-frame image super-resolution reconstruction method for implementing to return described in claim 1 based on spectrum mixed nucleus Gaussian process
TV image processing system.
10. a kind of single-frame images super-resolution rebuilding side for implementing to return described in claim 1 based on spectrum mixed nucleus Gaussian process
The Remote Sensing Image Processing System of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910194123.6A CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910194123.6A CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110097499A true CN110097499A (en) | 2019-08-06 |
| CN110097499B CN110097499B (en) | 2023-05-05 |
Family
ID=67443986
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910194123.6A Active CN110097499B (en) | 2019-03-14 | 2019-03-14 | Single-frame image super-resolution reconstruction method based on spectrum mixing kernel Gaussian process regression |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110097499B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112330541A (en) * | 2020-11-11 | 2021-02-05 | 广州博冠信息科技有限公司 | Live video processing method and device, electronic equipment and storage medium |
| CN112581367A (en) * | 2020-12-01 | 2021-03-30 | 南京理工大学 | GPR image super-resolution reconstruction method based on random sample classification amplification |
| CN113435487A (en) * | 2021-06-16 | 2021-09-24 | 中国电子科技集团公司第五十四研究所 | Deep learning-oriented multi-scale sample generation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107993194A (en) * | 2017-11-30 | 2018-05-04 | 天津大学 | A kind of super resolution ratio reconstruction method based on Stationary Wavelet Transform |
| CN108257093A (en) * | 2018-01-18 | 2018-07-06 | 洛阳理工学院 | The single-frame images ultra-resolution method returned based on controllable core and Gaussian process |
| US20180225807A1 (en) * | 2016-12-28 | 2018-08-09 | Shenzhen China Star Optoelectronics Technology Co., Ltd. | Single-frame super-resolution reconstruction method and device based on sparse domain reconstruction |
-
2019
- 2019-03-14 CN CN201910194123.6A patent/CN110097499B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180225807A1 (en) * | 2016-12-28 | 2018-08-09 | Shenzhen China Star Optoelectronics Technology Co., Ltd. | Single-frame super-resolution reconstruction method and device based on sparse domain reconstruction |
| CN107993194A (en) * | 2017-11-30 | 2018-05-04 | 天津大学 | A kind of super resolution ratio reconstruction method based on Stationary Wavelet Transform |
| CN108257093A (en) * | 2018-01-18 | 2018-07-06 | 洛阳理工学院 | The single-frame images ultra-resolution method returned based on controllable core and Gaussian process |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112330541A (en) * | 2020-11-11 | 2021-02-05 | 广州博冠信息科技有限公司 | Live video processing method and device, electronic equipment and storage medium |
| CN112581367A (en) * | 2020-12-01 | 2021-03-30 | 南京理工大学 | GPR image super-resolution reconstruction method based on random sample classification amplification |
| CN112581367B (en) * | 2020-12-01 | 2023-06-30 | 南京理工大学 | GPR image super-resolution reconstruction method based on random sample classification amplification |
| CN113435487A (en) * | 2021-06-16 | 2021-09-24 | 中国电子科技集团公司第五十四研究所 | Deep learning-oriented multi-scale sample generation method |
| CN113435487B (en) * | 2021-06-16 | 2022-08-12 | 中国电子科技集团公司第五十四研究所 | Deep learning-oriented multi-scale sample generation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097499B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yao et al. | Enhancing pseudo label quality for semi-supervised domain-generalized medical image segmentation | |
| Jiang et al. | Noise robust face image super-resolution through smooth sparse representation | |
| Wang et al. | Sparse representation-based MRI super-resolution reconstruction | |
| CN105069746B (en) | Video real-time face replacement method and its system based on local affine invariant and color transfer technology | |
| Li et al. | From beginner to master: A survey for deep learning-based single-image super-resolution | |
| CN107665492A (en) | Colon and rectum panorama numeral pathological image tissue segmentation methods based on depth network | |
| CN107784319A (en) | A kind of pathological image sorting technique based on enhancing convolutional neural networks | |
| CN110097499A (en) | The single-frame image super-resolution reconstruction method returned based on spectrum mixed nucleus Gaussian process | |
| Wei et al. | Medical image super-resolution by using multi-dictionary and random forest | |
| CN110414607A (en) | Classification method, device, equipment and the medium of capsule endoscope image | |
| Xiao et al. | Segmentation of multispectral high-resolution satellite imagery using log Gabor filters | |
| CN111932529A (en) | Image segmentation method, device and system | |
| CN108764368A (en) | A kind of image super-resolution rebuilding method based on matrix mapping | |
| Kanimozhi et al. | Brain MR image segmentation using self organizing map | |
| CN110648331A (en) | Detection method for medical image segmentation, medical image segmentation method and device | |
| CN108257093A (en) | The single-frame images ultra-resolution method returned based on controllable core and Gaussian process | |
| AU2015218184B2 (en) | Processing hyperspectral or multispectral image data | |
| CN109146785A (en) | A kind of image super-resolution method based on the sparse autocoder of improvement | |
| CN111695455A (en) | Low-resolution face recognition method based on coupling discrimination manifold alignment | |
| CN105512670A (en) | HRCT peripheral nerve cutting based on KECA feature dimension reduction and clustering | |
| Zhu et al. | Super resolution reconstruction method for infrared images based on pseudo transferred features | |
| Lu et al. | Infrared and visible image fusion based on tight frame learning via VGG19 network | |
| Mahapatra | Registration of histopathogy images using structural information from fine grained feature maps | |
| Zheng et al. | Overwater image dehazing via cycle-consistent generative adversarial network | |
| Luo et al. | Infrared and Visible Image Fusion: Methods, Datasets, Applications, and Prospects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |