CN106251297A - A kind of estimation based on multiple image fuzzy core the rebuilding blind super-resolution algorithm of improvement - Google Patents

Abstract

The invention discloses the method for rebuilding blind super-resolution estimated based on multiple image fuzzy core of a kind of improvement.Mainly comprise the steps that the first frame picture in the frame of video of the low resolution to input carries out Fuzzy Processing in various degree, obtain the image of two width difference fog-levels of Same Scene, obtain the picture of two different fog-levels；Utilize the picture of the different fog-levels of two Same Scene obtained above, its deconvolution algorithm using robustness is generated a rough fuzzy core, and by this fuzzy core, all of frame of video is carried out deblurring process, one group of sequence of pictures f after being processed^k, as the input of later process；Utilizing curvature difference operator extraction spatial structural form, then it is carried out cluster and obtain the adaptive weighted coefficient of regional space, this coefficient is for carrying out adaptive weighted to full variation and non-local mean regularization term；The adaptive weighted coefficient that utilization is previously obtained is to weight regularization term, so that it is determined that rebuild cost function；Utilize gradient descent method to carry out optimization and rebuild cost function, each iterative process wherein carries out a fuzzy core again and estimates, and carry out deblurring, finally give the high-definition picture sequence of output.

Description

A kind of estimation based on multiple image fuzzy core the rebuilding blind super-resolution algorithm of improvement

Technical field

The present invention relates to image super-resolution rebuilding technology, be specifically related to estimating based on multiple image fuzzy core of a kind of improvement The rebuilding blind super-resolution algorithm of meter, belongs to digital image arts.

Background technology

Image and video as the carrier of visual information, are mankind's important way of obtaining and transmitting information, therefore, research It is of great significance with video information tool with processing image.Along with the development of informationization technology, people's letter to receiving The requirement of breath is more and more higher, especially in the applications such as medical science, remote sensing, astronomy and video monitoring, is required for obtaining height The video of resolution.But when actual acquisition video, suffer from the aberration by light, lack sampling, atmospheric perturbation, defocus with And the impact of the factor such as system noise, the spatial resolution of the video obtained is the highest.Improve the hardware device in imaging system It is to improve the approach that video resolution is relatively simple, but cost is the highest, so it is contemplated that being improved by the method for software The resolution of picture and video is to meet our needs.

Fuzzy is one of the key factor of video degenerative process, and the video collected not only is obscured by sensor own optical Impact, simultaneously the most also, motion blur fuzzy by ambient atmosphere and defocusing blurring etc. are affected.At super-resolution rebuilding algorithm In, need fuzzy core is evaluated to can the degenerative process of analog imaging more accurately, to improve super-resolution further Rebuild the quality of video.Traditional fuzzy core based on boundary gradient change is estimated, need to be chosen on image two pieces uniform bright The straight border of dark areas is as sword limit, owing to being difficult to find enough strong edges in the low-resolution image of input, so The problem that fuzzy core is estimated becomes particularly difficult.In current most of super-resolution rebuilding algorithms, usually assume that imaging system The point spread function of system is previously known or assumes that fuzzy core has simple analytical form (such as Gaussian form), has even Do not account for dropping clear process, do not meet the real imaging model of optical device, therefore limit algorithm in different real scenes Application.

Summary of the invention

The purpose of the present invention is that provides estimating based on multiple image fuzzy core of a kind of improvement for solving the problems referred to above The rebuilding blind super-resolution algorithm of meter.

The rebuilding blind super-resolution algorithm estimated based on multiple image fuzzy core of a kind of improvement that the present invention proposes, specifically Can be divided into following step:

(1) the first frame picture in the frame of video of the low resolution of input is carried out Fuzzy Processing in various degree, obtain The image of two width difference fog-levels of Same Scene；

(2) blurred picture utilizing two the different fog-levels obtained in step (1) carries out fuzzy core estimation, and with raw The fuzzy core become carries out the pretreatment of deblurring, sequence of pictures y after being processed to all low resolution video frame_l, as The input of later process；

(3) utilizing curvature difference operator extraction spatial structural form, then it being carried out cluster, to obtain regional space adaptive Answering weight coefficient, this coefficient is for carrying out adaptive weighted to full variation and non-local mean regularization term；

(4) regularization term utilizing the weighting that adaptive weighted coefficient determined by step (3) obtains determines weight Build cost function；

(5) utilize gradient descent method to carry out optimization and rebuild cost function, wherein each iterative process uses in step (2) Fuzzy core method of estimation carry out fuzzy core again and estimate, and carry out deblurring, the high-definition picture finally exported Sequence.

Accompanying drawing explanation

Fig. 1 is the rebuilding blind super-resolution algorithm principle block diagram that the present invention estimates based on multiple image fuzzy core

Fig. 2 is video " Akiyo " low-resolution frames and the reconstructed results using algorithms of different

Fig. 3 is video " Teddy " low-resolution frames and the reconstructed results using algorithms of different

Fig. 4 is video " Suzie " low-resolution frames and the reconstructed results using algorithms of different

Fig. 5 is real video Broadcast original low-resolution frame and the reconstructed results using algorithms of different

Detailed description of the invention

The invention will be further described below in conjunction with the accompanying drawings:

In Fig. 1, the rebuilding blind super-resolution algorithm estimated based on multiple image fuzzy core of a kind of improvement, specifically can divide For following step:

(1) the first frame picture in the frame of video of the low resolution of input is carried out Fuzzy Processing in various degree, obtain The image of two width difference fog-levels of Same Scene；

(2) blurred picture utilizing two the different fog-levels obtained in step (1) carries out fuzzy core estimation, and with raw The fuzzy core become carries out the pretreatment of deblurring, sequence of pictures y after being processed to all low resolution video frame_l, as The input of later process；

(3) utilizing curvature difference operator extraction spatial structural form, then it being carried out cluster, to obtain regional space adaptive Answering weight coefficient, this coefficient is for carrying out adaptive weighted to full variation and non-local mean regularization term；

(4) regularization term utilizing the weighting that adaptive weighted coefficient determined by step (3) obtains determines weight Build cost function；

(5) utilize gradient descent method to carry out optimization and rebuild cost function, wherein each iterative process uses in step (2) Fuzzy core method of estimation carry out fuzzy core again and estimate, and carry out deblurring, the high-definition picture finally exported Sequence.

Specifically, in described step (1), we are to use a kind of fuzzy core algorithm for estimating in image restoration, and it needs The picture of the different fog-levels of Same Scene carries out fuzzy core estimation, so we are first to the low-resolution video obtained In the first frame picture carry out two kinds of in various degree fuzzy and obtain two groups of sequence of pictures in various degree.

Described step (2) utilizes two width pictures of the different fog-levels of the Same Scene that step (1) obtains, it is adopted Generate a rough fuzzy core with the deconvolution algorithm of robustness, then carry out deblurring process and obtain the one of corresponding scene The picture of width deblurring, then by this fuzzy core, remaining picture done sequence of pictures y after identical process is processed_l, as The input of later process, concrete fuzzy core solution procedure is as follows:

First introducing the process that degrades of image, high-definition picture f, after degradation model, finally creates low resolution Image sequence g_i, mathematic(al) representation corresponding to whole process is:

g_i=DB_iE_if+n_i, (i=1 ..., p) (1-1)

Wherein, D represents down-sampling process, B_iRepresent blurring process, E_iDenotation coordination conversion process (rotates, translation etc.), n_i Representing that blurring process is mainly estimated by noise and fuzzy core estimation procedure, the fuzzy core solution procedure in the present invention is such as Under:

The process of deblurring in first considering to rebuild, image expression formula after different fog-levels process is:

$g_i=f\⊗h_i+n_i,(i=1,2......n)---(1-2)$

Wherein, f is original high-definition image,For convolution algorithm, h_iFor different fuzzy core, relative with Fuzzy B hereinbefore Should, g_iFor through different Fuzzy Processing and add the image made an uproar.

Above formula is made such as down conversion:

$g_1\⊗h_2-g_2\⊗h_1=n_1\⊗h_2-n_2\⊗h_1-\mathrm{\ϵ}---(1-3)$

ε is proportional to noise variance n₁, n₂Error term, distinguishingly, without having in the case of making an uproar:? In the case of additive white Gaussian noise, work as n_i～(0, σ²), have:

ε～(0, σ²(cov(h1)+cov(h2))) (1-4)

Make Σ=σ²(cov(h₁)+cov(h₂)),By maximum a-posteriori estimation, the square that can solve Formation formula, it may be assumed that

$\hat{h}=\underset{h}{\arg \min }\left|\right|\mathrm{\ϵ}|{|}^2=h^2{\[G_2-G_1\]}^T{\Σ}^{-1}\[G_2,-G_1\]h---(1-5)$

Wherein, G_iFor g_iConvolution matrix.By (1-2) formula, f and h is associated, and the prior information of f can be about The solution of bundle h.Simultaneous (1-2) and (1-5) formula, meanwhile, in order to preferably keep image detail information and the morbid state solved in rebuilding Sex chromosome mosaicism, the present invention uses based on the adaptive full variation of regional space and non-local mean regularization term J (f) video oversubscription Resolution process of reconstruction carries out prior-constrained, then can obtain h_iSolution formula:

$\begin{array}{c}E\left(h_i\right)=\underset{i}{\Σ}\left|\right|f\⊗h_i-g_i|{|}^2\\ +{\mathrm{\αh}}^T{\[G_2,-G_1\]}^T{\Σ}^{-1}\[G_2,-G_1\]h\\ +\mathrm{\β}J\left(f\right)\end{array},i=1,2---(1-6)$

Wherein, relative single width is estimated, although can be solved accurate fuzzy core by (1-6) formula, but solution still may be deposited In uncertainty.Such as, for having an arbitrary function S of deconvolution, it may be assumed thatSo for any one groupThen for:

$\hat{h}=h\⊗s---(1-7)$

$\hat{f}=f\⊗s^{-1}---(1-8)$

Also it is the solution of equation (1-2), it is therefore necessary to further optimized fuzzy.

Based on this, (1-7) formula is carried out conversion has:

$\begin{array}{c}A\left(\right\{\widehat{h_i}\left\}\right)=g_1\⊗h_2\⊗s-g_2\⊗h_1\⊗s\\ =n_1\⊗h_2\⊗s-n_2\⊗h_1\⊗s\\ =(n_1\⊗h_2-n_2\⊗h_1)\⊗s\end{array}---(1-9)$

In view of the rough estimate of fuzzy core, the model that degrades of rough estimate initial for fuzzy core is written as:

$h_i^0=h_i\⊗s+{\mathrm{\ϵ}}_i---(1-10)$

Wherein,For the initial rough estimate to fuzzy core.Owing to motion blur core is often sparse, therefore solving Introduce sparse prior constraint in journey, add the constraint that h, s are always positive in addition, then the refinement of fuzzy core can be asked by following formula Solve:

$\begin{array}{c}E(s,h_i)=\left|\right|h_i\⊗s-h_0|{|}^2+\mathrm{\δ}\mathrm{\Ψ}\left(s\right)\\ +\mathrm{\δ}\mathrm{\Ψ}\left(h_i\right)+\mathrm{\γ}\left|\right|h|{|}_p^p\end{array}---(1-11)$

Wherein, Ψ () expression just retrains, and its expression formula is:

In described step 3, core is how to determine adaptive weight coefficient, specifically solves as follows:

First full variation uses isotropic model, and it is defined as:

$U\left(f\right)=\left|\right|\▿f|{|}_1^1=\underset{i\∈\mathrm{\Ω}}{\Σ}\sqrt{{{(\▿f_x)}_i}^2+{{(\▿f_y)}_i}^2}---(1-13)$

Wherein,Represent the gradient of image horizontal direction at pixel i,Represent that image hangs down at pixel i Nogata to gradient, Ω represents the set of pixels being weighted.

Non-local mean mainly utilizes the redundancy of non local image similarity block, by setting up non-centered by pixel Topography's block Similarity Measure function, solves current pixel point and its weights searching in window between similitude, two non-offices Image block the most similar then corresponding weight coefficient in portion's is the biggest.The denoising result of current pixel is the weighting of all similitudes in window Averagely.The image mathematics formula obtained by non-local mean is described as:

$f\left(x_i\right)=\underset{j\∈I}{\Σ}{w}_{i,j}^{N}g\left(x_j\right)---(1-14)$

Wherein, I represents the set of weighted pixel,For weight coefficient, and meetIts value By with x_iCentered by image block and with x_jCentered by image block similarity determine, be defined as:

$w_{i,j}^N=\frac{1}{D\left(i\right)}\exp (-\frac{d(i,j)}{h^2})---(1-15)$

Wherein,It is the Euclidean distance of two image blocks,For normalization Parameter, h is overall situation smoothing parameter.

Then we have employed a kind of curvature difference arithmetic operators proposed on the basis of second dervative, based on song Regularization weight coefficient is defined as by rate difference:

$w_i=\frac{1}{1+{\mathrm{\βC}}_i}---(1-16)$

In formula, β is constant, controls parameter intensity, C_iRepresent second derivative-based curvature difference.Use for reference in Denoising Problems Clustering algorithm, by w_iBeing divided into discontinuous region, the pixel with similar spatial weight constitutes a region, regularization coefficient Intensity is by of a sort cluster centre r_j(j=1,2 ..., n) control.

Can be by rebuilding constraint representation based on the adaptive regularization term of regional space finally:

$\begin{array}{c}J\left(f\right)={\mathrm{\λ}}_1\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\sqrt{{{(\▿f_x)}_i}^2+{{(\▿f_y)}_i}^2}\\ +{\mathrm{\λ}}_2\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\left|\right|f_i-{W_i}^N{N}_i|{|}_2^2\end{array}---(1-17)$

R_iFor RSA regularization weight coefficient, it is defined as:

$R_i=\{\begin{array}{cc}\mathrm{\τ}\×r_j,& i\∈\mathrm{\Ω}1\\ r_j,& i\∈\mathrm{\Ω}2\end{array}---(1-18)$

In formula, τ is constant, controls regularization parameter intensity, r_j(j=1,2 ..., n) it is the cluster centre of jth class, Ω 1 For the smooth region of image, Ω 2 is details area.

The cost function that in described step (4), we build is as follows:

$\hat{f}=\arg \min \left\{\underset{{\∂}_{*}\∈\mathrm{\Φ}}{\Σ}\underset{k=1}{\overset{p}{\Σ}}\right||{\mathrm{DB}}_{k}f-y_k|{|}_2^2+{\mathrm{\λ}}_1\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\sqrt{{{(\▿f_x)}_i}^2+{{(\▿f_y)}_i}^2}+{\mathrm{\λ}}_2\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\left|\right|f_i-{W_i}^N{N}_i\left|{|}_2^2\right\}---(1-19)$

Being found by Fig. 1, first we add two kinds of different fog-levels respectively to the first frame in low resolution sequence of pictures Fuzzy, obtain the picture of two kinds of different fog-levels of Same Scene, then carry out a fuzzy core rough estimate, illustrate In step (1), then by this fuzzy core, all frame of video are carried out a deblurring；And then we use the think of of sliding window Think, the most successively with reference frame as with reference to again four adjacent frames being carried out sequence of pictures y after motion registration is processed_k, And as the input of subsequent treatment.

Described step (5) utilizes gradient descent method carry out optimization and rebuild cost function, the high-resolution finally exported Rate image:

$f^{k+1}=f^k-(B^k{\mathrm{Df}}^k-y_l)D^{-1}{\left(B^k\right)}^{-1}-{\mathrm{\λ}}_1\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\sqrt{{{(\▿f_x)}_i}^2+{{(\▿f_y)}_i}^2}-{\mathrm{\λ}}_2\underset{i\∈\mathrm{\Ω}}{\Σ}{R}_i\left|\right|f-W_i^N{N}_i\left|{|}_2^2\right\}---(1-20)$

Wherein, B^kWhen representing+1 iteration of kth, for Df^kThe fuzzy core that the essence of (down-sampling picture) is estimated, y_lFor formula (1-18) y in_k, concrete estimation procedure is described in step (2).

In order to effectiveness of the invention is better described, the method using contrast experiment is shown reconstruction effect by the present invention Really.As a example by low-resolution video " Akiyo ", video " Teddy ", video " Suzie ", real video " Broadcast ", contrast Experiment is chosen the experimental result of 3 representative image super-resolution rebuilding methods and the present invention and is compared, and experiment is tied Fruit is as shown in Fig. 2, Fig. 3, Fig. 4, Fig. 5.These 3 representative image super-resolution rebuilding methods are:

The method that method 1:Xu et al. proposes, list of references " Xu L, Zheng S, et al.Unnatural l0sparse representation for natural image deblurring[C].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:1107-1114.”。

The method that method 2:Rajagopalan et al. proposes, list of references " Rajagopalan, A.N., and Rama Chellappa,eds.Motion Deblurring:Algorithms and Systems[M].Cambridge University Press,2014”。

A Video Enhancer software of method 3:nfognition Co.Ltd company exploitation, list of references “InfognitionCo.Ltd,Video Enhance[OL],http://www.infognition.com/video enhancer/.Version 2.0,23March 2016.”。

The content of contrast experiment is as follows:

Experiment 1, carries out 2 times of Super-resolution reconstructions by method 1, method 2, method 3 and the present invention to video " Akiyo " respectively Build, choose wherein that a frame is as a comparison.Super-resolution Reconstruction result is respectively such as Fig. 2 (a), Fig. 2 (b), Fig. 2 (c) and Fig. 2 (d) institute Show.

Experiment 2, carries out 2 times of Super-resolution reconstructions by method 1, method 2, method 3 and the present invention to video " Teddy " respectively Build, choose wherein that a frame is as a comparison.Super-resolution Reconstruction result is respectively such as Fig. 3 (a), Fig. 3 (b), Fig. 3 (c) and Fig. 3 (d) institute Show.

Experiment 3, carries out 2 times of Super-resolution reconstructions by method 1, method 2, method 3 and the present invention to video " Suzie " respectively Build, choose wherein that a frame is as a comparison.Super-resolution Reconstruction result is respectively such as Fig. 4 (a), Fig. 4 (b), Fig. 4 (c) and Fig. 4 (d) institute Show.

Experiment 4, carries out 2 times by method 1, method 2, method 3 and the present invention to real video " Broadcast " respectively and surpasses Resolution reconstruction, chooses wherein that a frame is as a comparison.Super-resolution Reconstruction result respectively as Fig. 5 (a), Fig. 5 (b), Fig. 5 (c) and Shown in Fig. 5 (d).

It can be seen that figure (a) rebuilds the sawtooth effect of frame of video marginal information more substantially from 4 groups of experiments, details is believed Breath occurs in that distortion；Figure (b) reconstructed results, compared with figure (a) reconstructed results, has recovered part detailed information, but ring is existing As the most obvious；It is fuzzyyer that figure (c) rebuilds frame of video entirety；Figure (d) is the reconstructed results of the present invention, rebuilds with contrast algorithm Frame of video is compared, and detailed information is more rich, and visual effect is more preferable.

Claims (3)

1. the rebuilding blind super-resolution algorithm estimated based on multiple image fuzzy core improved, it is characterised in that include following Step:

Step one: the first frame picture in the frame of video of the low resolution of input is carried out Fuzzy Processing in various degree, obtains The image of two width difference fog-levels of Same Scene；

Step 2: utilize the blurred picture of two the different fog-levels obtained in step one to carry out fuzzy core estimation, and with raw The fuzzy core become carries out the pretreatment of deblurring, sequence of pictures y after being processed to all low resolution video frame_l, as The input of later process；

Step 3: utilize curvature difference operator extraction spatial structural form, then it being carried out cluster, to obtain regional space adaptive Answering weight coefficient, this coefficient is for carrying out adaptive weighted to full variation and non-local mean regularization term；

Step 4: utilize the regularization term of the weighting that adaptive weighted coefficient determined by step 3 obtains to determine reconstruction Cost function；

Step 5: utilize gradient descent method to carry out optimization and rebuild cost function, wherein use in step 2 in each iterative process Fuzzy core method of estimation carry out fuzzy core again and estimate, and carry out deblurring, the high-definition picture finally exported Sequence.

The rebuilding blind super-resolution calculation estimated based on multiple image fuzzy core of a kind of improvement the most according to claim 1 Method, it is characterised in that the pretreatment that sequence of pictures has carried out a deblurring described in step 2, the method for pretreatment is The two width pictures utilizing in step 2 the different fog-levels to wherein Same Scene use the deconvolution algorithm of robustness to generate One rough fuzzy core, then utilizes the fuzzy core of generation that all low resolution video frame carry out a deblurring and processes, Obtain deblurring pretreated image sequence y_l, then with y_lInput as later process.

The rebuilding blind super-resolution calculation estimated based on multiple image fuzzy core of a kind of improvement the most according to claim 1 Method, it is characterised in that described in step 5, iterative process in, the picture after each iterative approximation is added once fuzzy obtaining The picture of two different fog-levels of Same Scene, then utilizes the algorithm in step 2 to carry out the estimation again of fuzzy core, And again carry out deblurring.