CN103824273B - Super-resolution reconstruction method based on compound motion and self-adaptive nonlocal prior - Google Patents

Abstract

The invention provides a super-resolution reconstruction method based on compound motion and self-adaptive nonlocal prior. The method comprises the following steps: selecting a reference frame image and a non-reference frame image from p low-resolution images; performing image registration by means of global motion parameters and local light stream to obtain a motion field mk(x) of the non-reference frame image relative to the reference frame image, and constructing a motion transformation matrix Mk by using mk(x); calculating an interpolation image X, nonlocal prior parameter hi,j and European threshold of the reference frame image; calculating the similarity weight wNLM[i, j; s, t] of each pixel to other pixels, and constructing a nonlocal weight matrix S related to a high-resolution image X by using wNLM; solving a target function shown in the specification by using the motion transformation matrix Mk and the nonlocal weight matrix S to obtain a reconstructed high-resolution estimation image. Compared with the prior art, the super-resolution reconstruction method has the advantages that the defects of high calculated amount, poor scalability and low accuracy in the conventional motion estimation are overcome effectively by adopting a compound motion model, and distortion of a reconstructed image is reduced by adopting self-adaptive nonlocal prior.

Description

Super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation

Technical field

The present invention relates in the technical field of image procossing, computer vision, more particularly to image sharpening field A kind of super resolution image reconstruction method based on compound motion and the non local priori of self adaptation of the spatial resolution improving image.

Background technology

In imaging field, the image of high spatial resolution is always one of target pursued.The figure of high spatial resolution As fully have recorded the detailed information of object, can give people to provide, with the reasoning of computer, judgements, decision-making, the letter more enriching Breath.Therefore, in many imaging applications, high-resolution image is generally very important, such as: video monitoring, medical science are examined The applications such as disconnected, military surveillance, remote sensing.The spatial resolution improving image can pass through two kinds of approach, " hardware approach " and " software Approach ".Under regular situation, people mainly obtain high score by improving the hardware devices such as high-precision ccd and cmos sensor The image distinguished.But, merely improving resolution by improvement hardware facility can be subject to many limitations, the electricity of such as sensor The lotus rate of transform, thermal noise, the Rayleigh entropy of optical lens, and hardware costs etc. limit.Therefore, for hardware device and economically Consideration, " software approach " becomes a kind of more feasible scheme.Tsai and huang proposes Super-resolution Reconstruction first within 1984 Problem is from " software approach ", improves the spatial resolution of image by theory of development, algorithm, it has also become image procossing is led One of domain research direction the most active.In recent years, super-resolution rebuilding is divided into the super-resolution based on multiframe by some scholars Rebuild and the super-resolution rebuilding based on study.The present invention is a kind of super-resolution reconstruction method based on multiframe.

Method for reconstructing based on frequency domain with based on spatial domain can be divided into again to the super-resolution rebuilding based on multiframe.Up-to-date Based on the method for reconstructing of frequency domain be rhee and kang in 1999 propose based on discrete cosine transform method for reconstructing, and chan exists The method for reconstructing based on wavelet transformation proposing for 2003.It is that theory is succinct, it is simple to calculate based on the method for reconstructing advantage of frequency domain, But the relative motion that inferior position is between multiframe can only be overall similar movement, and it is only applicable to when image fuzzy is linear not Situation about becoming.Therefore it is directed to these inferior positions, create the super-resolution reconstruction method of many classics based on the method for reconstructing in spatial domain, such as Non-uniform interpolation method, iterative backprojection method, projections onto convex sets, method of maximum likelihood, maximum a posteriori, mixing maximum a posteriori/convex set Sciagraphy.The present invention is a kind of method for reconstructing based on MAP estimation.

In the method for estimation based on maximum a posteriori, estimation and the design of priori item are two very important tasks. In most of the cases, the motion vector needed for image super-resolution is unknown, in order to solve these motion vectors, can have two Plant thinking: a kind of thinking is first to solve these motion vectors and then carry out super-resolution rebuilding again although this separately solve Mode calculates very simply, but significant limitation is all individually present；Another kind of thinking is motion vector and super-resolution figure As carrying out joint solution, than the estimation individually separating method for solving more accurately, reconstruction effect is good, but shortcoming is to ask for this kind of method Solution speed is slow, is difficult to practical application.For the estimation of motion vector, can be divided into again based on parameter model and be based on light stream Estimation.Estimation based on parameter model is fairly simple, but retractility is low；And the estimation based on light stream is stretched Property high, but estimated accuracy is low, rebuilds effect undesirable.For this reason, the present invention is based on separately solves motion vector and super resolution image Thinking it is proposed that a kind of have very high scalability and estimated accuracy, low operand compound motion model.

For priori item, namely regular terms design, have been proposed for many priori items in domestic and foreign literature, including Tikhonov priori, huber priori, tv priori, btv priori etc..These priori are all based on Image neighborhood difference and are described , fail the prior information of natural image is depicted exactly, lead to reconstruction image distortion.Recently, deposit based in natural image The fact the picture structure that bulk redundancy repeats, propose a kind of non local priori in the world, and be successfully applied to image Deconvolution.But, do many parameters needing artificial regulation it is impossible to accomplish the adaptive of parameter due to existing in non local priori Should.

Content of the invention

The problem existing for prior art, present invention is primarily targeted at providing a kind of reduction amount of calculation, improving figure The super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation as precision.

For achieving the above object, the invention provides a kind of super-resolution based on compound motion and the non local priori of self adaptation The embodiment of rate method for reconstructing, high-definition picture x is through the Procedure Acquisition p width low resolution observed image y that degrades_k(k= 1 ..., p), the size of every width observed image is m × n, and this utilizes p width low resolution observed image y_k(k=1 ..., p) rebuilds High-resolution estimates that the super resolution ratio reconstruction method of image comprises the steps (1) to step (5):

(1) choose reference frame image y in p width low resolution observed image_ref(1≤ref≤p) and non-reference frame image y_k(k=1 ..., ref-1, ref+1 ..., p) adopt for the sub-pel motion between reference frame image and non-reference frame image With the compound motion model of global parameter motion and local light stream, the relation between reference frame image and non-reference frame image represents For:Wherein m_kRepresent two dimensional motion field,Table Show that global parameter is moved,For local light stream campaign, θ_kFor globe motion parameter,Represent and use non-reference frame image The reference frame image of prediction, ε_kX () represents residual image；

(2) solve globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamUsing global motion Parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamMethod carry out image registration, obtain non-reference frame image phase Sports ground m for reference frame image_kX (), using m_kX () constructs motion transform matrices m_k；

(3) calculate reference frame image y_refR times of interpolation imageNon local Study first h_{I, j}(0≤i ＜ rm, 0≤j ＜ Rn) and similar image European threshold value；

(4) non local Study first, European threshold value are utilized, and with interpolation imageInitial graph as full resolution pricture x Picture, calculates interpolation imageIn each pixel (i, j) (wherein 0≤i ＜ rm, 0≤j ＜ rn) and other pixels (s, t) Similarity weight w of (wherein 0≤s ＜ rm, 0≤t ＜ rn)_nlm[i, j；S, t], using similarity weight w_nlmBuild non local power Weight matrix s；

(5) utilize motion transform matrices m_kSolve cost functional with non local weight matrix sWherein b_kFor observed image y_kThe clear function of corresponding fall, m_k For observed image y_kWith respect to the sub-pel motion of reference frame image, non local weight matrix s is a self adaptation high-resolution The non-local mean wave filter of image x, and ρ ＞ 0, minimize cost functional using conjugate gradient iterative procedure, obtain the height rebuild Resolution estimates image.

Further, the sub-pel motion that this step (1) is directed between reference frame image and non-reference frame image adopts complete Office's movement parameter and the compound motion model of local light stream, wherein motion vector are m_k(x)=[m_{K, u}(x) m_{K, v}(x)], x=[x_u x_v], two dimensional motion field m_kIt is expressed as m_k(x)=m_k ^g(x)+m_k ^l(x)=m_k ^g(x；θ_k)+m_k ^l(x), whereinRepresent global parameter fortune It is dynamic,For local light stream campaign, θ_kFor globe motion parameter.

Further, this step (2) solves globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) comprise the steps (21a) to step (22a):

(21a) adopt a₀, a₁, a₂, a₃, a₄, a₅The affine transformation of six parameters is as global parameter motion model:

(22a) set up least square standardSolve globe motion parameter θ_k= (a₀, a₁, a₂, a₃, a₄, a₅).This step (22a) sets up least square standardSolve complete Office kinematic parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) comprise the steps (221a) to step (224a): (221a) is by θ_kWrite as θ_k +δθ；(222a) least square cost functional is carried out with Taylor expansion and obtain the linear function with regard to δ θ；(223a) after to launching Function carry out a series of arithmetic operations and obtainWherein (224a) judge whether to meet | | δ θ | |≤∈, threshold value 0≤∈≤0.01, if not meeting, θ_k←θ_k+ δ θ, and return to step (223a)；If meeting, show parameter θ_kConvergence, stops iteration, obtains globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅), Global parameter motion finally can be obtained

Further, this step (2) solves local light streamComprise the steps (21b) to step (24b):

(21b) assumed using gradation of image constancy and image gradient constancy is assumed to obtain with regard to local light stream's Data confidence energy functionIts In,Function Wherein 0 ＜ τ≤0.01；

(22b) smooth penalty is obtained according to image segmentation smoothness assumption

(23b) obtaining whole energy function is e (u, v)=e_data+αe_smooth, wherein α ＞ 0 is regularization parameter；

(24b) adopt Nonlinear Numerical method for solving to whole energy function e (u, v)=e_data+αe_smoothSeek optimal value Just obtain local light streamSolution.

Further, this step (2) utilizes m_kX () constructs motion transform matrices m_kComprise the steps of (21c) to step Suddenly (23c):

(21c) calculate the sports ground m that non-reference frame image is with respect to reference frame image_k(x)=m_k ^g(x)+m_k ^l(x), and will Sports ground m_kX () carries out r times of linear interpolation, obtain the sports ground m after interpolation_kX (), this r is putting of the final high-definition image rebuild Big multiple；

(22c) calculate relative displacement δ x_k=m_k(x)-x=(δ x_{K, u}-δx_{K, v}), and d_k=δ x_{K, u}-floor(δ x_{K, u}), e_k=δ x_{K, v}-floor(δx_{K, v}), wherein operator floor (.) represents the maximum taking less than or equal to designated value Integer；

(23c) calculate motion transform matrices m_kIn each element value: m_k(j*m+i, floor (δ x_{K, u})+x_u+ceil (δx_{K, v}+x_v+ 1) * m)=d_k*(1-e_k), m_k(j*m+i, ceil (δ x_{K, u})+x_u+ceil(δx_{K, v}+x_v+ 1) * m)=d_k*e_k, m_k (j*m+i, floor (δ x_{K, u})+x_u+floor(δx_{K, v}+x_v+ 1) * m)=(1-d_k)*(1-e_k), m_k(j*m+i, ceil (δ x_{K, u}) +x_u+floor(δx_{K, v}+x_v+ 1) * m+x)=(1-d_k)*e_k, wherein operator ceil (.) represent take more than or equal to specify The smallest positive integral of value, for matrix m_kJth * m+i row in except this four row in addition to other elements value be zero.

Further, this step (3) calculates non local Study first h_{I, j},Wherein std(n_{I, j}) it is centered on pixel (i, j), region of search is n_{I, j}Standard deviation, β is the constant (1 ＜ β ＜ 5) more than zero, σ²For reference frame image y_refNoise variance, r be image magnification, this reference frame image y_refNoise variance σ²It is estimated as:Wherein n is the pixel sum of image,n(x_i) it is picture Vegetarian refreshments x_iFour neighborhoods.

Further, the maximum weighted Euclidean distance that in this step (3), the European threshold value of similar image adopts is 4 σ², Namely:

Further, this step (4) utilizes non local Study first, European threshold value, and with interpolation imageAs height The initial pictures of resolution image x, calculate interpolation imageIn each pixel (i, j) and other pixels (s, t) similar Degree weight w_nlm[i, j；S, t], using similarity weight w_nlmBuild non local weight matrix s and comprise the steps of (41) to step (43):

(41) in calculating and in each block of pixels centered on coordinate (i, j) and its neighborhood n (i, j) with coordinate (s, t) being The similarity of the block of pixels of the heart

Wherein r_{I, j}It is the image block extraction operator of pixel centered on (i, j),Represent two images The weighted euclidean distance of block, wherein a ＞ 0 are the standard deviations of gaussian kernel function, h_{I, j}For filter smoothing parameter, depend on image Noise size and image itself, f is the normal function of the geometric distance depending on two central pixel point；

(42) all pixels Similarity value in normalization pixel (i, j) and neighborhood n (i, j),

(43) build non local weight matrix

Further, in this step (5) whenWhen, iteration ends, wherein n are repeatedly Generation number.

With respect to prior art, first, non local priori is applied to the multiframe Super-resolution reconstruction of image by the present invention first Build it is proposed that a kind of parameter adaptive method for solving of non local priori；Secondly, by using compound motion model, effectively Solve the computationally intensive of current estimation, the shortcoming that scalability is not strong, precision is not high；Again, using adaptive non- Local priori more can automatically describe the prior information of image exactly, decreases the distortion of reconstruction image.For this reason, the method is done Arrive telescopic high-precision estimation and accurately adaptive priori description, reality can be more effectively applied to Characteristics of image sharpening engineering.

Brief description

Fig. 1 is the embodiment based on compound motion and the super resolution ratio reconstruction method of the non local priori of self adaptation of the present invention Flow chart

Fig. 2 is the solution globe motion parameter θ of the present invention_k=(a₀, a₁, a₂, a₃, a₄, a₅) embodiment flow chart

Fig. 3 is the solution local light stream of the present inventionEmbodiment flow chart

Specific embodiment

Below in conjunction with the accompanying drawings, describe the specific embodiment of the present invention in detail.

The super-resolution rebuilding of image, will revert to ideal image by observed image.Observed image is a series of low point Resolution image, the i.e. required high-definition picture of ideal image.Give the high-definition picture x of certain scene, through a series of The process that degrades of geometry motion, optical dimming, sub-sampling and additional noise, produces p width low resolution observed image y_k, use one Individual conventional image observation model describes the relation between ideal image and observed image, and this observation model is: y_k=db_km_kx+ n_k, k=1 ..., p, wherein m_kFor motion change matrix, b_kFor fuzzy matrix, d is down-sampling matrix, n_kFor additional noise.

Based on above-mentioned observation model, the present invention is estimated to ideal image x using maximum a posteriori method.P width low resolution figure As being expressed asBased on the theoretical Super-resolution Reconstruction problem representation of MAP estimation it is: x=arg maxp (x | y), through simple calculations operation, can obtain: P (y herein_k| x)=p (n_k) illustrating the type of observation model noise, common hypothesis noise is average is 0, and variance is σ_k ²Height This noise, that is,Wherein c₁For constant.Image prior probability density general type For:Wherein c₂For constant, η is control parameter, and u (x) is the priori energy function with regard to image x. Through simple abbreviation, following cost function may finally be obtained: WhereinHere, present invention assumes that the fuzzy core of image is it is known that namely fuzzy matrix b_kKnown.

According toUnderstand, ideal image x will be obtained, need to solve Motion transform matrices m_kWith priori energy u (x), i.e. image registration and the design of self adaptation non local priori.Final employing is commonly used Conjugate gradient method is to cost functionalCarry out solving and obtain ideal image x.

Therefore, the invention mainly comprises: (1) multiframe low-resolution image registration when using globe motion parameter add local The method of light stream carries out image registration, and builds motion transform matrices m_k；(2) design the non local priori of self adaptation, build non-office Portion weight matrix s；(3) set up cost functional, and solved using conjugate gradient method.

As shown in figure 1, the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation for the present invention Embodiment flow chart.The embodiment of super resolution ratio reconstruction method comprises the steps s1 to step s8:

The low resolution observed image y that s1, acquisition p width are formed through the process that degrades by high resolution graphics x_k, every width observation The size of image is m × n.Obtain image viewing model y_k=db_km_kx+n_k, k=1 ..., p, wherein m_kFor motion change matrix, b_kFor fuzzy matrix, d is down-sampling matrix, n_kFor additional noise, x is high-definition picture, and k is the p width low resolution figure obtaining The numbering of picture.

S2, in p width low-resolution image choose reference frame image y_ref(1≤ref≤p) and non-reference frame image y_k(k =1 ..., ref-1, ref+1 ..., p), for the sub-pel motion between reference frame image and non-reference frame image using complete Office's movement parameter adds the compound motion model of local light stream, and the relation between reference frame image and non-reference frame image is expressed as:Wherein m_kRepresent two dimensional motion field,Represent complete Office's movement parameter,For local light stream campaign, namely nonparametric local compensation campaign, θ_kFor globe motion parameter,Represent the reference frame image with non-reference frame image prediction, ε_kX () represents residual image.In global parameter motion In the compound motion model of local light stream, motion vector is m_k(x)=[m_{K, u}(x) m_{K, v}(x)], estimate this motion vector Purpose is used to build motion transform matrices m_k, wherein x=[x_ux_v], two dimensional motion field m_kIt is expressed asWhereinRepresent global parameter motion,For local light stream campaign, θ_kFor globe motion parameter.

S3, solution globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamUsing global motion Parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamMethod carry out image registration, obtain non-reference frame image phase Sports ground m for reference frame image_kX (), using m_kX () constructs motion transform matrices m_k；

Solve globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) when, initially set up least square standardWherein Then by θ_kWrite as θ_k+ δ θ, with the side of increment Formula solves θ_k, namely θ_k←θ_k+δθ.Least square cost functional is carried out with the linear function that Taylor expansion obtains with regard to δ θ, right Function after expansion carries out a series of arithmetic operations and obtainsIts In When kinematic parameter increment δ θ is less than certain threshold value ∈ (0≤∈≤0.01), namely | | δ θ | |≤∈, parameter θ_kReceive Hold back, stop iteration, obtain globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅), finally can obtain global parameter motionOtherwise update θ_k, namely θ_k←θ_k+ δ θ, and recalculate δ θ.

Solve local light streamWhen, assume first with gradation of image constancy and image gradient constancy is assumed To with regard to local light streamData confidence energy functionWherein,FunctionThen smooth penalty is obtained according to image segmentation smoothness assumptionFinally giving whole energy function is e (u, v)=e_data+αe_smooth, wherein α ＞ 0 is regularization parameter.Optimal value is asked just to obtain local light stream whole energy function using Nonlinear Numerical method for solvingSolution.

Finally give the sports ground that non-reference frame image is with respect to reference frame imageAnd Motion transform matrices m is built with this_k.First by sports ground m_kX () carries out r times (amplification of the final high-definition image rebuild) Linear interpolation, obtains the sports ground m after interpolation_k(x).Then calculate relative displacement δ x_k=m_k(x)-x=(δ x_{K, u}-δx_{K, v}), And d_k=δ x_{K, u}-floor(δx_{K, u}), e_k=δ x_{K, v}-floor(δx_{K, v}), wherein operator floor (.) represents to take and is less than Or it is equal to the maximum integer of designated value.Finally calculate motion transform matrices m_kIn each element value: m_k(j*m+i, floor (δx_{K, u})+x_u+ceil(δx_{K, v}+x_v+ 1) * m)=d_k*(1-e_k), m_k(j*m+i, ceil (δ x_{K, u})+x_u+ceil(δx_{K, v}+x_v + 1) * m)=d_k*e_k, m_k(j*m+i, floor (δ x_{K, u})+x_u+floor(δx_{K, v}+x_v+ 1) * m)=(1-d_k)*(1-e_k), m_k(j* M+i, ceil (δ x_{K, u})+x_u+floor(δx_{K, v}+x_v+ 1) * m+x)=(1-d_k)*e_k, wherein operator ceil (.) represent take big In or be equal to designated value smallest positive integral, for matrix m_kJth * m+i row in except this four row in addition to other elements value It is zero.

Step s2, s3 completes image registration and kinematic matrix m_kStructure.

S4, calculating reference frame image y_refR times of bicubic interpolation imageNon local priori smoothing parameter h_{I, j}With similar The European threshold value of image.Because ideal image x is unknown in advance, the non local weight matrix just reference frame image according to low resolution y_refInterpolation imageEstimation obtains.Therefore, smoothing parameter h_{I, j}It it is one with regard to reference frame image y_refNoise, picture number According to itself, the function of down-sampling multiple.Wherein std (n_{I, j}) it is to be with pixel (i, j) The heart, region of search is n_{I, j}Standard deviation, be the constant (1 ＜ β ＜ 5) more than zero, σ²For the noise variance of reference frame image, r is Image magnification.This reference frame image y_refNoise variance σ²It is estimated as:Wherein n is image Pixel sum,n(x_i) it is pixel x_iFour neighborhoods.Finally adopt European threshold Value carries out dissimilar pixel removal, and the maximum weighted Euclidean distance that European threshold value (i.e. similarity threshold) adopts is 4 σ², namely:

S5, calculate each pixel (i, j) (wherein 0≤i ＜ rm, 0≤j ＜ rn) with other pixels (s, t) (wherein 0≤s ＜ rm, 0≤t ＜ rn) similarity weight w_nlm[i, j；S, t], build with regard to interpolation imageAs initial high-resolution The non local weight matrix s of image x comprises the steps of: calculates the pixel in each pixel (i, j) and its neighborhood n (i, j) The similarity of (s, t)

Wherein r_{I, j}Be pixel centered on (i, j) image block extract operator, generally the square of q × q (q=5,7, 9...),Represent the weighted euclidean distance of two image blocks, wherein a ＞ 0 is the standard deviation of gaussian kernel function, h_{I, j} For filter smoothing parameter, depend on noise size and the image itself of image, f is the geometry depending on two central pixel point The normal function (monotone non-increasing function) of distance；All pixels Similarity value in normalization pixel (i, j) and neighborhood n (i, j),Finally build non local weight matrix

The s of self adaptation solution and non local weight matrix that step s4, s5 completes non local Study first builds.

S6, utilize motion transform matrices m_kSolve cost functional with non local weight matrix sWherein ρ ＞ 0.Mesh is minimized using conjugate gradient iterative procedure Mark functional；

S7, judge whether to meet stopping criterion for iteration:Wherein n is iterationses, If not meeting, return to step s6, if meeting, showing solution convergence, iteration ends, then entering step s8；

S8, the high-resolution obtaining rebuilding estimate image.

Step s6 to step s8 completes solution cost functional.

As shown in Fig. 2 the solution globe motion parameter θ for the present invention_k=(a₀, a₁, a₂, a₃, a₄, a₅) embodiment stream Cheng Tu, comprises the steps s31a to step s32a:

S31a, adopt a₀, a₁, a₂, a₃, a₄, a₅The affine transformation of six parameters is as global parameter motion model:

S32a, set up least square standardSolve globe motion parameter θ_k= (a₀, a₁, a₂, a₃, a₄, a₅), comprise the steps s321a to step s324a:

S321a, because least square cost functional is with regard to θ_kBe nonlinear need to be by θ_kWrite as θ_kThe form of+δ θ, is changed using increment Solve θ for mode_k, namely θ_k←θ_k+δθ.And least square cost functional is carried out with Taylor expansion obtain the linear function with regard to δ θ； S322a, the function after launching is carried out by arithmetic operation obtains Wherein

S323a, judge whether to meet | | δ θ | |≤∈, threshold value 0≤∈≤0.01, if not meeting, θ_k←θ_k+ δ θ, and Return to step s322a, if meeting, showing that parameter θ is restrained, stopping iteration, entering step s324a；S324a, obtain the overall situation fortune Dynamic parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅).

As shown in figure 3, the solution local light stream for the present inventionEmbodiment flow chart.Obtaining global motion Parameter θ=(a₀, a₁, a₂, a₃, a₄, a₅), namely behind global motion field, if affine motion can approximate object well Motion model, now just can omit the solution of the local optical flow field that sports ground is compensated.But if overall affine mould Type fails accurately approximately whole sports ground, just needs to solve local optical flow field.However, the global parameter sports ground having solved To a great extent close to mass motion field, contribute to ensuing local optical flow field solves the local so that solving Optical flow field is more accurate.This just embodies the good scalability of compound motion model proposed by the present invention, high-precision fortune Dynamic estimation and low operand.In order to try to achieve local light streamSolution, the present invention utilize thomas brox propose light Stream solving method.Thomas brox light stream solving method is based primarily upon three hypothesis: gradation of image constancy is assumed, image gradient is permanent Perseverance is assumed, image segmentation smoothness assumption.

Solve local light streamComprise the steps s31b to s34b:

S31b, using gradation of image constancy assume and image gradient constancy assume obtain with regard to local light stream Data confidence energy function Wherein,Function

S32b, smooth penalty is obtained according to the image segmentation smoothness assumption of light stream

S33b, obtain whole energy function be e (u, v)=e_data+αe_smooth, wherein α ＞ 0 is regularization parameter；

S34b, optimal value is asked just to obtain local light stream whole energy function using Nonlinear Numerical method for solving Solution.

It is described above a kind of super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation.The present invention Be not limited to above example, any without departing from technical solution of the present invention, only ordinary skill people is carried out to it Improvement or change that member is known, belong within protection scope of the present invention.

Claims (10)

1. a kind of super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation, high-definition picture x passes through Degrade Procedure Acquisition p width low resolution observed image y_k(k=1 ..., p), the size of every width observed image is m × n, its feature It is, described utilization p width low resolution observed image y_k(k=1 ..., p) rebuilds the super-resolution that high-resolution estimates image Method for reconstructing comprises the steps:

(1) choose reference frame image y in p width low resolution observed image_ref(1≤ref≤p) and non-reference frame image y_k(k= 1 ..., ref-1, ref+1 ..., p), for the sub-pel motion between reference frame image and non-reference frame image using the overall situation The compound motion model of movement parameter and local light stream, the relation between reference frame image and non-reference frame image is expressed as:Wherein m_kRepresent two dimensional motion field,Represent complete Office's movement parameter,For local light stream campaign, θ_kFor globe motion parameter,Represent and use non-reference frame image prediction Reference frame image, ε_kX () represents residual image；

(2) solve globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamUsing globe motion parameter θ_k =(a₀, a₁, a₂, a₃, a₄, a₅) and local light streamMethod carry out image registration, obtain non-reference frame image with respect to ginseng Examine the sports ground m of two field picture_kX (), using m_kX () constructs motion transform matrices m_k；

(3) calculate reference frame image y_refR times of interpolation imageNon local Study first h_{I, j}(0≤i ＜ rm, 0≤j ＜ rn) European threshold value with similar image；

(4) non local Study first, European threshold value are utilized, and with interpolation imageAs the initial pictures of full resolution pricture x, count Calculate interpolation imageIn each pixel (i, j) and other pixels (s, t) similarity weight w_nlm[i, j；S, t], wherein 0≤i ＜ rm, 0≤j ＜ rn, 0≤s ＜ rm, 0≤t ＜ rn, using similarity weight w_nlmBuild non local weight matrix s；

(5) utilize motion transform matrices m_kSolve cost functional with non local weight matrix sWherein d is down-sampling matrix, b_kFor observed image y_kThe clear letter of corresponding fall Number, m_kFor observed image y_kWith respect to the sub-pel motion of reference frame image, non local weight matrix s is a self adaptation high score The non-local mean wave filter of resolution image x, and ρ ＞ 0, minimize cost functional using conjugate gradient iterative procedure, are rebuild High-resolution estimate image.

2. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: the sub-pel motion that described step (1) is directed between reference frame image and non-reference frame image adopts global parameter Motion and the compound motion model of local light stream, wherein motion vector are m_k(x)=[m_{K, u}(x) m_{K, v}(x)], x=[x_ux_v], Two dimensional motion field m_kIt is expressed asWhereinRepresent global parameter motion,For local light stream campaign, θ_kFor globe motion parameter.

3. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: described step (2) solves globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) comprise the steps:

(21a) adopt a₀, a₁, a₂, a₃, a₄, a₅The affine transformation of six parameters is as global parameter motion model:

(22a) set up least square standardSolve globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅).

4. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 3, its It is characterised by: described step (22a) sets up least square standardSolve global motion Parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅) comprise the steps:

(221a) by θ_kWrite as θ_k+δθ；

(222a) least square cost functional is carried out with Taylor expansion and obtain the linear function with regard to δ θ；

(223a) carry out a series of arithmetic operations to the function after launching to obtain Wherein

(224a) judge whether to meet | | δ θ | |≤∈, threshold value 0≤∈≤0.01, if not meeting, θ_k←θ_k+ δ θ, and return Step (223a)；If meeting, show parameter θ_kConvergence, stops iteration, obtains globe motion parameter θ_k=(a₀, a₁, a₂, a₃, a₄, a₅), finally can obtain global parameter motion

5. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: described step (2) solves local light streamComprise the steps:

(21b) assumed using gradation of image constancy and image gradient constancy is assumed to obtain with regard to local light streamData Confidence level energy functionWherein,FunctionIts In 0 ＜ τ≤0.01；

(22b) smooth penalty is obtained according to image segmentation smoothness assumption

(23b) obtaining whole energy function is e (u, v)=e_data+αe_smooth, wherein α ＞ 0 is regularization parameter；

(24b) adopt Nonlinear Numerical method for solving to whole energy function e (u, v)=e_data+αe_smoothOptimal value is asked just to obtain Local light streamSolution.

6. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 5, its It is characterised by: described step (2) utilizes m_kX () constructs motion transform matrices m_kComprise the steps of:

(21c) calculate the sports ground that non-reference frame image is with respect to reference frame imageAnd will transport Dynamic field m_kX () carries out r times of linear interpolation, obtain the sports ground m after interpolation_kX (), described r is putting of the final high-definition image rebuild Big multiple；

(22c) calculate relative displacement δ x_k=m_k(x)-x=(δ x_{K, u}-δx_{K, v}), and d_k=δ x_{K, u}-floor(δx_{K, u}), e_k =δ x_{K, v}-floor(δx_{K, v}), wherein operator floor (.) represents the maximum integer taking less than or equal to designated value；

(23c) calculate motion transform matrices m_kIn each element value:

m_k(j*m+i, floor (δ x_{K, u})+x_u+ceil(δx_{K, v}+x_v+ 1) * m)=d_k*(1-e_k),

m_k(j*m+i, ceil (δ x_{K, u})+x_u+ceil(δx_{K, v}+x_v+ 1) * m)=d_k*e_k,

m_k(j*m+i, floor (δ x_{K, u})+x_u+floor(δx_{K, v}+x_v+ 1) * m)=(1-d_k)*(1-e_k),

m_k(j*m+i, ceil (δ x_{K, u})+x_u+floor(δx_{K, v}+x_v+ 1) * m+x)=(1-d_k)*e_k, wherein operator ceil (.) Represent the smallest positive integral taking more than or equal to designated value, for matrix m_kJth * m+i row in except this four row in addition to its His element value is zero.

7. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: described step (3) calculates non local Study first h_{I, j},Wherein std (n_{I, j}) It is centered on pixel (i, j), region of search is n_{I, j}Standard deviation, β is the constant (1 ＜ β ＜ 5) more than zero, σ²It is reference Two field picture y_refNoise variance, r be image magnification, described reference frame image y_refNoise variance σ²It is estimated as:Wherein n is the pixel sum of image,n(x_i) it is picture Vegetarian refreshments x_iFour neighborhoods.

8. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: the maximum weighted Euclidean distance that in described step (3), the European threshold value of similar image adopts is 4 σ², namely:Wherein r_{I, j}It is the image block extraction of pixel centered on (i, j) Operator, σ²For reference frame image y_refNoise variance,For σ²Estimated value.

9. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its It is characterised by: described step (4) utilizes non local Study first, European threshold value, and with interpolation imageAs full resolution pricture x Initial pictures, calculate interpolation imageIn each pixel (i, j) and other pixels (i, j) similarity weight w_nlm [i, j；S, t], using similarity weight w_nlmBuild non local weight matrix s to comprise the steps of:

(41) calculate in each block of pixels centered on coordinate (i, j) and its neighborhood n (i, j) centered on coordinate (i, j) The similarity of block of pixels, WhereinFor σ²Estimated value, σ²For reference frame image y_refNoise variance, r_{I, j}It is the figure of pixel centered on (i, j) As block extracts operator,Represent the weighted euclidean distance of two image blocks, wherein a ＞ 0 is Gauss The standard deviation of kernel function, h_{I, j}For filter smoothing parameter, depend on noise size and the image itself of image, f is to depend on The normal function of the geometric distance of two central pixel point；

(42) all pixels Similarity value in normalization pixel (i, j) and neighborhood n (i, j),

(43) build non local weight matrix

10. the super resolution ratio reconstruction method based on compound motion and the non local priori of self adaptation according to claim 1, its Be characterised by: in described step (5) whenWhen, iteration ends, wherein n are iterationses.