CN103312941A - Joint de-noising and super-resolution method and joint de-noising and super-resolution system on basis of convex optimization theories for videos - Google Patents

Abstract

The invention provides a joint de-noising and super-resolution method on the basis of convex optimization theories for videos. The joint de-noising and super-resolution method includes steps of learning from a plurality of sharp images according to a sparse coding theory on the basis of image blocks to obtain a group of over-complete low-resolution bases and over-complete high-resolution bases; respectively creating constraints related to noise, a reconstruction coefficient matrix and original signals; creating an objective function according to the constraints; solving the objective function by an augmented Lagrangian optimization process to obtain the original signals and the reconstruction coefficient matrix; performing super-resolution operation by the aid of the reconstruction coefficient matrix and the over-complete high-resolution bases to obtain de-noised sharp high-resolution videos. The joint de-noising and super-resolution method has the advantages that signal-related de-noising and super-resolution operation can be effectively performed by the method, sharp images and the sharp videos can be obtained, algorithms are high in universality, and a video optimization effect is obvious. The invention further discloses a joint de-noising and super-resolution system on the basis of the convex optimization theories for the videos.

Description

Video associating denoising and super-resolution method and system based on protruding optimum theory

Technical field

The present invention relates to calculate shooting field, particularly a kind of video associating denoising and super-resolution method and system based on protruding optimum theory.

Background technology

In recent years, the denoising of signal dependent noise and image super-resolution always are the research focus that calculates in shooting and the computer vision.

The research of tradition denoising aspect mainly concentrates on the noise denoising irrelevant with signal, but this camera system with reality does not conform to.People such as Alessandro Foi studies show that, primary signal and CCD take between the noise average and have quadratic relationship.The denoising of signal correction now more and more becomes computer vision and the research focus that calculates in shooting.Enumerate some main flows and denoising method signal correction below:

(1), people such as Danielyan and G.Boracchi proposes is converted to frequency domain with the block of pixels in the original image by the signal conversion and carries out denoising.

(2), the method for the utilization principal component analysis of people such as Li Zhang and Sundeep Vaddadi proposition and tensor analysis realizes the denoising of many noise images.

(3), people such as Keigo Hirakawa and Thomas W.Parks regards primary signal the combination of series of noise block of pixels as, proposes a kind of denoising method of probabilistic type.

There have been many comparatively ripe methods the image super-resolution aspect, generally speaking may be summarized to be based on the spatial domain interpolation, based on the frequency domain processing, based on regularization with based on study base four big classes.Compare with its excess-three class methods, the method basic based on study has less constrained aspect input picture and the optimization model.Nearest result of study shows, the natural image block of pixels can be crossed complete base and sparse linear matrix with one group and rebuild and represent, therefore need learn the complete base of mistake of one group of low resolution and one group of corresponding high-resolution based on the method for study base and cross complete base and be used for super-resolution.Owing to need less constraints, the method has attracted more and more researchers' interest at present.People such as Emmanuel Candes, Jianchao Yang and Jinjun Wang good authentication the exploitativeness of the method.

In high-speed camera and high frame-rate video, strong noise and low resolution are two problems that need solve always.It is very crucial how to solve these two problems simultaneously.Under noise was Gaussian Profile hypothesis, people such as Yi Xu and M.B.Chappalli at first were transformed into signal frequency domain, and utilize frequency domain method to finish image denoising and super-resolution.People such as Dekeyser utilize the spatial domain method simultaneously image to be carried out denoising and super-resolution.

But also nobody proposes a kind of effective method at present, can carry out denoising and super-resolution simultaneously to image and the video that contains signal dependent noise.

Summary of the invention

The present invention is intended to solve at least one of technical problem that exists in the prior art.

For this reason, one object of the present invention is to propose a kind of video associating denoising and super-resolution method based on protruding optimum theory, can obtain picture rich in detail and video effectively to carrying out the operation of signal correction denoising and super-resolution, the algorithm highly versatile, video optimized effect is obvious.

Second purpose of the present invention is to propose a kind of video associating denoising based on protruding optimum theory and super-resolution system.

For achieving the above object, the embodiment of first aspect present invention has proposed a kind of video associating denoising and super-resolution method based on protruding optimum theory, may further comprise the steps: according to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base; Set up the constraint formula about noise, reconstructed coefficients matrix and primary signal respectively; Set up target function according to described constraint formula; Utilize augmentation lagrangian optimization method that described target function is found the solution, obtain described primary signal and described reconstructed coefficients matrix; Utilize the complete high-resolution base of described reconstructed coefficients matrix and described mistake to carry out super-resolution operation, obtain the clear high-resolution video after the denoising.

Video associating denoising and super-resolution method based on protruding optimum theory according to the embodiment of the invention can effectively carry out the operation of signal correction denoising and super-resolution to high frame-rate video simultaneously, obtain picture rich in detail and video, the algorithm highly versatile, high frame-rate video under the natural scene all is suitable for, in optimizing process, accomplish signal correction denoising and super-resolution simultaneously, reduced the possibility of deviation iteration, video optimized effect is significantly better than additive method, this method is with the sparse coding theory, protruding optimum theory solves high speed camera capture video noise height preferably for supporting from the software aspect, the problem that bandwidth is little.

In one embodiment of the invention, describedly obtain one group of complete low resolution base of described mistake and the complete high-resolution base of described mistake further comprises: from a plurality of described pictures rich in detail, select a plurality of block of pixels, adopt sparse coding in a plurality of described block of pixels, to learn, obtain the complete low resolution base of described mistake and the complete high-resolution base of described mistake, wherein, a plurality of described pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

In one embodiment of the invention, according to the complete high-resolution base of described mistake and the complete low resolution base of described mistake, utilize described reconstructed coefficients matrix to rebuild and obtain corresponding described low-resolution image and described high-definition picture.

In one embodiment of the invention, the complete low resolution base of described mistake and the complete high-resolution base of described mistake are pervasive to natural scene under statistical significance.

In one embodiment of the invention, describedly set up described target function according to described constraint formula and further comprise: utilize optical flow algorithm to carry out the light stream alignment by relative displacement to described primary signal, obtain photographed data.

In one embodiment of the invention, described constraint formula comprises: described photographed data is made up of primary signal and shooting noise.

In one embodiment of the invention, described constraint formula comprises: the described noise of taking the photograph is in the 3 δ scopes that with described primary signal are average.

In one embodiment of the invention, described constraint formula comprises: described primary signal can be built with the complete low resolution basic weight of described mistake and be represented.

In one embodiment of the invention, described constraint formula comprises: overlapping if two described block of pixels have, then described block of pixels lap is identical.

In one embodiment of the invention, describedly obtain described primary signal and described reconstructed coefficients matrix further comprises: utilize described augmentation lagrangian optimization method, described target function is decomposed into the iteration optimization of described primary signal and the iteration optimization of described reconstructed coefficients, and changes corresponding parameter.

In one embodiment of the invention, described augmentation lagrangian optimization method comprises that described constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

In one embodiment of the invention, described super-resolution operation further comprises: the described reconstructed coefficients matrix that will obtain and the complete high-resolution base of described mistake multiply each other, and obtain corresponding described super-resolution image and video.

In one embodiment of the invention, according to described relative displacement described super-resolution image and video are implemented reflective flow algorithm, obtain the described clear high-resolution video after the denoising.

The embodiment of second aspect present invention has proposed a kind of video associating denoising based on protruding optimum theory and super-resolution system, comprises complete basic study module, constraint formula module, target function module, finds the solution module, result-generation module and reconstruction module.

Wherein, cross complete basic study module and be used for according to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base; Constraint formula module is used for setting up respectively the constraint formula about noise, reconstructed coefficients matrix and primary signal; The target function module is used for setting up target function according to described constraint formula; Find the solution module and be used for utilizing augmentation lagrangian optimization method that described target function is found the solution, obtain described primary signal and described reconstructed coefficients matrix; Result-generation module is used for utilizing described reconstructed coefficients matrix and the complete high-resolution base of described mistake to carry out the super-resolution operation, obtains the clear high-resolution video after the denoising.

Video associating denoising and super-resolution system based on protruding optimum theory according to the embodiment of the invention can effectively carry out the operation of signal correction denoising and super-resolution to high frame-rate video simultaneously, obtain picture rich in detail and video, the algorithm highly versatile, high frame-rate video under the natural scene all is suitable for, in optimizing process, accomplish signal correction denoising and super-resolution simultaneously, reduced the possibility of deviation iteration, video optimized effect is significantly better than other system, native system is with the sparse coding theory, protruding optimum theory solves high speed camera capture video noise height preferably for supporting from the software aspect, the problem that bandwidth is little.

In one embodiment of the invention, crossing complete basic study module also is used for selecting a plurality of block of pixels from a plurality of described pictures rich in detail, adopt sparse coding in a plurality of described block of pixels, to learn, obtain the complete low resolution base of described mistake and the complete high-resolution base of described mistake, wherein, a plurality of described pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

In one embodiment of the invention, also comprise the reconstruction module, be used for according to the complete high-resolution base of described mistake and the complete low resolution base of described mistake, utilize described reconstructed coefficients matrix to rebuild and obtain corresponding described low-resolution image and described high-definition picture.

In one embodiment of the invention, the complete low resolution base of described mistake and the complete high-resolution base of described mistake are pervasive to natural scene under statistical significance.

In one embodiment of the invention, the target function module also is used for, and utilizes optical flow algorithm to carry out the light stream alignment by relative displacement to described primary signal, obtains photographed data.

In one embodiment of the invention, described constraint formula comprises: described photographed data is made up of primary signal and shooting noise.

In one embodiment of the invention, described constraint formula comprises: the described noise of taking the photograph is in the 3 δ scopes that with described primary signal are average.

In one embodiment of the invention, described constraint formula comprises: described primary signal can be built with the complete low resolution basic weight of described mistake and be represented.

In one embodiment of the invention, described constraint formula comprises: overlapping if two described block of pixels have, then described block of pixels lap is identical.

In one embodiment of the invention, the described module of finding the solution also is used for, and utilizes described augmentation lagrangian optimization method, described target function is decomposed into the iteration optimization of described primary signal and the iteration optimization of described reconstructed coefficients, and changes corresponding parameter.

In one embodiment of the invention, described augmentation lagrangian optimization method comprises that described constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

In one embodiment of the invention, described result-generation module also is used for, and described reconstructed coefficients matrix and the complete high-resolution base of described mistake that obtains multiplied each other, and obtains corresponding described super-resolution image and video.

In one embodiment of the invention, described result-generation module also is used for, and according to described relative displacement described super-resolution image and video is implemented reflective flow algorithm, obtains the described clear high-resolution video after the denoising.

Additional aspect of the present invention and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.

Description of drawings

Above-mentioned and/or additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment in conjunction with following accompanying drawing, wherein:

Fig. 1 be according to the embodiment of the invention based on the video associating denoising of protruding optimum theory and the flow chart of super-resolution method;

Fig. 2 a is the structural representation according to the complete low resolution base of mistake of the embodiment of the invention;

Fig. 2 b is the structural representation according to the complete high-resolution base of mistake of the embodiment of the invention;

Fig. 3 is according to parameter in the iteration optimization process of the embodiment of the invention and iteration results change schematic diagram;

Fig. 4 takes optimization result schematic diagram under the noise in various degree according to the embodiment of the invention;

Fig. 5 is the different natural scene experimental result data analysis contrast schematic diagrames according to the embodiment of the invention;

Fig. 6 is the optimization result schematic diagram according to the actual photographed video of the embodiment of the invention; With

Fig. 7 be according to the embodiment of the invention based on the video associating denoising of protruding optimum theory and the structural representation of super-resolution system.

Embodiment

Describe embodiments of the invention below in detail, the example of described embodiment is shown in the drawings, and wherein identical or similar label is represented identical or similar elements or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.

Below with reference to video associating denoising and the super-resolution method based on protruding optimum theory of Fig. 1 description according to the embodiment of the invention, may further comprise the steps:

Step S110: according to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base.

Obtaining one group crosses complete low resolution base and crosses complete high-resolution base and further comprise: select a plurality of block of pixels from a plurality of pictures rich in detail, adopt sparse coding in a plurality of block of pixels, to learn, must be complete low resolution base and cross complete high-resolution base, wherein, a plurality of pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

Wherein, complete low resolution base and the complete high-resolution base of mistake are pervasive to natural scene under statistical significance excessively.

Step S120: set up the constraint formula about noise, reconstructed coefficients matrix and primary signal respectively.

Step S130: set up target function according to the constraint formula.

Setting up target function according to the constraint formula further comprises: utilize optical flow algorithm to carry out the light stream alignment by relative displacement to primary signal, obtain photographed data.

The constraint formula comprises: photographed data is made up of primary signal and shooting noise; Take the photograph noise in the 3 δ scopes that with the primary signal are average; Primary signal can be built with the complete low resolution basic weight of mistake and be represented; If it is overlapping that two block of pixels have, then the block of pixels lap is identical.

Step S140: utilize augmentation lagrangian optimization method that target function is found the solution, obtain primary signal and reconstructed coefficients matrix.

Obtain primary signal and the reconstructed coefficients matrix further comprises: utilize augmentation lagrangian optimization method, target function is decomposed into the iteration optimization of primary signal and the iteration optimization of reconstructed coefficients, and change corresponding parameter.

Wherein, augmentation lagrangian optimization method comprises that constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

Step S150: utilize the reconstructed coefficients matrix and cross complete high-resolution base and carry out the super-resolution operation, obtain the clear high-resolution video after the denoising.

The super-resolution operation further comprises: the reconstructed coefficients matrix and the complete high-resolution base of mistake that obtain are multiplied each other, obtain corresponding super-resolution image and video.According to relative displacement super-resolution image and video are implemented reflective flow algorithm, obtain the clear high-resolution video after the denoising.

In one embodiment of the invention, according to crossing complete high-resolution base and crossing complete low resolution base, utilize the reconstructed coefficients matrix to rebuild and obtain corresponding low-resolution image and high-definition picture.

For example this method is described below, be understandable that, following explanation is not limited thereto according to embodiments of the invention only for illustrative purposes.

Step S210: according to the sparse coding theory, based on image block, in a plurality of picture rich in detail data, learn, obtain one group and cross complete low resolution base and high-resolution base.Wherein, complete low resolution base and high-resolution base are pervasive to natural scene under statistical significance excessively.

Wherein, will cross complete low resolution base note and be B _L, high-resolution base note is B _H, photographic images is at B _LOn projection coefficient have sparse attribute, and can utilize B _HCarry out super-resolution.A plurality of pictures rich in detail comprise a plurality of high-definition pictures and corresponding a plurality of low-resolution images.

For a plurality of block of pixels in a plurality of pictures rich in detail, adopt sparse coding to learn, must be complete low resolution base and cross complete high-resolution base.Simultaneously, according to crossing complete high-resolution base and crossing complete low resolution base, utilize corresponding coefficient matrix to rebuild and obtain corresponding low resolution and high-definition picture.

Utilize optical flow algorithm to the input data, carry out the light stream alignment as high frame-rate video consecutive frame, before the output data, the result is carried out corresponding reflective flow operation.Utilize optical flow algorithm to carry out the light stream alignment input data of algorithm that can be optimized, thereby satisfy the condition of input data matrix address.

Step S220: set up the constraint formula about noise N, reconstructed coefficients matrix A and primary signal L respectively.

The constraint formula comprises: capture video is made up of primary signal and shooting noise; Primary signal and take the relation that the noise average exists square is so according to 3 δ criterions, take noise in the 3 δ scopes that with the primary signal are average;

Primary signal can be crossed complete basic weight with low resolution and build and represent; If it is overlapping that two block of pixels have, then two block of pixels laps are identical.

Step S230: generate the primary signal low-rank, and according to constraint formula and the sparse characteristics of reconstructed coefficients, set up target function.Target function is included denoising and super-resolution in the same optimization framework simultaneously.

Step S240: utilize augmentation lagrangian optimization method that target function is found the solution, obtain primary signal L and reconstructed coefficients A;

Obtain primary signal and reconstructed coefficients further comprises: utilize augmentation lagrangian optimization method, target function is decomposed into the iteration optimization of primary signal and the iteration optimization of reconstructed coefficients matrix, and changes corresponding parameter, obtain primary signal and reconstructed coefficients.With reconstructed coefficients matrix and shooting noise separation.

Augmentation lagrangian optimization method comprises that constraints is carried out augmentation Lagrange to be launched, and obtains the target function that can separate, and adopts protruding optimum theory to carry out iterative.

Step S250: utilize reconstructed coefficients A and cross complete high-resolution base B _HCarry out the super-resolution operation, obtain high-definition picture and video clearly.

The reconstructed coefficients that obtains is multiplied each other with the complete high-resolution base of mistake, obtain super-resolution image and the video of corresponding alignment.The relative displacement that utilizes above-mentioned light stream to obtain is done reflective flow algorithm to super-resolution image and the video of alignment, obtains the clear high-resolution video after the final denoising.

Be the concrete example of this explanation below, be understandable that the present invention is not limited to following example.

Fig. 2 crosses complete base and the corresponding complete high-resolution base of mistake for the one group of low resolution that obtains by sparse coding study.Fig. 2 a is the low resolution base, and 7 * 7pixels, Fig. 2 b are the high-resolution base, 21 * 21pixels.As shown in Figure 2, come reconstructed image and video in order to use same reconstructed coefficients matrix, cross complete high-resolution base and low resolution and cross complete base and will keep corresponding on the locus.

The data of utilizing optical flow algorithm that shooting is obtained are carried out the light stream alignment, obtain photographed data, are expressed as D.And according to primary signal L, shooting noise N, the complete high-resolution base B of mistake _H, cross complete low resolution base B _L, the reconstructed coefficients matrix A sets up following optimization model:

$\min {\left|\right|L\left|\right|}_{*}+\mathrm{\λ}{\left|\right|A\left|\right|}_1$

$s.t.\left\{\begin{array}{c}D=L+N\\ L=B_{L}A\\ \mathrm{IL}=\mathrm{<figure-callout\; id="2"\; label="C\; N"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">C</figure-callout>}& \\ N^{}{}^2\&le;\mathrm{\&alpha;L}+\mathrm{\&beta;}\end{array}\right.$

Wherein C is constant matrices, and α, β are constant; D=L+N represents to take the data that obtain and is made of primary signal L and shooting noise N.L=B _LA represents that primary signal L can use the complete basic B of mistake of low resolution _HRebuild; IL=C represents that the lap of adjacent pixel blocks is identical, and this restrictive condition is in order to eliminate the blocking effect in the reconstructed results.N ²≤ α L+ β shows the non-linear relation of taking between noise N and primary signal L.

Based on protruding optimum theory, utilize augmentation lagrangian optimization method that above-mentioned model is carried out iterative.Above-mentioned model conversation is that Augmented Lagrangian Functions is:

$\mathrm{Lag}={\left|\right|L\left|\right|}_{*}+<{\mathrm{<figure-callout\; id="1"\; label="Y"\; filenames="HDA00003373785800023.png"\; state="\{\{state\}\}">Y</figure-callout>}}_1,D-L-N>+\frac{\mathrm{\&mu;}}{2}{\left|\right|D-L-N\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+<{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">Y</figure-callout>}}_2,L-\mathrm{BA}>+\frac{\mathrm{\&mu;}}{2}{\left|\right|L-\mathrm{BA}\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+<{\mathrm{\&lambda;}}_1,\mathrm{IL}-C>+\frac{\mathrm{\&mu;}}{2}{\left|\right|\mathrm{IL}-C\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+<{\mathrm{\&lambda;}}_2,N^2-(\mathrm{kL}+\mathrm{con})+\mathrm{\&epsiv;}\&CenterDot;^2>+\frac{\mathrm{\&mu;}}{2}{\left|\right|N-(\mathrm{kL}+\mathrm{con})+\mathrm{\&epsiv;}.^2\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

The update rule of important parameter is as follows in each iterative process:

(1), upgrade L:

$L^{n+1}=\arg {\min }_L{\left|\right|L\left|\right|}_{*}+\frac{{\mathrm{\&mu;}}_1}{2}{\left|\right|L-(x-{{\mathrm{\&mu;}}_1}^{-1}\&dtri;f_1\left(x\right))\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$={\mathrm{Us}}_{{{\mathrm{\&mu;}}_1}^{-1}}[L\_\mathrm{temp}]V^T$

UL_tempV wherein ^TBe $x-{{\mathrm{\&mu;}}_1}^{-1}{\&dtri;f}_1\left(x\right)$ The SVD decomposed form, $S_{{{\mathrm{\&mu;}}_1}^{-1}}\left[x\right]=\left\{\begin{array}{cc}x-{{\mathrm{\&mu;}}_1}^{-1},& \mathrm{ifx}>{{\mathrm{\&mu;}}_1}^{-1}\\ x+{{\mathrm{\&mu;}}_1}^{-1},& \mathrm{ifx}<-{{\mathrm{\&mu;}}_1}^{-1}\\ 0,& \mathrm{others},\end{array}\right.$

$\&dtri;f_1\left(x\right)=\mathrm{\&mu;x}-\mathrm{\&mu;}(D-N+{\mathrm{\&mu;}}^{-1}{Y}_1)$

$+\mathrm{\&mu;x}-\mathrm{\&mu;}(B_{L}A-{\mathrm{\&mu;}}^{-1}{Y}_2)$

$+{\mathrm{\&mu;I}}^T\mathrm{Ix}-\mathrm{\&mu;}{I}^T(C-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_1)$

$+{\mathrm{\&mu;\&alpha;}}^{2}x-\mathrm{\&mu;\&alpha;}(N^2-\mathrm{\&beta;}+\mathrm{\&epsiv;}.^2+{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2),$ X represents the L that last iteration obtains.

(2), upgrade A:

$A^{n+1}=\mathrm{ar}g{\min }_A\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+\frac{\mathrm{\&mu;}}{2}{\left|\right|B_{L}A-(L+{\mathrm{\&mu;}}^{-1}{Y}_2)\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$=\arg {\min }_A\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+\frac{{\mathrm{\&mu;}}_2}{2}(A-(x-{{\mathrm{\&mu;}}_2}^{-1}\&dtri;f_2\left(x\right)))$

$=s_{{\mathrm{\&lambda;}/\mathrm{\&mu;}}_2}[x-{{\mathrm{\&mu;}}_2}^{-1}\&dtri;f_2\left(x\right)]$

Wherein ${\&dtri;f}_2\left(x\right)={{\mathrm{\&mu;B}}_L}^T{B}_{L}x-{{\mathrm{\&mu;B}}_L}^T(L+{\mathrm{\&mu;}}^{-1}{Y}_2),$ X represents the A that last iteration obtains.

(3), upgrading N:N is $2N.^3+(1-2(\mathrm{kL}+\mathrm{con}-\mathrm{\&epsiv;}\&CenterDot;^2-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2))\&CenterDot;*N-(D-L+{\mathrm{\&mu;}}^{-1}{Y}_1)=0$ The positive number solution of equation of n th order n.

(4), undated parameter ε: ε equation $\mathrm{\&epsiv;}.^3=(\mathrm{\&alpha;L}+\mathrm{\&beta;}-N^2-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2)\&CenterDot;*\mathrm{\&epsiv;}$ The positive number solution.

Fig. 3 has shown the variation of important parameter and iteration result in the iterative process.Along with the increase of iterations, target function diminishes gradually as seen from the figure, and the constraints error diminishes gradually, shows that optimizing alternative manner can find the solution this model effectively.

Carry out model solution according to above-mentioned iteration update rule, the reconstructed coefficients matrix A after can being optimized.Utilize A and cross complete high-resolution base B _HMultiply each other and to obtain high-resolution video frame to it.The relative displacement that light stream obtains before utilizing is done reflective flow algorithm to these alignment frame of video, just can obtain the clear high-resolution video after the final denoising.

Fig. 4 is for taking the optimization result under the noise in various degree.As seen from the figure, when the input image sequence noise hour, this method can be removed noise fully; When input noise sequence noise was big, this method solving result noise had increase to a certain degree, but still can remove most noise.Fig. 5 is that this method is to different natural scene solving result data analysis contrast schematic diagrames.Under the situation of the identical setting of algorithm parameter, visible this algorithm of lateral comparison all can obtain result preferably to different natural scenes, illustrates that this algorithm robustness is stronger; As seen longitudinal comparison was low to moderate 11 o'clock at signal noise ratio (snr) of image, and this paper algorithm still can promote signal to noise ratio significantly, illustrated that this paper algorithm is highly effective to the quality that improves image.Fig. 6 is the optimization result schematic diagram of actual photographed video, can effectively improve signal to noise ratio and the resolution of actual photographed video by the visible this paper method of the details comparison diagram in (e), has improved the quality of capture video greatly.

Utilized high frame-rate video consecutive frame to change characteristics such as less, that the reconstructed coefficients matrix is sparse when crossing complete basic weight and building image according to the embodiment of the invention based on the video associating denoising of protruding optimum theory and super-resolution method, signal dependent noise denoising and super-resolution are placed same optimization framework, obtain one group by sparse coding study and cross the complete high-resolution base of the complete low resolution base mistake corresponding with a group.Based on protruding optimum theory, utilize augmentation lagrangian optimization method that target function is found the solution, thus the clear low-resolution video that recovers and high-resolution video clearly.This method can effectively be removed signal dependent noise, obtains picture rich in detail and video; The algorithm highly versatile all is suitable for the high frame-rate video under the natural scene; Accomplish signal correction denoising and super-resolution simultaneously, video optimized effect is significantly better than additive method.

Below with reference to video associating denoising and the super-resolution system 100 based on protruding optimum theory of Fig. 7 description according to the embodiment of the invention, comprised complete basic study module 110, constraint formula module 120, target function module 130, find the solution module 140, result-generation module 150 and rebuild module 160.

Cross complete basic study module 110 and be used for according to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base; Constraint formula module 120 is used for setting up respectively the constraint formula about noise, reconstructed coefficients matrix and primary signal; Target function module 130 is used for setting up target function according to the constraint formula; Find the solution module 140 and be used for utilizing augmentation lagrangian optimization method that target function is found the solution, obtain primary signal and reconstructed coefficients matrix; Result-generation module 150 is used for utilizing the reconstructed coefficients matrix and crosses complete high-resolution base and carry out the super-resolution operation, obtains the clear high-resolution video after the denoising.

Crossing complete basic study module 110 also is used for selecting a plurality of block of pixels from a plurality of pictures rich in detail, adopt sparse coding in a plurality of block of pixels, to learn, must be complete low resolution base and cross complete high-resolution base, wherein, a plurality of pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

Rebuilding module 160 is used for utilizing the reconstructed coefficients matrix to rebuild and obtaining corresponding low-resolution image and high-definition picture according to crossing complete high-resolution base and crossing complete low resolution base.

Wherein, complete low resolution base and the complete high-resolution base of mistake are pervasive to natural scene under statistical significance excessively.

Target function module 130 also is used for utilizing optical flow algorithm to carry out the light stream alignment by relative displacement to primary signal, obtains photographed data.

The constraint formula comprises: photographed data is made up of primary signal and shooting noise; Take the photograph noise in the 3 δ scopes that with the primary signal are average; Primary signal can be built with the complete low resolution basic weight of mistake and be represented; If it is overlapping that two block of pixels have, then the block of pixels lap is identical.

Find the solution module 140 and also be used for, utilize augmentation lagrangian optimization method, target function is decomposed into the iteration optimization of primary signal and the iteration optimization of reconstructed coefficients, and change corresponding parameter.Augmentation lagrangian optimization method comprises that constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

Result-generation module 150 also is used for, and reconstructed coefficients matrix and the complete high-resolution base of mistake that obtains multiplied each other, and obtains corresponding super-resolution image and video.According to relative displacement super-resolution image and video are implemented reflective flow algorithm, obtain the clear high-resolution video after the denoising.

For example this method is described below, be understandable that, following explanation is not limited thereto according to embodiments of the invention only for illustrative purposes.

Cross complete basic study module 110 according to the sparse coding theory, based on image block, in a plurality of picture rich in detail data, learn, obtain one group and cross complete low resolution base and high-resolution base.Wherein, complete low resolution base and high-resolution base are pervasive to natural scene under statistical significance excessively.

Wherein, will cross complete low resolution base note and be B _L, high-resolution base note is B _H, photographic images is at B _LOn projection coefficient have sparse attribute, and can utilize B _HCarry out super-resolution.A plurality of pictures rich in detail comprise a plurality of high-definition pictures and corresponding a plurality of low-resolution images.

Particularly, cross complete basic study module 110 for a plurality of block of pixels in a plurality of pictures rich in detail, adopt sparse coding to learn, must be complete low resolution base and cross complete high-resolution base.According to crossing complete high-resolution base and crossing complete low resolution base, reconstruction module 160 is utilized corresponding coefficient matrix to rebuild and is obtained corresponding low resolution and high-definition picture.

Cross complete basic study module 110 and utilize optical flow algorithm to the input data, carry out the light stream alignment as high frame-rate video consecutive frame, before the output data, the result is carried out corresponding reflective flow operation.Utilize optical flow algorithm to carry out the light stream alignment input data of algorithm that can be optimized, thereby satisfy the condition of input data matrix address.

Constraint formula module 120 is set up the constraint formula about noise N, reconstructed coefficients matrix A and primary signal L respectively.

The constraint formula comprises: capture video is made up of primary signal and shooting noise; Primary signal and take the relation that the noise average exists square is so according to 3 δ criterions, take noise in the 3 δ scopes that with the primary signal are average;

Primary signal can be built with the complete low resolution basic weight of mistake and be represented; If it is overlapping that two block of pixels have, then two block of pixels laps are identical.

According to primary signal low-rank, constraint formula and the sparse characteristics of reconstructed coefficients, target function module 130 is set up target function.Target function is included denoising and super-resolution in the same optimization framework simultaneously.

Find the solution module 140 and utilize augmentation lagrangian optimization method that target function is found the solution, obtain primary signal L and reconstructed coefficients A;

Wherein, find the solution that module 140 obtains primary signal and reconstructed coefficients further comprises: utilize augmentation lagrangian optimization method, target function is decomposed into the iteration optimization of primary signal and the iteration optimization of reconstructed coefficients matrix, and changes corresponding parameter, obtain primary signal and reconstructed coefficients.With reconstructed coefficients matrix and shooting noise separation.

Augmentation lagrangian optimization method comprises that constraints is carried out augmentation Lagrange to be launched, and obtains the target function that can separate, and adopts protruding optimum theory to carry out iterative.

Result-generation module 150 utilizes reconstructed coefficients A and crosses complete high-resolution base B _HCarry out the super-resolution operation, obtain high-definition picture and video clearly.

Result-generation module 150 multiplies each other the reconstructed coefficients that obtains with the complete high-resolution base of mistake, obtain super-resolution image and the video of corresponding alignment.The relative displacement that utilizes above-mentioned light stream to obtain is done reflective flow algorithm to super-resolution image and the video of alignment, obtains the clear high-resolution video after the final denoising.

Be the concrete example of this explanation below, be understandable that the present invention is not limited to following example.

Fig. 2 was one group of complete high-resolution base of mistake of crossing complete low resolution base and correspondence that complete basic study module 110 obtains by sparse coding study.Fig. 2 a is the low resolution base, and 7 * 7pixels, Fig. 2 b are the high-resolution base, 21 * 21pixels.As shown in Figure 2, come reconstructed image and video in order to use same reconstructed coefficients matrix, cross complete high-resolution base and the complete low resolution base of mistake and will keep corresponding on the locus.

Cross the data that complete basic study module 110 utilizes optical flow algorithm that shooting is obtained and carry out the light stream alignment, obtain photographed data, be expressed as D.Constraint formula module 120 and target function module 130 are according to primary signal L, shooting noise N, the complete high-resolution base B of mistake _H, cross complete low resolution base B _L, the reconstructed coefficients matrix A sets up following optimization model:

$\min {\left|\right|L\left|\right|}_{*}+\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1$

$s.t.\left\{\begin{array}{c}D=L+N\\ L=B_{l}A\\ \mathrm{IL}=\mathrm{<figure-callout\; id="2"\; label="C\; N"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">C</figure-callout>}& \\ N^{}{}^2\&le;\mathrm{\&alpha;L}+\mathrm{\&beta;}\end{array}\right.$

Wherein C is constant matrices, and α, β are constant; D=L+N represents to take the data that obtain and is made of primary signal L and shooting noise N.L=B _LA represents that primary signal L can use the complete basic B of mistake of low resolution _HRebuild; IL=C represents that the lap of adjacent pixel blocks is identical, and this restrictive condition is in order to eliminate the blocking effect in the reconstructed results.N ²≤ α L+ β shows the non-linear relation of taking between noise N and primary signal L.

Based on protruding optimum theory, find the solution module 140 and utilize augmentation lagrangian optimization method that above-mentioned model is carried out iterative.Above-mentioned model conversation is that Augmented Lagrangian Functions is:

$\mathrm{Lag}={\left|\right|L\left|\right|}_{*}+<{\mathrm{<figure-callout\; id="1"\; label="Y"\; filenames="HDA00003373785800023.png"\; state="\{\{state\}\}">Y</figure-callout>}}_1,D-L-N>+\frac{\mathrm{\&mu;}}{2}\left|\right|D-L-N{\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+<{\mathrm{<figure-callout\; id="2"\; label="Y"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">Y</figure-callout>}}_2,L-\mathrm{BA}>+\frac{\mathrm{\&mu;}}{2}{\left|\right|L-\mathrm{BA}\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+<{\mathrm{\&lambda;}}_1,\mathrm{IL}-C>+\frac{\mathrm{\&mu;}}{2}{\left|\right|\mathrm{IL}-C\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$+<{\mathrm{\&lambda;}}_2,N^2-(\mathrm{kL}+\mathrm{con})+\mathrm{\&epsiv;}\&CenterDot;^2>+\frac{\mathrm{\&mu;}}{2}{\left|\right|N-(\mathrm{kL}+\mathrm{con})+\mathrm{\&epsiv;}.^2\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

The update rule of finding the solution module 140 important parameter in each iterative process is as follows:

(1), upgrade L:

$L^{n+1}=\arg {\min }_L{\left|\right|L\left|\right|}_{*}+\frac{{\mathrm{\&mu;}}_1}{2}{\left|\right|L-(x-{{\mathrm{\&mu;}}_1}^{-1}\&dtri;f_1\left(x\right))\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$={\mathrm{Us}}_{{{\mathrm{\&mu;}}_1}^{-1}}[L\_\mathrm{temp}]V^T$

UL_tempV wherein ^TBe $x-{{\mathrm{\&mu;}}_1}^{-1}{\&dtri;f}_1\left(x\right)$ The SVD decomposed form, $S_{{{\mathrm{\&mu;}}_1}^{-1}}\left[x\right]=\left\{\begin{array}{cc}x-{{\mathrm{\&mu;}}_1}^{-1},& \mathrm{ifx}>{{\mathrm{\&mu;}}_1}^{-1}\\ x+{{\mathrm{\&mu;}}_1}^{-1},& \mathrm{ifx}<-{{\mathrm{\&mu;}}_1}^{-1}\\ 0,& \mathrm{others},\end{array}\right.$

${\&dtri;f}_1\left(x\right)=\mathrm{\&mu;x}-\mathrm{\&mu;}(D-N+{\mathrm{\&mu;}}^{-1}{Y}_1)$

$+\mathrm{\&mu;x}-\mathrm{\&mu;}(B_{L}A-{\mathrm{\&mu;}}^{-1}{Y}_2)$

${+\mathrm{\&mu;I}}^T\mathrm{Ix}-{\mathrm{\&mu;I}}^T(C-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_1)$

$+{\mathrm{\&mu;\&alpha;}}^{2}x-\mathrm{\&mu;\&alpha;}(N^2-\mathrm{\&beta;}+\mathrm{\&epsiv;}\&CenterDot;^2+{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2),$ X represents the L that last iteration obtains.

(2), upgrade A:

$A^{n+1}=\arg {\min }_A\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+\frac{\mathrm{\&mu;}}{2}{\left|\right|B_{L}A-(L+{\mathrm{\&mu;}}^{-1}{Y}_2)\left|\right|}_{\mathrm{<figure-callout\; id="2"\; label="F"\; filenames="HDA00003373785800033.png"\; state="\{\{state\}\}">F</figure-callout>}}^2$

$=\arg {\min }_A\mathrm{\&lambda;}{\left|\right|A\left|\right|}_1+\frac{{\mathrm{\&mu;}}_2}{2}(A-(x-{{\mathrm{\&mu;}}_2}^{-1}{\&dtri;f}_2\left(x\right)))$

$=S_{\mathrm{\&lambda;}/{\mathrm{\&mu;}}_2}[x-{{\mathrm{\&mu;}}_2}^{-1}{\&dtri;f}_2\left(x\right)]$

Wherein ${\&dtri;f}_2\left(x\right)={{\mathrm{\&mu;B}}_L}^T{B}_{L}x-{{\mathrm{\&mu;B}}_L}^T(L+{\mathrm{\&mu;}}^{-1}{Y}_2),$ X represents the A that last iteration obtains.

(3), upgrading N:N is $2N.^3+(1-2(\mathrm{kL}+\mathrm{con}-\mathrm{\&epsiv;}\&CenterDot;^2-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2))\&CenterDot;*N(D-L+{\mathrm{\&mu;}}^{-1}{Y}_1)=0$ The positive number solution of cubic equation.

(4), undated parameter ε: ε equation $\mathrm{\&epsiv;}\&CenterDot;^3=(\mathrm{\&alpha;L}+\mathrm{\&beta;}-N^2-{\mathrm{\&mu;}}^{-1}{\mathrm{\&lambda;}}_2)\&CenterDot;*\mathrm{\&epsiv;}$ The positive number solution.

Fig. 3 has shown the variation of important parameter and iteration result in the iterative process.Along with the increase of iterations, target function diminishes gradually as seen from the figure, and the constraints error diminishes gradually, shows that optimizing alternative manner can find the solution this model effectively.

Carry out model solution according to above-mentioned iteration update rule, the reconstructed coefficients matrix A after can being optimized.Result-generation module 150 utilizes A and crosses complete high-resolution base B _HMultiply each other and to obtain high-resolution video frame to it.The relative displacement that light stream obtains before utilizing is done reflective flow algorithm to these alignment frame of video, just can obtain the clear high-resolution video after the final denoising.

Fig. 4 is for taking the optimization result under the noise in various degree.Fig. 5 is artificial generated data experimental result.Be the optimization result of actual photographed video as Fig. 6.

Utilized high frame-rate video consecutive frame to change characteristics such as less, that the reconstructed coefficients matrix is sparse when crossing complete basic weight and building image according to the embodiment of the invention based on the video associating denoising of protruding optimum theory and super-resolution system, signal dependent noise denoising and super-resolution are placed same optimization framework, obtain one group by sparse coding study and cross the complete high-resolution base of the complete low resolution base mistake corresponding with a group.Based on protruding optimum theory, utilize augmentation lagrangian optimization method that target function is found the solution, thus the clear low-resolution video that recovers and high-resolution video clearly.This method can effectively be removed signal dependent noise, obtains picture rich in detail and video; The algorithm highly versatile all is suitable for the high frame-rate video under the natural scene; Accomplish signal correction denoising and super-resolution simultaneously, video optimized effect is significantly better than additive method.

In the description of this specification, concrete feature, structure, material or characteristics that the description of reference term " embodiment ", " some embodiment ", " example ", " concrete example " or " some examples " etc. means in conjunction with this embodiment or example description are contained at least one embodiment of the present invention or the example.In this manual, the schematic statement to above-mentioned term not necessarily refers to identical embodiment or example.And concrete feature, structure, material or the characteristics of description can be with the suitable manner combination in any one or more embodiment or example.

Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification to these embodiment that scope of the present invention is by claims and be equal to and limit.

Claims (26)

1. video associating denoising and super-resolution method based on a protruding optimum theory is characterized in that, may further comprise the steps:

According to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base;

Set up the constraint formula about noise, reconstructed coefficients matrix and primary signal respectively;

Set up target function according to described constraint formula;

Utilize augmentation lagrangian optimization method that described target function is found the solution, obtain described primary signal and described reconstructed coefficients matrix; And

Utilize the complete high-resolution base of described reconstructed coefficients matrix and described mistake to carry out super-resolution operation, obtain the clear high-resolution video after the denoising.

2. the method for claim 1, it is characterized in that, describedly obtain one group of complete low resolution base of described mistake and the complete high-resolution base of described mistake further comprises: from a plurality of described pictures rich in detail, select a plurality of block of pixels, adopt sparse coding in a plurality of described block of pixels, to learn, obtain the complete low resolution base of described mistake and the complete high-resolution base of described mistake, wherein, a plurality of described pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

3. the method for claim 1 is characterized in that, according to the complete high-resolution base of described mistake and the complete low resolution base of described mistake, utilizes described reconstructed coefficients matrix to rebuild and obtains corresponding described low-resolution image and described high-definition picture.

4. the method for claim 1 is characterized in that, the complete low resolution base of described mistake and the complete high-resolution base of described mistake are pervasive to natural scene under statistical significance.

5. the method for claim 1 is characterized in that, describedly sets up described target function according to described constraint formula and further comprises: utilize optical flow algorithm to carry out the light stream alignment by relative displacement to described primary signal, obtain photographed data.

6. method as claimed in claim 5 is characterized in that, described constraint formula comprises: described photographed data is made up of primary signal and shooting noise.

7. the method for claim 1 is characterized in that, described constraint formula comprises: the described noise of taking the photograph is in the 3 δ scopes that with described primary signal are average.

8. the method for claim 1 is characterized in that, described constraint formula comprises: described primary signal can be built with the complete low resolution basic weight of described mistake and be represented.

9. the method for claim 1 is characterized in that, described constraint formula comprises: overlapping if two described block of pixels have, then described block of pixels lap is identical.

10. the method for claim 1, it is characterized in that, describedly obtain described primary signal and described reconstructed coefficients matrix further comprises: utilize described augmentation lagrangian optimization method, described target function is decomposed into the iteration optimization of described primary signal and the iteration optimization of described reconstructed coefficients, and changes corresponding parameter.

11. the method for claim 1 is characterized in that, described augmentation lagrangian optimization method comprises that described constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

12. the method for claim 1 is characterized in that, described super-resolution operation further comprises: the described reconstructed coefficients matrix that will obtain and the complete high-resolution base of described mistake multiply each other, and obtain corresponding described super-resolution image and video.

13. the method for claim 1 is characterized in that, according to described relative displacement described super-resolution image and video is implemented reflective flow algorithm, obtains the described clear high-resolution video after the denoising.

14. video associating denoising and super-resolution system based on a protruding optimum theory is characterized in that, comprising:

Cross complete basic study module, be used for according to the sparse coding theory, based on image block, from a plurality of pictures rich in detail, learn, obtain one group and cross complete low resolution base and cross complete high-resolution base.

Constraint formula module is used for setting up respectively the constraint formula about noise, reconstructed coefficients matrix and primary signal;

The target function module is used for setting up target function according to described constraint formula;

Find the solution module, be used for utilizing augmentation lagrangian optimization method that described target function is found the solution, obtain described primary signal and described reconstructed coefficients matrix; And

Result-generation module is used for utilizing described reconstructed coefficients matrix and the complete high-resolution base of described mistake to carry out the super-resolution operation, obtains the clear high-resolution video after the denoising.

15. system as claimed in claim 14, it is characterized in that, crossing complete basic study module also is used for, from a plurality of described pictures rich in detail, select a plurality of block of pixels, adopt sparse coding in a plurality of described block of pixels, to learn, obtain the complete low resolution base of described mistake and the complete high-resolution base of described mistake, wherein, a plurality of described pictures rich in detail comprise a plurality of high-definition pictures and corresponding low-resolution image.

16. system as claimed in claim 14, it is characterized in that, also comprise the reconstruction module, be used for according to the complete high-resolution base of described mistake and the complete low resolution base of described mistake, utilize described reconstructed coefficients matrix to rebuild and obtain corresponding described low-resolution image and described high-definition picture.

17. system as claimed in claim 14 is characterized in that, the complete low resolution base of described mistake and the complete high-resolution base of described mistake are pervasive to natural scene under statistical significance.

18. system as claimed in claim 14 is characterized in that, the target function module also is used for, and utilizes optical flow algorithm to carry out the light stream alignment by relative displacement to described primary signal, obtains photographed data.

19. system as claimed in claim 18 is characterized in that, described constraint formula comprises: described photographed data is made up of primary signal and shooting noise.

20. system as claimed in claim 14 is characterized in that, described constraint formula comprises: the described noise of taking the photograph is in the 3 δ scopes that with described primary signal are average.

21. system as claimed in claim 14 is characterized in that, described constraint formula comprises: described primary signal can be built with the complete low resolution basic weight of described mistake and be represented.

22. system as claimed in claim 14 is characterized in that, described constraint formula comprises: overlapping if two described block of pixels have, then described block of pixels lap is identical.

23. system as claimed in claim 14, it is characterized in that the described module of finding the solution also is used for, utilize described augmentation lagrangian optimization method, described target function is decomposed into the iteration optimization of described primary signal and the iteration optimization of described reconstructed coefficients, and changes corresponding parameter.

24. system as claimed in claim 14 is characterized in that, described augmentation lagrangian optimization method comprises that described constraints is carried out the augmentation Lagrange to be launched, and obtains separating target function, and adopts protruding optimum theory to carry out iterative.

25. system as claimed in claim 14 is characterized in that, described result-generation module also is used for, and described reconstructed coefficients matrix and the complete high-resolution base of described mistake that obtains multiplied each other, and obtains corresponding described super-resolution image and video.

26. system as claimed in claim 14 is characterized in that, described result-generation module also is used for, and according to described relative displacement described super-resolution image and video is implemented reflective flow algorithm, obtains the described clear high-resolution video after the denoising.

Images