CN104050652A - Super-resolution reconstruction method for constructing pyramid in self-learning mode - Google Patents

Abstract

The invention relates to a super-resolution reconstruction method for constructing pyramid self-learning. The method includes the following steps that registering is conducted on an obtained original image sequence, and registering is within the accuracy of one pixel; a downsampling is conducted on the original image sequence, and a downsampling image sequence is obtained; the relation between a downsampling image and an adjacent frame image is established, the pixel of the downsampling image is made to correspond to the pixel of the original image, and a learning relation is established; Omega k is solved to serve as the relation between a super-resolution image and a low resolution image; a super-resolution image sequence is obtained according to the original image sequence through the Omega k. According to the super-resolution reconstruction method for constructing pyramid self-learning, an image sequence of a lower resolution is obtained through downsampling imaging, and a downward pyramid is constructed, so that a known reconstruction result is equaled, a reconstruction law is learnt, then the pyramid is extended upwards, and an image of a higher resolution is obtained. By means of the super-resolution reconstruction method, the image of the higher solution can be obtained through the self-learning, and the method had higher adaptability.

Description

The ultra-resolution ratio reconstructing method of structure pyramid self study

Technical field

The invention belongs to computer vision, image processing field, particularly a kind of ultra-resolution ratio reconstructing method of constructing pyramid self study.

Background technology

Super resolution image (Super Resolution, SR) reconstruct is by multiple (Low Resolution that degenerates continuously, LR) image rebuilds a kind of image reconstruction technique of high resolving power (High Resolution, HR) image.The reason that forms LR image is due to the restriction of sampling hardware condition, adds the impact of various degeneration factors (such as motion deformation, optical dimming, random noise), there are differences of acquired original image and desirable image object.In moving pedestal (as Aeronautics and Astronautics, mobile monitor) photoelectric measurement field, the size of photoelectric measurement equipment and weight have strict restriction, how in the situation that not changing existing hardware condition, it is significant that the ability of high image resolution is more obtained in lifting, how to utilize existing resolution sequence image information to reconstruct more high-resolution image and have important theoretical significance and actual application value widely.

Traditional super-resolution algorithms is not in the time there is no high-definition picture, need to carry out a priori assumption to its model, then be reconstructed by several low-resolution images, this a priori assumption can obtain by the method for estimating or learn, but the result of estimating does not often square with the fact, and method versatility and the real-time of study are not strong.

Summary of the invention

The present invention will solve technical matters of the prior art, provide a kind of can self study draw high-definition picture and have stronger adaptive, the ultra-resolution ratio reconstructing method of structure pyramid self study.

In order to solve the problems of the technologies described above, technical scheme of the present invention is specific as follows:

A ultra-resolution ratio reconstructing method of constructing pyramid self study, comprises the following steps:

Step 1: the original sequence obtaining is carried out to registration, and registration accuracy reaches in 1 pixel;

Step 2: original sequence is carried out down-sampled, obtains down-sampled image sequence;

Step 3: set up the relation of down-sampled image and consecutive frame image, down-sampled image picture elements is corresponding with original image pixel, set up study relation, represented by linear representation:

$p_h^{i,j}=\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}$

Wherein, low-resolution image pixel value for high-definition picture the neighborhood pixel that position is corresponding, ω _kfor weighting coefficient;

Step 4: ask for ω _kbe the relation between high-definition picture and low-resolution image, make:

$\mathrm{\ϵ}=\min {\left|\right|p_h^{i,j}-\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}\left|\right|}^2;$

Step 5: utilize ω _ktry to achieve high-definition picture sequence according to original sequence.

In technique scheme, in step 5, image block being merged in piece image process, the piece that matches adopts average fusion method.

In technique scheme, in step 5, obtaining after optimal estimation, first deblurring, then, in conjunction with overall Reconstruction Constraints, Optimized Iterative obtains result:

$\hat{X}=\arg \min \{\underset{k=1}{\overset{p}{\mathrm{\Σ}}}{\left|\right|Y-H_{k}X\left|\right|}^2+\mathrm{\β}\underset{c\∈C}{\mathrm{\Σ}}{\mathrm{\ρ}}_{\mathrm{\α}}\left({\mathrm{dx}}_c\right)\}.$

The present invention has following beneficial effect:

The ultra-resolution ratio reconstructing method of structure pyramid of the present invention self study, obtains the more image sequence of low resolution by down-sampled imaging, constructs downward pyramid, equivalent so known reconstruction result, study reconstruct rule, then pyramid is upwards extended, more high-resolution image obtained.The ultra-resolution ratio reconstructing method of structure pyramid of the present invention self study can self study draw high-definition picture, has stronger adaptability.

Brief description of the drawings

Below in conjunction with the drawings and specific embodiments, the present invention is described in further detail.

Fig. 1 is inverted pyramid recursive relation view of the present invention.

Fig. 2 is the correspondence position schematic diagram of area-of-interest in image sequence that need to carry out super-resolution reconstruction.

Fig. 3 is the schematic diagram of the ultra-resolution ratio reconstructing method of structure pyramid of the present invention self study.

Embodiment

Invention thought of the present invention is:

The super-resolution imaging model formula that degrades represents, can write:

Y _k＝D _kH _kF _k X+ V _k,1≤k≤N

N width low-resolution image is expressed as size is [M _k× M _k], (k=1,2,3), high-definition picture to be asked is expressed as x, size is [L × L], and by image by row vector representation.Have this model just can be further expressed as vector form:

$\begin{array}{c}\left[\begin{array}{c}{\underset{\‾}{Y}}_1\\ \·\\ \·\\ \·\\ {\underset{\‾}{Y}}_N\end{array}\right]\\ \⇔\underset{\‾}{Y}=H\underset{\‾}{X}+\underset{\‾}{E}\end{array}=\left[\begin{array}{c}{D}_1{C}_1{F}_1\\ \·\\ \·\\ \·\\ D_N{C}_N{F}_N\end{array}\right]\underset{\‾}{X}+\left[\begin{array}{c}{\underset{\‾}{E}}_1\\ \·\\ \·\\ \·\\ {\underset{\‾}{E}}_N\end{array}\right]=\left[\begin{array}{c}{H}_1\\ \·\\ \·\\ \·\\ H_N\end{array}\right]\underset{\‾}{X}+\underset{\‾}{E}$

For above-mentioned imaging model, the process that solves the high-definition picture X that is gone out to need by known parametric solution in fact exactly of super-resolution problem, is an inverse problem Solve problems.But in fact, this equation can not directly be tried to achieve by inverse operation, except noise E item is difficult to accurately estimate, the transformation matrix H in equation is a singular matrix often, is that an ill-condition problem solves.But can find out from equation, if acted in the similar situation of the model of low-resolution image, between high-definition picture local unit and low-resolution image local unit, have linear changing relation.

The image sequence that camera acquisition is arrived is as HR (High Resolution high-definition picture), first carry out mating between image, find the correspondence position of the area-of-interest that need to carry out super-resolution reconstruction in image sequence, carry out down-sampled to them, obtain the image of a series of LR resolution, as shown in Figure 2, by the relation between the correlativity between each LR and LR and HR, relation between study low resolution and high-definition picture, finally utilizes HR and reconstructs more high-resolution target image SR.It should be noted that, learning process is the study of interframe intersection, because the relation of high resolving power and low-resolution image is to be determined by imaging model, can not be from the down-sampled study of single image out, and though inter frame image is by very strong correlativity, but be not simple down-sampled relation, but all results after optical dimming, atmospheric agitation impact of each two field picture, therefore interframe study more can embody the true relation between high-definition picture and low-resolution image compared with single width study.

Below in conjunction with accompanying drawing, the present invention is described in detail.The ultra-resolution ratio reconstructing method of structure pyramid of the present invention self study as Figure 1-3, it comprises the following steps:

1. first the original sequence obtaining is carried out to registration, registration accuracy reaches 1 pixel (has several different methods to realize with interior at present, as corners Matching, yardstick extraneous features point (SIFT) coupling etc., need the interesting target of coupling to there is obvious outline herein, therefore intend adopting the method for previous work: based on the Scene matching method based of characteristic curve and polar coordinate transform);

2. original sequence is carried out down-sampledly, obtains down-sampled image sequence;

3. set up the relation of down-sampled image and consecutive frame image, down-sampled image picture elements is corresponding with original image pixel, set up study relation, can be represented by linear representation:

$p_h^{i,j}=\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}$

Wherein low-resolution image pixel value for high-definition picture the neighborhood pixel that position is corresponding, ω _kfor weighting coefficient;

4. ask for ω _kbe the relation between high-definition picture and low-resolution image, make:

$\mathrm{\ϵ}=\min {\left|\right|p_h^{i,j}-\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}\left|\right|}^2$

5. utilize ω _ktry to achieve high-definition picture sequence according to original sequence, image block is merged in piece image process, the piece that matches adopts average fusion method.Because local average process can produce fuzzy and unnecessary distortion, therefore obtaining after optimal estimation, first deblurring, then, in conjunction with overall Reconstruction Constraints, Optimized Iterative obtains result:

$\hat{X}=\arg \min \{\underset{k=1}{\overset{p}{\mathrm{\Σ}}}{\left|\right|Y-H_{k}X\left|\right|}^2+\mathrm{\β}\underset{c\∈C}{\mathrm{\Σ}}{\mathrm{\ρ}}_{\mathrm{\α}}\left({\mathrm{dx}}_c\right)\}.$

Obviously, above-described embodiment is only for example is clearly described, and the not restriction to embodiment.For those of ordinary skill in the field, can also make other changes in different forms on the basis of the above description.Here without also giving exhaustive to all embodiments.And the apparent variation of being extended out thus or variation are still among the protection domain in the invention.

Claims (3)

1. a ultra-resolution ratio reconstructing method of constructing pyramid self study, is characterized in that, comprises the following steps:

Step 1: the original sequence obtaining is carried out to registration, and registration accuracy reaches in 1 pixel;

Step 2: original sequence is carried out down-sampled, obtains down-sampled image sequence;

Step 3: set up the relation of down-sampled image and consecutive frame image, down-sampled image picture elements is corresponding with original image pixel, set up study relation, represented by linear representation:

$p_h^{i,j}=\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}$

Wherein, low-resolution image pixel value for high-definition picture the neighborhood pixel that position is corresponding, ω _kfor weighting coefficient;

Step 4: ask for ω _kbe the relation between high-definition picture and low-resolution image, make:

$\mathrm{\ϵ}=\min {\left|\right|p_h^{i,j}-\underset{k}{\mathrm{\Σ}}{\mathrm{\ω}}_k{p}_l^{m,n}\left|\right|}^2;$

Step 5: utilize ω _ktry to achieve high-definition picture sequence according to original sequence.

2. the ultra-resolution ratio reconstructing method of structure pyramid according to claim 1 self study, is characterized in that, in step 5, image block is merged in piece image process, the piece that matches adopts average fusion method.

3. the ultra-resolution ratio reconstructing method of structure pyramid according to claim 1 self study, is characterized in that, in step 5, obtaining after optimal estimation, and first deblurring, then, in conjunction with overall Reconstruction Constraints, Optimized Iterative obtains result:

$\hat{X}=\arg \min \{\underset{k=1}{\overset{p}{\mathrm{\Σ}}}{\left|\right|Y-H_{k}X\left|\right|}^2+\mathrm{\β}\underset{c\∈C}{\mathrm{\Σ}}{\mathrm{\ρ}}_{\mathrm{\α}}\left({\mathrm{dx}}_c\right)\}.$