CN100382567C - Method for rebuilding super resolution image from reduced quality image caused by interlaced sampling - Google Patents

Abstract

A method for rebuilding super resolution image in image at low quality caused by interlaced sampling includes separately picking up odd and even line of each frame image in low resolution image sequence being at size of N1xN2 and being formed by interlaced sampling and forming them separately to be odd and even line image at size of N1 x N2 / 2, forming obtained odd and even line images at size of N1xN2 / 2 to be new image sequence then carrying out super resolution treatment on said new sequence for obtaining result image with high resolution ratio.

Description

From cause the method for rebuilding super resolution image the image that degrades because of partiting row sampling

Technical field

The present invention relates to a kind of from cause the method for rebuilding super resolution image the image that degrades because of partiting row sampling.

Background technology

Along with being extensive use of of various imaging sensors, picture quality becomes the crux in the practical application.Partiting row sampling imaging (collection) technology is the technology that forms for the video acquisition frequency that obtains more at a high speed, and what use in a large amount of application all is the transducer of partiting row sampling imaging or the capture card of partiting row sampling.But the shortcoming of this technology is: when bigger motion occurring between scenery in the scene and the imaging platform, partiting row sampling can cause occurring degrading of lattice-shaped in the image, is called " partiting row sampling grid ".As shown in Figure 1, occurred significantly " partiting row sampling grid " in the image, had a strong impact on the quality of image.

The operation principle of image partiting row sampling, as shown in Figure 2, row in the left side of schematic diagram are original images of input, through partiting row sampling, obtained strange/even number line image sequence of middle column, the output of partiting row sampling is the complete image that row on right side are synthesized by the capable image of adjacent odd and even number.When having bigger motion in the scene that is taken (automobile of a motion as shown in Figure 2), tangible partiting row sampling grid can appear, and this result has caused decrease in image quality.Though partiting row sampling can be to write down than higher frame rate and to show that the partiting row sampling grid has caused decrease in image quality, this phenomenon can not be ignored.At present also there is not effective method can overcome this defective that partiting row sampling brings.

Summary of the invention

The present invention aims to provide a kind of from cause the method for rebuilding super resolution image the image that degrades because of partiting row sampling, finally eliminates " partiting row sampling grid " to guarantee the partiting row sampling image, and improves the resolution of image.

Provided by the invention this from cause the method for rebuilding super resolution image the image that degrades because of partiting row sampling, may further comprise the steps:

A, the size that partiting row sampling is obtained are N ₁* N ₂Sequence of low resolution pictures { Y _i| i=0 ..., the odd and even number of every two field picture is capable among the P} proposes respectively, and the composition size is N respectively ₁* N ₂/ 2 odd-numbered line image and even number line image;

B, the size that step a is obtained are N ₁* N ₂/ 2 odd-numbered line image and even number line image are formed new image sequence { S _i| i=0 ..., 2P};

C, to new sequence { S _i| i=0 ..., 2P} carries out SUPERRESOLUTION PROCESSING FOR ACOUSTIC and obtains the high-resolution result images.

D, the image sequence that described step b is obtained carry out motion detection and cut apart, with { S _iIn scene be divided into some consistent zones of moving;

E, by described step c, SUPERRESOLUTION PROCESSING FOR ACOUSTIC is carried out in each consistent moving region; Each high-resolution region image that will obtain at last carries out amalgamation, returns to the high-definition picture consistent with original scene.

According to the principle of partiting row sampling, the new sequence { S that finishes from step a of the present invention to step b _iFoundation { S _iIn, there be not " partiting row sampling grid ", utilize existing super-resolution technique to handle again, the high-definition picture of therefore can be eliminated " partiting row sampling grid ".Fig. 6 is that the inventor utilizes the inventive method to carry out the image of handling to Fig. 5, compares this two width of cloth figure, and obviously there be " partiting row sampling grid " in Fig. 5, and Fig. 6 does not have, and the very clear wash rice of image has embodied the significant beneficial effect of the present invention.This has important practical value to fields such as civilian security protection supervision, military surveillance, space flight earth observations.

Description of drawings

Fig. 1 is the pattern (ship) with the image capture device acquisition of partiting row sampling;

Fig. 2 is the operation principle schematic diagram of image partiting row sampling;

Fig. 3 is the fundamental diagram that the image interlacing is decomposed with method of the present invention;

Fig. 4 is the high-definition picture of method of the present invention to image sequence restoration and reconstruction shown in Figure 1.

Embodiment

When motion is because the mass motion of the image that the factors such as shake of camera cause, promptly when only have a kind of motion in the image, new sequence { S _iIn the motion of scene be consistent.In this case, from cause the method for rebuilding super resolution image the image that degrades because of partiting row sampling, only need to carry out the super-resolution rebuilding processing in turn and get final product by following three steps:

A, the size that partiting row sampling is obtained are N ₁* N ₂Sequence of low resolution pictures { Y _i| i=0 ..., the odd and even number of every two field picture is capable among the P} proposes respectively, and the composition size is N respectively ₁* N ₂/ 2 odd-numbered line image and even number line image;

B, the size that step a is obtained are N ₁* N ₂/ 2 odd-numbered line image and even number line image are formed new image sequence { S _i| i=0 ..., 2P};

C, to new sequence { S _i| i=0 ..., 2P} carries out SUPERRESOLUTION PROCESSING FOR ACOUSTIC and obtains the high-resolution result images.

As new image sequence { S _iIn exist the motion of scenery, except above-mentioned three steps, after executing the b step, the image sequence that step b obtains should be carried out motion detection and cut apart, { S _iIn scene be divided into some consistent zones of moving; By above-mentioned steps c, SUPERRESOLUTION PROCESSING FOR ACOUSTIC is carried out in each consistent moving region; Each high-resolution region image that will obtain at last carries out amalgamation, returns to the high-definition picture consistent with original scene.

Be at low resolution (N below with the inventive method from the partiting row sampling imaging ₁* N ₂) image sequence { Y _iCarry out the detailed step of rebuilding super resolution image:

At first each image is carried out the interlacing decomposition of single image.With image Y _iOdd-numbered line to take out and form a height in order be half (N of original size ₁* N ₂/ 2) image is designated as Y _OiEqually, with image Y _iEven number line to take out and form a height in order be half (N of original size ₁* N ₂/ 2) image is designated as Y _EiNote image sequence { Y _OiY _Ei/ i=0 .., P} are { S _i/ i=0 .., 2P}, { S _iAs new image sequence.Follow-up processing all is at { S _iCarry out.According to the principle of partiting row sampling, at { S _iIn do not had the partiting row sampling grid.As shown in Figure 4.

Follow new sequence { S to obtaining _iCarry out motion detection and cut apart (when motion is because the mass motion of the image that the factors such as shake of camera cause, promptly when only having a kind of motion in the image, this step can be saved).

If new sequence { S _iIn exist the motion of scenery, in the sampling interval of continuous several two field pictures, the image-region of the scenery of each self-movement can be approximated to be has the consistency motion, the motion of different self-movement scenery there are differences.Follow-up super-resolution rebuilding is to carry out at the zone with consistent affine transformation motion, therefore, need cut apart the scenery (image-region) of self-movement, so that the follow-up SUPERRESOLUTION PROCESSING FOR ACOUSTIC of carrying out.Motion detection is a lot of in the prior art with the method for cutting apart, and for example, can use frame-to-frame differences motion detection and dividing method.

Motion segmentation is divided into some consistent zone { R that move with whole scene _k| k=0 ..., K}.Super-resolution rebuilding carries out each consistent zone of moving.As processing region R _kThe time, the new sequence { S that interlacing is decomposed at first _iEvery two field picture in R _kCorresponding zone keeps, and other parts unification in the image is changed to 0.Like this, obtain an inclusion region R _kSequence { SR _k| k=0 ..., 2P}.{ SR with each zone _iCarry out the high-resolution high-resolution result images that SUPERRESOLUTION PROCESSING FOR ACOUSTIC obtains this zone, will obtain the high-resolution result images of whole scene after these image combinations.

The super-resolution technique that the present invention uses is a prior art, and the present invention comprises two basic steps for SUPERRESOLUTION PROCESSING FOR ACOUSTIC:

At first need image sequence is carried out estimation accurately.The basis of image sequence estimation is the estimation between two two field pictures.Present embodiment adopts the method for estimating of single order Taylor expansion.

Hypothetical reference image f ₁(x is y) with image f _k(x, y) relation table between is shown:

[formula 1]

f _k(x，y)＝f ₁(x?cos(θ _k)-y?sin(θ _k)+h _k，x?sin(θ _k)+y?cos(θ _k)+v _k)

Motion hour is carried out Taylor expansion, and is similar to when between two two field pictures.Obtain

[formula 2]

f _k(x，y)＝f ₁(x，y)+(h _k-yθ _k)g _x(x，y)+(v _k+xθ _k)g _y(x，y)

G in the formula _x(x, y), g _y(x, y) this f ₁(x, partial differential y).

Use least square method can find the solution above equation and obtain translational movement h, v and anglec of rotation θ, following formula can only be found the solution when side-play amount and the anglec of rotation are less, for the bigger side-play amount and the anglec of rotation, can find the solution by the method for iteration.

After having obtained the accurate movement estimation, just can carry out super-resolution rebuilding.The high-definition picture method for reconstructing has developed multiple.The common trait of these methods is to make the minimized high-definition picture of error function by the optimal method approximate solution.These methods all are used in the present invention, for example, and following target function:

[formula 3]

$E\left(z\right)=\underset{m=1}{\overset{\mathrm{PM}}{\mathrm{\Σ}}}{(f_m-\underset{r=1}{\overset{N}{\mathrm{\Σ}}}{W}_{m,r}{Z}_r)}^2+\mathrm{\λ}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\left(\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_{\mathrm{ij}}{Z}_j\right)}^2$

F wherein _iBe I pixel in the image sequence in the sequence; PM is the total pixel number of image sequence; W _{M, r}R the pixel that is high-definition picture Z is to f _mContribution, obtain by estimation; N is the total pixel number among the Z; λ is the experience constant; Parameter alpha is a regularization parameter, is defined as follows:

[formula 4]

Use the gradient descent method, can find the solution and make the high-definition picture Z of target function minimum.

The x direction multiplication factor N of common SUPERRESOLUTION PROCESSING FOR ACOUSTIC middle high-resolution image ₁Multiplication factor N with the y direction ₂Be consistent.Used N among the present invention ₂=2 * N ₁, like this image that guarantees to recover and original image are had consistent and wide ratio.

Referring to Fig. 4, this is the image that image shown in Figure 1 obtains after the inventive method is handled, and has eliminated the sampling grid, returns to the high-definition picture consistent with original scene.

The present invention is with regard to the related technology of each step, all be to accomplish with the general knowledge in present technique field, but these steps combine and form a kind of the recovery and the technical method of rebuilding super resolution image from the low-resolution images that has partiting row sampling grid effect that the image capture device of partiting row sampling obtains, really significant contribution has been made in this area, it can guarantee that the partiting row sampling image finally eliminates " partiting row sampling grid ", and returns to the high-definition picture consistent with original scene.

Claims (1)

1. the method for a rebuilding super resolution image from the image that causes to degrade because of partiting row sampling may further comprise the steps:

A, the size that partiting row sampling is obtained are N ₁* N ₂Sequence of low resolution pictures { Y _i| i=0 ..., the odd and even number of every two field picture is capable among the P} proposes respectively, and the composition size is N respectively ₁* N ₂/ 2 odd-numbered line image and even number line image;

B, the size that step a is obtained are N ₁* N ₂/ 2 odd-numbered line image and even number line image are formed new image sequence { S _i| i=0 ..., 2P};

C, to new sequence { S _i| i=0 ..., 2P} carries out SUPERRESOLUTION PROCESSING FOR ACOUSTIC and obtains the high-resolution result images;

D, the image sequence that described step b is obtained carry out motion detection and cut apart, with { S _iIn scene be divided into some consistent zones of moving;

E, by described step c, SUPERRESOLUTION PROCESSING FOR ACOUSTIC is carried out in each consistent moving region; Each high-resolution region image that will obtain at last carries out amalgamation, returns to the high-definition picture consistent with original scene.

Images