CN102324104B - Space structure modeling method and system based on single-image defocusing information - Google Patents

Abstract

The invention discloses a space structure modeling method and system based on single-image defocusing information. The method comprises the steps of: 1, partitioning a single image; and 2, determining a clear image block through calculation, assigning a value to the clear image block, the value is the distance u from a scene to a lens, f is a focus, sigmate is the diameter of a blur circle, N is equal to f/D, D is the diameter of aperture, and u0 is a distance from a focusing plane to the lens. The invention can be used for establishing a space structure model of the scene by only needing one image with small focus depth, has a wide application range, and can realize real-time calculation because a linear operation is mainly adopted.

Description

A kind of space structure modeling method and system based on the single image defocus information

Technical field

The present invention relates to computer vision and image processing field, relate in particular to a kind of space structure modeling method and system based on the single image defocus information.

Background technology

The space structure of asking for scene according to the defocus information of image is a kind of important method in the monocular stereo technology, is proposed in 1987 by Pentland the earliest.Imaging system needs to increase aperture in order to increase light-inletting quantity, causes away from the object image-forming of focussing plane fuzzy.For diffracting effect is not clearly imaging system, and the point spread function of imaging system forms the circle of confusion.The diameter of the circle of confusion that produces according to pointolite and the parameter of imaging system can calculate object distance.Generally speaking, foreground object is positioned on the focussing plane.The diameter σ of the circle of confusion and the relation derivation of object distance u are as follows:

$\frac{\mathrm{\σ}}{D}=\frac{v_0-v}{v}---\left(1\right)$

Wherein v is an image distance, v ₀Be the distance of camera lens to sensor plane.According to the lens imaging formula

(f is a focal length) and (1) obtain

$u=\frac{{\mathrm{fv}}_0}{v_0-f-\mathrm{\σN}},$ $N=\frac{f}{D}.---\left(2\right)$

Wherein D is a diaphragm diameter, and N is the f number.The f of imaging system and N can know, but v ₀The unknown, therefore it is changed to camera lens to focussing plane apart from u ₀:

$u=\frac{f^2{u}_0}{(f+\mathrm{\σN})f-{\mathrm{\σNu}}_0}.---\left(3\right)$

The validity of asking for the space structure of scene according to the defocus information of image has obtained proof in theory.Out-of-focus image calculating scene depth by the large aperture lens shooting is of equal value with calculating scene depth with binocular camera, and the diameter of monocular camera aperture is equivalent to the base length of binocular camera.Difficult point is how to calculate the fog-level of picture point.If jump in brightness point position in the known scene, the second derivative that can pass through brightness of image is estimated the diameter of the circle of confusion.But generally the information of scene is unknown, and the multiple image of therefore general same viewpoint of employing and brightness, different focal compares.At present the problem of asking for the space structure of scene according to the defocus information of image has been defaulted as two width of cloth by same viewpoint with the epigraph restoration scenario degree of depth.This conditional request scene must be static, has therefore limited this The Application of Technology prospect.

Summary of the invention

At the above-mentioned problems in the prior art, the invention provides a kind of space structure modeling method and system based on the single image defocus information.

The invention provides a kind of space structure modeling method, comprising based on the single image defocus information:

Step 1 is carried out piecemeal to single image;

Step 2 is determined the distinct image piece by calculating, and to this distinct image piece assignment; This value for scene arrive camera lens apart from u,

F is a focal length, and σ is the diameter of the circle of confusion, and N=f/D, D are diaphragm diameter, u ₀Be the distance of focussing plane to camera lens.

In one example, step 2 comprises:

Step 20, the diameter of set point spread function;

Step 21 for the point spread function of setting diameter, is calculated the rectangular projection image that the image block of choosing arrives the space of its generation;

Step 22 is subtracted each other rectangular projection image and selected image block, if difference greater than predetermined value, then this selected image block is the distinct image piece;

Step 23 is to distinct image piece assignment.

In one example, in the step 1, single image is carried out piecemeal according to the principle that attribute is similar.

In one example, in the step 20, the diameter of point spread function is n, point spread function H _n(x, y)=φ _n(x) φ _n(y), function phi _n(t) obtain φ by generating function φ (t) convergent-divergent _n(t)=φ (t/n); Generating function φ (t) is Gaussian function or Battle-Lemari é wavelet function or Airy Disk function.

In one example, in the step 21, the rectangular projection image

I is selected image block,

Be analysis filter, h is a reconstruction filter.

In one example, h=h (p) h (q),

P, q are natural number;

H (p) and h (q) by

Obtain through Fourier inversion;

With

By $\hat{\tilde{h}}\left(\mathrm{\ω}\right)=\frac{{\mathrm{\Σ}}_{k\∈Z}\widehat{\mathrm{\φ}}(\mathrm{\ω}+2\mathrm{k\π}){\widehat{\mathrm{\φ}}}^{*}(\mathrm{n\ω}+2\mathrm{k\π})}{n{\mathrm{\Σ}}_{k\∈Z}{\left|\widehat{\mathrm{\φ}}(\mathrm{n\ω}+2\mathrm{k\π})\right|}^2}$ Obtain through Fourier inversion.

The invention provides a kind of space structure modeling, comprising based on the single image defocus information:

Divide module unit, be used for single image is carried out piecemeal;

The assignment unit is used for determining the distinct image piece by calculating, and to this distinct image piece assignment; This value for scene arrive camera lens apart from u,

F is a focal length, and σ is the diameter of the circle of confusion, and N=f/D, D are diaphragm diameter, u ₀Distance for focussing plane.

In one example, the assignment unit is used for the diameter of set point spread function; For the point spread function of setting diameter, calculate the rectangular projection image that the image block of choosing arrives the space of its generation; Rectangular projection image and selected image block are subtracted each other, if difference greater than predetermined value, then this selected image block is the distinct image piece.

In one example, divide module unit, be used for single image being carried out piecemeal according to the similar principle of attribute.

In one example, the diameter of point spread function is n, point spread function H _n(x, y)=φ _n(x) φ _n(y), function phi _n(t) obtain φ by generating function φ (t) convergent-divergent _n(t)=φ (t/n); Generating function φ (t) is Gaussian function or Battle-Lemari é wavelet function or Airy Disk function.

In one example, rectangular projection image

I is selected image block,

Be analysis filter, h is a reconstruction filter.

In one example, h=h (p) h (q),

P, q are natural number;

H (p) and h (q) by

Obtain through Fourier inversion;

With

By $\hat{\tilde{h}}\left(\mathrm{\ω}\right)=\frac{{\mathrm{\Σ}}_{k\∈Z}\widehat{\mathrm{\φ}}(\mathrm{\ω}+2\mathrm{k\π}){\widehat{\mathrm{\φ}}}^{*}(\mathrm{n\ω}+2\mathrm{k\π})}{n{\mathrm{\Σ}}_{k\∈Z}{\left|\widehat{\mathrm{\φ}}(\mathrm{n\ω}+2\mathrm{k\π})\right|}^2}$ Obtain through Fourier inversion.

The present invention only needs the image of the little depth of field of a width of cloth just can set up the space-filling model of scene, applied range, and mainly adopt linear operation, thereby can realize real-time calculating.

Description of drawings

Come the present invention is described in further detail below in conjunction with accompanying drawing, wherein:

Fig. 1 is the space structure modeling method flow process based on the single image defocus information provided by the invention;

Fig. 2 is the test picture that the present invention uses;

Fig. 3 a-Fig. 3 f is based on the Error Graph at different levels of test picture.

Embodiment

A kind of space structure modeling method based on the single image defocus information that the present invention proposes may further comprise the steps:

At first, according to the similar principle of attribute image is carried out piecemeal;

Secondly, set the ascending variation of diameter of the point spread function of optical imaging system; For the point spread function of given diameter, computed image promptly obtains the image of certain fog-level in the rectangular projection of its span; Subtract each other with original image then, extract the clear part of image, thereby image is divided into step by step from clear to fuzzy zone;

At last, by the calculation of parameter scene of the diameter of point spread function and optical imaging system distance, give the not assignment piecemeal under the clear area to camera lens.

A kind of space structure modeling method detailed process based on the single image defocus information of the present invention may further comprise the steps as shown in Figure 1:

Step 101 is carried out piecemeal according to the principle that attribute is similar to image.Entire image is divided into some difform zonules, and the similar neighbor of feature (as color, texture, histogram etc.) incorporates the same area into.

Step 102, set-point spread function diameter;

Step 103, for the point spread function of given diameter, computed image is in the rectangular projection of its span;

Step 104, projected image and former figure subtract each other, and extract the clear area;

Step 105 according to point spread function diameter and the optical imaging system calculation of parameter scene distance to camera lens, is given the not assignment piecemeal under the clear area;

Step 106 increases the point spread function diameter, execution in step 103, until all image blocks all by assignment.Preferably, after increase point spread function diameter 2-3 time,, then carry out assignment according to predetermined value for these piecemeals if still exist not by the assignment piecemeal.

In the step 102,, calculate the analysis filter and the reconstruction filter of its span according to the point spread function of given diameter.

If diameter is the point spread function H of n _n(x y) can be separated into

H _n(x,y)＝φ _n(x)φ _n(y)。(4)

And function phi _n(t) get by generating function φ (t) convergent-divergent:

φ _n(t)＝φ(t/n)。(5)

Generating function can be measured by experiment, then is Airy Disk function for desirable circular iris.But adopt Gaussian function usually

For convenience of calculation adopts Battle-Lemari é small echo (shown in the following embodiment).

The Fourier transform of analysis filter is:

$\hat{\tilde{h}}\left(\mathrm{\ω}\right)=\frac{{\mathrm{\Σ}}_{k\∈Z}\widehat{\mathrm{\φ}}(\mathrm{\ω}+2\mathrm{k\π}){\widehat{\mathrm{\φ}}}^{*}(\mathrm{n\ω}+2\mathrm{k\π})}{n{\mathrm{\Σ}}_{k\∈Z}{\left|\widehat{\mathrm{\φ}}(\mathrm{n\ω}+2\mathrm{k\π})\right|}^2}.---\left(6\right)$

The Fourier transform of reconstruction filter is:

$\hat{h}\left(\mathrm{\ω}\right)=\frac{\widehat{\mathrm{\φ}}\left(\mathrm{\ω}\right)}{n\widehat{\mathrm{\φ}}\left(\mathrm{n\ω}\right)}.---\left(7\right)$

Doing Fourier inversion can obtain

Corresponding two dimensional filter is

h(p,q)＝h(p)h(q), $\tilde{h}(p,q)=\tilde{h}\left(p\right)\tilde{h}\left(q\right),$ P, q are natural number.(8)

In the step 103, image and analysis filter are carried out convolution, and then carry out convolution, intercepting and the equal-sized center section of original image, the rectangular projection of asking exactly with reconstruction filter.If image array is that (x, y), then projected image is I

$I_n=I*\tilde{h}*h---\left(9\right)$

Error is:

$\mathrm{err}=I-I_n=I-I*\tilde{h}*h---\left(10\right)$

Wherein, analysis filter

Can obtain by formula (6)-(8) with the frequency-domain expression of reconstruction filter h.According to Frame Theory, be the point spread function H of n with diameter _n(x, y) I in Biao Shi the image _nError be minimum.

Error is greater than preset value (be generally clear value 20%), and then regarding as this part is distinct image.

3) by the calculation of parameter scene of the diameter of point spread function and the optical imaging system distance to camera lens, computing formula is as follows, gives the not assignment piecemeal under the clear area then.

$u=\frac{f^2{u}_0}{(f+\mathrm{\σN})f-{\mathrm{\σNu}}_0}.---\left(11\right)$

Embodiment of the present invention adopts one shown in Figure 2 1310 * 874 image that the space structure modeling method that the present invention is based on the single image defocus information is verified:

1) according to the similar principle of attribute image is carried out piecemeal.Entire image is divided into some difform zonules, and the similar neighbor of feature (as color, texture, histogram etc.) incorporates the same area into.

2) diameter of setting the point spread function of optical imaging system increases progressively with 2 power, and promptly 2,4,8,16 ...Adopt separable point spread function H _n(x, y)=φ _n(x) φ _n(y), wherein

$\widehat{\mathrm{\φ}}\left(\mathrm{\ω}\right)=\frac{{\hat{B}}_m\left(\mathrm{\ω}\right)}{{\mathrm{\Σ}}_{k\∈Z}{\left|{\hat{B}}_m(\mathrm{\ω}+2\mathrm{k\π})\right|}^{1/2}}.---\left(12\right)$

B wherein _mBe m rank B-spline function.This makes

Choose m=1, corresponding filter coefficient is

Because the diameter of point spread function increases progressively with 2 power, so corresponding wave filter can obtain with the method for inserting 0.Note h _k(n) insert 2 between per two points of wave filter h (n) ^k-10 wave filter that obtains.Image decomposed step by step obtain

$a_k=a_{k-1}*{\tilde{h}}_k---\left(13\right)$

A wherein ₀=I.Rebuild the rectangular projection image:

$I_{2^k}=a_k*h_k---\left(14\right)$

Error Graph at different levels are shown in Fig. 3 a-Fig. 3 f.

3), give the not assignment piecemeal under the clear area then by the calculation of parameter scene of the diameter of point spread function and optical imaging system distance to camera lens.

The above only is a preferred implementation of the present invention, but protection domain of the present invention is not limited thereto.Any those skilled in the art all can carry out suitable change or variation to it in technical scope disclosed by the invention, and this change or variation all should be encompassed within protection scope of the present invention.

Claims (8)

1. space structure modeling method based on the single image defocus information is applied to the monocular stereo technology of computer vision and image processing field, it is characterized in that, comprising:

Step 1 is carried out piecemeal to single image;

Step 2 is determined the distinct image piece by calculating, and to this distinct image piece assignment; This value for scene arrive camera lens apart from u,

F is a focal length, and σ is the diameter of the circle of confusion, and N=f/D, D are diaphragm diameter, u ₀Be the distance of focussing plane to camera lens,

Wherein, step 2 comprises:

Step 20, the diameter of set point spread function;

Step 21 for the point spread function of setting diameter, is calculated the rectangular projection image that the image block of choosing arrives the space of its generation;

Step 22 is subtracted each other rectangular projection image and selected image block, if difference greater than predetermined value, then this selected image block is the distinct image piece;

Step 23 is to distinct image piece assignment.

2. space structure modeling method as claimed in claim 1 is characterized in that, in the step 1, according to the principle that attribute is similar single image is carried out piecemeal.

3. space structure modeling method as claimed in claim 1 is characterized in that, in the step 20, the diameter of point spread function is n, point spread function H _n(x, y)=φ _n(x) φ _n(y), function phi _n(t) obtain φ by generating function φ (t) convergent-divergent _n(t)=φ (t/n); Generating function φ (t) is Gaussian function or Battle-Lemari é wavelet function or Airy Disk function.

4. space structure modeling method as claimed in claim 1 is characterized in that, in the step 21, and the rectangular projection image

I is selected image block,

Be analysis filter, h is a reconstruction filter.

5. space structure modeling based on the single image defocus information is applied to the monocular stereo technology of computer vision and image processing field, it is characterized in that, comprising:

Divide module unit, be used for single image is carried out piecemeal;

The assignment unit is used for determining the distinct image piece by calculating, and to this distinct image piece assignment; This value for scene arrive camera lens apart from u, F is a focal length, and σ is the diameter of the circle of confusion, and N=f/D, D are diaphragm diameter, u ₀Be the distance of focussing plane to camera lens,

Wherein, the assignment unit is used for the diameter of set point spread function; For the point spread function of setting diameter, calculate the rectangular projection image that the image block of choosing arrives the space of its generation; Rectangular projection image and selected image block are subtracted each other, if difference greater than predetermined value, then this selected image block is the distinct image piece.

6. space structure modeling as claimed in claim 5 is characterized in that, divides module unit, is used for according to the similar principle of attribute single image being carried out piecemeal.

7. space structure modeling as claimed in claim 5 is characterized in that, the diameter of point spread function is n, point spread function H _n(x, y)=φ _n(x) φ _n(y), function phi _n(t) obtain φ by generating function φ (t) convergent-divergent _n(t)=φ (t/n); Generating function φ (t) is Gaussian function or Battle-Lemari é wavelet function or AiryDisk function.

8. space structure modeling as claimed in claim 5 is characterized in that, the rectangular projection image

I is selected image block,

Be analysis filter, h is a reconstruction filter.

Images