CN104063854B - Refraction and reflection omnidirectional image restoration method with mirror surface parameter constraining blurring kernels - Google Patents

Abstract

The invention discloses a refraction and reflection omnidirectional image restoration method with mirror surface parameter constraining blurring kernels. Under the framework of the maximum posterior probability image restoration theory, according to the characteristics of a refraction and reflection imaging system, blurring kernel constraints, namely the rotating symmetry constraint and the spatial transformation constraint, for a refraction and reflection imaging model are provided. According to the refraction and reflection omnidirectional image restoration method, the good omnidirectional image restoration effect is obtained through new blurring kernel constraint terms.

Description

A kind of refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core

Technical field

The present invention relates to digital image processing field, particularly a kind of refractive-reflective all image recovery method.

Background technology

Due to catadioptric Imaging Small and integrated feature, panorama application based on refractive-reflective all imaging is also got more and more, typical application relates to moonfall detection, intelligent panoramic video monitoring, Visual Navigation of Mobile Robots, Remote Video Conference, dummy scene roaming etc., as shown in Figure 1.But the underlying issues such as along with going deep into of application, the intrinsic spatial resolution of refractive-reflective all imaging is low, resolution is uneven, image blur are more and more outstanding, particularly defocusing blurring problem, as shown in Figure 2.

At present, the method for maximum a posteriori probability image restoration is most widely used method in image restoration field.The method based on probability statistics, according to maximum a posteriori probability distribution find best can convergence scheme, obtain an optimum solution, the picture rich in detail namely restored.Under the framework of described maximum a posteriori probability image restoration theory, the statistical models of image restoration problem can be expressed as:

$\arg \max p(k,l|b)=\arg \max \frac{p(b,l|k)p(l,k)}{p\left(b\right)}---\left(1\right)$

Wherein, b is blurred picture, and l is picture rich in detail, and k is fuzzy core.Suppose between l, b, k separate here, and consider that all information of blurred picture are all known, so have

p(k,l|b)∝p(b|l,k)p(l)p(k) (2)

In formula (2), the item on the left side is desired Posterior estimator.Negative logarithm is got to formula (2) the right and left simultaneously, then can obtain

-logp(k,l|b)＝-logp(b|k,l)-logp(l)-logp(k) (3)

Formula (3), by getting negative logarithm, can be converted into following based on maximum a posteriori probability image restoration theory function:

f(l)＝||k*l-b|| ²+ρ _l(l)+ρ _k(k) (4)

Wherein, Section 1 represents the constraint to picture noise, and Section 2 and Section 3 represent priori and the constraint condition of natural image l and fuzzy core k respectively, also referred to as regular terms.

Compared with normal image deblurring, the bottleneck problem that catadioptric defocusing blurring problem does not obtain solving very well is estimation and the analysis of omni-directional image fuzzy core, the Section 3 namely in formula (4).The whether accurate of fuzzy core estimation determines image restoration effect quality.The defocusing blurring of common imaging system is only relevant with scene depth with aperture size, and mirror-lens system is by curved reflector imaging, and the estimation of fuzzy core also needs integrated consideration reflecting surface equation and angle of incident light.Therefore traditional image restoration technology is difficult to analyze accurately omni-directional image defocusing blurring and describe, and causes the deviation that fuzzy core is estimated, thus limits the effect of image restoration.

Summary of the invention

Technical matters to be solved by this invention is, according to the minute surface feature of catadioptric imaging system, under the framework of maximum a posteriori probability image restoration theory, propose a kind of refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core, the recovery effect of omni-directional image is improved.

For realizing above-mentioned technique effect, the technical solution used in the present invention is, a kind of refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core, comprise the steps: the omni-directional image b (x that first doubling reflection imaging system collects, y) carry out point ring and point block operations, then draw the horizontal direction Grad of each image block with vertical direction Grad and then draw the prior imformation ρ of place image block _ll (), by described prior imformation ρ _ll () and fuzzy core retrain ρ _k(l) substitute in following formula based in maximum a posteriori probability image restoration theory function, draw picture rich in detail block l:

f(l)＝||k*l-b|| ²+ρ _l(l)+ρ _k(l)

Wherein l is the picture rich in detail block needing to solve, and b is the blurred picture block that imaging system obtains, and k is fuzzy core, finally the picture rich in detail block of trying to achieve is carried out splicing and merges, restore and obtain final picture rich in detail, the invention is characterized in, described fuzzy core constraint ρ _kl () comprises rotational symmetry constraint and spatial alternation constraint, i.e. ρ _k(l)=ρ ₁(l) ρ ₂(l), rotational symmetry constraint ρ ₁l ()=φ, wherein φ represents position angle, and l represents image block, and spatial alternation retrains wherein l represents image block, scale parameter σ=f (S of defocusing blurring core _d)=1/S _d, wherein S _dfor the corresponding degree of depth of object point.

Preferred version according to embodiment, the prior imformation ρ of described image block _l(l) be: ${\mathrm{\ρ}}_l\left(l\right)={\left|\right|{\▿}_{x}b\left|\right|}^{0.8}+{\left|\right|{\▿}_{y}b\left|\right|}^{0.8}.$

Preferred version according to embodiment, the method for solving of picture rich in detail block l is: differentiate based on l in maximum a posteriori probability image restoration theory function formula to described, and make it equal 0, i.e. df (l)/dl=0, draws picture rich in detail block l.

As preferred embodiment, the present invention adopts the fusion method average to overlapping sub-region right to carry out image mosaic fusion, finally tries to achieve picture rich in detail.

Below with reference to principle, detailed process of the present invention is described:

The refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core of the present invention, comprises the steps:

A refractive-reflective all image recovery method for minute surface restriction on the parameters fuzzy core, comprises the steps:

1) the omni-directional image b (x, y) that doubling reflection imaging system collects carries out point ring and point block operations, and wherein x represents omni-directional image horizontal coordinate, and y represents the vertical coordinate of omni-directional image.

The detailed process of described point of ring operation is: with the initial point of omni-directional image, i.e. the center of omni-directional image, centered by, radius is r _kbe r to radius _kthe region of+Δ r is a kth ring, and the width Delta r of each ring is all equal, wherein r _krepresent the inner ring radius of a kth ring;

The detailed process of described point of block operations is: to cross the horizontal line at omni-directional image center for datum line, with the center of omni-directional image for starting point makes ray, ray is rotated counterclockwise, and every fixing radian θ, carries out piecemeal;

2) again to dividing the omni-directional image b (x, y) of ring piecemeal to ask horizontal direction Grad with vertical direction Grad wherein represent that image asks partial derivative about horizontal direction, represent that image asks partial derivative about vertical direction.In actual computation process, the Grad computing method of pixel (x, y) horizontal direction are: the i.e. difference of adjacent two pixel point values.In like manner, the Grad computing method of vertical direction are: the i.e. difference of adjacent two pixel point values;

3) in conjunction with the horizontal direction Grad obtained in above-mentioned steps and vertical gradient value, the prior imformation ρ of computed image block _l(l).The computing method of above-mentioned image prior have a variety of, as the sparse prior computing method that the people such as Levin propose, at list of references: A.Levin, R.Fergus, F.Durand, and W.Freeman.Image and depth from a conventional camera with acoded aperture.SIGGRAPH, associated description in 2007:1-8..Also has the total variation method that the people such as Chan propose, at list of references: T.F.Chan and C.K.Wong.Total variationblind deconvolution.IEEE Transaction on Image Processing, Volume.7, Issue.3, pp.370-375,1998. have associated description.

4) in conjunction with the minute surface parameter of catadioptric imaging system, the fuzzy core constraint ρ of each image block is calculated _kl (), described fuzzy core constraint comprises rotational symmetry constraint and spatial alternation constraint, i.e. ρ _k(l)=ρ ₁(l) ρ ₂(l).

Rotational symmetry retrains: the fuzzy core shape on same annulus is the same, but due to the point-symmetry property of catadioptric imaging system, the fuzzy core on same annulus is about omni-directional image Central Symmetry, and the computing method of rotational symmetry constraint are: ρ ₁l ()=φ, wherein φ represents position angle, and l represents image block.

Spatial alternation retrains: in omni-directional image, the fuzzy core of each position is not identical.With image block be base unit to study the change of defocusing blurring core, spatial variability has 2 rules: (1) i-th image block P _icorresponding defocusing blurring core P _ithe fog-level of k is relevant with the distance d at distance omni-directional image center; (2) fog-level of defocusing blurring core increases gradually with the increase of d, and change size is correlated with the minute surface parameter of catadioptric imaging system.

The computing method of spatial alternation constraint: wherein l represents image block, and σ represents the scale parameter of defocusing blurring core, σ=f (S _d)=1/S _d, wherein S _dfor the corresponding degree of depth of object point.

5) the image prior constraint ρ will obtained _ll () and fuzzy core retrain ρ _k(l) substitute in following formula based in maximum a posteriori probability image restoration theory function, draw picture rich in detail block l:

f(l)＝||k*l-b|| ²+ρ _l(l)+ρ _k(l)

Wherein l is the picture rich in detail block needing to solve, and b is the blurred picture block that imaging system obtains, and k is fuzzy core, ρ _ll () is image prior constraint, ρ _kl () is fuzzy core constraint;

6) the picture rich in detail block of trying to achieve is carried out splicing to merge, restore and obtain complete picture rich in detail.

Compared with going defocusing blurring technology with existing catadioptric image, the beneficial effect that the present invention has is:

The present invention is under the theoretical frame of maximum a posteriori probability image restoration, have employed especially for the constraint of catadioptric imaging system feature design fuzzy core, namely comprise rotational symmetry constraint and spatial alternation constraint, carry out image restoration, improve omni-directional image deblurring effect.

Accompanying drawing explanation

Fig. 1 is the main application fields of refractive-reflective all imaging system;

Fig. 2 is the fuzzy schematic diagram of refractive-reflective all image defocus, and focal plane is arranged on inner ring, inner ring picture rich in detail, and outer shroud is fuzzy;

Fig. 3 is that omni-directional image divides ring and piecemeal schematic diagram;

Fig. 4 is omni-directional image fuzzy core rotational symmetry schematic diagram;

Fig. 5 is the process flow diagram of the method for the invention;

Fig. 6 is horizontal catadioptric imaging system Experimental equipment;

Fig. 7 is the omni-directional image of the defocusing blurring gathered;

Omni-directional image after Fig. 8 is restored by the method for the invention.

Embodiment:

As shown in Figure 5, the present embodiment is by catadioptric imaging system acquires view data as shown in Figure 6.In imaging system, camera is Canon5D Mark II, camera lens is Canon's camera lens of 50mm F/1.8, aperture is set to F/1.8, and gather the omni-directional image of 7,500,000 pixels, focal point settings is in the front end of catadioptric minute surface, catadioptric minute surface is hyperboloid, parameter is a=22.3mm, b=22.3mm, c=31.5mm.The omni-directional image of the defocusing blurring collected as shown in Figure 7.

As shown in Figure 3, point ring and a point block operations are carried out to the omni-directional image gathered.For a point ring operation, its detailed process is: centered by image origin, and radius is r _kbe r to radius _kthe region of+Δ r is a kth ring, and the width of each ring is all equal.Wherein r _krepresent the inner ring radius of a kth ring, the width of Δ r representative ring, the sequence number of k representative ring.For a point block operations, its detailed process is: to cross the horizontal line of image center for datum line, be that starting point makes ray with picture centre, ray is rotated counterclockwise, and every fixing radian θ, carries out piecemeal.

Omni-directional image is specifically divided into 8 rings by the present embodiment, the width of ring, i.e. the value of Δ r, determines according to image size, and the present embodiment value is 1 centimetre.After dividing loops bundle, piecemeal is carried out to each ring: to cross the horizontal line at omni-directional image center for datum line, with the center of omni-directional image for starting point makes ray, ray is rotated counterclockwise, and every fixing radian 45 degree, carries out piecemeal, is divided into 8 blocks by each ring-type image.

Again to dividing omni-directional image block b (x, y) of ring piecemeal to obtain horizontal gradient value with vertical gradient value wherein represent that image asks partial derivative about horizontal direction, represent that image asks partial derivative about vertical direction.In actual computation process, the Grad computing method of pixel (x, y) horizontal direction are: the i.e. difference of adjacent two pixel point values.In like manner, the Grad computing method of vertical direction are: the i.e. difference of adjacent two pixel point values;

In conjunction with the above-mentioned Grad drawn with calculate image prior constraint ρ _l(l), ${\mathrm{\ρ}}_l\left(l\right)={\left|\right|{\▿}_{x}b\left|\right|}^{0.8}+{\left|\right|{\▿}_{y}b\left|\right|}^{0.8}.$

Then, Binding experiment platform parameters and minute surface parameter, calculate fuzzy core constraint ρ _kl (), this step is also the emphasis of the present embodiment, described fuzzy core constraint ρ _kl () comprises rotational symmetry constraint and spatial alternation retrains two aspects.

Rotational symmetry retrains: the fuzzy core shape on same annulus is the same, but due to the point-symmetry property of catadioptric imaging system, the fuzzy core on same annulus about omni-directional image Central Symmetry, as shown in Figure 4.Rotational symmetry constraint ρ ₁l ()=φ, wherein φ represents position angle.

Spatial alternation retrains: in omni-directional image, the fuzzy core of each position is not identical.With image block be base unit to study the change of defocusing blurring core, as shown in Figure 3.Spatial variability has 2 rules: (1) i-th image block P _icorresponding defocusing blurring core P _ithe fog-level of k is relevant with the distance d at distance omni-directional image center; (2) fog-level of defocusing blurring core increases gradually with the increase of d, and change size is correlated with the minute surface parameter of catadioptric imaging system.Spatial alternation retrains wherein l represents image block, and σ represents the scale parameter of defocusing blurring core, σ=f (S _d)=1/S _d, wherein S _dfor the corresponding degree of depth of this image block.

To sum up, fuzzy core constraint ${\mathrm{\ρ}}_k\left(l\right)={\mathrm{\ρ}}_1\left(l\right)\·{\mathrm{\ρ}}_2\left(l\right)=\frac{\mathrm{\φ}}{\mathrm{\σ}\sqrt{2\mathrm{\π}}}\exp \left({\frac{\left|\right|\▿l\left|\right|}{2{\mathrm{\σ}}^2}}^2\right),$ Wherein σ=f (S _d)=1/S _d, S _dfor the corresponding degree of depth of this image block, determined by minute surface parameter and image block present position.Wherein φ is fuzzy core position angle, is determined by image block present position, and value is between 0-90 degree.

By the image prior constraint ρ calculated _ll () and fuzzy core retrain ρ _kl () is updated to based in maximum a posteriori probability image restoration theoretical frame:

f(l)＝||k*l-b|| ²+ρ _l(l)+ρ _k(l)

Wherein l is the picture rich in detail block needing to solve, and b is the blurred picture block that imaging system obtains, and k is fuzzy core, ρ _ll () is image prior constraint, ρ _kl () is fuzzy core constraint.

Above-mentioned majorized function being differentiated about l, and makes it equal 0, be i.e. df (l)/dl=0, by solving above-mentioned equation, finally trying to achieve picture rich in detail block l.

Finally adopt the average (list of references: Szeliski and Richard, Imagealignment and stitching:A tutorial.Foundations and of overlapping sub-region right inComputer Graphics and Vision, 2006, vol.2:1-104) method the image block that all recoveries obtain is spliced, be fused into complete omni-directional image clearly, as shown in Figure 8, substantially increase the recovery effect of omni-directional image.

Claims (3)

1. the refractive-reflective all image recovery method of a minute surface restriction on the parameters fuzzy core, comprise the steps: the omni-directional image b (x that first doubling reflection imaging system collects, y) carry out point ring and point block operations, then calculate the horizontal direction Grad of each image block after point ring piecemeal with vertical direction Grad and then draw the prior imformation ρ of place image block _ll (), by described prior imformation ρ _ll () and fuzzy core retrain ρ _k(l) substitute in following formula based in maximum a posteriori probability image restoration theory function, draw picture rich in detail block l:

f(l)＝||k*l-b|| ²+ρ _l(l)+ρ _k(l)

Wherein l is the picture rich in detail block needing to solve, and b is the blurred picture block that imaging system obtains, and k is fuzzy core, finally the picture rich in detail block of trying to achieve is carried out splicing and merges, restore and obtain final picture rich in detail, it is characterized in that, described fuzzy core constraint ρ _kl () comprises rotational symmetry constraint and spatial alternation constraint, i.e. ρ _k(l)=ρ ₁(l) ρ ₂(l), rotational symmetry constraint ρ ₁(l)=φ, wherein φ represents position angle, and l represents picture rich in detail block; Spatial alternation retrains wherein l represents picture rich in detail block, scale parameter σ=f (S of defocusing blurring core _d)=1/S _d, wherein S _dfor the corresponding degree of depth of object point.

2. the refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core according to claim 1, is characterized in that, the prior imformation ρ of described image block _l(l) be: wherein with be respectively horizontal gradient value and the vertical gradient value of described image block.

3. the refractive-reflective all image recovery method of minute surface restriction on the parameters fuzzy core according to claim 1 or 2, it is characterized in that, the method for solving of picture rich in detail block l is: differentiate based on l in maximum a posteriori probability image restoration theory function formula to described, and make it equal 0, i.e. df (l)/dl=0, draws picture rich in detail block l.