CN101354790A - Omnidirectional camera N surface perspective panorama expanding method based on Taylor series model - Google Patents

Abstract

The invention discloses a method for expanding N-surface perspective panoramic images of an omni-directional camera, which is based on Taylor series models. The method comprises the following steps that: the omni-directional camera is calibrated by use of a Taylor series model, so as to obtain internal parameters of the camera; the size of an N-surface perspective panoramic image is estimated; and re-projection and interpolation are carried out so as to obtain the N-surface perspective panoramic image. The method is applicable to various omni-directional vision sensors, has the characteristic of wide application range, and can obtain parameters through calibration under the condition that the parameters of the omni-directional vision sensors are unknown and carry out effective panoramic image expansion. Every surface in the generated N-surface perspective panoramic image is equivalent to an image generated by a common perspective camera, thereby ensuring that the application of the prior algorithm applied to the prior perspective camera can be continued.

Description

Omnidirectional camera N surface perspective panorama expanding method based on Taylor series model

Technical field

The present invention relates to a kind of the annular that is obtained by omnidirectional camera be looked like to carry out the method for N surface perspective panorama expanding, be specifically related to a kind of omnidirectional camera N surface perspective panorama expanding method based on Taylor series model.

Background technology

In recent years, have benefited from the wide visual angle characteristic of omnidirectional camera, it is applied in the growing field, as the robot navigation, and video monitoring etc.Early stage panoramic picture mainly obtains by multiple-camera splicing or the rotation of single camera.But said method all needs correction and registration between the common fluoroscopy images, is not suitable for the real time imagery application scenario.By comparison, 360 ° of environmental informations of the omnidirectional camera disposable acquisition of energy are ideal transducers that vision is used.Omnidirectional camera generally can be made up of general camera and protruding two secondary mirror, needs proper alignment to place to satisfy the single view constraint between the two.Because the rotational symmetry of two secondary mirror, environment is ' compressed ' an annular section that images in general camera all around.This image does not meet perception and the understanding of people to environment, return the application of many classical vision algorithms and make troubles, so generally the re-projection that need be correlated with to it and expansion so that further handle.At present to the expansion of omni-directional image mainly based on two kinds of thinkings: one is based on the Direct Transform between polar coordinate system and the Cartesian coordinates.This method is analyzed annular image under polar coordinate system, the cylindrical panoramic figure that re-projection launches then represents that with flute card system the position angle in the annular image is mapped as the transverse axis of cylindrical panoramic figure, and radial distance then is mapped as Z-axis.This conversion is that simple plane is launched, and by the imaging model of camera, image and three-dimensional mapping relations are not left in the basket.Two re-projections that are based on the accurately image parameter launch.These class methods are before expansion, must carry out the demarcation of imaging modeling and camera parameter to omnidirectional camera, satisfying under the single viewpoint constraint prerequisite, the point in the image can accurately find the corresponding light in the three-dimensional world, and the feasible reconstruction based on cylindrical panoramic figure of definite object-image relation becomes possibility.This parsing expansion mode depends on camera model strongly, and it is very responsive to make that it changes parameter, and when having bigger alignment error between minute surface and the camera, this method will lose efficacy.In addition, under many circumstances, the accurate model parameter of system of omnidirectional is to be difficult to obtain.Thus, the present invention proposes omnidirectional camera N surface perspective panorama expanding method based on Taylor series model.

Summary of the invention

The object of the present invention is to provide a kind of omnidirectional camera N surface perspective panorama expanding method based on Taylor series model, the image on each side of the N face body after the expansion can be regarded as by a virtual perspective camera and produce.Each pixel is all corresponding with a light in the 3d space in the panoramic picture, thereby traditional perspective camera algorithm can both be suitable on the panorama sketch that re-projection launches.

The step of the technical solution used in the present invention comprises:

(1) camera calibration: utilize Taylor series model that omnidirectional camera is demarcated, obtain the camera confidential reference items;

(2) size of estimation N surface perspective panorama;

(3) re-projection and interpolation obtain the N surface perspective panorama.

N in the N surface perspective panorama of described expansion meets following condition: N 〉=3.

It is known three-dimensional point and other corresponding point of utilizing on the plane on image that the described omnidirectional camera that utilizes Taylor series model is demarcated, carry out two step least-squares calculation, the confidential reference items of the rotation on camera and plane and translation, camera have successively been obtained, just the coefficient of Taylor series and order.

The size of described estimation N surface perspective panorama comprises the following step:

(1) height of calculating N surface perspective panorama: H=R (tan θ ₁+ tan θ ₂), wherein R is the projection radius, and is definite apart from the distance of axis of symmetry by the maximum elevation place of catadioptric minute surface, θ ₁And θ ₂Be maximum elevation and maximum depression

(2) face of determining the N surface perspective panorama is counted N, obtains the angle theta between adjacent two virtual perspective cameras ₀=2 π/N;

(3) determine the wide of N surface perspective panorama: W=N * L ₀, L wherein ₀=2Rtan θ ₀

Described re-projection and interpolation obtain the N surface perspective panorama and comprise the following step:

(1), calculates point and the mapping relations between annular picture the on the N surface perspective panorama according to calibrating parameters;

(2) bilinear interpolation obtains the pixel value of picture point on the N surface perspective panorama.

The beneficial effect that the present invention has is:

1. the omnidirectional camera N surface perspective panorama expanding method based on Taylor series model of the present invention's proposition applicable to various omnidirectional vision sensors, has widely applicable characteristics;

2. the omnidirectional camera N surface perspective panorama expanding method based on Taylor series model of the present invention's proposition can obtain its parameter by demarcating, and carry out effective panorama expanding under omnidirectional vision sensor parameters condition of unknown;

3. the omnidirectional camera N surface perspective panorama expanding method that proposes of the present invention based on Taylor series model, each face in the N surface perspective panorama that is generated, all image equivalences that is become with common perspective camera, thus make the algorithm that is applied to tradition perspective camera originally can continue to be used.

Description of drawings

Fig. 1 is the process flow diagram of the inventive method.

Fig. 2 is based on omnidirectional's imaging model of Taylor series.

Fig. 3 is the projection relation between i side of figure of annular omnidirectional and N surface perspective panorama.

Fig. 4 is the projection relation on annular figure of omnidirectional and the N surface perspective panorama horizontal direction.

Fig. 5 is the common perspective camera network with the equivalence of N surface perspective panorama.

Embodiment

Below in conjunction with the drawings and specific embodiments the present invention is described in further detail.

Fig. 1 has provided according to the present invention and has carried out omnidirectional camera N surface perspective panorama expanding method process flow diagram based on Taylor series model.

As shown in Figure 1, in step 101, employed omnidirectional vision sensor is demarcated, the Taylor series omnidirectional imaging model that is adopted can be represented with Fig. 2.Suppose the some P in the 3D world _w=[X _w, Y _w, Z _w] on virtual minute surface, be projected as P _m=[X _m, Y _m, Z _m], the point of the respective sensor and the plane of delineation is respectively p _s=[u _s, v _s] and p _i=[u _i, v _i], have a few all at minute surface coordinate system O _mMiddle expression, then the perfect imaging process is:

$\mathrm{\λ}\·\stackrel{\→}{O_m{P}_m}=\mathrm{\λ}\·g\left(p_s\right)=\mathrm{\λ}\·g({\mathrm{Ap}}_i+t)={\mathrm{RP}}_w+T=M{[P_w,1]}^T$

Wherein, M=[R, T] be projection matrix; A and t are sensor plane O _sWith plane of delineation O _iBetween affined transformation.Function g () is the minute surface equation, has represented from a p _sTo vector

Mapping:

g(p _s)＝(u _s，v _s，f(ρ _s)) ^T＝(u _s，v _s，a _Nρ ^N+a _N-1ρ ^N-1+…+a ₁ρ+a ₀) ^T

${\mathrm{\ρ}}_s=\sqrt{u_s^2+v_s^2}$ Be the radial distance of picture point to projection centre.In conjunction with last two formulas, after having found the solution one four step linear least-squares problem, can demarcate and obtain confidential reference items a _N, a _N-1A ₁, a ₀With outer ginseng M=[R, T].This scaling method based on Taylor series has avoided accurately finding the solution the problem of camera parameter, and the reflection model that obtains is a virtual minute surface.Have the following advantages in actual applications: under the situation of camera model unknown parameters, this method has been approached imaging process with Taylor series, for the catadioptric camera, this method is insensitive with the deviation of aliging between the minute surface to camera, and the parameter variation that any deviation causes only is embodied on the coefficient of Taylor series.Simultaneously, the distortion of camera lens is also contained in this intrinsic parameter.

But scaling method list of references 1:D.Scaramuzza about Taylor series model, A Toolbox forEasy Calibrating Omnidirectional Cameras, Proceedings of the IEEE InternationalConference on Intelligent Robots and Systems, Beijing, China, October 2006.

As shown in Figure 1, in step 102, estimate the size of N surface perspective panorama.The angle of N surface perspective panorama adjacent surface is ${\mathrm{\θ}}_0=\frac{2\mathrm{\π}}{N},$ Suppose the projection radius R, it depends on the distance (in fact R be equivalent to the focal length of desirable pinhole camera) of the maximum elevation place of catadioptric minute surface apart from axis of symmetry, and the size of N face body panoramic picture can be obtained by following formula:

Figure image height H=R (tan θ ₁+ tan θ ₂)

Figure image width W=N * L ₀

Wherein, R is the projection radius, θ ₁And θ ₂Be maximum elevation and maximum depression; L ₀=2Rtan θ ₀, and θ ₀=2 π/N.

As shown in Figure 1, in step 103, re-projection and interpolation obtain the N surface perspective panorama.As shown in Figure 3 and Figure 4, for the some P of rotation angle ω on i the side and height h (ω, h), connect initial point O and P (ω, straight line h) is:

$f\left(\mathrm{\ρ}\right)=\frac{h\cos \mathrm{\ω}}{R}\mathrm{\ρ}$

Can obtain subpoint P on the minute surface with the minute surface equations simultaneousness ₀(ρ ₀, f (ρ ₀)), corresponding some u and v on annular omni-directional image is:

$\left[\begin{array}{c}u\\ v\end{array}\right]={\mathrm{\ρ}}_0\left[\begin{array}{c}\cos ({\mathrm{\θ}}_0\·(i-1)+\mathrm{\ω})\\ \sin ({\mathrm{\θ}}_0\·(i-1)+\mathrm{\ω})\end{array}\right]+\left[\begin{array}{c}{c}_x\\ c_y\end{array}\right]$

Wherein, θ ₀(i-1)+ω puts P (ω, the h) position angle on annular image, c on i the side _x, c _yBe the coordinate of projection centre at the plane of delineation.Because (ω h) may be non-integer coordinates, can be obtained by bilinear interpolation method to put P.

Work as N=1,2 o'clock, because single plane can not comprise 360 ° image information fully, so the projection number of sides must satisfy 3≤N≤∞.

The N surface perspective panorama that obtains at last is equivalent to the image connection that N photocentre overlapped and do not have the perspective camera set generation of public view field, as shown in Figure 5.

Claims (5)

1. the omnidirectional camera N surface perspective panorama expanding method based on Taylor series model is characterized in that, the step of this method is as follows:

(1) camera calibration: utilize Taylor series model that omnidirectional camera is demarcated, obtain the camera confidential reference items;

(2) size of estimation N surface perspective panorama;

(3) re-projection and interpolation obtain the N surface perspective panorama.

2. a kind of omnidirectional camera N surface perspective panorama expanding method based on Taylor series model according to claim 1 is characterized in that: the N in the N surface perspective panorama of described expansion meets following condition: N 〉=3.

3. a kind of omnidirectional camera N surface perspective panorama expanding method according to claim 1 based on Taylor series model, it is known three-dimensional point and other corresponding point of utilizing on the plane on image that the described omnidirectional camera that utilizes Taylor series model is demarcated, carry out two step least-squares calculation, the confidential reference items of the rotation on camera and plane and translation, camera have successively been obtained, just the coefficient of Taylor series and order.

4. a kind of omnidirectional camera N surface perspective panorama expanding method based on Taylor series model according to claim 1 is characterized in that, the size of described estimation N surface perspective panorama comprises the following step:

(1) height of calculating N surface perspective panorama: H=R (tan θ ₁+ tan θ ₂), wherein R is the projection radius, and is definite apart from the distance of axis of symmetry by the maximum elevation place of minute surface, θ ₁And θ ₂Be maximum elevation and maximum depression;

(2) face of determining the N surface perspective panorama is counted N, obtains the angle theta between adjacent two virtual perspective cameras ₀=2 π/N;

(3) determine the wide of N surface perspective panorama: W=N * L ₀, L wherein ₀=2Rtan θ ₀

5. a kind of omnidirectional camera N surface perspective panorama expanding method based on Taylor series model according to claim 1 is characterized in that, described re-projection and interpolation obtain the N surface perspective panorama and comprise the following step:

(1), calculates point and the mapping relations between annular picture the on the N surface perspective panorama according to calibrating parameters;

(2) bilinear interpolation obtains the pixel value of picture point on the N surface perspective panorama.

Images