KR100867731B1 - Method for estimation of omnidirectional camera motion - Google Patents

Abstract

A position estimating method of an omnidirectional camera is provided to estimate position information of a camera accurately with regard to an actual image so that the information can be used efficiently in 3D structure restoration and 3D synthesization of a virtual object. Location information parameter of a camera is obtained(S10). A project model of the omni-directional camera is estimated. A locus of a contour orthogonally projected on 2D image plane of the camera is calculated after a 3D straight line is mapped on a hemisphere of the camera. An angle error function is calculated by using a 3D vector and a normal vector. The 3D vector corresponds to a point on plural contours of two images. Predetermined points at two image viewing points are reverse-projected to make the 3D vector. The normal vector of an epipolar plane is formed by a viewing point of the image and the vector. An initial value of a position information variable number of the camera is calculated. The position information variable number has the angle error function value as a minimum value. The position of the omni-directional camera is determined(S20).

Description

How to estimate the position of the omnidirectional camera {METHOD FOR ESTIMATION OF OMNIDIRECTIONAL CAMERA MOTION}

The present invention relates to a method for estimating the position of the omnidirectional camera, and more particularly, to a method for estimating the position of the omnidirectional camera using a two-stage algorithm related to minimizing the error function.

The omnidirectional camera system having a wide viewing angle can acquire a lot of information about the surrounding scene even with a small number of images, and thus is widely used for camera calibration and three-dimensional reconstruction of space. Camera position estimation and 3D reconstruction techniques from image sequences are important research topics in computer vision field. For this, feature detection and matching processes such as points, lines, and curves corresponding to multi-view images are required. Is required.

However, if the camera moves a lot, the overlapping area in the two image frames is reduced, making accurate matching difficult. Since the images acquired from the omnidirectional camera having a wide viewing angle of 180 degrees or more can acquire a lot of information about the surrounding environment, the matching between the images according to the movement of the camera can be derived relatively easily. Actively done.

Initially, a method of estimating the projection relationship of a camera by converting an omnidirectional camera into a pinhole camera model has been mainly proposed. However, in the case of using the model, a straight line existing in space is projected as a contour in the omnidirectional image, and a method of estimating the projection model of the camera by converting the distorted contour into the original straight line is used. It does not represent all of the field of view.

Alternatively, Xiong implemented panorama images by automatically correcting the distortion and field of view of the images obtained from four fisheye lenses in US370280. However, initial setup information for the camera is required, and incorrect calibration may be performed depending on the type of lens.

Sato [I. Sato, Y. Sato, and K. Ikeuchi, "Acquiring a radiance distribution to superimpose virtual objects onto a real scene," IEEE Trans . on Visualization and Computer Graphics , vol. 5, no. 1, pp. 1-12. 1999.] simplifies the user's direct input required for 3D spatial construction using omnidirectional stereo algorithm. However, this method requires advance information about the camera's position and internal parameters.

Other studies on omnidirectional camera models have been proposed in conjunction with automatic calibration, but most of the studies focus on automatic calibration methods rather than model estimation of omnidirectional cameras. Kannal [J. Kannala and SS Brandt, "A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses," IEEE Trans. on Pattern Analysis and Machine Intelligence , Vol. 28, No. 8, pp. 1335-1340, 2006.] et al. Proposed a general method for accurately modeling general cameras as well as omnidirectional cameras using correction patterns, but has a disadvantage in that a pattern must be produced in advance and an image can be acquired.

We propose a method for estimating the omni-directional camera including a simple algorithm that can be applied to the omni-directional camera for effective registration of three-dimensional images.

   The present invention has been made to improve the prior art as described above, and an object of the present invention is to provide a method for estimating the position of the omnidirectional camera using a contour component corresponding to an image.

It is still another object of the present invention to provide a method for estimating an omnidirectional camera, which minimizes an angular error with respect to an epipolar plane and a distance error of a contour.

According to the present invention, a minimized two-stage algorithm is provided for estimating the position information of the omni-directional camera.

In addition, according to the present invention, it is possible to accurately estimate the position information of the camera with respect to the actual image.

In addition, according to the present invention, it can be effectively used for the restoration of three-dimensional structure and three-dimensional synthesis of virtual objects.

In order to achieve the above object and solve the problems of the prior art, the present invention estimates the projection model of the omnidirectional camera, calculates the trajectory of the projected contour on the image plane of the camera, and calculates a plurality of contour points on the two images. A first step of obtaining an initial value of the position information variable of the camera by calculating an angle error function for the corresponding point as a corresponding point, and setting the position of the camera of the first reference point with respect to the reference point using the initial value. Determining a position of the camera from a position information variable value of the camera which forms a three-dimensional straight line and minimizes a distance error between the contour projected at the second reference point and the contour projected on the image plane of the camera. Provide a second step.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention;

1 is a flowchart illustrating a method for estimating a position of a omnidirectional camera according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the method of estimating the position of the omnidirectional camera according to an embodiment of the present invention includes obtaining a position information variable of the omnidirectional camera (S10) and determining a position of the omnidirectional camera (S20). .

FIG. 2 is a detailed flowchart illustrating an operation S10 of obtaining the position information variable of the omnidirectional camera of FIG. 1.

Referring to FIG. 2, the step S10 of obtaining the positional information variable of the camera may include estimating the projection model of the omnidirectional camera (S11), and a three-dimensional straight line is mapped to the hemisphere model of the camera. Computing the trajectory of the orthogonal projected contour on the two-dimensional image plane of (S12), the three-dimensional vector and the inverse of the arbitrary points in the two image points as the corresponding point on the plurality of the contour of the two images Calculating an angular error function using a viewpoint of an image and a normal vector of an epipolar plane formed by the vector (S13), and calculating an initial value of a position information variable of the camera that minimizes the angular error function value Step S14 is included.

Since the general perspective projection model cannot express the projection relationship of the omnidirectional image acquired by the fisheye lens, a radially symmetrical projection model is required according to the characteristics of the omnidirectional image. The pinhole camera model is represented by a perspective projection model shown in Equation 1 below.

Here, r is a distance from a principal point of the image, θ is an angle between the principal axis of the camera and the incident light, f is a focal length.

FIG. 3 illustrates an orthogonal projection model composed of an incidence vector P and a height value z of a pixel separated by a distance r in an omnidirectional image having a radius rmax.

Since the perspective projection model of Equation 1 cannot be used as an omnidirectional camera model when r is infinite when θ is 90 °, the projection models of Equations 2 to 5 are generally used.

In order to estimate the projection model of the omni-directional camera, a method of obtaining a function represented by nine variables from the image obtained by obtaining a calibration pattern for the projection model is used. A general projection model having radial symmetry may be represented as in Equation 6.

In the process of digitizing the image on the CCD, internal variables including the center point and the aspect ratio of the omnidirectional camera can also be estimated. In this case, the nine variables for the projective model would include the five variables as well as the camera's four internal variables.

Various linear components existing in artificial buildings, indoor environments, etc. are projected in a contour form on the image by the omnidirectional camera model. Therefore, the trajectory of the projected contour on each image can be calculated by using the estimated projection model of the camera and the point on the contour, for example, both end point coordinates, and this relationship can be usefully applied for estimating the movement and rotation parameters of the camera. .

4 shows the trajectory of the contour (c) projecting the linear component (1) in the three-dimensional space to the two-dimensional image plane (IP; IP) of the camera.

Referring to FIG. 4, the contour (c) trajectory of the image plane IP is represented by orthogonal projection after a straight line l on a three-dimensional space is mapped to the hemisphere S. In order to obtain the two-dimensional coordinates of the contour (c) of the image plane (IP), a vector intersecting the hemisphere model and the projected projection model is used.

FIG. 5 is a conceptual diagram illustrating a principle of obtaining two-dimensional coordinates of the contour (c) projected on the image plane IP.

First, an interpretation plane (Π) consisting of the three-dimensional straight line l shown in FIG. 4 and the center of the camera is defined, and the normal vector ( m ) of the analysis plane (Π) = (m _x , _y m, _z m) are external (cross product of the two end point direction vector (p ₁₎ (p ₂₎ of: is calculated from p ₁ X p _2). Contour (c) is an orthogonal component of the line on the analysis plane (Π) and sphere (S), and the three-dimensional vector of the line segment on sphere (S) is the vector ( p ₁ ) and the normal vector ( m ). It can be obtained by rotating transformation to vector ( p ₂ ) based on.

Projection relationships of omnidirectional cameras with fisheye lenses are generally not modeled as hemispheres, so the cameras are first viewed at their maximum viewing angle in order to obtain a vector ( p _curve ) between the analysis plane (Π) and the estimated camera projection model. Find the intersection vector ( p _sphere ) with a _sphere whose distance from the center (r _max ) is half the current. Then, the intersection vector ( p _sphere ) with the _{sphere is} rotated about the normal vector ( m ), and the rotated vector is projected onto the estimated camera projection model to calculate the trajectory c of the contour.

As shown in FIG. 5, the trajectory of the contour (c) may be obtained by projecting a vector ( p _sphere ) intersecting the old model onto a projected projection model (not shown) of the camera using the triangle proportional equation. From the estimated projective model of the camera, the image plane (IP) projected distance (r _curve ) of the vector ( p _curve ) with respect to the incident angle (θ) is obtained, and the two-dimensional contour (c) projected on the image plane (IP) The coordinate is determined by the equation (7).

By estimating the projection model of the omni-directional camera and obtaining the trajectory of the projected contour on the image plane of the camera, the epipolar plane for each corresponding point is estimated to estimate the rotation and relative position information of the camera. The initial value of the location information variable can be estimated by minimizing the angle error function between the three-dimensional vectors.

First, a step (S13) of calculating an angular error function regarding a corresponding point on a contour will be described.

A method of roughly estimating the position information, that is, the movement and the rotation of the camera, is proposed using the two ends of the contour of two arbitrary images as corresponding points. In several omnidirectional images, a start point of the initial image is set as a base view corresponding to the origin of the world coordinate system. Subsequent omnidirectional images of the frames (k = 1, ..., n-1) have relative positions (C _k = s _k Xt _k ), considering the scale element (s _k ) and the unit direction vector (t _k ) from the origin. It is regarded as an image acquired from a reference view converted by and a rotation matrix Rk. Here, the position C ₀ and the rotation matrix R ₀ of the reference viewpoint are (0,0,0) and I, respectively.

6 is a conceptual diagram illustrating a geometric relationship between two images of a reference view and an arbitrary reference view. After setting m end points corresponding to two images to 2m corresponding points, an angle error function E _angualr at each point may be calculated.

Referring to FIG. 6, and are three-dimensional vectors inverted with respect to the i- th corresponding point at the reference viewpoint and the reference viewpoint among the two images, and n _i0 is the unit direction vector t ₀ of the position vector C ₁ of the and reference viewpoint. The normal vector of the epipolar plane () with respect to the reference time point calculated by n _{i 1} is The normal vector of the epipolar plane () with respect to the reference view point computed as the unit direction vector ( t ₁ ) of the position vector ( C _{1 )} of the _first reference point.

Here, since the camera is installed on a tripod to acquire an image, the camera moves only the XZ plane and only rotation about the Y axis is considered. The angle error function for the corresponding points can be obtained as shown in Equation 8.

Here, m is the number of contours, Θ and Φ are the rotation angles of the rotation matrix R1 at the reference point and the y-axis of t1, respectively.

As shown in Equation 9, when the angle error function value E _angualr is minimum, θ and Φ are determined as initial rotation angles θ _init and Φ _init of R1 and t1 of Equation 10.

Here, R ₁ and t ₁ are the initial rotation matrix (R _init ) is I, the initial motion vector (t _init ) is (0,1,0), θ _max is π, and [w _x ] is skew symmetry of the rotation axis w vector Where e is the exponent operator of the matrix, the unit vector of w is (0, 1, 0), and the size of w ∥w∥ is the rotation angle about the y axis.

If the initial rotation matrix (R _init ), the motion vector (t _init ), and the maximum viewing angle (θ _max ) are set to I, (1,0,0), and π, respectively, the angle error is calculated in all regions as shown in FIG. Initially set the angle (Θ, Φ) at the position with the rotation angle of R ₁ and t ₁ respectively. The search method is summarized in Table 1 below, and the rotation angle search pseudo code shown in Table 1 is 1 + 4n when the loop is n, and the search area is reduced to θ _max / 2 ^n-1 at every step. .

n = 1 Em = E _angular (0,0) Loop Search range R = θ _max / 2 ^n-1 Ek = Eangular (± R, ± R) Em = min {Em, Ek} n = n + 1 loop end Calculate the angle error Em at the initial search position Calculate the angle error Ek (k = 1,2,3,4) of the 4 outermost positions

As described above, when the initial value of the position information variable of the omnidirectional camera, that is, the motion vector and the rotation matrix is obtained, the process proceeds to step S20 of determining the position of the camera of FIG. 1.

FIG. 8 is a flowchart illustrating a step (S20) of determining the position of a camera in the method of estimating the position of the omnidirectional camera according to the embodiment of FIG. 1 in more detail.

Referring to FIG. 8, in the determining of the position of the camera (S20), the position C of the camera at the first reference time may be multiplied by a predetermined scale by a movement unit vector of the first reference time among the camera variables. ₁ ) setting (S21), determining a three-dimensional linear component (p _w ) from the contour corresponding to the image of the reference time point and the first reference view (S22), the three-dimensional linear component (p calculating a camera matrix P ₂ of the second reference time point from _w ) and the projected contour ci (t) calculated from the camera matrix P ₂ and an actual value of the second reference time point. Determining the position of the camera by obtaining the camera position information variable including the position and rotation information of the first and second reference viewpoints when the distance error D between the contours ci (s) is minimum ( S24).

A second estimation method for estimating more accurate rotation and position information from unit direction vectors for rotation and movement of the camera described with reference to FIGS. 1 to 7 is proposed.

9 is a conceptual diagram illustrating a step of determining the position of the camera using the second estimation method of the position estimation method of the omni-directional camera according to the embodiment of the present invention.

Referring to FIG. 9, three images of a reference view and first and second reference views appear first in a three-dimensional space. If the position C1 of the first reference point is determined by multiplying the movement unit vector (t ₁ ) estimated in the initial stage by an arbitrary scale (s1), three-dimensional linear components (L) can be obtained for respective contours in the two images. have. In more detail, when the position C1 of the first reference point is set, the direction vector p _w (= p _w1 - p _w0 ) on the world coordinate system of the intersection where the two planes Π ₀ and Π ₁ are in contact is determined. The camera matrix P ₂ is expressed by converting the ₃ × 4 camera matrix P ₂ of the second reference point into a local coordinate system as shown in Equation 11 below.

The normal vector n ₂ of (Π ₂ ) can be obtained from the contour corresponding to the second reference time point, and equation (12) can be derived using equation (11).

When deployed the equation (12) to generalize for the m corresponding contour camera matrix at the second reference point, the method was m is greater than or equal to deploy the expression than 12 to SVD (singular value decomposition) equal to Equation 13 (P ₂ ) Can be obtained.

The trajectory of the projected contour is calculated from the estimated camera matrix P ₂ at the second reference time point, and the contour present in the actual image is calculated from Equation 7. The distance error D between the actual contour ci (s) and the estimated contour ci (t) existing in the omnidirectional image may be expressed by Equation 14. In Equation 14, P is a camera matrix at the second reference point calculated by the rotation angles Θ and Φ, i is a total m as an index of the corresponding contour, and t (s) does not monotonically decrease. It is a monotonic non-decreasing function, with t (0) = 0 and t (1) = 1.

The two-dimensional coordinates in the image plane of the actual contour (ci (s)) of the second reference point is a vector ( p _sphere ) of the point where the ray incident to the center of the camera meets the hemisphere model of the camera. When the vector intersecting the projective model is ci (s) , it can be obtained by using Equation 15 using a triangular proportional expression.

Here, ci (s) is the coordinates (x _curve , y _curve ) orthogonal to the image plane, and p _sphere is the coordinates (x, y) orthogonal to the image plane.

The calculated distance error D is used as a measure to verify the accuracy of the estimated omnidirectional camera rotation and position information. Rotation angle (θ _init , Φ _init ) of initial estimated rotation matrix (R ₁ ) and moving unit vector ( t ₁ ) In the center, the double area of the final search area is pre-defined sampling and searched, and the rotation and position information (Θ ₁ , Φ ₂ ) of the first reference point when the distance error (D) is the smallest and the second reference point Each of the camera matrices P ₂ is determined as the final result.

Hereinafter, a result of verifying through a test the method of estimating the position of the omni-directional camera according to the embodiment of the present invention described above.

We verify the accuracy of the estimated camera model and the calculation of the contour trajectory using both end points of the contour in the image. A Nikon Coolpix 995 digital camera was fitted with a fisheye converter FC-E8 with a viewing angle of 183 ° to shoot a pre-prepared calibration pattern at 1600 × 1200 resolution. We estimated an omni-directional camera model consisting of nine variables, and the calibration pattern used a 2 × 3 m ² flat plate composed of white circles with a radius of 60 mm on a black background.

Fig. 10 shows the contour trajectories calculated with both end point coordinates and the estimated camera model constituting the projected contour.

Referring to FIG. 10, the endpoint of the contour used as an input is represented by a red circle, and the 18 contour trajectories estimated by the proposed method are represented by a blue contour. It can be seen that the trajectory of the estimated contour exactly matches the contour of the real image, and the projection model of the two target images acquired from different places in order to quantitatively express the degree of agreement between the actual contour and the calculated result can be confirmed. The average contour distance error according to the above is shown in Table 2.

Projective Model A B C Experiment 1 4.20 0.10 0.05 Experiment 2 6.70 0.15 0.07

Among the projection models used in the experiment, A is an equidistant projection model, B is an omnidirectional model estimating method using epipolar geometry, and C is a method using nine variables as a model presented in the detailed description of the present invention.

As shown in Table 2, it can be confirmed that the most accurate result is obtained by applying the camera model estimated by the method C, which is the proposed contour estimation method. In the experiments of Table 2, the contours of the contours actually present in the image can be accurately calculated from the coordinate values of both endpoints using the proposed method, which is used as an input to the matching experiment of the projected contour.

In order to verify the accuracy of the proposed algorithm, we experiment with three omnidirectional images (768 × 768) obtained by randomly rotating and moving the omnidirectional camera using POV-Ray rendering software. FIG. 11 illustrates input images using a total of 20 corresponding contours with respect to a reference time point and first and second reference time points, and the experimental results are shown in Table 3. FIG.

Applying the method of the first step of the method according to the embodiment of the present invention using the minimization of the error angle function, and then superimposing the epipolar curve calculated by the end points of the contour and the rotation angles of the estimated R ₁ and t ₁ It is shown in FIG. 11 (c). Here, it can be seen that the red dots representing both ends of the contour are close to the epipolar curve, and the total number of loops is 9 times. In the first initial value estimating step, only a unit direction vector for rotation and movement between two images is estimated and set as an initial value of the second estimating step. The final parameter is a camera matrix that minimizes the contour matching between the third images by dividing the area of ± 0.70 ° × 2 by 100 with initial values of R ₁ and t ₁ as initial values. The contour actually present in the image and the projected contour by the final estimated camera matrix are superimposed and shown in FIG. 12 (c), where it can be seen that there is almost no difference in distance between the two contours.

R ₁ (˚) C1 R ₂ (˚) input -10.00 (60.00,0.00,0.00) -15.00 1st estimate -9.84 Unit vector (1.00,0,0.02) - 2nd estimate -10.13 (60.00,0.00, -0.06) -15.01 C2 Search area error input (80.00,0.00,20.00) - - 1st estimate - ± 0.70˚ Angle error (E): 0.01 2nd estimate (78.79,0.95,19.80) ± 0.01˚ Distance error (D, pixel): 0.78

FIG. 13 shows three omnidirectional images (1600 × 1200) acquired by attaching a fisheye converter FC-E8 to a Nikon Coolpix 995 digital camera in order to apply the method according to an exemplary embodiment of the present invention.

As shown, 20 corresponding contours were used and the camera was mounted on a dolly track without rotation and moved to the Y axis, and the distance ratio of the first and second reference viewpoints from the reference viewpoint was 1: 2 was adjusted.

FIG. 14 is a diagram illustrating an epipolar curve of a corresponding point superimposed on the image of FIG. 13. In the first initial value estimation step, the total number of loops is nine. It can be seen from the figure that the red dots representing both ends of the contour are relatively close but slightly off the epipolar curve.

The contour matching was performed by dividing the area of ± 0.70 ° × 2 into 100 equal parts as in the experiment on the composite image with the estimated R ₁ and t ₁ as initial values. Contours are shown in Table 4 and FIG. 14, respectively.

R ₁ (˚) R ₂ (˚) C ₁ : C ₂ Distance Ratio Search area error input 0.00 0.00 1: 2 - - 1st estimate 1.40 - - ± 0.70˚ Angle error (E) 2nd estimate 0.00 0.40 1: 1.95 ± 0.01˚ Distance error (D, pixel): 0.91

The distance error between the two contours was relatively higher than that of the synthesized image, which occurs during the actual image shooting process including the camera's horizontal maintenance, the camera's exact different setting, and the projection model estimation process. However, from the experimental results, a very small contour matching error of 1 pixel or less was finally obtained, and it was confirmed that the two contours were almost overlapped with each other in FIG.

That is, it can be seen that the position of the moved and rotated camera can be accurately estimated from the position estimation method of the omnidirectional camera according to the embodiment of the present invention.

The above-described calibration method of the omnidirectional camera of the present invention can be executed by a computer program including a high-level language code that can be executed by a computer using an interpreter, as well as machine code made by a compiler, and a computer storing the same. It can be implemented by a readable recording medium.

Examples of the recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic-optical such as floppy disks. Media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

The present invention can be effectively used for the restoration of three-dimensional structure and three-dimensional synthesis of virtual objects, because it uses the projection relationship of real cameras. It can be widely used in.

While many details are set forth in the foregoing description, they should be construed as illustrative of preferred embodiments, rather than to limit the scope of the invention.

For example, one of ordinary skill in the art to which the present invention pertains may further include a more powerful algorithm for selecting an inlier set as well as a suitable omnidirectional projection model according to the technical idea of the present invention. As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. It is obvious that modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

1 is a flowchart illustrating a method for estimating a position of a omnidirectional camera according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating in more detail a step S10 of obtaining an initial value of a position information variable of a camera of FIG. 1;

3 is a view showing an orthographic projection model of the omnidirectional camera used in the method of estimating the omnidirectional camera according to the embodiment of the present invention;

4 is a conceptual view showing a contour projected on a projective model in the method of estimating the position of the omnidirectional camera according to the embodiment of the present invention;

5 is a conceptual diagram illustrating a method of calculating a trajectory of a contour projected on a two-dimensional image plane in a method of estimating a position of an omnidirectional camera according to an embodiment of the present invention;

6 is a view showing a geometrical relationship between each component and a method of obtaining a zeroed vector of an i-th corresponding point from a reference point and a reference point in a method of estimating a position of an omnidirectional camera according to an embodiment of the present invention;

7 is a graph illustrating a method of searching for a rotation angle having a minimum angle error when loop n = 3;

8 is a flow chart illustrating in more detail the step (S20) of determining the position of the camera of FIG.

9 is a view showing a geometric relationship of three omnidirectional images for obtaining a second camera matrix to determine the position of the camera in the method of estimating the position of the omnidirectional camera according to an embodiment of the present invention;

10 is a view showing a contour calculated according to a projection model of the method for estimating the position of the omnidirectional camera according to the embodiment of the present invention;

11 is a view showing (a) an omnidirectional input image of a reference point, (b) an omnidirectional input image of a first reference point, (c) an omnidirectional input image of a second reference point in an equidistant projection model;

12 shows (a) both endpoints of the contour of the first reference point and the estimated epipolar curve, (b) both endpoints of the contour of the second reference point and the estimated epipolar curve, and (c) the actual and estimated contours. Drawing.

13 is (a) an actual omnidirectional input image of a reference time point, (b) an actual omnidirectional input image of a first reference time point, (c) an actual omnidirectional input image of a second reference time point,

14 shows (a) both endpoints of the contour of the first reference point and the estimated epipolar curve, (b) both endpoints of the contour of the second reference point and the estimated epipolar curve, and (c) the actual and estimated contours. Visible drawings.

Claims (12)

Estimating the projection model of the omni-directional camera to calculate the trajectory of the contour projected on the image plane of the camera, using the points on the plurality of contours of the two images as the corresponding points, and calculating the angle error function for the corresponding points A first step of obtaining an initial value of the information variable; And By forming the three-dimensional straight line by setting the position of the camera of the first reference point with respect to the reference point using the initial value, the contour projected to the second reference point and the contour projected on the image plane of the camera and Determining a location of the camera from a location information variable of the camera to minimize a distance error of the camera; Position estimation method of the omni-directional camera, characterized in that it comprises a. The method of claim 1, wherein the first step, Estimating the projection model of the omnidirectional camera; Calculating a trajectory of a contour projected orthogonally to a two-dimensional image plane of the camera after a three-dimensional straight line is mapped to the hemisphere model of the camera; Angle error using a three-dimensional vector in which two points on the contour of the two images are corresponding points and inverted for any corresponding point at two image points, and a normal vector of the epipolar plane formed by the vector and the view point of the image. Calculating a function; And Calculating an initial value of a positional information variable of the camera that minimizes the angle error function value; Position estimation method of the omni-directional camera, characterized in that it comprises a. The method of claim 2, The projection model of the camera is a method of estimating the position of the omni-directional camera, characterized in that the radially symmetrical projection model. The method of claim 3, The projection model of the camera is a projection method of the omni-directional camera, characterized in that the projection model represented by the following equation (16).

Where r is the distance from the center of the image and θ is the angle between the camera's central axis and the incident light The method of claim 2, The two-dimensional coordinates of the trajectory of the contour projected on the image plane may be a vector ( p _curve ) where a vector ( p _sphere ) of a point where a ray incident to the center of the camera meets a hemisphere model and intersects the projected model. And satisfying Equation 17 using a triangular proportional expression.

Here, the two-dimensional coordinates of p _curve orthogonal to the image plane are (x _curve , y _curve ), and the coordinates of p _sphere orthogonal to the image plane are (x, y) The method of claim 2, And the angle error function (E _angualr ) satisfies Equation 18 when the camera moves on the xz plane and rotates about the y-axis.

Where

Is a three-dimensional vector inverted for the i- th correspondence point from the reference and reference viewpoints, and n _i0 is the reference viewpoint computed as the unit direction vector ( t ₀ ) of the position vector ( C ₁ ) of and the reference viewpoint. Normal of epipolar plane (), n _{i 1} is And the normal vector of the epipolar plane () with respect to the reference point computed as the unit direction vector ( t ₁ ) of the position vector ( C ₁₎ of the _first reference point, m is the number of contours, Θ and Φ are respectively Rotation angle about the y-axis of the rotation matrix (R1) and t ₁ . The method of claim 6, When the angle error function value (E _angualr ) is the minimum position θ and Φ as the initial rotation angle (Θ _init , Φ _init ) of R1 and t1 satisfying the following equation (19) Estimation method.

Here, R ₁ and t ₁ are the initial rotation matrix (R _init ) is I, the initial motion vector ( t _init ) is (0,1,0), θ _max is π, and [w _x ] is skew symmetry of the rotation axis w vector Where e is the exponential operator of the matrix, the unit vector of w is (0,1,0), and the size of w ∥w∥ is the rotation angle about the Y axis. The method of claim 7, wherein The angle error function value (E _angualr ) is the number of calculations when the loop (n) is 1 + 4n and the search region (R) satisfy the following equation (20).

The method of claim 1, And the camera position information variable for minimizing the angle error function comprises a motion vector and a rotation matrix. The method of claim 1, wherein the second step Setting a position C ₁ of the camera at the first reference time by multiplying a movement unit vector of the first reference time among the camera variables by an arbitrary scale; Determining a three-dimensional linear component ( p _w ) from a contour corresponding to the image of the reference time point and the first reference time point; Calculating a camera matrix P ₂ of the second reference time point obtained from Equation 21 from the three-dimensional linear component p _w ; And

Equation 22 when the distance error D between the projected contour ci (t) calculated from the camera matrix P ₂ and the actual contour ci (s) of the second reference point is minimum Determining the position of the camera by obtaining the camera position information variable including the position and rotation information of the first and second reference viewpoints using;

Position estimation method of the omni-directional camera, characterized in that it comprises a. The method of claim 10, The two-dimensional coordinates in the image plane of the actual contour (ci (s)) of the second reference point may be a vector of a point ( p _sphere ) at which the ray incident to the center of the camera meets the hemisphere model of the camera. The method of estimating the position of the omni-directional camera, characterized by satisfying the following equation (23) using a triangular proportional expression when the vector intersecting the model is ci (s) .

Where, ci (s) is orthogonal to the image plane (x _curve , y _curve ), and p _sphere is orthogonal to the image plane (x, y) A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 12.

Images