JP2000182051A - Camera calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce work time in the setting of information required for camera calibration by calculating the relative direction/position of a camera from the correspondence of two corresponding points and three vertical straight lines or from the correspondence of the corresponding points and two groups of two parallel straight lines in two photographs. SOLUTION: For creating the three-dimensional model of an object with the principle of stereo view from two photographs 1 and 2 where the object is taken by changing the position and the direction of a camera, the relative direction/position of the camera is calculated from the correspondence of the two corresponding points and three straight lines which are mutually perpendicular to each other on the two photographs 1 and 2 or the relative direction/ position of the camera is calculated form the correspondence of the two corresponding points and the two groups of two parallel straight lines in the two photographs 1 and 2 in a camera calibration method for obtaining information of the direction and the position of the camera from the corresponding points on the two photographs 1 and 2. Thus, calibration from the photograph with few corresponding points is realized and the positioning work of the corresponding points or the like is simplified.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of creating a three-dimensional model from a plurality of photographs of equipment and facilities by using the principle of stereoscopic vision. It relates to a camera calibration method for obtaining.

[0002]

2. Description of the Related Art With the improvement of computer graphics processing performance, three-dimensional computer graphics (3D
Various three-dimensional simulation systems using CG) have begun to be applied to industry. In particular, for industrial use, a real device is modeled (three-dimensional model is created), and various simulation functions and status display functions are realized by a three-dimensional application.

A problem when creating a three-dimensional model is a large number of man-hours. For example, when creating a device model, by dividing the device to be modeled into primitive shapes such as a rectangular parallelepiped and a cylinder, it is possible to create a model more easily than by modeling with a free-form surface, etc. Acquiring the spatial information (vertex coordinates) of a primitive takes time.

As a method for easily extracting the spatial information of a primitive, a model creation method using a photograph has been proposed.

[0005] This method is a method of creating a three-dimensional model by applying primitives to photographs taken of objects of equipment and facilities from a plurality of directions. This method does not require numerical input of a model, and since a photograph is used, the image of the model creator is easily boiled, thereby simplifying the model creation. There is also an advantage that a realistic three-dimensional model can be created by using a photograph as a texture. Spatial information is acquired from a plurality of photographs using the principle of stereo vision. At this time, information on the orientation and position of the camera at the time of shooting each target object is required, and obtaining such information is called camera calibration.

As a technique for easily performing camera calibration, there is a method of using corresponding points on a plurality of photographs.
A geometric relationship called an epipolar relationship is established between corresponding points on two photographs of the object taken from different directions. If there are eight or more corresponding points on two photographs, the relative camera orientation and position (based on the coordinate system of the camera that took one of the two photographs) using the epipolar relationship There are the following documents regarding camera calibration for estimating the camera orientation and position of the other photo when the above is performed.

Reference 1) Longgest-Higgin
s, H .; C, "A Computer A1gorit
hm for Restructuring a S
Cene from Two Projection
s, "Nature, Vol. 293, pp. 133-135 (1
981) Reference 2) Tasi, R .; Y. and Huang, T .;
S. , "Uniqueness and estima
Tion of 3-D motion parame
ters of rigid bodies with
curvedsurfaces, "IEEE Tra
ns Pattern Anal. Machine I
nell. vol. PAMI-6, pp. 13-27 (198
4) Reference 3) Weng, J. et al. et al. , "Motion
and Structure from Two P
erotic Views: Algorith
ms, Error Analysis, and Err
or Estimation, "lEE Trans
Pattern Anal. Machine Int
ell., vol. 11, no. 5, pp. 451-476 (1989) Reference 4) Kanatani, K. et al. , "Renormal
Ization for Motion Ana1ys
is: Statistically Optimal
Algorithm ", IEICE Trans. In
f. & Syst., Vol. E77-D, no. 11 pp. 1
233-1239 (1994) The applicant of the present application has already proposed a camera calibration method using information on corresponding points on three or more photographs (Japanese Patent Application No. 9-45389).

[0008]

The above "prior art"
In the camera calibration method using the corresponding points on a plurality of photographs described above, many accurate corresponding points are required. For example, in order to linearly calculate the relative camera relationship between two photographs, eight or more corresponding points are required. Therefore, camera calibration cannot be performed on a photograph for which the required number of corresponding points cannot be obtained.

[0009] When a person manually performs the operation of accurately locating the corresponding point, a considerable amount of time is required. The reason for this is that although it is possible to automatically perform corresponding scoring by computer processing between adjacent images, there is no effective algorithm for corresponding scoring between non-adjacent images.

An object of the present invention is to provide a camera calibration method which enables camera calibration from a photograph having a small number of corresponding points and simplifies the work of locating the corresponding points.

[0011]

According to the present invention, when an artificial object such as a building or an industrial product is used as an object for creating a three-dimensional model, a set of parallel straight lines, a vertex of a rectangular parallelepiped, By utilizing the fact that information such as three straight lines is selected, from the correspondence between two corresponding points and three straight lines mutually perpendicular in two photographs, or from the correspondence between two corresponding points and two sets of parallel two straight lines This is a method for calculating the relative camera direction and position, and is characterized by the following method. (First invention)
To create a three-dimensional model of an object using the principle of stereo vision from two photographs taken of the object while changing the position and orientation of the camera, the direction and position of the camera from the corresponding points on the two photographs Is characterized in that the relative camera direction and position are calculated from the correspondence between two corresponding points in two photographs and three straight lines perpendicular to each other.

(Second Invention) Since a three-dimensional model of an object is created from two photographs of the object by changing the position and orientation of the camera using the principle of stereo vision, the two In the camera calibration method for obtaining information on the direction and position of the camera from the corresponding points, the relative camera direction and position are calculated from the correspondence between two corresponding points in two photographs and two sets of two parallel straight lines. It is characterized by the following.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Basic matters, symbols, and the like used in the description of the embodiments of the present invention will be described.

The transformation when a three-dimensional point (X, Y, Z) is projected on a point (u, v) on a photograph is formulated by the following equation.

[0015]

(Equation 1)

Here, the three-dimensional rotation matrix R indicates the direction of the camera, and the three-dimensional vector T indicates the position. Also, the focal length of the camera is normalized to 1. Also, the point (u ₁ , v ₁ ) on Photo ₁ and the point (u ₂ , v ₂ ) on Photo 2 are the same 3
When the images are points on a dimension, these points are called corresponding points, and the following epipolar relationship is established.

[0017]

(Equation 2)

Here, R ₁ ^T and R ₁ ^T (T ₂ −T ₁ ) are the values of the camera (camera 2) that took picture 2 when viewed in the coordinate system of the camera (camera 1) that took picture 1. Orientation, position. That is, the direction and position of the camera 2 relative to the camera 1.

(First Embodiment) In this embodiment, relative camera orientation and position (excluding scale) are calculated from two corresponding points and three straight lines perpendicular to each other in two photographs. Camera calibration method.

As shown in FIG. 1, two photographs of the object taken at different positions are represented as photograph 1 and photograph 2, respectively. Now, based on the coordinate system of the camera (camera 1) that took the photograph 1, the position of the camera 2 when viewed in this coordinate system is represented by a three-dimensional vector T, and the direction is represented by a three-dimensional rotation matrix R. The position of the camera 1 is a zero vector, and the direction is a three-dimensional unit matrix.

In the three-dimensional space, three mutually perpendicular straight lines are denoted by a,
The images b and c on the photograph 1 are denoted by l, m and n. In addition, let three straight lines d, e, and f in the three-dimensional space be images s, t, and u on the photograph 2, respectively. At this time, a and d are the same or parallel, and b and e and c and f are also the same or parallel. That is, ((a = d or a //
d) and (b = e or b // e) and (c = f
or c // f)).

FIG. 2 is a flowchart showing an algorithm for performing camera calibration. Hereinafter, steps S1 to S6 will be described.

(S1) Six mutually perpendicular three straight lines in the three-dimensional space are selected, two points on the images 1, m and n on the photograph 1 of a, b and c, two for each straight line, a total of six. .

The points in the actual three-dimensional space where these points are represented are represented by P _a1 , P _a2 , P _b1 , P _b2 , P _c1 , P _c2 , and their three-dimensional coordinates are represented by (X _a1 , Y _a1). , Z _a1 ),…,
(X _c2 , Y _c2 , Z _c2 ) and the image coordinates are (u _a1 , v
_a1 ), ..., ( _uc2 , _vc2 ).

Each point has three vectors P _a1 P _a2 , P
_{b1 Pb2} and _{Pc1 Pc2} are selected so as to form a right-handed system in this order. The vector (u _a1 , v _a1 , 1) ^T is represented by q _a1 , and q _a2 ,..., Q _c2 are similarly defined. At this time, there are certain λ _a1 ,..., Λ _c2 from the above equation (1), and the following equation (3) is established.

[0026]

(Equation 3)

(S2) Two mutually perpendicular three straight lines in the three-dimensional space are selected from d, e, and f on the image s, t, and u of the image 2 on the photograph 2, two for each straight line, that is, six in total. .

The points in the actual three-dimensional space where these points are represented are represented by P _d1 , P _d2 , P _e1 , P _e2 , P _f1 , and P _f2 , and their three-dimensional coordinates are represented by (X _d1 , Y _d1). , Z _d1 ),…,
(X _f2 , Y _f2 , Z _f2 ) and the image coordinates are (ud ₁ , v
_d1 ), ..., ( _uf2 , _vf2 ).

Each point has three vectors P _d1 P _d2 , P
_e1 P _e2, P _f1 P _f2 is forms a right-handed in this order, also a vector _{P a1 P a2, P d1 P} d2 is the same direction, the vector P _b1 P _b2,
P _e1 P _e2 are the same direction, the vector _{P c1 P c2, P f1 P} f2 is chosen to be in the same direction. In the same manner as in the above (S1), vectors q _d1 ,..., Q _f2 are defined. At this time, there are certain λ _c1 ,..., Λ _f2 from equation (1), and the following equation is established.

[0030]

(Equation 4)

(S3) Three vectors λ _a2 q _a2 −λ _a1 q
_{a1, λ b2 q b2 -λ b1} q b1, λ c2 q c2 -λ c1 q c1 are perpendicular to each other, from the fact that form a right-handed in this order, λ _a2 /
The values of λ _a1 , λ _b2 / λ _b1 , λ _c2 / λ _c1 are calculated.

(S4) Three vectors λ _d2 q _d2 −λ _d1 q
_{d1, λ e2 q e2 -λ e1} q e1, λ f2 q f2 -λ f1 q f1 are perpendicular to each other, from the fact that form a right-handed in this order, λ
The values of _d2 / _λd1 , _λe2 / _λe1 , and _λf2 / _λf1 are calculated.

(S5) The rotation matrix R is calculated. From the above equations (3) and (4), the following equations hold.

[0034]

(Equation 5)

From this, the rotation matrix R is obtained as follows.

[0036]

(Equation 6)

(S6) The position vector T is calculated from the two corresponding points and the epipolar relationship and the above equation (2).

(Second Embodiment) In this embodiment, the relative camera direction and position (excluding the scale) are determined based on the correspondence between two corresponding points and two sets of two parallel straight lines in two photographs. This is a camera calibration method to be calculated.

As shown in FIG. 3, two photographs of the object taken at different positions are represented as photograph 1 and photograph 2, respectively. Now, based on the coordinate system of the camera (camera 1) that took the photograph 1, the position of the camera 2 when viewed in this coordinate system is represented by a three-dimensional vector T, and the direction is represented by a three-dimensional rotation matrix R. The position of the camera 1 is a zero vector, and the direction is a three-dimensional unit matrix.

In a three-dimensional space, two sets of parallel two straight lines a and b (a / b, a （b) and c and d (c // d, c ≠)
The image on the photograph 1 of d) is k, l, m, n. Also, 3
Two sets of parallel two straight lines e and f (e //
f, e ≠ f) and g, h (g // h, g ≠ h) on the photograph 2 are denoted by s, t, u, v.

At this time, e is parallel to a and b or the same as either a or b, and (e // a // bo
re = a or a = b). Similarly, f is a,
It is parallel to b or the same as either one of a and b, and (f // a // borf = aorf = b). Similarly, for c, d, g, and h, (g // c // d
org = c or g = d, h // c // d or h = c
or h = d).

In perspective projection, a point at which the images of a group of parallel straight lines intersect at one point in a three-dimensional space is called a vanishing point. FIG. 4 is a flowchart showing an algorithm for performing camera calibration. Hereinafter, steps S11 to S14 will be described.

(S11) Image coordinates of two vanishing points on the photograph 1 are obtained. Image k, on photo 1, of two parallel straight lines a, b,
point l intersects the image coordinates of (vanishing point) and (u _kl, v _kl), represents a vector _{(u kl, v kl, 1} ) a ^T at q _kl.
Similarly, the image coordinates of the point at which the images m and n of the two parallel straight lines c and d on the photograph 1 intersect are defined as ( _umn , _vmn ), and the vector (u
_mn , v _mn , 1) ^T is represented by q _mn .

(S12) Image coordinates of two vanishing points on the photograph 2 are obtained. Images s, on photograph 1 of two parallel straight lines e, f
t intersect point the image coordinates of (vanishing point) and (u _st .v _st), represents a vector _{(u st .v st, 1)} T with q _st.
Similarly, the image coordinates of the point where the images u and v of the two parallel straight lines g and h intersect on the photograph 1 are (u _uv , v _uv ), and the vector (u
_uv, the v _uv, 1) ^T represented by q _uv.

(S13) The rotation matrix R is calculated. Since the vanishing point is an image at a point at infinity, q _kl ∝Rq _st , q _mn ∝Rq
_uv holds. Than this

[0046]

(Equation 7)

Then, the rotation matrix R is obtained by the following equation.

[0048]

(Equation 8)

(S14) The position vector T is calculated from the two corresponding points and the epipolar relationship and the above equation (2).

[0050]

As described above, according to the present invention, the correspondence between two corresponding points and three straight lines perpendicular to each other, or the correspondence between two corresponding points and two sets of two parallel straight lines in two photographs. Since the relative camera direction and position are calculated,
The work time in setting information necessary for camera calibration can be reduced as compared with the conventional method.

The setting of three mutually perpendicular lines and the parallel lines can be performed by a method in which the user can select a straight line extracted from edges extracted by computer image processing, thereby reducing work time.

Further, by using information of a group of three straight lines and a group of parallel straight lines, camera calibration can be performed even for a photograph in which the number of corresponding points is not large.

[Brief description of the drawings]

FIG. 1 is a diagram showing the relationship between two corresponding points in photographs 1 and 2 and three vertical straight lines.

FIG. 2 illustrates a camera calibration algorithm according to the first embodiment.

FIG. 3 is a diagram showing the relationship between two corresponding points in photographs 1 and 2 and two vertical straight lines.

FIG. 4 illustrates a camera calibration algorithm according to the second embodiment.

Claims (2)

[Claims] 1. A three-dimensional model of an object is created from the two photographs of the object by changing the position and orientation of the camera using the principle of stereoscopic vision. A camera calibration method for obtaining camera orientation and position information, comprising calculating a relative camera orientation and position from a correspondence between two corresponding points in two photographs and three mutually perpendicular lines. Camera calibration method. 2. A three-dimensional model of an object is created from the two photographs of the object by changing the position and orientation of the camera using the principle of stereoscopic vision. A camera calibration method for obtaining camera orientation and position information, comprising calculating a relative camera orientation and position from correspondence between two corresponding points in two photographs and two sets of two parallel straight lines. Camera calibration method.