JP2000076417A - Camera calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make improvable the camera calibration accuracy by combining the corresponding points of photographs where the feature points are common to each other at three or more points into a one via the rotation and translation of the said corresponding points and deciding the direction and positional vector of a camera. SOLUTION: The three-dimensional coordinates of a set A are calculated, and the camera direction and position are decided for a set B of a photograph including six or more feature points and deleted from the set A. The camera direction and position are decided for a pair of two sheets of photographs including eight or more corresponding points to define a set C of paired two sheets of photographs. When one of both photographs is an element of the set B, the other photograph is also added to the set B to decide the relative camera direction and position. Then the three-dimensional coordinates of a new set B are calculated, and the camera direction and position are decided based on the remainder of the set A for a photograph including six or more calculation points, added to the set B and deleted from the set A. The three-dimensional coordinates of a set D whose camera direction and position are known from the remainder of the set A are calculated and the set B is added to the set D. The corresponding points of photographs where three or more feature points are common to each other in a new set D are combined into one via the rotation and translation of the said corresponding points, and the camera direction and position are found.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera calibration for obtaining information on the position and orientation of a camera that has taken each picture in creating a three-dimensional model from a plurality of pictures using the principle of stereo vision. About the method.

[0002]

2. Description of the Related Art With the improvement of computer graphics processing performance, three-dimensional computer graphics (3D
Various three-dimensional simulation systems using CG) have begun to be applied to industry. In particular, the target for industrial use is to model real equipment (create a three-dimensional model), and to provide various simulation functions and status display functions.
Implemented in a two-dimensional application.

[0003] An issue in creating a three-dimensional model is the large number of man-hours to create. For example, when creating a device model, by dividing the device to be modeled into primitive shapes such as a rectangular parallelepiped and a cylinder, it is possible to create a model more easily than by modeling with a free-form surface, etc. Acquiring the spatial information (vertex coordinates) of a primitive takes time.

As a method for easily extracting the spatial information of a primitive, a model creation method using a photograph has been proposed. This method is a method of creating a three-dimensional model by applying primitives to photographs taken of an object from a plurality of directions. It is not necessary to input numerical values for the model, and since the photograph is used, the image of the model creator is easy to understand, and the model creation is simplified. There is also an advantage that a realistic three-dimensional model can be created by using a photograph as a texture.

[0005] Spatial information is acquired from a plurality of photographs using the principle of stereo vision. At this time, information on the direction and position of the camera that has taken each photo is required, and obtaining such information is called camera calibration. The present applicant has already proposed a camera calibration method using information of corresponding points on a plurality of photographs.

This camera calibration method has three
The orientation and position vector of the camera are obtained for all combinations of the two-piece set having eight or more corresponding points of f or more photos, and the camera orientation and the position vector for each corresponding point of the two-piece set of photos , A straight line connecting the viewpoint and the image is obtained, a distance between the two straight lines is obtained, an average value of each distance for each corresponding point is obtained as a camera calibration error, and the direction of each camera is sorted by ascending order of the error. _1, R _2, ... R
seeking _f, the position of each camera which minimizes the evaluation function with the position vector of the feature points and camera and orientation of an object or the like on the photograph as variable vector T _1, T _2, obtained as ... T _f.

[0007]

In some cases, additional information such as a drawing exists in addition to a photograph of an object, and three-dimensional coordinates of some feature points of the object can be extracted from the additional information. In such a case, it is expected that the accuracy of camera calibration will be improved by using the three-dimensional coordinates of the feature points in addition to the information on the corresponding points on a plurality of photographs.

However, according to the conventional camera calibration method, information of corresponding points on a plurality of photographs cannot be used together with three-dimensional coordinates of feature points, and additional information such as drawings is wasted.

An object of the present invention is to provide a camera calibration method for improving camera calibration accuracy using additional information such as drawings.

[0010]

(First Invention) The present invention provides a three-dimensional model of an object using the principle of stereo vision from a plurality of photographs of the object taken by changing the position and orientation of a camera. In the camera calibration method for obtaining information on the position and orientation of the camera that captured the target from the feature points of the target on each photograph, two or more f
For the set A, which is a single photograph, the three-dimensional coordinates of the object can be calculated from additional information such as drawings.
The dimensional coordinates are calculated in advance, and among the f photos, a set B, which is a photo including six or more feature points calculated from the additional information, is set based on a three-dimensional point projected on the photo. The direction and the position vector are obtained, and the set B is deleted from the set A, and the same 3 is set for all the sets of the two sets of photos having eight or more corresponding points among the f photos.
A relative camera direction and a position vector are obtained by using the epipolar condition of the corresponding point which becomes a point on the dimension, and a set C which becomes the two-piece photograph is obtained. , One of the pictures is the set B
, The other photograph is also added to the set B, and the relative camera direction and position vector of the photograph added to the set B are obtained. Is used to determine the three-dimensional coordinates of the feature points appearing in each photograph, and among the photographs remaining in the set A, those photographs in which six or more points for which the three-dimensional coordinates have been calculated appear on the photographs are projected. The camera direction and the position vector are obtained from the three-dimensional points to be obtained, and the photograph is added to the set B and deleted from the set A. Of the photos remaining in the set A, the relative camera direction and A set D of pictures whose position vectors are known is obtained, relative three-dimensional coordinates of feature points are calculated for the pictures of the set D using the principle of stereo vision, and a new set D is obtained by adding the set B to the set D. Feature points in Corresponding points pictures that are on common coalescing in rotation and translation finally 1 Tsunishi, and obtaining the orientation and the position vector of the camera.

(Second invention) The directions R ₁ ,
R _2, ... R _f and the position vector T _1, T _2, a ... T _f as the initial value, the following expression evaluation function H

[0012]

(Equation 3)

Here, P _i is a vector representing the three-dimensional coordinates of the feature point calculated from the additional information such as the drawing, and the other feature points are represented by the following equations. q _fi is a three-dimensional vector of feature points P _i on the f-th photograph.

[0014]

(Equation 4)

It is characterized in that the values of the position and orientation of each camera are corrected by minimization.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In this embodiment, camera calibration is realized by using information of corresponding points on a plurality of photographs and three-dimensional coordinates of feature points extracted from additional information such as drawings. Hereinafter, after describing basic matters, symbols, and the like, the embodiment will be described.

(Description of Basic Matters, etc.) A three-dimensional point (XY
The transformation when Z) is projected onto the point (u, v) on the photograph is formulated by the following equation.

[0018]

(Equation 5)

Here, the three-dimensional rotation matrix R represents the direction of the camera, and the three-dimensional vector T represents the position.

As shown in FIG. 2, when the point (u ₁ , v ₁ ) on the photograph ₁ and the point (u ₂ , v ₂ ) on the photograph 2 are the same three-dimensional point images, These points are called corresponding points, and the following epipolar condition is satisfied.

[0021]

(Equation 6)

Here, R ₁ ^T R ₂ and R ₁ ^T (T ₂ −T ₁ ) are
The direction and the position of the camera (camera 2) that took the picture 2 when viewed in the coordinate system of the camera (camera 1) that took the picture 1 respectively. That is, the direction and position of the camera 2 relative to the camera 1.

The number of photographs is f (f ≧ 2), and the photographs are represented as photograph 1, photograph 2,... Photograph f. Feature points and the number of feature points is N the i-th object represented by P _i. A set of indices of photographs in which P _i appears is defined as I _i . Feature point P _i
When There that is reflected in the f th photographs, Nachi Suwa when a F∈I, the position of the image of the feature point P _i on the f-th photo (u
_fi , v _fi ), and the three-dimensional vector (u _fi , v _fi , 1) is represented by q _fi . Also, the direction of the camera that took the f-th photograph is represented by R _f , and the position is represented by T _f .

(First Embodiment) FIG. 1 shows a processing procedure of a camera calibration method according to an embodiment of the present invention. ~ S11,
This will be described in detail below.

(S1) Consider a set having Photo 1, Photo 2,... Photo f as elements.

[0026] A = {photo 1, photo 2, ..., photo f} (S2) feature point P _1, P _2, ... of the P _N, for it can calculate the three-dimensional coordinates from additional information such as drawings, Calculate three-dimensional coordinates.

(S3) Photo 1, Photo 2,... Photo f
For a photograph in which six or more points for which the three-dimensional coordinates have been calculated in the processing S2 appear (need not to be on the same plane), camera calibration is performed using the above equation (1) (camera direction, position Seek). In addition, a set of photographs on which camera calibration has been performed is set to B, and elements that are elements of the set B are deleted from the set A. For example, B = {Photo 2, Photo 5} A = {Photo 1, Photo 3, Photo 4, Photo 6,..., Photo f} Relative camera calibration (calculating the orientation and position of the camera that took the other photo with reference to the camera coordinate system of one of the photos) using the epipolar condition of equation (2) Do. Also, a set of two sets of photographs on which the relative camera calibration has been performed is denoted by C. For example, C = ｛(Photo 1, Photo 2), (Photo 1, Photo 4), (Photo 3, Photo 6)... (S5) One of the photos in the element of the set C (two photos) Is a member of the set E, the other photograph is also added to the set E. Further, the position of the photograph of the photo newly added to the set B is determined (the camera direction and position of the photo which were originally elements of the set B, and the relative camera direction and position of the photo newly added to the set E). Calculated from).

The elements of the set C, which consist of a set of two photographs included in the set B, are deleted. Elements that are elements of the set B are deleted from the set A.

The above operation is repeated until there is no more data to be added to the set E. For example, A = {Photo 1, Photo 3, Photo 4, Photo 6,..., Photo f} B = {Photo 2, Photo 5} C = (Photo 1, Photo 2), (Photo 1, Photo 4), (Photo 3, Photo 6) Then, sets A, B, and C are: A = {Photo 3, Photo 6,..., Photo f} B = {Photo 1, Photo 2, Photo 4, Photo 5} C = {(Photo 6, Photo 7)} (S6) Calculate the three-dimensional coordinates of the feature points appearing in each photo of the set E using the principle of stereo vision (the three-dimensional coordinates were calculated in the above process S2). Except the ones).

(S7) Of the photographs remaining in the set A, the processing S
For a photograph in which six or more points whose three-dimensional coordinates have been calculated in steps 2 and 6 are present (need not to be on the same plane), camera calibration is performed using the above equation (1). Find the position). The camera-calibrated photograph is added to the set B and deleted from the set A. If there is a photo newly added to the set B, the process returns to step S5.

(S8) A set of photographs in which the orientation and position of the camera relative to each other are known among the elements of the photographs remaining in the set A is defined as D (determined by the same method as in the above-described processes S5 to S7). For example, A = {Photo 3, Photo 6, Photo 7, Photo 8, Photo f} C = {(Photo 6, Photo 7), (Photo 3, Photo 7), (Photo 8, Photo f) D = {Photo 3, Photo 6, Photo 7}, {Photo 8, Photo f} (S9) For each element of the set D, the relative 3 Calculate dimensional coordinates (three-dimensional coordinates in the coordinate system of each element).

(S10) The set B is added to the set D. For example, if B = {Photo 1, Photo 2, Photo 4, Photo 5} D = {Photo 3, Photo 6, Photo 7}, {Photo 8, Photo f}, the set D becomes D = { Photo 1, Photo 2, Photo 4, Photo 5｝, {Photo 3, Photo 6, Photo 7}, {Photo 8, Photo f} (S11) Three-dimensional (not on the same straight line) For two elements having three or more points in common, conversion (rotation, translation) between the respective coordinate systems is found.
These will coalesce and eventually 1 Tsunishi, camera position _{T 1, T 2, ... T} f, the orientation R _1, R _2, ..., determining the R _f.

(Second Embodiment) In this embodiment, the direction and position of the camera are obtained from a plurality of photographs and additional information such as drawings according to the same procedure as in the first embodiment. Thereafter, these camera positions T ₁ , T ₂ ,..., T _f , directions R ₁ , R _{2 ,.}
Is used as the initial value, and the camera position and orientation values are corrected by minimizing the evaluation function H in the following equation to improve the accuracy.

[0034]

(Equation 7)

Here, P _i is a vector representing the three-dimensional coordinates of the feature point for which the three-dimensional coordinates have been calculated in the above-described processing S2, and the other feature points are represented by the following equations. . As the minimization method, a quasi-Newton method, a conjugate gradient method, or the like is used.

[0036]

(Equation 8)

[0037]

As described above, according to the present invention, the following effects can be obtained.

(1) There is no need for a person to specify the position where each photograph was taken and the approximate value of the direction of the camera. Specifying the camera position and orientation is a complicated operation, and has the advantage that this work can be omitted.

(2) By using three-dimensional coordinates calculated from additional information such as drawings, in addition to information on corresponding points on a plurality of photographs, it is possible to improve the camera calibration accuracy of photographs.

(3) Since the position and orientation of each camera are corrected by minimizing the evaluation function with the obtained position and orientation of the camera as initial values, camera calibration is performed with high accuracy.

[Brief description of the drawings]

FIG. 1 is a processing procedure of camera calibration according to an embodiment of the present invention.

FIG. 2 is a diagram showing a relationship between a photograph and corresponding points.

Claims (2)

[Claims] 1. A feature point of an object on each photograph to create a three-dimensional model of the object from a plurality of photographs of the object by changing the position and orientation of a camera by using the principle of stereo vision. In a camera calibration method for obtaining information on the position and orientation of a camera that has captured an object from a set, three-dimensional coordinates of the object are calculated from additional information such as drawings with respect to a set A including two or more f-photos As far as possible, the three-dimensional coordinates of the feature points are calculated in advance, and a set B that is a photograph including six or more feature points calculated from the additional information out of the f photos is projected on the photograph. The camera direction and the position vector are obtained from the three-dimensional points, and the set B is deleted from the set A. Of the f photos, all the sets of two photos having eight or more corresponding points are obtained. On the same 3D The relative camera direction and the position vector are obtained by using the epipolar condition of the corresponding point, and the set C that becomes the two-piece photograph is obtained. When one of the photos is an element of the set B, the other photo is also added to the set B, and a relative camera direction and a position vector are determined for the photos added to the set B. For each photograph, the three-dimensional coordinates of the feature points appearing in each photograph are obtained by using the principle of stereo vision. Of the photographs remaining in the set A, six or more points at which the three-dimensional coordinates have been calculated appear. The camera orientation and the position vector are obtained from the three-dimensional point projected on the photograph for the photograph that is present, and the photograph is added to the set B and deleted from the set A. ,Relative A set D of photographs in which the direction and the position vector of the camera are known is obtained. Relative three-dimensional coordinates of feature points are calculated for the photographs of the set D using the principle of stereo vision, and the set B is added to the set D. Camera calibration characterized in that corresponding points of a photograph having three or more characteristic points in the new set D are merged by rotation and translation to finally obtain one and obtain a camera direction and a position vector. Option. Wherein said orientation R _1, R ₂ of each camera, ... R _f and the position vector T _1, T _2, a ... T _f as the initial value, the evaluation function H [Number 1] of the formula Here, _Pi is a vector representing the three-dimensional coordinates of the feature point calculated from the additional information such as the drawing, and the other feature points are represented by the following equation. q _fi is a three-dimensional vector of feature points P _i on the f-th photograph. (Equation 2) 2. The camera calibration method according to claim 1, wherein the position and orientation values of each camera are corrected by minimizing.