JPH07159167A - Calculation of camera position - Google Patents

Abstract

PURPOSE:To easily obtain the focus position without using complicate equation by supposing a plane vertical to the camera optical axis in a space and detecting the min. error position by a means for projecting the reference points of at least three points, on the plane. CONSTITUTION:The camera focus position is set at FC, and the three dimensional coordinate values are represented by Fx, Fy, and Fz, and the vertical plane P relative to the camera optical axis is set. A plurality of characteristic points on an object having the well-known measurement values of the dimension and shape are projected on the plane, setting the camera focus position FC as the center. Each projected point Si' is represented, having the camera focus coordinates as parameters. The camera focus position FC is scanned (shifted) in a space by using a computer, etc., and the min. error position is searched, and the position is set as the camera focus position. The plane P is set, and since the distance between two points in two pairs in the projection of the reference point on the object and the corresponding camera position can be obtained, only three points are enough as the reference points. Further, the error is easy to understand since the error is calculated as a simple numerical value through the comparison of the distance, and the necessity of the complicated calculation is obviated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional shape measuring method for measuring a three-dimensional shape of a subject from an image taken by a camera or the like.

[0002]

2. Description of the Related Art An object is measured using a camera,
To obtain the positional relationship between the camera and the outside world, the position of the camera must be known accurately. To show the camera position,
The image plane of the camera or the focal position of the lens is generally represented as coordinate values on a coordinate system centered on the measurement object. However, the position of the camera is natural when the camera itself is moved, but even when the lens used is changed, the positional relationship between the image and the measurement object changes,
It is necessary to calibrate each time. The coordinate system used for the measurement target is called the "object coordinate system",
The coordinate system with the origin at the camera is called the "camera coordinate system". Obtaining the conversion parameter between these two coordinate systems is camera calibration. Hereinafter, a method of obtaining conversion parameters by conventional camera calibration will be described.

FIG. 3 shows a perspective transformation model based on a pinhole camera. This perspective transformation model in which the camera is idealized has a pinhole opened at the center of the lens surface, and the line of sight is defined as a straight line passing through this point.
A general imaging system using a glass lens can also be represented by this simple model if the distortion aberration is small enough to be ignored. In an actual camera, the target, the lens, and the image plane are arranged in this order, but this makes the image reverse and is difficult to see. Therefore, the image plane is virtually placed in front of the lens, and the object, image plane, and lens They are arranged in order. In the figure, F is the center of the lens, f is the focal length of the lens, and I is the image plane. Since the image plane is considered as the reference of the coordinate system fixed to the camera, the center of I is the origin of the coordinate axis.

A point S (Xc, Yc, Zc) obtained by perspective-transforming a certain point V (X, Y, Z) in space into an image plane I, that is, the line of sight toward the measuring point V and the image plane. The intersection is

[0005]

[Equation 1]

Is given by Also, if you change the expression

[0007]

[Equation 2]

Can be expressed as Although the perspective transformation is a non-linear transformation as described above, it can be linearized by adding one variable mediated by three-dimensional coordinates and using a one-dimensionally enhanced expression. This is called the homogeneous coordinate system. Three-dimensional point (X, Y, Z)

[0009]

[Equation 3]

As shown in (4), homogeneous coordinates are expressed by four-dimensional points (Xh, Yh, Zh, Wh) mediated by Wh. With this homogeneous coordinate system, the perspective transformation is

[0011]

[Equation 4]

As described above, it can be described by a 4 × 4 matrix operation. The perspective transformation described above can be applied when it is represented in a coordinate system fixed to the camera. When the point P that is the measurement target is represented by another independent coordinate system, there must be an "object coordinate system" that is the coordinate system used for the measurement target and a "camera coordinate system" that is the coordinate system with the origin of the camera. As described above, the transformation T that relates these two coordinate systems is expressed by the homogeneous coordinate system expression including rotation and translation.

[0013]

[Equation 5]

Can be represented by The conversion from the point V in the object coordinate system to the point S in the camera coordinate system is

[0015]

[Equation 6]

Can be expressed as Since the image plane I in the camera coordinate system is Zch = 0, the two-dimensional coordinates (Xc, Y
c), that is, simplifying the previous equation by the position of the pixel in the input image,

[0017]

[Equation 7]

It can be described as follows. The 3 × 4 C matrix in this equation is called a “camera parameter”, and the principal point position Fc (focal point position) in FIG. 3 can be obtained from the C matrix. Calibration is performed to obtain the camera parameters. The reference point (X, Y,
Z) and the corresponding position (Xc, Y on the camera image plane)
If you know one c),

[0019]

[Equation 8]

Is satisfied. Therefore, in order to obtain twelve unknowns C11 to C34 of the camera parameters, it is sufficient to use six reference points that are not on the same plane. In order to improve the accuracy of calibration, it is usual to use six or more reference points and identify the parameters by the least squares method.

[0021]

However, the camera parameter calculation method described in "Three-dimensional image measurement" (Iguchi, Sato) described above requires a minimum of six reference points, and further the inverse matrix. Needs to be calculated. However, the solution of the inverse matrix is not always obtained, and if the number of reference points is increased in order to improve the accuracy, the determinant becomes larger and the calculation becomes complicated and time consuming.
Further, since the coordinate value on the camera image is an integer value, there is no intermediate value and it contains an error, and it must be solved by using the least square method.

The conventional method has the above problems,
Therefore, an object of the present invention is to easily obtain the focal position with a minimum of three reference points without using complicated equations and inverse matrix calculations.

[0023]

Therefore, according to the present invention, in an apparatus for calculating a three-dimensional positional relationship between a camera and an object for obtaining the image from an image showing an object of known size and shape,
Assuming a plane perpendicular to the optical axis of the camera in space,
By means of projecting a minimum of three reference points onto the plane, the coordinates of the projected points and the corresponding points on the camera image are scanned using a computer or the like while scanning the camera focal point position. The present invention has been found to be able to solve the problems by using the method of calculating the position and detecting the position with the least error to obtain the camera position coordinates on the coordinate system of the reference object. I arrived.

[0024]

[Function] A cube that is relatively far from the viewpoint has the same length on all sides, but when viewed close, the side on the near side looks long and the side on the back looks short. By changing the viewing position,
The perspective of an object changes. By utilizing this change, the positional relationship between the camera and the object can be known.

An outline of the apparatus is shown in FIG. The object (reference object) of known dimensions and shape is projected by the camera, the object is recognized by the image processing unit, and the coordinate value (on the camera image) of the reference point on the object (a point whose position is known in the object such as a vertex) is extracted. To do. From the coordinate values and the shape data of the reference object, the camera position calculation unit calculates the camera coordinate position on the object coordinates. The principle will be described with reference to FIG. The focus position of the camera is Fc, and the three-dimensional coordinate value of Fc is Fx, Fy, Fz, and a plane P perpendicular to the optical axis of the camera is set. A plurality of feature points on a measurement object whose dimensions and shape are known (a reference object for camera calibration, etc.), for example, a vertex Vi (hereinafter, i and j are feature point numbers), are used as the focus position Fc of the camera. Is projected on this plane. Each projected point Si ′ is represented with the focus coordinates (Fx, Fy, Fz) of the camera as a parameter. The camera focus coordinates Fc may be obtained so that the points Si ′ projected on the plane P and the points Si on the camera image corresponding to those points have the same positional relationship. Since the unit of the point Si is a two-dimensional pixel value and the unit of the point Si ′ is represented by a three-dimensional coordinate value, here, the position ratio is calculated using the ratio of the distances between the corresponding two points. I will seek a relationship. If Dij is the distance between points Si and Sj and Dij 'is the distance between points Si' and Sj ', there is a proportional relationship between Dij and Dij'.

[0026]

[Equation 9]

The following relationship holds. Here, Dij 'is also represented by using the focus coordinates of the camera as a parameter. However, in the above method, since the value on the camera image is an integer value and includes an error in measurement, the camera position is not always found even if the equation is solved. Therefore, in the present invention, the camera focus position Fc is scanned (moved) in space using a computer or the like, and the position with the smallest error is searched for and set as the camera focus position.

As described above, by setting the plane P and comparing the distance between two sets of two points on which the reference point on the object is projected with the distance between the two sets of two points on the corresponding camera image, Since the camera position can be obtained, the minimum number of reference points is 3, and the error is calculated as a simple numerical value by comparing the distances, which is easy to understand and does not require complicated calculation.
However, each reference point must not be on a plane perpendicular to the optical axis of the camera.

Hereinafter, the present invention will be described in more detail by way of examples with reference to the drawings, but the present invention is not limited thereto.

[0030]

EXAMPLE FIG. 4 shows the case where stereo measurement is performed by two cameras. In this method, it is usually necessary to know the positional relationship (direction of the optical axis) of the two cameras, but it is difficult to find the two optical axis directions. There is also a method of placing two cameras in parallel for simplification, but the measurement range is limited. By using the present invention, the cameras can be installed at arbitrary positions and the positional relationship between the cameras can be easily obtained. The same applies when using a plurality of cameras.

In principle, the simplest example is that the optical axis of a camera passes through a certain vertex on a rectangular parallelepiped of known dimensions, and that point is the origin of the object coordinate system, and the three sides of the cube are respectively the x-axis and the y-axis. The case where the axes are z and z will be described. Figure 5
At, the optical axis LF of the camera passes through a point Vo on the object. In the object coordinate system, if Vo is the origin and the axes passing through V1, V2, and V3 are the x-axis, y-axis, and z-axis, each point on the cube can be represented by a value on the object coordinate system. The plane P perpendicular to the optical axis of the camera is assumed to include the origin Vo here in order to simplify the equation of the plane. Here, the focal coordinate of the camera is used as a parameter.

[0032]

[Equation 10]

It is represented by In FIG. 5, a point Vi on the cube (i is the vertex number of the cube, and the coordinate values are Vix, Vi
A point Si ′ obtained by projecting y, Viz) onto the plane P is a straight line Lvi passing through the focal point of the camera and the point Vi.

[0034]

[Equation 11]

And the intersection with the plane P

[0036]

[Equation 12]

It is The coordinate value of Si 'is also represented by the focal coordinate of the camera as a parameter. Let the length between the points Si and Sj be Dij, and let the length between the corresponding Si 'and Sj' be Dij '.
Then, there is a proportional relationship between Dij and Dij 'for any i, j. Since the units of Dij and Dij 'are different,

[0038]

[Equation 9]

It will be expressed as follows. By solving this equation, the focus coordinate value Fc which is a parameter can be obtained. However, since Dij is a value (integer value) on the image, this equation is not actually an equal sign,

[0040]

[Equation 13]

It becomes as follows. Therefore, the error K

[0042]

[Equation 14]

The focus position coordinates, which are the parameters, are scanned in space and the coordinate values are actually substituted.
Find the position with the least error. The obtained position is used as the focus coordinate. The space to be scanned can be limited by the direction in which the object coordinate system is set. The case in which the optical axis of the camera passes through the origin of the object coordinate system has been described above. However, even if the optical axis does not pass through a known point, if the above-described formula of the plane P is obtained, the focus is similarly set. The position can be calculated. In this case, the passing point may be obtained from the positional relationship with another known point (reference point), and the focus position may be obtained by the method described above. As shown in FIG. 6, by adding a device that can freely change the direction of the camera based on the camera position information, the camera system itself can search for a reference object, and the reference point can be used to determine the light of the camera. The camera position can be calculated faster because it can be aligned with the axis.
Further, when there are a plurality of reference objects, the positional relationship between them can be obtained.

[0044]

Since the positional relationship with the object can be recognized, it can be used for acquiring the positional information of the mobile device (car, airplane). Most of the objects installed on the ground, not only on the road, are manufactured according to a certain standard and size, and can be used as a reference object. Therefore, a new dedicated marker for position information is not necessarily required.

[Brief description of drawings]

FIG. 1 is a schematic diagram according to an embodiment of the present invention.

FIG. 2 is a principle diagram of this embodiment.

FIG. 3 is a diagram of a perspective transformation model based on a pinhole camera.

FIG. 4 is a diagram of stereo measurement.

FIG. 5 is a diagram illustrating a case where an optical axis of a camera passes through a known vertex of a rectangular parallelepiped according to the present embodiment.

FIG. 6 is an example of the present embodiment. that's all

Claims (1)

[Claims] 1. From an image showing an object of known size and shape,
In a device for calculating a three-dimensional positional relationship between a camera and an object that obtains the image, a plane that is perpendicular to the optical axis of the camera is assumed in space, and means for projecting a minimum of three reference points onto the plane. Then, based on the positional relationship between the projected point position and the corresponding point on the camera image, the coordinate position is calculated using a computer or the like while scanning the camera focal point position, and the position with the least error is detected. The camera position calculation method is characterized in that the camera position coordinates on the coordinate system of the reference object are obtained by