JP2000283719A - Method and instrument for measuring position of object and recording medium recording program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a position measuring instrument, etc., which can measure the distance and direction to an object to be measured even when the number of actually measured values is few. SOLUTION: A position measuring instrument is provided with a camera section 100 which takes one or a plurality of pictures; a feature point tracking section 200 which detects the coordinate values of one or a plurality of feature points contained in the photographed pictures; and a coordinate matrix generating section 300 which normalizes the coordinate values with the focal distance, calculates the differences between the normalized coordinate values and the mean values of the normalized coordinate values in the pictures calculated at every feature point, and generates a matrix using the calculated difference values as elements. The measuring instrument is also provided with a matrix resolving section 400 which decides certain two matrixes which are the closet to the matrix generated by means of the generating section 300; a normalizing section 500 which multiplies one of the two matrixes by a certain value, adds a fixed value to the product, and divides the other matrix by a certain value so that the calculated value may correspond to actually measured values; and a coordinate calculating section 600 which calculates the distance and direction to each feature point based on the matrixes.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for calculating a distance and a direction to a target object using an image captured by a camera.

[0002]

2. Description of the Related Art Conventionally, an image (2
As a method of extracting three-dimensional information from (dimensions), for example,
Tomita Fumiaki, “Stereo Vision for Intelligent Robots”, 22nd Image Engineering Conference, 1991 (hereinafter referred to as a conventional example).

[0003] In the conventional example, images are taken by a camera from two or more positions, and for a target object appearing in an image taken at each position, a position where a vector directed from the camera to the target object intersects is calculated. The distance and direction to the target object are obtained.

In the case of multi-view measurement, the position of an object is measured from the position and orientation of each camera and the direction of the object viewed from each camera. Here, when there are three or more eyes, two of them
Mainly on the eyes, after the third eye, not the measurement of the object position,
It is mainly used to verify the correspondence of feature points. Also,
In the case of moving vision, the camera is moved in parallel at a known speed at a known speed, and the distance is obtained from the locus of the feature point.

[0005]

However, when the position is measured by the method shown in the conventional example, when measuring with multiple eyes, it is necessary to accurately grasp the orientation and positional relationship of each camera. is there. Even if there are three or more cameras, the three or more cameras are not directly used for measuring the position of the object. In addition, in the case of moving vision, it is necessary to move the camera in parallel. For example, in order to measure the positions of objects distributed over a width of 5 m, a mechanism for moving the camera by a distance of 5 m at a constant speed is required. Become. Therefore, to measure by both methods, many highly accurate measurement values are required.

Therefore, it has been desired to realize a method for measuring the position of an object such that the distance and direction to the target object can be obtained from an image captured by a camera even if the measured values are small.

[0007]

According to a method for measuring the position of an object according to the present invention, one or more images outside the circumference are photographed by a camera while moving on a circumference at a fixed distance from a center of rotation. A step of detecting one or a plurality of feature points included in all of one or a plurality of images, and detecting a coordinate value in a moving direction of each feature point in each image; and a movement of each feature point in each image The coordinate values in the direction are normalized by the focal length based on the camera that has taken the image, and the average of the normalized coordinate values in a plurality of images is calculated for each feature point to calculate the difference from each normalized coordinate value Creating a matrix having the calculated difference values as components; determining two matrices so that the product of the two matrices is the matrix closest to the created matrix; Values and values corresponding to the measured values A value is calculated, one of two matrices is multiplied by a certain value to add a constant value, and the other of the two matrices is divided by a certain value. Calculating a distance and a direction to the feature point.

Further, in the object position measuring method according to the present invention, when all of the detected feature points are not included in all of a plurality of images, an image is measured based on an actually measured value measured in advance. Creating, as a basic group, a combination of a feature point and an image from which the value of the direction of the camera at the time of shooting and the distance and direction from the center to each feature point can be calculated; and For the combination, calculate the value of the direction of the camera when the image was taken and the distance and direction from the center to each feature point,
A step of storing the calculation result, and a combination of a combination of a feature point and an image that can calculate a distance and a direction from the center to each feature point and a value of a camera direction when the image is captured based on the stored calculation result. The process of creating as a group,
Calculating the value of the camera orientation when the image was taken and the distance and direction from the center to each feature point for the combination of the feature points and images included in the chained group, and storing the calculation results; and A chain group is created for the point until the distance and direction from the center to each feature point are calculated, and the value of the camera direction at the time of capturing the image and the distance and direction from the center to each feature point are calculated. And storing the calculation result.

Further, the object position measuring device according to the present invention moves on a circumference at a fixed distance L from the center of rotation,
A camera unit for taking T images outside the circumference, detecting N feature points included in all of the T images, and detecting coordinate values in the moving direction of each feature point i in each image t U _ti (t = 1, 2,..., T, i) in which the coordinate values of the moving direction of each feature point i in each image t are normalized by the focal length based on the camera unit that captured the image. = 1,2,…,
N), and

(Equation 8) And a coordinate matrix creating unit that creates a matrix U ′ having T rows and N columns, and a matrix Θ ″ and a matrix A ′ for a matrix Θ ″ having T rows and one column and a matrix A ′ having one row and N columns Is a matrix decomposer that determines a matrix Θ ″ and a matrix A ′ so as to be regarded as a matrix U ′,
A value q that corresponds to a previously measured actual value
And a constant value θ ₀ , multiplying the matrix Θ ″ by q and adding θ ₀ , and calculating a matrix 成分 having the orientation of the camera unit when capturing T images as a component, and a matrix A ′ Is divided by q to obtain a matrix A, and the components of the matrix Θ calculated by the normalization unit are represented by θ _t, and each component of the matrix A is represented by α _i , and the distance from the rotation center to the feature point a r _i r _i = L /
(1-1 / α _i ), and the direction φ _i from the rotation center to the feature point is

(Equation 9) And a coordinate calculation unit for calculating the coordinates.

Further, the object position measuring apparatus according to the present invention, when not all of the feature points detected by the feature point tracking unit are included in all of the plurality of images, calculates the measured values in advance. Based on this, a combination of a feature point and an image in which the value of the direction of the camera at the time of capturing the image and the distance and direction from the center to each feature point can be calculated as a basic group, a coordinate matrix creating unit, a matrix decomposing unit A normalization unit and a coordinate calculation unit, a basic group creation unit for calculating the value of the direction of the camera when capturing the image and the distance and direction from the center to each feature point, and the calculated camera direction when capturing the image And a position storage unit that stores the distance and direction from the center to each feature point as a calculation result, and based on the calculation result stored in the position storage unit, the direction of the camera when the image is captured. And creating a combination of distance and direction can be calculated feature points and images to each feature point as a chain group from the center, the coordinate matrix creating unit, matrix decomposition unit,
The normalization unit and the coordinate calculation unit calculate the value of the direction of the camera at the time of capturing the image and the distance and direction from the center to each feature point, and store the calculation result in the position storage unit. A chain group creation unit that calculates a value of the direction of the camera at the time of shooting, and creates a chain group until the distance and direction from the center to each feature point are calculated for all feature points.

A recording medium on which a program for the method for measuring the position of an object according to the present invention is recorded is moved on a circle at a fixed distance from the center of rotation, and one or a plurality of images on the outside of the circle are taken by a camera. To detect one or a plurality of feature points included in all of the one or more images, to detect a coordinate value of a moving direction of each feature point in each image, and to detect a moving direction of each feature point in each image. Is normalized by the focal length based on the camera that captured the image, and the average of the normalized coordinate values in a plurality of images is calculated for each feature point to calculate the difference from each normalized coordinate value Then, a matrix having the calculated difference values as components is created, and two matrices are determined so that the product of the two matrices is the matrix closest to the created matrix. A value to correspond to A constant value is calculated, one of two matrices is multiplied by a certain value to add a constant value, and the other of the two matrices is divided by a certain value. And a program for causing a computer to calculate a distance and a direction from the image to each feature point.

[0012] Further, in the recording medium on which the program of the method for measuring the position of an object according to the present invention is recorded, if all the detected feature points are not included in all of the plurality of images, the measurement is performed in advance. Based on the actually measured values, a combination of a feature point and an image in which the value of the direction of the camera at the time of capturing the image and the distance and direction from the center to each feature point can be calculated as a basic group, and are included in the basic group For the combination of the feature point and the image, the value of the direction of the camera when the image was taken and the distance and direction from the center to each feature point are calculated, and the calculation result is stored,
Based on the calculation result, a combination of the value of the direction of the camera at the time of capturing the image and the feature point and the image from which the distance and the direction from the center to each feature point can be calculated are created as a chain group and included in the chain group. For the combination of the feature points and the image, the value of the direction of the camera when the image was taken and the distance and direction from the center to each feature point are calculated, and the calculation results are stored. A chain group is created until the distance and direction to each feature point are calculated, the value of the direction of the camera at the time of capturing the image and the distance and direction from the center to each feature point are calculated, and the calculation result is obtained. A program for causing a computer to store the program is recorded.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 FIG. FIG. 1 shows the first embodiment of the present invention.
It is a block diagram of an apparatus for realizing the position measuring method according to the embodiment. In FIG. 1, 100 is a camera unit, 200 is a feature point tracking unit, 300 is a coordinate matrix creation unit,
400 is a matrix decomposition unit, 500 is a normalization unit, and 600 is a coordinate calculation unit. Before explaining the specific configuration and operation,
The principle of position measurement according to the present embodiment will be described.
In the present embodiment, at least two of the captured images
Calculates the distance and direction from the center of rotation to a feature point (for example, a point that is characterized to represent the shape of a target object) when the camera orientation value is actually measured for a single image It is.

FIG. 2 is a diagram showing an example of an installation state of the camera unit 100. In the present embodiment, as shown in FIG. 2, a camera serving as a photographing part is installed so as to move on a circumference having a constant radius L and photograph the outside of the circumference. Here, in the present embodiment, it is assumed that feature points are captured in all captured images.

FIG. 3 is a diagram for explaining the approximation used in the present embodiment. Here, the feature point is the camera unit 1
From the rotation center 00 intended to be on the circumference of the radius r _i. If the feature point is close enough to the front of the camera,
Even if the feature point is approximated to a point moved by l _i perpendicularly to the extension line from a point at a distance r _i on the extension line of the line connecting the rotation center and the front of the camera, there is no large error. Therefore, this embodiment uses this approximation. The present invention obtains a distance and a direction from a rotation center to a feature point from a plurality of images in different directions of the camera and a small number of actually measured values by devising a camera installation condition and approximation.

FIG. 4 is a diagram showing the relationship between the camera unit 100 and the characteristic points as viewed from above the camera unit 100. In FIG. 4, each relationship between the camera unit and the feature points is represented by a parameter.
When the horizontal position of a feature point on a captured image is normalized by a focal length having a value more specific than the camera, the normalized value is represented by the following equation (1). _{u ti = x ti / z ti} = α i θ t + β i ... (1) _{where, α i = 1 / (1} -L / r i), β i = φ i / (1-
L / r _i)

What is obtained from the actually photographed image is u _ti in the equation (1), and the object is to obtain r _i and φ _i based on this u _ti . Here, i is a number assigned to each feature point. In addition, t is a number given to each of the captured images.

For u _{ti in} equation (1), each feature point (i =
.., N: N is a natural number (t = 1, 2,...)
The average value in (T: T is a natural number) is represented by the following equation (2).
In equation (2), all values are treated equally and calculated by a simple average value, but may be calculated by weighting in some cases.

[0019]

(Equation 10)

When the value of equation (1) is subtracted from the average value calculated by equation (2), the following equation (3) is obtained.

[0021]

[Equation 11]

This is performed for all u _ti , and when expressed by a matrix, it is expressed by U ′ = よ う ′ A as in the following equation (4). Here, since U ′ is calculated only from the positions of the feature points obtained from the actually photographed image, the left side of this equation can be obtained by performing some kind of image processing on the image. Therefore,
Based on U ′ obtained by the image processing, Θ ′ and A are calculated under certain conditions so that Expression (4) holds. Then, theta 'and based on A, it can be the distance r _i and the image to the feature point obtaining and camera orientation when shooting. Further, it is possible to calculate the beta _i from equation (2), it is possible to substitute that beta _i in Equation (1), obtain phi _i.

[0023]

(Equation 12)

Next, a specific configuration of the apparatus shown in FIG. 1 will be described. As shown in FIG. 2, the camera unit 100 is installed so as to photograph outward on the circumference of the radius L, and captures and outputs an image at an arbitrary point on the circumference. The feature point tracking unit 200 includes, for each image captured by the camera 100,
For one or more feature points whose distance is to be obtained,
Search for the coordinates of the camera rotation direction. The feature point tracking unit 200 only needs to be able to associate each feature point on each image, and a specific mark attached to a target object is identified by a generally known method such as pattern matching (pattern matching). You may search. Even when no mark is attached, the search may be performed using a generally known image processing method such as an optical flow.

The coordinate matrix creating unit 300 includes the feature point tracking unit 2
With respect to the feature point i in the image t searched in 00, _uti is calculated by normalizing the horizontal coordinate (coordinate in the camera rotation direction) by the focal length with the image center as the origin. And (3)
U _ti ′ is calculated based on the equation, and a matrix U ′ is calculated. Therefore, assuming that N feature points are searched on each image and T images are captured, the matrix U ′ becomes a matrix of T rows and N columns.

The matrix decomposition section 400 generates a matrix of T rows and 1 column Θ ″
The matrix Θ ″ and the matrix A ′ are calculated such that the product of the matrix '″ and the matrix A ′ of one row and N columns is close to the matrix U ′. And realize it.
When the above approximation is sufficiently established, the singular value of U ′ can be approximated to 0 except for the maximum singular value k. Therefore, the singular value decomposition can be performed as U ′ = Θ “kA”.
Here, the matrix A 'is calculated as A' = kA ", and the matrix Θ"
And the calculation of the matrix A ′.

In the normalizing section 500, when the camera orientation value is actually measured for at least two of the captured images, the difference in the camera orientation becomes closest to the previously measured value. Multiplying the matrix Θ ″ by q and the matrix A ′ by 1 / q,
And matrix A. q does not appear in the matrix U ′ because q applied to the matrix Θ ″ and 1 / q applied to the matrix A ′ cancel each other out. Also, for the matrix カ メ ラ ′, the value of the camera orientation when the image was taken Is added to each θ _t ′, which is a component of Θ ′, by adding a constant value θ ₀ to obtain θ _t (a component of the matrix Θ), where q and θ
₀ is an adjustment value for adjusting so as to reduce the difference between the actually measured value and the calculated matrix component corresponding thereto.
The coordinate calculation unit 600 calculates the distance r _i of each feature point from the camera rotation center based on α _i which is each component of A based on the following equation (5). Further, the position φ _i of each feature point is calculated based on the following equation (6).

[0028]

(Equation 13)

FIG. 5 is a diagram showing the feature points of the target object and the photographing situation. Next, the operation of the apparatus according to the present embodiment will be described based on specific numerical values. The description is made on the assumption that the distance L from the rotation center of the camera to the camera is 0.6 m. The camera section of the camera section 100 is rotated about the camera rotation center as an axis. At this time, the measured values of the distance and direction of each feature point are shown in Table 1 (the purpose of the present embodiment is to calculate a value that approaches this value as much as possible).

[0030]

[Table 1]

The feature point tracking section 200 detects the position of each feature point in each image by a general method such as pattern matching or optical flow. In this case, for each image, the position of the rotation direction of the camera at each feature point (in the installation as shown in FIG. 2, the camera rotates horizontally (horizontally)) is calculated as a value u _ti normalized by the focal length of the camera. Are shown in Table 2 below.

[0032]

[Table 2]

The coordinate matrix creating section 300 calculates an average of coordinate values (distances) derived from each image for each feature point, and derives a matrix in which a difference between the average and coordinate values is calculated. For example, in the case of the feature point 1, the average value of each coordinate from t = 1 to 11 is −0.166. Subtract this from each coordinate,
At t = 1 and i = 1, −0.309 − (− 0.166) =
−0.143, which is a component of the first row and first column of the created matrix. The matrix calculated in this way is shown in the following equation (7).

[0034]

[Equation 14]

The matrix decomposing unit 400 decomposes the matrix of the formula (7) into a matrix Θ ″ and a matrix A ′. Specifically, in the procedure exemplified below, it was approximated that the values other than the largest singular value were 0. First, the singular value decomposition is performed to realize the decomposition.First, the product of the matrix of equation (7) and its transposed matrix is calculated, and the matrix is shown in the following equation (8).

[0036]

(Equation 15)

Next, the maximum eigenvalue of this matrix and the eigenvector corresponding thereto are obtained. This is shown in the following equation (9). Many methods for calculating eigenvalues and eigenvectors are generally known, such as the power method.

[0038]

(Equation 16)

The product of the matrix of the equation (7) and the eigenvector divided by the square root of the eigenvalue is defined as a matrix Θ ″.
The product of the square root of the eigenvalue and the eigenvector is defined as a matrix A ′. The matrix Θ ″ and the matrix A ′ are shown in the following equations (10) and (11).

[0040]

[Equation 17]

Here, for example, when the images at the time t = 1 and t = 11 are photographed, the directions of the cameras are -0.872, respectively.
It is assumed that the measured values are 7 rad and 0.8727 rad. The normalizing unit 500 calculates q based on the following equation (12), multiplies Θ ″ by q to calculate a matrix Θ ′, and multiplies matrix A ′ by 1 / q to calculate matrix A. The components of the calculated matrix Θ ′ and the components of the matrix A are shown in Table 3. q = {0.08727 − (− 0.08727)} / (θ ₁₁ ″ −θ ₁ ″) (12)

[0042]

[Table 3]

Further, usually, θ ₁ ′ is the measured value −0.87
27 rad, and a constant value is added to all θ _t ′ with the adjustment. In this example, θ ₁ ′ becomes θ ₁
Since θ _i ′ already matches, θ _i ′ becomes θ _i as it is.

The coordinate calculation section 600 calculates the distance r ₁ from the camera focal point to the feature point 1 based on the equation (1) as shown in the following equation (13) for the feature point 1, for example. Similarly, based on equation (1), β ₁ = −0.
166-1.648325 × 0 = −0.166, φ ₁ =
-0.166 can be calculated. Table 4 shows the calculation results of the other feature points calculated in this manner. When these values are compared with the measured values in Table 1, it can be seen that the calculated values are close to the measured values. _{r 1 = 0.6 / (1-1 /} 1.648325) = 1.525462 ... (13)

[0045]

[Table 4]

As described above, according to the first embodiment, the camera unit 1 has been devised so as to rotate and move on the circumference at a fixed distance from the center to capture an image outside the circumference.
The feature point tracking unit 20 determines the T images
0 detects N feature points included in all images, detects coordinate values in the rotation direction of each feature point in each image, and sets the coordinate matrix creation unit 300 to the camera unit 100 that has captured those coordinate values. _Is calculated by normalizing with a focal length unique to
Further, for each feature point, the average of u _ti of the T images is calculated, the difference from u _ti is calculated, a matrix U ′ having the calculated difference value as a component is created, and the matrix decomposition unit 400 But T row 1
The matrix Θ ″ and the matrix A ′ are determined so that the product of two matrices of the column matrix Θ ″ and the 1-row N-column matrix A ′ is the matrix closest to the matrix U ′. When the value of the direction of the camera at the time of capturing the image is actually measured, the normalization unit 500 multiplies the matrix q by a value q so as to correspond to the actually measured value to obtain a constant value θ ₀ . In addition, the matrix Θ is calculated, and the matrix Θ represents the value of the direction of the camera at the time of capturing the image.
Is calculated, and the coordinate calculation unit calculates the distance and direction from the center to each feature point based on the matrix A. Therefore, the value of the camera direction when at least two images are captured is calculated. Only by actual measurement, the distance and direction from the rotation center to the feature point and the value of the direction of the camera when each image is photographed can be accurately calculated from the small actual measured values. Further, the matrix decomposing unit 400 approximates singular values other than the largest singular value in the matrix U ′ created by the coordinate matrix creating unit 300 to 0, performs singular value decomposition, and performs a matrix 行 ″ and 1 Since two matrices, that is, the matrix A ′ having N rows and N columns are determined, if the approximation shown in FIG. 3 is sufficiently established, a quick and effective calculation can be performed.

Embodiment 2 In the first embodiment, the value of the direction of the camera when at least two images have been photographed has to be measured. In this embodiment, a description will be given of a position measurement method in a case where the measured value is not the value of the orientation of at least two cameras, but the value of the distance and direction from the rotation center is measured for at least one feature point. .

An apparatus for realizing the position measuring method according to the present embodiment is substantially the same as that shown in FIG. 1 described in the first embodiment. However, this is performed using the normalization unit 501 instead of the normalization unit 500. The normalization unit 500 calculates q based on the orientation of the camera measured when capturing at least two images, but the normalization unit 501 calculates the distance from the rotation center measured for at least one feature point. And normalizing unit 5 in calculating q based on the direction.
Different from 00.

Before describing the specific operation, the principle of the normalizing section 501 of the present embodiment will be described. here,
It is assumed that the distance and the direction of at least one feature point from the rotation center are actually measured. Here, the value of q described in the first embodiment is determined so that the value calculated from the actually measured distance and the corresponding component of the matrix A are the closest. For example, when the position (distance and direction from the rotation center) is measured for one feature point, the ratio of the corresponding component of A ′ to the distance is defined as q. Then, as in the first embodiment, the matrix Θ ″ is multiplied by q to calculate the matrix Θ ′. Further, the matrix A ′ is multiplied by 1 / q to calculate the matrix A.

Also, the following equation (14) is used such that the actually measured direction and the direction calculated by the equation (6) best match.
Then, a constant value θ ₀ is added to each θ _t ′, which is a component of Θ ′, to obtain θ _t . For example, if there is one measured feature point, θ ₀ is calculated as in the following equation (15).

[0051]

(Equation 18)

Next, the operation of the normalizing section 501 which operates differently from the first embodiment will be described based on specific numerical values. Here, the distance r ₄ of feature points 4 based on Table 1 is 2, assumed to be measured with the direction phi ₄ is zero.
In this case, α ₄ = 1 / (1-0.
6/2) = 1.428571. The component of the fourth column in the matrix A ′ calculated by the matrix decomposition unit 400 is 0.
262202, the value of q is q = 0.2622
02 / 1.428571 = 0.183541. Based on the value of q, Θ ″ is multiplied by q to calculate a matrix Θ ′. Matrix A ′ is multiplied by 1 / q to calculate a matrix A. The components of the calculated matrix Θ ′ and the matrix A The components are shown in Table 5.

[0053]

[Table 5]

Next, since φ ₄ = 0, in the first and second terms on the right side of the equation (9), 0 + (− 0.125 + (− 0.
1) + (− 0.075) + (− 0.05) + (− 0.0
25) + 0 + 0.025 + 0.05 + 0.075 + 0.
1 + 0.125) = 0. Also, for the third term, the average value of each component of Θ ′ is 0. Therefore, θ ₀ =
0−0 = 0, and each component θ _t ′ of the matrix Θ ′ becomes each component θ _{t of the} matrix Θ. Table 6 shows the calculation results of the position of each feature point calculated in the present embodiment.

[0055]

[Table 6]

As described above, in the second embodiment, the normalization section 501 devises the camera installation conditions to rotate at least one of the feature points captured by the camera. If the distance and the direction from the center are actually measured, the distance and the direction from the rotation center can be calculated for all the feature points. Therefore, highly accurate position measurement can be performed even with a small measured value. You.

Embodiment 3 In the first embodiment, the direction of the camera at the time of capturing at least two images needs to be actually measured. In this embodiment, the actually measured value is not the value of the orientation of at least two cameras,
A position measurement method in the case where the directions from the rotation center are actually measured as values for at least two feature points will be described.

An apparatus for realizing the position measuring method according to the present embodiment is almost the same as that shown in FIG. 1 described in the first embodiment. However, the process is performed using the normalization unit 502 instead of the normalization unit 500. Although the normalization unit 500 calculates q based on the camera orientation value measured when at least two images are captured, the normalization unit 502
Is different from the normalization unit 500 in that q is calculated based on the value of the direction from the rotation center measured for at least two feature points.

Before giving a specific description of the operation, the principle of the normalizing section 502 of the present embodiment will be described. here,
It is assumed that the directions of at least two feature points from the rotation center are actually measured. Q and θ ₀ are determined such that the sum of squares of the difference between the direction from the rotation center of the feature point calculated based on the equation (6) and the direction of the actually measured feature point is minimized. For example, assuming that the directions from the rotation center are actually measured for two feature points i and j, the following equation (16) is obtained.
Is calculated, and the matrix 及 び ′ and the matrix A are calculated. Here, α _i ′ and α _j ′ are calculated by the matrix decomposition section 400. Based on the component α ₁ of the i-th column of the matrix A obtained by the equation (16), θ ₀ is determined by the above-described equation (15).

[0060]

[Equation 19]

Next, the operation of the normalizing section 502 which operates differently from the first embodiment will be described based on specific numerical values. Here, based on Table 1, the direction φ _{1 of} feature point ₁ =
0.09967, direction φ ₇ of feature point ₇ = 0.09967
It is assumed that it is actually measured. Based on equation (16), q
Is calculated, the following equation (17) is obtained.

[0062]

(Equation 20)

Based on the value of q, a matrix Θ ′ is calculated by multiplying Θ ″ by q. A matrix A is calculated by multiplying matrix A ′ by 1 / q. Table 7 shows the components of the matrix A.

[0064]

[Table 7]

Here, the first component of the matrix A is 1.6
Therefore, when θ ₀ is calculated based on this, θ ₀ = 0.09967 + (− 1.826) / (11)
× 1.665502) = 0, so that each component θ _t ′ of the matrix Θ ′ becomes each component θ _{t of the} matrix Θ. Table 8 shows the calculation result of the position of each feature point calculated in the present embodiment.

[0066]

[Table 8]

As described above, in the third embodiment, the normalization section 502 devises the camera installation conditions so that at least two of the feature points photographed by the camera are obtained.
If the directions from the rotation center are actually measured for one feature point, the distances and directions from all the feature points to the rotation center can be calculated, so that highly accurate position measurement can be performed even with a small measured value.

Embodiment 4 FIG. 6 is a diagram showing a case where the feature points are arranged over a wide range. The first mentioned above
In the third to third embodiments, a case has been described in which all feature points to be measured are included in all images.

FIG. 7 is a diagram showing an example of the relationship between feature points included in each image. Here, consider a case where not all images necessarily include all feature points. For example,
When the feature points are arranged over a wide range, FIG.
In some cases, the feature point is observed (or not included) only at the location indicated by “○”. In the present embodiment, a description will be given of a method of processing so that position measurement can be performed for all feature points even in such a case.

FIG. 8 is a block diagram of an apparatus for realizing the position measuring method according to the fourth embodiment of the present invention.
In the figure, the camera unit 100 and the feature point tracking unit 200 perform the same operations as those described in the first embodiment, and a description thereof will be omitted. Also, a coordinate matrix creation unit 10300
And 20300, matrix decomposition units 10400 and 2040
0, the coordinate calculation units 10600 and 20600 also perform the same operations as the coordinate matrix creation unit 300, the matrix decomposition unit 400, and the coordinate calculation unit 600 based on the input data. Also, normalization units 10500 and 2
0500, the normalizing unit 500, the normalizing unit 501, or the normalizing unit 502 described in the first to third embodiments.
The same operation as is performed. However, here, the normalization unit 20500
Is not the actual measured value, but the coordinate calculation unit 106
00 or 20600, and the position storage unit 900
Matrix A and matrix A based on the camera orientation and the distance and direction from the camera rotation center to the camera stored in
Is calculated.

In the figure, a basic group creation unit 700
In the feature point tracking unit 200, first, among feature points of each image, the first, second, or third embodiment (hereinafter, first to second embodiments)
One combination of an image and a feature point (hereinafter, referred to as a basic group) that can perform the processing described in the third embodiment) is selected and created. For this basic group, a coordinate matrix creation unit 10300 and a matrix decomposition unit 104
00, normalization unit 10500 and coordinate calculation unit 10600
Performs the processing described in the first to third embodiments, and calculates the distance and direction of the feature points included in the basic group from the rotation center, and the direction of the camera of each image. The calculated distance and direction of the feature point from the rotation center and the camera orientation (calculation result) of each image are stored in the position storage unit 900.
Is stored.

The method of selecting a basic group means that, when the camera orientation is actually measured for at least two images, feature points that are commonly included in at least two of the images One or more feature points are selected from. In this method, a combination of data on the selected feature point and an image including all the selected feature points is used as a basic group (in this basic group, processing can be performed by the method described in the first embodiment).

Further, for at least one feature point,
When the distance and the direction from the camera rotation center are actually measured, a plurality of images including at least one certain feature point in common among those feature points are selected. There is also a method in which a combination of data on feature points commonly included in the selected image and all the selected images is used as a basic group (in this basic group, processing is performed by the method described in the second embodiment). it can).

Further, when the directions from the camera rotation center are actually measured for at least two feature points,
From those feature points, a plurality of images that include at least two feature points in common are selected. There is also a method in which a combination of data on feature points commonly included in the selected image and all the selected images is used as a basic group (in this basic group, processing is performed by the method described in the third embodiment). it can). In this selection method, since the number of data used for processing can be increased, for example, the product of the number of images and the number of feature points is maximized (coordinate matrix creation unit 103
It seems that it is common to select the basic group so that the number of components of the matrix created at 00 is the largest)
It is not particularly limited to this.

The linkage group creation unit 800 includes at least one feature point or at least two images of the feature points and images for which the calculation results have already been stored in the location storage unit 900, and the calculation results are still stored in the location storage unit 900. A combination including a feature point or an image for which a result is not stored is selected so that the processing described in the first to third embodiments can be performed, and a combination is created. This chain group is also selected and created by the same method as the above-described basic group selection method. Also in the chain group selection method, since the number of data used for processing can be increased, for example, the product of the number of images and the number of feature points is maximized (the number of components of the matrix created by the coordinate matrix creating unit 20300 is maximized). In general, it is considered that a linkage group is selected, but the present invention is not limited to this.

For this chain group, a coordinate matrix creating section 20300, a matrix decomposing section 20400, a normalizing section 205
00 and the coordinate calculation unit 20600 perform the processing described in the first to third embodiments, and calculate the distance and direction from the rotation center of the feature points included in the chain group, and the direction of the camera of each image. I do. The calculated distance and direction of the feature point from the rotation center and the camera orientation of each image (calculation result) are stored in the position storage unit 900. In the processing from the creation of the linkage group by the linkage group creation unit 800 to the storage of the calculation result of the coordinate calculation unit 20600 in the position storage unit 900, all feature points and images are included in any group (all features). Until the calculation result is calculated for points and images).

For example, among the feature points whose calculation results have already been stored in the position storage unit 900, the feature point at the left end is shared with the basic group or the chain group whose calculation results have already been calculated. Point. Select this feature point and one or more feature points to the left of it,
The combination of a plurality of images including all the selected feature points and data on the selected feature points is processed as a new chain group. This is also performed for the rightmost feature point among the feature points for which the calculation results have already been stored in the position storage unit 900.

Next, the operation of the apparatus of this embodiment will be described based on specific numerical values. Here, the portion indicated by “「 ”in FIG.
0 indicates that each feature point is observed in each image. It is also assumed that the values of the camera orientation when the images 1 and 4 are photographed are actually measured.

On the basis of the image captured by the camera of the camera section 100, the feature point tracking section 200 searches the coordinates of each feature point in the camera rotation direction in each image. First, since the values of the camera direction are actually measured in the images 1 and 4, the basic group creation unit 700 detects feature points that are commonly included in the images 1 and 4. Here, feature point 1, feature point 2, feature point 3, and feature point 4 (hereinafter, referred to as feature points 1 to 4 in such a case) are detected. As images including all the detected feature points 1 to 4, not only image 1 and image 4, but also image 2 and image 3 are detected. The product of the number of images and the number of feature points at this time is 4 × 4 = 16
It is. Here, as the image including the feature points 3 and 4, various combinations such as detection of images 1 to 6 can be selected. However, even in this case, the product of the number of images and the number of feature points is 2 × 6 = 12, and when the number of feature points is 1 to 4, the product of the number of pixels and the number of feature points is the largest. A combination of points 1 to 4 and images 1 to 4 is selected as a basic group.

Based on the data on the feature points 1 to 4 and the images 1 to 4 selected as the basic group, the coordinate matrix creation unit 10300, the matrix decomposition unit 10400, the normalization unit 10500, and the coordinate calculation unit 10600 The processing described in the embodiment is performed. As a result, the distances and directions of the feature points 1 to 4 from the rotation center and the directions of the cameras of the images 1 to 4 are stored in the position storage unit 900 as calculation results. At this point, the position storage unit 900 stores data on the feature points 1 to 4 and the images 1 to 4.

The chain group creation unit 800 first obtains the feature points 1 and 4 at the end of the feature points stored in the position storage unit 900. Here, since no more results can be obtained from the feature point 1, only the feature point 4 is targeted. For example, feature points 4 to 6 are detected from feature points 5 and after including feature point 4, and images 4 to 7 are detected as images including all of them. The product of the number of feature points and the number of images at this time is 3 × 4 = 12. Since the product of the number of feature points and the number of images is the largest among the combinations from the feature points including the feature point 4 and thereafter, this is set as the chain group 1. Here, in order to perform processing based on the third embodiment, a method of selecting feature points 3 to 6 and images 4 to 6 depending on the directions of the feature points 3 and 4 from the rotation center is also conceivable. . Although the calculation result of the new feature point is not large, in order to perform the processing based on the first embodiment,
The feature points 3 to 3 are determined by the values of the images 3 and 4 in the camera direction.
5. A method of selecting images 3 to 7 is also conceivable.

The feature point 4 selected as this chain group
6, a coordinate matrix creating unit 20300, a matrix decomposing unit 20400, a normalizing unit 2
0500 and the coordinate calculation unit 20600 perform the processing described in the second embodiment. The normalization unit 20500 performs a process using the distance and the direction from the center of the feature point 4 stored in the position storage unit 900 as the actually measured values. As a result, the distances and directions of the feature points 5 to 6 from the rotation center and the images 5 to 7
Of the camera of the position storage unit 90 as a new calculation result.
0 is stored. At this point, the data regarding the feature points 1 to 6 and the images 1 to 7 are stored in the position storage unit 900.

Here, as for the feature points 7 to 9 (images 8 and 9), since the calculation results have not been stored yet, the chained group creating unit 800 creates a chained group again. Among the feature points stored in the position storage unit 900, the feature point 6 at the end is obtained. For example, feature points 6 to 9 are detected from feature points 7 and later including feature point 6, and images 7 to 9 are detected as images including all of them. The product of the number of feature points and the number of images at this time is 4 × 3 = 12. Including the feature point 6, in the combination from the subsequent feature points,
Since the product of the number of feature points and the number of images is the largest, this is set as a chain group 2. Based on the data on the feature points 6 to 9 and the images 7 to 9 selected as the chain group, a coordinate matrix creation unit 20300 and a matrix decomposition unit 2040
0, the normalization unit 20500 and the coordinate calculation unit 20600,
The processing described in the second embodiment is performed, and the calculation result is stored in the position storage unit 900. As described above, the distance and the direction from the rotation center of the camera are obtained for all the feature points. In addition, the camera orientation can be obtained for all images.

As described above, in the fourth embodiment, if all the feature points detected by the feature point tracking unit 200 are not included in all the images, the basic group creation unit 700
However, according to the first to third embodiments, a combination of a feature point and an image for which the position of the feature point can be measured is created as a basic group based on actually measured values measured in advance, and a coordinate matrix creation unit 10300 , Matrix decomposition unit 1040
0, the normalization unit 10500 and the coordinate calculation unit 10600 calculate the value of the direction of the camera when capturing the image, and the distance and direction from the center to each feature point as described in the first to third embodiments. Then, it is stored in the position storage unit 900 as a calculation result, and the chain group creation unit 800 can determine the position of the feature point based on the calculation result according to the first to third embodiments. A combination of images is created as a chain group, and the coordinate matrix creating unit 20300, the matrix decomposing unit 20400, the normalizing unit 20500, and the coordinate calculating unit 20600 obtain the values of the camera orientation when the image was captured and the values from the center to each feature point The distance and the direction are calculated as described in the first to third embodiments, and are stored in the position storage unit 900 as the calculation result. Is repeated until the distance and direction are calculated, so that the distance and direction from the rotation center can be calculated for all the feature points even if not all the feature points are included in all the images. , For all images,
The value of the direction of the camera at the time of capturing the image can be calculated.

According to the fourth embodiment, when the measured value is the value of the camera orientation when two or more images are captured, the basic group creation unit 700 A plurality of feature points included in both of the two images whose values are actually measured are selected, and a combination of the selected plurality of feature points and an image including all of the selected plurality of feature points is created as a basic group. Therefore, based on the processing described in the first embodiment, it is possible to calculate the distance and the direction from the rotation center for all the feature points in the basic group. The value of the direction of the camera at the time of capturing the image can be calculated.

According to the fourth embodiment, if the measured value is the value of the distance and direction from the rotation center of at least one feature point, the basic group creation unit 700
Selects multiple images containing the measured feature points,
Since a combination of the feature points included in all of the selected plurality of images and the selected plurality of images is created as a basic group, the basic group is created based on the processing described in the second embodiment. The distance and the direction from the rotation center can be calculated for all the feature points in the group, and the value of the direction of the camera when the image is shot can be calculated for all the images.

Further, according to the fourth embodiment, chained group creation section 800 detects a position-most end point among the characteristic points stored in position storage section 900, and detects the characteristic point. And an image including all the feature points that are not stored in the position storage unit 900, and a combination of the feature points and the selected image is created as a chained group, which will be described in the second embodiment. Based on the above processing, the distance and direction from the rotation center can be calculated for all the feature points in the chain group, and the value of the direction of the camera when the image was taken for all the images. Can be calculated.

According to the fourth embodiment, among the combinations of selected feature points and images, the combination of the number of images and the number of feature points included in the image is determined by the basic group and the chain. Since the data is created as a group, the calculation result can be calculated for the feature points and images in the basic group or the chained group based on a larger number of data.

Embodiment 5 For example, it is assumed that the coordinates in the camera rotation direction of each feature point in each image are searched as shown in Table 9 below. Here, when the directions of the feature point 1 and the feature point 7 from the rotation center of the camera are actually measured, there is no image including the feature point 1 and the feature point 7 at the same time, so that the basic group and the chain group cannot be created. , Fourth
The position measurement based on the embodiment cannot be performed. Therefore, in the present embodiment, after setting a temporary actual measurement value, the processing described in the fourth embodiment is performed, and the calculation result that best matches the actual actual measurement value is finally determined as the calculation result, and the position is calculated. A method for performing the measurement will be described.

[0090]

[Table 9]

FIG. 9 is a block diagram of an apparatus for realizing the position measuring method according to the fifth embodiment of the present invention.
In the figure, a camera unit 100, a feature point tracking unit 200, a basic group creation unit 700, a chain group creation unit 800,
Position storage unit 900, coordinate matrix creation units 10300 and 20
300, matrix decomposition units 10400 and 20400, coordinate calculation units 10600 and 20600, and normalization unit 2050
Since 0 performs the same operation as that described in the fourth embodiment, the description is omitted. The normalization unit 10501 also performs substantially the same operation as the normalization unit 10500, but does not use the measured values but one or a plurality of temporary camera directions assigned to the two images provided from the temporary camera direction providing unit 1000. The difference is that the matrix Θ and the matrix A are calculated based on the values.

The image selection section 1300 selects two images to set a temporary camera orientation. Image selection unit 13
As the image selection method in 00, any method may be used as long as two images that can perform the first embodiment can be selected. For example, the image selected first (first image) is selected as one of the two images. A plurality of feature points are selected from the feature points included in the image, and an image obtained by photographing a position farthest from the first image among images including all the selected feature points is selected as another image. Here, a plurality of feature points are selected such that the product of the selected number of feature points and an image including all of the feature points is maximized.

The provisional camera direction providing unit 1000 calculates one or a plurality of provisional camera direction values for the image selected by the image selection unit 1300 and transmits the values to the normalization unit 10501. The provisional camera direction providing unit 1000 predicts and calculates the provisional camera direction by an appropriate method based on the measured direction of the feature point from the rotation center. For example, for two actually measured feature points, assuming that the direction of the camera of an image of which each feature point is closest to the center coincides with the direction from the actually measured rotation center of the feature point, By extrapolation or interpolation, each feature point is calculated and estimated as being distributed at equal distance intervals. When a plurality of temporary camera directions are transmitted, position measurement processing is performed for each of them, and each calculation result is output.

The measured value comparison unit 1100 compares the measured value of the two feature points in the direction from the rotation center with the calculated value. As a result, if it is determined that a calculation result having an error smaller than the required accuracy has been calculated, or if it is determined that the error is not reduced even if the correction is performed by the temporary camera direction correction unit 1200, the calculation result is set as a final calculation result. . Temporary camera orientation correction unit 120
In the case of 0, a better temporary camera direction is set in the temporary camera direction providing unit 1000 based on the comparison result between the actually measured value in the actually measured value comparison unit 1100 and the calculated value, and the calculation is performed again.
For example, when the directions of the two feature points from the rotation center are actually measured, their values are respectively set as φ _i φ _j , and the calculated values are set as φ _ci and φ _cj . Multiplied by φ _i −φ _j / (φ _ci −φ _cj ) to the direction of the temporary camera from which the value was obtained, and (φ _{j φci} −φ _i φ _cj ) / (φ _i −φ _j ) was added. The object is transmitted to the provisional camera direction providing unit 1000 as a newly corrected provisional camera direction. φ _i −φ _j / (φ _ci −
φ _cj ) represents the ratio of the angle difference between the two camera directions in the measured value and the angle difference between the camera directions in the calculation result. Also,
(Φ _j φ _ci −φ _i φ _cj ) / (φ _i −φ _j ) represents the difference between the measured value and the calculation result of the reference rotation angle for the camera.

Next, the operation of the apparatus according to the present embodiment will be described based on specific numerical values. Here, the description will be given on the assumption that the tracking result of the feature point is as shown in Table 13 because the viewing angle of the camera of the target object in FIG. 5 is narrower than that used in the first embodiment. The feature points whose directions from the rotation center are actually measured are feature points 1 and 7, and the measured values are 0.009967 r, respectively.
ad and -0.009967 rad. Further, the measured value comparison unit 1100 aims at an accuracy of 0.001 rad or less.

On the basis of the image captured by the camera of the camera unit 100, the feature point tracking unit 200 searches for the coordinates of each feature point in the camera rotation direction in each image. Image selection unit 13
00 selects image 1 as one of the two images for which the temporary camera orientation is set. Feature point 4 included in image 1
7 are all images 1 to 6 and the product of the number of feature points and the number of images is 4 × 6 = 24, which is the maximum. Therefore, all the feature points 4 to 7 are included and the image is farthest from image 1. The selected image 6 is selected as the second image.

The provisional camera direction providing section 1000 calculates a value of the provisional camera direction. The image 11 is taken at the feature point 1 closest to the center. Therefore, suppose image 11
Is set to 0.009967. In addition, image 1 is the one where feature point 7 is photographed closest to the center.
Therefore, suppose that the camera direction of image 1 is -0.009967.
And Here, the image 1 selected by the image selection unit 1300 has the temporary camera direction set to −0.009967. The image 6 is calculated by interpolation based on the image 1 and the image 11. Therefore, $ 0.009967-(-0.
0099967)｝ × (6-1) / (11-1) + (−
0.009967) = 0 rad, which is the value of the image 6 for the temporary camera. Temporary camera orientation providing unit 1000
Transmits these values to the normalization unit 10501.

Basic group creating section 700, chained group creating section 800, position storage section 900, coordinate matrix creating section 10
300 and 20300, matrix decomposition units 10400 and 20
400, the normalizing units 10501 and 20500, and the coordinate calculating units 10600 and 20600 perform the same operations as described in the fourth embodiment, and calculate the calculation results for all the feature points and images. At this time, the basic group and the chain group are divided into groups as shown by the dotted lines in Table 9. The result of calculation based on these values is shown in the “first time” row of Table 10. Although the calculation values of the directions from the rotation center were obtained for the feature points 1 and 7 by the calculation, the error was 0.014 for each.
815 and 0.01461, and the measured value comparison unit 1100
Does not reach the target value of 0.001 rad or less.

[0099]

[Table 10]

The temporary camera direction correcting section 1200 calculates the temporary camera direction of the image 1 and the image 6 as −0.009967,0.
{0.009967-(-0.009967)} /
｛0.1144848-(− 0.1142805) =
Multiply by 0.87137. Also, $ 0.1144848
× (-0.009967)-(-0.1142805)
× 0.009967｝ / ｛0.1144848-(-
0.1142805)｝ = − 0.000089 is added. As a result, the provisional camera directions of the corrected image 1 and image 6 are -0.086938 and -0.000, respectively.
089. This value is transmitted to the provisional camera direction providing unit 1000 as a new provisional camera direction, and the position measurement processing is performed again. The result of this calculation is shown in "After correction" in Table 10.
Line. The measurement comparison unit 1100 compares the calculation result with the measurement value. The error of both feature point 1 and feature point 7 is 0.
Since it is less than 001 rad, the process is terminated here, and the calculation result at this time is output as a final calculation result. Table 11 shows the calculation result of the position of each feature point calculated in the present embodiment.
Shown in

[0101]

[Table 11]

As described above, according to the fifth embodiment,
The image selection unit 1300 selects two images, and selects those images.
Calculated by provisional camera orientation providing unit 1000 based on the image
Processed by the normalization unit 10501 based on the temporary camera orientation
Based on the calculated result, the measured value comparison unit 11
00 sets the temporary camera orientation again based on the measured values.
To determine whether to perform the calculation again, and perform the calculation again.
If the camera orientation correction unit 1200 determines that the
Perform the calculation to set the camera orientation again, and then
The most accurate calculation result as the calculation result.
Output, so even if the direction from the center of rotation
Includes at least two feature points measured at the same time
Even if no image exists, the positions of all feature points
Can be measured. Also, because of this,
Feature points do not have to be included in the same screen.
Restrictions on point selection are relaxed. Also for camera
Correction unit 1200 for the temporary camera orientation
(Φ_i −φ_j ) / (Φ_ci−φ _cj) And (φ_jφ_ci−
φ_iφ_cj) / (Φ_i −φ_j ) Is added to the modified temporary
Of the camera, so the more accurate temporary
Can be provided to the normalization unit 10501,
High position measurements can be made.

Embodiment 6 FIG. In the present embodiment, the camera of the camera unit 100 is rotated by about one rotation to obtain about 2πrad (360
Let's consider a case where an image is taken over the same degree). A feature point shot on the first image and the same feature point on another image shot when the camera is rotated about one turn are defined as different feature points separated by 2π from the rotation center of the two feature points. Is the measured value in the direction of. The measured value comparison unit 1100 according to the fifth embodiment performs the position measurement processing according to the fifth embodiment by using the measured values as the measured values to be compared with the calculation results.

FIG. 10 is a block diagram of an apparatus for realizing the position measuring method according to the sixth embodiment of the present invention. In the figure, the same operations as those denoted by the same reference numerals in FIG. 9 are performed except for the same point detection unit 1400 and the actual measurement value holding unit 1500, and a description thereof will be omitted because it has already been described in the fifth embodiment. The same point detection unit 1400 includes, for example, the feature points included in the first few images and the coordinates of the coordinates searched by the feature point tracking unit 200, which are included in the first few images. The same feature point is detected from the feature points included in the image. In this detection, the same feature point may be searched for by simple pattern matching, or the same feature point may be specified by the operator. The measured value holding unit 1500 stores, in a set of feature points detected as the same feature point by the same point detection unit 1400, a measured value in the direction from the rotation center of the feature point included in the first image to 0 rad. And Also,
The actual measured value of the direction from the rotation center of the feature point included in the image after the camera has been rotated about 1 is treated as another feature point as 2πrad, and two actual measured values for comparison with the actual measured value comparison unit 1100 are included. I do.

FIG. 11 is a diagram for explaining the operation of the device according to the sixth embodiment. The operation of the apparatus according to the present embodiment will be described using specific numerical values with reference to FIG. The camera of the camera unit 100 is rotated about one turn, and
It is assumed that an image has been captured over πrad (360 degrees). Here, as shown in FIG. 24, the image at the camera position 1 is referred to as an image 1, and the image at the camera position T is referred to as an image T. It is assumed that feature point 1 is detected as feature point A in image 1 and feature point B in image T.

For example, when the operator designates the feature point A and the feature point B to be the same feature point on the same point detection unit 1400 based on the image 1 and the image T, the same point detection unit 14
00 transmits to the measured value holding unit 1500 data indicating that the feature points A and B are the same feature point. The measured value holding unit 150 sets the measured value in the direction from the rotation center of the feature point A (the feature point 1 in the image 1) to 0 rad. Also, the measured value of the direction of the feature point B (the feature point 1 in the image T) from the rotation center is set to 2πrad, and the feature point B is newly added to the feature point (N
+1) is output to the measured value comparison unit 1100. The measured value comparison unit 1100 that has obtained the measured values of the two feature points in the direction from the rotation center can perform comparison using the measured values. Here, although not particularly precise actual measurement, the fifth
As described in the embodiment, the position measurement processing can be performed based on the temporary camera direction, and the calculation result can be obtained.

As described above, according to the sixth embodiment,
When the camera unit 100 shoots for one round, the same point detection unit detects an image obtained by shooting 0 rad and an image obtained by shooting 2πrad for a certain feature point,
At 500, the measured value of one feature point in the direction from the rotation center is set to 0 rad, and the measured value of the other feature point in the direction from the rotation center is treated as 2πrad as another feature point. , The actual measurement value of the comparison object is set,
Since the directions from the rotation center of the two feature points of rad and 2πrad can be obtained, the distance and the direction from the rotation center can be calculated for all the feature points without particularly requiring precise measurement.

Embodiment 7 FIG. In the first embodiment described above, the normalization unit 500 can calculate q and θ ₀ using the least square sum. For example, in the first embodiment, when the orientation of the camera is actually measured in three or more images, θ ₁ ′ is obtained by a general method such as the least square method.
And θ ₁ by the least square sum so that
Can be determined.

Embodiment 8 FIG. Further, in the above-described second embodiment, the normalization unit 501 can calculate q using the least square sum. For example, when the distance r _i and the direction φ _i from the rotation center are actually measured at two or more feature points, 1/1 is calculated for each actually measured distance r _i .
(1-L / r _i) is calculated to calculate a minimum square sum, which is referred to as q. Further, θ ₀ is obtained by Expression (15).

Embodiment 9 FIG. In the third embodiment, the normalizing unit 502 can calculate q and θ ₀ using the least square sum. For example, when the direction φ _i from the rotation center is actually measured at two or more feature points, the component of the matrix Θ ″ is set to θ _t ″, and the component of the matrix A ′ is set to α
_The following equation (18) is calculated for each feature point as _i ′,
Q and θ ₀ are determined so that the sum of squares is minimized.

[0111]

(Equation 21)

Embodiment 10 FIG. In the above-described embodiment, each unit is provided and each calculation is performed. However, regarding each unit other than the camera unit 100, a computer and a program for causing the computer to perform the operations of the above-described embodiment and a program for the computer are actually used. This is realized by a recording medium for recording.

[0113]

As described above, according to the present invention, one or a plurality of images obtained by rotating and moving on the circumference at a fixed distance from the center and devising conditions so as to capture an image outside the circumference are obtained. For one image, one or more feature points included in all images are detected, the coordinate value of each feature point in each image in the rotation direction is detected, and the coordinate value is normalized by the focal length of the captured camera.
Also, for each feature point, a matrix is created by calculating the average of the normalized values and calculating the difference, and two certain matrices are determined such that the product of the two matrices is closest to the created matrix. Then, a certain value and a certain value are calculated so as to correspond to the actually measured value, a certain value is multiplied by one of the matrices, a certain value is added, and the other matrix is divided by a certain value. , The distance and direction to each feature point are calculated, so that the measured values are small (for example, the distance and direction from the rotation center of one feature point, the direction from the rotation center of two feature points, It is possible to accurately calculate the distances and directions from the rotation center to all the feature points and the camera orientation values at the time of capturing each image.

Further, according to the present invention, even if all the detected feature points are not included in all the images, the positions of the feature points can be measured based on the actually measured values measured in advance. A combination of feature points and images is created as a basic group, the values of the direction of the camera and the distance and direction from the center to each feature point are calculated and stored for the basic group, and the calculation result calculated in the basic group is calculated. A combination of feature points or images included in a part is created as a chain group, and the value of the camera direction and the distance and direction from the center to each feature point are calculated and stored for the chain group, and this is calculated for all the features. As the distance and direction from the rotation center are calculated for the point,
Even if all the feature points are not included in all the images, the distance and the direction from the rotation center can be calculated for all the feature points, and when all the images are photographed, The value of the camera direction can be calculated.

[Brief description of the drawings]

FIG. 1 is a block diagram of an apparatus for realizing a position measuring method according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of an installation state of a camera unit 100.

FIG. 3 is a diagram for describing approximation used in the present embodiment.

FIG. 4 is a diagram illustrating a relationship between the camera unit 100 and feature points as viewed from above the camera unit 100.

FIG. 5 is a diagram illustrating a feature point of a target object and a shooting state.

FIG. 6 is a diagram showing a case where feature points are arranged over a wide range.

FIG. 7 is a diagram illustrating a relationship example of feature points included in each image.

FIG. 8 is a block diagram of an apparatus for realizing a position measuring method according to a fourth embodiment of the present invention.

FIG. 9 is a block diagram of an apparatus for realizing a position measuring method according to a fifth embodiment of the present invention.

FIG. 10 is a block diagram of an apparatus for realizing a position measuring method according to a sixth embodiment of the present invention.

FIG. 11 is a diagram for explaining an operation of the device according to the sixth embodiment.

[Explanation of symbols]

Reference Signs List 100 camera unit 200 feature point tracking unit 300, 10300, 20300 coordinate matrix creation unit 400, 10400, 20400 matrix decomposition unit 500, 501, 502, 10500, 10501, 2
0500 Normalization unit 600, 10600, 20600 Coordinate calculation unit 700 Basic group creation unit 800 Chained group creation unit 900 Position storage unit 1000 Temporary camera direction providing unit 1100 Actual measured value comparison unit 1200 Temporary camera direction correction unit 1300 Image selection unit 1400 Same inspection Output unit 1500 Actual value holding unit

Claims (21)

[Claims] 1. a step of moving on a circumference at a fixed distance from a center of rotation and photographing one or more images outside the circumference with a camera; and including all of the one or more images Detecting one or a plurality of feature points to be detected, and detecting the coordinate value of each feature point in each image in the movement direction, and the coordinate value of each feature point in each image in the movement direction in the image-captured camera. Normalized by the focal length based on, the average of the normalized coordinate values in the plurality of images is calculated for each of the feature points to calculate a difference between the normalized coordinate values, and the calculated difference Creating a matrix having values as components; determining the two matrices so that the product of the two matrices is the matrix closest to the created matrix; and A certain value and a constant value to correspond Multiplying one of the two matrices by a certain value to add a constant value, and dividing the other of the two matrices by the certain value, and based on those matrices, from the center, Calculating a distance and a direction to each of the feature points. 2. When all of the detected feature points are not included in all of the plurality of images, a camera orientation at the time of capturing the image based on the previously measured actual measurement value. Creating a combination of feature points and images from which the values and distances and directions from the center to the feature points can be calculated as a basic group; and for the combination of feature points and images included in the basic group, Calculating the value of the orientation of the camera at the time of shooting and the distance and direction from the center to each of the feature points, and storing the calculation result, based on the stored calculation result, when the image is shot Creating a combination of a feature point and an image for which a value of a camera direction and a distance and a direction from the center to each of the feature points can be calculated as a chain group; For the combination of feature points and images included in the group, calculate the value of the direction of the camera when the image was captured and the distance and direction from the center to each of the feature points, and store the calculation results. Creating a chain group until a distance and a direction from the center to each of the feature points are calculated for the feature points,
A step of calculating a value of a direction of a camera at the time of capturing an image, a distance and a direction from the center to each of the feature points, and storing a calculation result.
An apparatus for measuring the position of an object as described. 3. A camera unit that moves on a circumference at a fixed distance L from a rotation center and captures T (T is a natural number) images outside the circumference, and a camera unit that captures all of the T images. A feature point tracking unit that detects N (N is a natural number) included feature points and detects coordinate values in the moving direction of each feature point i in each image t; U _ti (t = 1, 2,..., T, i = 1, 2,..., N) in which coordinate values in the moving direction are normalized by the focal length based on the camera unit at which the image was taken.
Is calculated, and And a coordinate matrix creating unit that creates a matrix U ′ having T rows and N columns, and a matrix Θ ″ having T rows and 1 column and a matrix A ′ having 1 row and N columns. A matrix decomposer that determines the matrix Θ ″ and the matrix A ′ so that the product of the matrix U ′ can be regarded as the matrix U ′, and a value q that corresponds to a previously measured actual value
And a constant value θ ₀ , multiplying the matrix Θ ″ by the q, and adding the θ ₀ to calculate a matrix と す る with the orientation of the camera unit when the T images have been taken as components. A normalizing unit that divides the matrix A ′ by the q to calculate a matrix A; a component of the matrix Θ calculated by the normalizing unit is θ _t, and each component of the matrix A is α _i , the distance r _i from the rotational center to the feature points calculated by _{r i = L / (1-1 /} α i),
In addition, the direction φ _i from the rotation center to the feature point is given by An object position measuring device, comprising: a coordinate calculating unit that calculates the position of the object. 4. The matrix decomposing unit sets a singular value other than the largest singular value in a matrix created by the coordinate matrix creating unit to 0.
4. An apparatus according to claim 3, wherein the matrix Θ ″ and the matrix A ′ are determined by performing singular value decomposition by approximating. 5. The method according to claim 1, wherein the previously measured actual value is 2
When the values of the orientations of the respective cameras at the time of capturing more than one image are obtained, the normalization unit minimizes the sum of squares of the difference between the measured value and the component of the matrix す る corresponding to the measured value. The object position measuring apparatus according to claim 3, wherein the q and the θ ₀ are calculated so that 6. The method according to claim 1, wherein the previously measured actual value is 2
When the values of the orientations of the two cameras at the time of capturing the two images are taken, the normalization unit calculates the difference between the measured values of the orientations of the two cameras and the measured value of the orientations of the two cameras. The ratio of the difference between the components of the matrix Θ ″ to the corresponding component is defined as q, and any one of the measured values of the directions of the two cameras,
4. The object position measuring apparatus according to claim 3, wherein a difference from a value obtained by multiplying the component of the matrix Θ ″ corresponding to the actually measured value by q is defined as θ ₀ . 7. The method according to claim 6, wherein the previously measured actual value is small.
Distance r from the center of rotation of at least one feature point_i And who
Direction φ_i When the value is the value of 1 / (1
-L / r_i ) Is calculated, and the value of the corresponding matrix A is calculated.
Component α_i Is calculated so that the sum of squares of the difference between
Out, and each measured φ_i Was calculated forEach component when the average of the values of
The difference from the average value of θ ₀ Claims characterized by
Item 3. An apparatus for measuring a position of an object according to Item 3. 8. The method according to claim 1, wherein the previously measured actual value is 1
Distance r from the center of rotation of one feature point_i And direction φ_i The value of the
When it is, the normalization unit calculates 1 / (1-L / r
_i ) Is calculated, and the corresponding component α of the matrix A is calculated._i 
Is calculated as the above q, and each actually measured φ_i 
Was calculated forEach component when the average of the values of
The difference from the average value of θ ₀ Claims characterized by
Item 3. An apparatus for measuring a position of an object according to Item 3. 9. The method according to claim 8, wherein the previously measured actual value is small.
Direction φ from the center of rotation of at least two feature points_i With the value of
If there is, the corresponding component of the matrix Θ ″ is θ_i”
The corresponding component of the matrix A 'is α_i’, Then:All measured φ of_i Minimizes the sum of squares for
So that q and θ ₀ Is characterized by calculating
An object position measuring apparatus according to claim 3. 10. The pre-measured actual value is:
When the values of the directions of the two feature points from the rotation center are φ _i and φ _j , the normalization unit calculates the following components from α _i ′ and α _j ′ corresponding to the components of the matrix A ′. 6] Is calculated based on the components θ _t ′ of the matrix Θ ′ created by multiplying the matrix Θ ″ by the q and the components α _i of the matrix A created by dividing the matrix A ′ by the q. Then, 4. The θ ₀ that satisfies the following is calculated.
An apparatus for measuring the position of an object as described. 11. When all the feature points detected by the feature point tracking unit are not included in all of the plurality of images, when the image is shot based on the previously measured actual value. A combination of a feature point and an image for which the value of the camera direction and the distance and direction from the center to each of the feature points can be calculated are created as a basic group, and the coordinate matrix creation unit, the matrix decomposition unit, and the normalization And a basic group creation unit that calculates a value of the direction of the camera when the image is captured by the unit and the coordinate calculation unit, and a distance and a direction from the center to each of the feature points. A position storage unit that stores, as a calculation result, a value of an orientation and a distance and a direction from the center to each of the feature points; A combination of a feature point and an image in which the value of the direction of the camera when shadowed and the distance and direction from the center to each of the feature points can be calculated are created as a chain group, the coordinate matrix creating unit, the matrix decomposing unit, The normalization unit and the coordinate calculation unit calculate the value of the direction of the camera when capturing the image and the distance and direction from the center to each of the feature points, store the calculation result in the position storage unit, and The value of the direction of the camera at the time of shooting for the image is calculated, and a chain group is created until the distances and directions from the center to the respective feature points are calculated for all the feature points. The object position measuring apparatus according to claim 3, further comprising a unit. 12. The method according to claim 12, wherein the previously measured actual value is:
If the values of the camera orientations at the time of capturing two or more images are the same, the basic group creation unit sets the plurality of images included in both of the two images whose camera orientation values are actually measured. The position of the object according to claim 11, wherein a feature point is selected, and a combination of the selected plurality of feature points and an image including all of the selected plurality of feature points is selected and created as a basic group. measuring device. 13. The pre-measured actual value is:
When the value of the distance and the direction from the rotation center of at least one feature point is the value of the one or more feature points including the value of the distance and the direction from the rotation center which is actually measured. 12. The image processing method according to claim 11, further comprising: selecting a combination of feature points included in all of the plurality of selected images and the selected plurality of images to create a basic group. Object position measuring device. 14. The basic group, wherein, among combinations of selected feature points and images, a group having the largest product of the number of images and the number of feature points included in the image is created as a basic group. 14. The object position measuring apparatus according to claim 12, wherein: 15. The linkage group creation unit detects a feature point located at the end of a position among feature points stored in the position storage unit, and detects the detected feature point and a distance and a distance from the center. An image including all the one or more feature points for which the direction has not been calculated is selected, and the detected image and the selected feature point and one or more feature points for which the distance and direction from the center have not been calculated are selected. 12. The object position measuring apparatus according to claim 11, wherein the combination is selected and created as a chain group. 16. The chain group, wherein, among combinations of selected feature points and images, a group having the largest product of the number of images and the number of feature points included in the image is created as a chain group. The object position measuring apparatus according to claim 15, wherein 17. The method according to claim 16, wherein the previously measured actual value is:
If there is no image that includes at least two feature points in the direction from the rotation center and includes these feature points at the same time, two of the images of the basic group created by the basic group creation unit are An image selection unit to be selected, and 1 to each of the two images selected by the image selection unit.
Or, a plurality of temporary camera directions are set, and a temporary camera direction providing unit that causes the normalization unit to process the actual measurement value, and the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit, and the coordinate calculation unit When the direction from the rotation center is calculated for at least two feature points whose directions from the rotation center are actually measured, the calculated value is compared with the previously measured actual value, and based on the comparison, the temporary value is again determined. An actual measurement value comparison unit that determines whether the coordinate matrix creation unit, the matrix decomposition unit, the normalization unit, and the coordinate calculation unit calculate a calculation result; and the actual measurement value comparison unit If it is determined that the temporary camera direction is set again and the calculation result is calculated, the calculated value and the constant calculation result based on the previously measured actual measurement value are provided to the temporary camera direction. A feed, a position measuring device of an object according to claim 11, characterized in that a temporary camera direction correcting section for setting the camera facing new tentative. 18. The measured value is φ_i , Φ_j And calculated
The direction from the rotation center of the two feature points is φ_ci, Φ
_cj, And (φ_i −φ
_j ) / (Φ_ci−φ_cj) And (φ_jφ_ci−φ_iφ_cj)
/ (Φ_i −φ _j ) Is added to the new temporary camera
18. The position of an object according to claim 17, wherein the position is an orientation.
Position measuring device. 19. In the camera unit, at least 2
When photographing over πrad, based on at least two images, the same point detection unit that detects an image obtained by capturing 0 rad and an image obtained by capturing 2πrad for a certain feature point, The measured value of one feature point in the direction from the rotation center is set to 0 rad, and the measured value of the other feature point in the direction from the rotation center is set to 2πrad as another feature point. 18. The object position measuring apparatus according to claim 17, further comprising an actual measurement value. 20. The camera moves on a circumference at a fixed distance from the center of rotation to cause a camera to capture one or more images outside the circumference, and includes one or more images included in all of the one or more images. Or detecting a plurality of feature points, detecting the coordinate value of each feature point in each image in the movement direction, and calculating the coordinate value of each feature point in each image in the movement direction, a focal length based on the photographed camera. In addition, the average of the normalized coordinate values in the plurality of images is calculated for each of the feature points to calculate a difference from each of the normalized coordinate values, and the calculated difference value is used as a component. The two matrices are determined so that the product of the two matrices is the matrix closest to the matrix created by the coordinate matrix creating unit, and the two matrices correspond to actual measured values measured in advance. A certain value and a constant value Calculation, multiplying one of the two matrices by a certain value to add a certain value, and dividing the other of the two matrices by the certain value, based on those matrices, A recording medium recording a program of an object position measurement method for causing a computer to calculate a distance and a direction from the center to each of the feature points. 21. When all of the detected feature points are not included in all of the plurality of images, a camera orientation at the time of capturing the image based on the previously measured actual measurement value. And a combination of feature points and images from which the distance and the direction from the center to each of the feature points can be calculated are created as a basic group, and for the combination of feature points and images included in the basic group, an image is taken. The value of the direction of the camera and the distance and direction from the center to each of the feature points are calculated, and the calculation result is stored.Based on the calculation result, the direction of the camera when the image is captured is calculated. A combination of a feature point and an image for which a value and a distance and a direction from the center to each of the feature points can be calculated are created as a chain group, and are included in the chain group. For each combination of feature points and images, the value of the direction of the camera when the image was taken and the distance and direction from the center to each of the feature points are calculated, and the calculation results are stored. A chain group is created until a distance and a direction from the center to each of the feature points are calculated, and a value of a camera direction when an image is captured and a distance and a direction from the center to each of the feature points are calculated. Let me calculate
21. A recording medium storing a program for the method for measuring a position of an object according to claim 20, wherein the program causes a computer to store the calculation result.