JP2018124107A - Measurement device and measurement method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a measurement device and a measurement method capable of accurately measuring using simultaneous localization and mapping (SLAM).SOLUTION: The measurement method includes the steps of: attaching plural markers in which the relationship of the size and/or position is known to a measurement object; sequentially picking up the markers by a camera; carrying out limited-optimization by using the three-dimensional points (marker) the relative positional relationship of which is known in the image coordinates of the plural three-dimensional points which are picked up by the camera; and calculating the three-dimensional coordinates of the plural three-dimensional points.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a measurement apparatus and a measurement method including a camera that photographs a plurality of markers.

  Conventionally, various measuring devices (three-dimensional position measuring devices) and measuring methods (three-dimensional position measuring methods) for performing SLAM (Simultaneous Localization and Mapping) have been proposed (for example, see Patent Document 1). .

A bundle adjustment technique has been used for these measurement apparatuses and measurement methods. As is generally known, the three-dimensional coordinate of a three-dimensional point i is Xi, the three-dimensional position and orientation of the camera j is Pj, and the image coordinates of the three-dimensional point i appearing in the image of the camera j are known. Xij and Pj and Xi that minimize the following equation are calculated when f is a function for projecting the three-dimensional point Xi onto the image of the camera j.

JP 2011-043419 A

  However, there is a need to perform measurement using SLAM as described above with higher accuracy.

  An object of the present invention is to provide a measuring device and a measuring method capable of solving the above-described problems.

A first aspect of the present invention is illustrated in FIG. 1, and includes a camera (j) that sequentially captures a plurality of markers (1) whose sizes and / or positional relationships are known,
The three-dimensional coordinate of the three-dimensional point i is Xi, the three-dimensional coordinate of the three-dimensional point m is Xm, the three-dimensional coordinate of the three-dimensional point n is Xn, the position and orientation of the camera (j) is Pj, and the camera (j ) Where xij is the image coordinate of the three-dimensional point i reflected in (), f is a function for projecting the three-dimensional point Xi onto the image of the camera (j), and the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm When the relative positional relationship is known, e _nm = 1, and when the relative positional relationship is not known, e _nm = 0, and the following equations (1) and (2) are simultaneously limited and optimal Calculation means (6) for calculating Pj and Xi by minimizing in the framework of
It is related with the measuring device (7) characterized by comprising.

  A second aspect of the present invention is illustrated in FIG. 2, and the relative positional relationship is a distance between two of the four corners of the marker (1) having a known size, or When the distance between two markers (1) is known, the distance is between the markers (1).

A third aspect of the present invention is a step of attaching a plurality of markers (1) having a known size and / or positional relationship to a measurement object (2);
Photographing the marker (1) with a camera (j);
The three-dimensional coordinate of the three-dimensional point i is Xi, the three-dimensional coordinate of the three-dimensional point m is Xm, the three-dimensional coordinate of the three-dimensional point n is Xn, the position and orientation of the camera (j) is Pj, and the camera (j ) Where xij is the image coordinate of the three-dimensional point i reflected in (), f is a function for projecting the three-dimensional point Xi onto the image of the camera (j), and the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm When the relative positional relationship is known, e _nm = 1, and when the relative positional relationship is not known, e _nm = 0, and the following equations (3) and (4) are simultaneously limited and optimized. And a step of calculating Pj and Xi by minimizing in the framework of the conversion.

  According to a fourth aspect of the present invention, the relative positional relationship is such that the distance between two of the four corners in the marker (1) having a known size, or the distance between the two markers (1). When it is known, it is a distance between the markers (1).

  Note that the numbers in parentheses are for the sake of convenience indicating the corresponding elements in the drawings, and therefore the present description is not limited to the descriptions on the drawings.

  According to the first to fourth aspects described above, measurement can be performed with higher accuracy than when the bundle adjustment is simply used.

FIG. 1 is a block diagram for explaining a measuring apparatus and a measuring method according to the present invention. FIG. 2 is a plan view illustrating an example of a marker arrangement state. FIG. 3 is a plan view showing an example of the configuration of the marker used in the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 3.

The measuring apparatus according to the present invention is illustrated by reference numeral 7 in FIG.
A camera j that sequentially captures a plurality of markers 1;
A calculating means 6 for calculating the position and orientation Pj of the camera j and the three-dimensional coordinates Xi of the three-dimensional point i;
It has.

Here, the plurality of markers 1 photographed by the camera j are known in size, the positional relationship between the markers is known, or the positional relationship between the size and the markers is known. Further, the calculation means 6 sets the three-dimensional coordinate of the three-dimensional point m to Xm, the three-dimensional coordinate of the three-dimensional point n to Xn, the three-dimensional position and orientation of the camera j as Pj, and the three-dimensional image reflected in the camera j. The image coordinate of the point i is xij, the function of projecting the three-dimensional point Xi onto the image of the camera j is f, the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm , and the relative positional relationship If e _nm = 1 is known, and if the relative positional relationship is not known, e _nm = 0, and the following equations (1) and (2) are simultaneously minimized in a limited optimization framework. Pj and Xi are calculated. In other words, the distance “Xn−Xm” between the two points is originally Cnm, and therefore the accuracy is improved by calculating so that the following expression (2) is minimized.

Here, examples of the relative positional relationship include the following. That is,
When the size of the marker 1 is known, the distance between two of the four corners of the marker 1 (that is, the length of one side of the marker 1 or the length of the diagonal line)
・ When the distance between two markers to be attached is known (for example, when two markers are printed simultaneously on a piece of paper and the distance between the markers is known, or two markers printed on separate papers) Even so, the distance between the markers (for example, when the operator actually measures the distance between the centers of the markers) is the distance between the markers.

The measurement method according to the present invention includes:
Attaching a plurality of markers of known size and / or positional relationship to the measurement object 2;
A step of photographing the marker with a camera;
The three-dimensional coordinate of the three-dimensional point i is Xi, the three-dimensional coordinate of the three-dimensional point m is Xm, the three-dimensional coordinate of the three-dimensional point n is Xn, the position and orientation of the camera j is Pj, and is reflected in the camera j The image coordinate of the three-dimensional point i is xij, the function that projects the three-dimensional point Xi onto the image of the camera j is f, the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm , and the relative When the positional relationship is known, e _nm = 1, and when the relative positional relationship is not known, e _nm = 0, and the following equations (3) and (4) are simultaneously minimized in the framework of limited optimization: And calculating Pj and Xi,
It has.

  That is, in the above equation (4), enm is 1 when the relative positional relationship between the three-dimensional point n and the three-dimensional point m is known, and is 0 when the relative positional relationship is not known. The above equation (4) is designed as a constraint when the relative positional relationship is known. That is, in the measurement method according to the present invention, Pj and Xi are calculated as described above by adding the constraint that the relative positional relationship is known to the above-described equation (3) of the bundle adjustment. .

Here, examples of the relative positional relationship include the following. That is,
When the size of the marker 1 is known, the distance between two of the four corners of the marker 1 (that is, the length of one side of the marker 1 or the length of the diagonal line)
・ When the distance between two markers to be attached is known (for example, when two markers are printed simultaneously on a piece of paper and the distance between the markers is known, or two markers printed on separate papers) Even so, the distance between the markers (for example, when the operator actually measures the distance between the centers of the markers) is the distance between the markers.

  Xi can be obtained from the above equation (1) alone, but according to the present invention, since the above equation (2) is also calculated at the same time, the position and orientation of camera j (that is, Pij in the above equation) The three-dimensional coordinates of the point i can be obtained with high accuracy Xi.

For example, as illustrated in FIG. 2, when two markers 1 each having a length of 20 mm are arranged at a distance of 10,000 mm, the following expression may be added to the constraint.

  As illustrated in FIG. 1, markers 1 having various sizes may be arranged at various locations, and images may be taken from various directions while the arranged markers 1 are moved by a camera j held by hand. Thereby, each marker 1 is recognized on the image photographed by each camera j, and the three-dimensional position of each marker 1 is calculated at any time by SLAM optimization using information on the size of each marker 1.

The marker used in the present invention is, as illustrated by reference numeral 1 in FIG.
A target display area 10 which is an area for displaying a measurement target 10a;
A code image display area 11, 12, 13, 14 which is an area for displaying at least its own ID information (that is, marker identification information capable of identifying the marker 1 from other markers 1) as a code image;
It has. The marker 1 is used by being attached to some measurement object 2, and the target 10a is photographed by a camera (camera for photographing a moving image or a still image) j and the position of the target 10a (two-dimensional position or This is for specifying the position of the measurement object 2 by calculating (three-dimensional position).

  The marker 1 is preferably in the form of a sheet (that is, a flexible thin plate), but is not limited to this, and does not exclude shapes other than the sheet. Moreover, as a method of attaching the marker 1 to the measurement object 2, sticking using an adhesive or the like is preferable, but other known methods other than sticking may be used.

In the example shown in FIG. 3, the target 10a is displayed with a shading pattern (that is, by arranging a plurality of corners of dark parts such as black so as to indicate one point, Although the target 10a is specified at one point), of course, it is not limited to this,
・ Even if it is done by color-coded pattern,
-You can do it with both shading and color-coded patterns, or
-You may make it carry out by the difference in material (for example, arrangement state of a member with high reflectance, and a member with low reflectance).

In the example shown in FIG. 3, the code image is displayed in a shade pattern (that is, a pattern formed by a dark portion and a light portion such as a barcode or QR code (registered trademark)). Of course, but not limited to this,
・ Even if it is done by color-coded pattern,
-You can do it with both shading and color-coded patterns, or
-You may make it carry out by the difference in material (for example, arrangement state of a member with high reflectance, and a member with low reflectance). Further, the code image may carry not only the ID information of the marker itself but also other information (for example, information on the shape, dimensions, etc. of the measurement object 2 to which the marker is attached). Further, in the example shown in FIG. 3, the code image display areas 11, 12, 13, and 14 themselves and the patterns inside thereof have a rectangular shape (that is, a rectangular shape or a square shape). Instead, the shape may be other than a rectangle (for example, a circle).

  By the way, the above-described code image display areas 11, 12, 13, and 14 are preferably arranged at predetermined positions with reference to the target display area 10. That is, the code image display area may be arranged at any position as long as the position is based on the target display area and can be clearly recognized by using the target display area as a reference. In such a case, the display position of the code image display area can be accurately specified by using the target display area as a reference, and the data carried by the code image display area can be read in a short time. Here, as illustrated in FIG. 3, the target display area 10 is formed by a plurality of strip-shaped display areas 10 b and 10 c arranged so as to intersect with each other, and the code image display areas 11, 12, 13, 14 may be disposed at a position sandwiched between a pair of adjacent band-like display areas 10b and 10c. In the example shown in FIG. 3, the code image display areas 11, 12, 13, and 14 are formed in a substantially cross shape by two belt-like display areas 10b and 10c arranged so as to intersect. However, the present invention is not limited to this, and it may be formed in a substantially X shape, or may be formed in another shape (that is, a shape in which three or more strip-shaped display areas intersect). In the example shown in FIG. 3, the code image display areas 11, 12, 13, and 14 are formed at all positions (that is, four parts) sandwiched between the belt-like display areas 10b and 10c, respectively. Of course, the present invention is not limited to this, and it may be formed in at least one portion.

  On the other hand, an operator who attaches the marker 1 to the measurement object 2 can clearly distinguish the target display area 10 and the code image display areas 11, 12, 13, and 14 (hereinafter referred to as “identification marks”). 4) may be arranged at least one of the upper, lower, left and right sides of the target display area 10. When such an identification mark 4 is provided, the operator can attach the marker 1 to the measurement object 2 so that the vertical and horizontal directions are correct. As a result, a measurement device (specifically, a device that analyzes an image taken by the camera j) can accurately specify the display positions of the code image display areas 11, 12, 13, and 14 at high speed. The data carried by the code image display areas 11, 12, 13, and 14 can be read at high speed. In the example shown in FIG. 3, the identification mark 4 can be identified by the operator by the difference in color shading, but of course it is not limited to this, and depending on the color, the shape of the mark itself, etc. You may make it distinguishable.

  On the other hand, auxiliary lines (horizontal lines and vertical lines) 5a, 5b, 5c, and 5d are provided so as to indicate the center of the marker itself (preferably, the position of the target 10a described above) and sandwich the target display area 10. It is good to keep. In such a case, the operator can easily know the horizontal and vertical of the marker itself and the center of the marker, and the operator can place the marker 1 in an appropriate state (that is, appropriate) Position and posture).

  According to the present invention, it is possible to perform measurement with higher accuracy than when simply using bundle adjustment.

1 Marker 6 Calculation means 7 Measuring device j Camera

Claims (4)

A camera that sequentially captures a plurality of markers having a known size and / or positional relationship;
The three-dimensional coordinate of the three-dimensional point i is Xi, the three-dimensional coordinate of the three-dimensional point m is Xm, the three-dimensional coordinate of the three-dimensional point n is Xn, and the position and orientation of the camera j is Pj. The image coordinate of the three-dimensional point i is xij, the function for projecting the three-dimensional point Xi onto the image of the camera j is f, the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm , and the relative position When the relationship is known, e _nm = 1, and when the relative positional relationship is not known, e _nm = 0, and the following equations (1) and (2) are simultaneously minimized in a limited optimization framework. Calculating means for calculating Pj and Xi;
A measuring device comprising:

The relative positional relationship is a distance between two of the four corners of a marker having a known size, or a distance between the markers when the distance between the two markers is known.
The measuring apparatus according to claim 1. Attaching a plurality of markers of known size and / or positional relationship to the measurement object;
Photographing the marker with a camera;
The three-dimensional coordinate of the three-dimensional point i is Xi, the three-dimensional coordinate of the three-dimensional point m is Xm, the three-dimensional coordinate of the three-dimensional point n is Xn, and the position and orientation of the camera j is Pj. The image coordinate of the three-dimensional point i is xij, the function for projecting the three-dimensional point Xi onto the image of the camera j is f, the relative positional relationship between the three-dimensional point n and the three-dimensional point m is C _nm , and the relative position When the relationship is known, e _nm = 1, and when the relative positional relationship is not known, e _nm = 0, and the following equations (3) and (4) are simultaneously minimized by the limited optimization framework. Calculating Pj and Xi,
A measurement method characterized by comprising:

The relative positional relationship is a distance between two of the four corners of a marker having a known size, or a distance between the markers when the distance between the two markers is known.
The measuring method according to claim 3.

Images