JPH06101023B2 - Robot 3D measurement system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] A system for three-dimensionally measuring an object by mapping a plurality of image data taken by a camera having a moving mechanism while translating the image data by a calculation means, and a measurement object of a robot. In order to enable three-dimensional measurement when is a cylindrical body, when the measurement target is a cylindrical body, the calculating means obtains the central axis direction from the contour line of the cylindrical body, and is perpendicular to the central axis direction. Then, by obtaining the left and right tangents tangent to the cross-section circle that cuts the cylinder in a plane passing through the center of the camera, and further extracting the accumulative intersections generated by generating a plurality of bisectors of the tangents by the translation of the camera. The configuration is such that the central axis position of the cylindrical body is obtained.

[Industrial application field]

The present invention relates to a system in which a robot measures an object three-dimensionally, and particularly, the object is three-dimensionally measured by mapping a plurality of image data taken by a camera having a moving mechanism while translating it. It is about the system.

In recent years, the use range of robots has been expanded not only in factories but also in environments where humans cannot enter, such as in nuclear power generation facilities. In order for a robot to work in such a complicated environment, it is necessary to accurately measure the state of the external world and adapt to it as needed. Especially in the nuclear power generation facility, many pipes are installed, and in order for the robot to move and work in it,
The function of measuring the position and orientation of an object in three dimensions is essential.

[Conventional technology]

As a powerful means to realize this environment recognition function,
Various techniques for measuring a three-dimensional shape of an object by using an image input by a camera (for example, a spherical camera) have been proposed so far.

That is, Japanese Patent Application Laid-Open No. Sho 59-184973 discloses the extraction of a straight line segment of an object and measurement of its orientation, and Japanese Patent Application No. 61-258141 discloses the distance between the straight line segment and the extracted straight line segment. It has already been proposed by the applicant. Each principle will be described below.

Extraction of straight line segment of object (Japanese Patent Laid-Open No. 59-184973): For this, as shown in FIG. 14, the solid 21 is not projected by the plane projection method in which the external solid 21 is projected on the plane 22. Spherical surface 23
The spherical projection method is used for projecting onto. In this case, if a monocular spherical camera is used, this camera will be located at the center of the spherical surface 23.

FIG. 15 shows an example in which a point is made to correspond to a straight line (a great circle on a spherical surface) by spherical projection, and a point P in the three-dimensional space is on the spherical surface 23 by spherical projection as shown in FIG. It is projected on the point P '. Then, as shown in FIG. 7B, the point P'is mapped to the straight line l which is a great circle by the spherical mapping.
This corresponds to the relation that the point P'is the North Pole and the great circle l is the equator.

This is used to map a straight line segment. In FIG. 16, the straight line L in the three-dimensional space is spherically projected onto the straight line L ′ on the spherical surface 23,
The points P ₁ , P ₂ , P ₃ , ... On the straight line L are replaced by the points P ′ ₁ ,
Project onto P ′ ₂ , P ′ ₃ ,. Each of these points P ′ ₁ ,
P ′ ₂ , P ′ ₃ , ... Are groups of straight lines that are great circles due to spherical mapping
Mapped to l ₁ , l ₂ , l ₃ , ..., these great circles are the origin O
Intersect at two points (anti-center) that are symmetric about. These two points are mathematically one point, and one of them is represented as a mapping accumulation point S. As a result, the information on the straight line L is condensed into one point S by the spherical mapping.

Measurement of orientation of extracted line segment (Japanese Patent Laid-Open No. 59-184973): Now, in order to obtain the orientation of the extracted line segment, as shown in FIG. ), The extracted line segments should be parallel lines P, Q, and R as shown in the figure. In this case, the parallel lines P, Q, R are spherically mapped to obtain the above accumulation points S _P , S _R , S _Q one by one, and when they are spherically mapped, the drawn great circle group is one point.
Meet at SS. The vector direction connecting the intersection SS and the origin O coincides with the directions of the parallel lines P, Q, and R.

Therefore, this direction indicates the direction of the extracted line segment.

Measurement of distance from line segment from which direction is extracted (Japanese Patent Application No. 61-25)
No. 8141 (Japanese Patent Laid-Open No. 63-113782): Now, in order to obtain the distance between the line segment L and the camera position C,
Explaining with reference to the drawing, first, the line segment L is spherically mapped to obtain the line segment L '. If this line segment L'is obtained, the accumulation point S is obtained in the same manner as above.

On the other hand, the azimuth of the line segment L is also required to be parallel to the vector connecting the point C and the pole SS as described above. Therefore,
When the integration point S located on the great circle l _SS of the pole SS rotated 90 ° along a great circle l _SS, H point on the line segment L 'is obtained,
In this case, the line segment L is vertically positioned on the extension line of the straight line h that connects the origin O and the point H.

Therefore, if the intersection of the straight line h and the line segment L is D, the distance between the points D and C corresponds to the distance between the line segment L and the line segment L.

In order to obtain this point D, as shown in FIG. 19, the center C of the camera is relatively moved to C ⁰ , C ¹ , ... C ^{i to} obtain respective lines of sight Sh ⁰ , Sh ¹ obtained. , ... Sh ⁱ 's intersection point P is obtained, and since this intersection point P corresponds to the point when the line segment L is viewed from above, the distance between the camera and the line segment L can be obtained.

In obtaining the intersection point P, a generally known peak extraction method (method of multiplying a filter coefficient to raise the peak portion) is used. The applicant of the present invention is concerned with this peak extraction method separately from Japanese Patent Application No. 61-258136 (Japanese Patent Application Laid-Open No. 63-113777).
), Japanese Patent Application No. 61-258138 (Japanese Patent Application Laid-Open No. 63-113779), and Japanese Patent Application No. 61-258139 (Japanese Patent Application Laid-Open No. 63-113780), which propose peak extraction processing on a spherical surface. There is.

[Problems to be solved by the invention]

However, even if any of the above-mentioned conventional methods is used, measurement is difficult when the object to be measured is a cylindrical body.

The reason is that the contour (for example, a vertical line segment) of the cylindrical body viewed from the camera changes due to the movement (horizontal movement) of the camera, so that the distance cannot be measured.

Therefore, the present invention is a system for three-dimensionally measuring an object by mapping a plurality of image data taken by translating a camera having a moving mechanism by a calculation means, and the object to be measured by the robot is a cylindrical body. The purpose is to enable three-dimensional measurement at a certain time.

[Means for solving problems]

FIG. 1 is a block diagram conceptually showing a three-dimensional measuring system for a robot according to the present invention for achieving the above object. Reference numeral 1 is a camera having a moving mechanism, and 2 is a calculation means. The means 2 is, as shown in FIG.
The central axis azimuth SS is obtained from the contour lines l _R and l _L of the cylindrical body B with the plane Ω perpendicular to the central axis azimuth SS and passing through the center C of the camera 1,
The left and right tangent lines h _R and h _L tangent to the cross-section circle that is cut are obtained, and the intersection point Q generated by generating a plurality of bisectors h _b of both tangent lines by translation of the camera 1 is extracted to obtain a cylindrical body. B
Find the central axis position of.

[Action]

In the present invention, if the cylindrical body B is imaged by a camera and the image is spatially differentiated, a contour straight line on the side surface of the cylindrical body can be obtained. L _R and l _L shown in FIG. 2 are examples of contour straight lines obtained by imaging the cylindrical body B on the imaging surface Λ of the camera 1.

Here, the central axis azimuth SS of the cylindrical body B is first obtained, and then the plane Ω perpendicular to SS and passing through the camera center C is considered. Plane Ω and line
l _R, intersection H each _R with l _L, and H _L, the camera center C and the point H _R on a plane Omega, a straight line passing through the H _L respectively h _R, if h _L, h _R, h _L Contact the cylindrical body B together. Cylinder B is a plane Ω
Since the cross section cut by is a circle, the straight lines h _R and h _L are the left and right tangents drawn from the camera to the circle at the same time.

This also applies to the cylindrical body B, which is obtained by subjecting the input image shown in FIG. 3 to the spatial differentiation processing and spherical projection, and if the azimuth SS of the central axis of the cylindrical body B is known, it is regarded as the pole. The great circle l _SS is determined, and the accumulation point S for the contour lines l _R and l _L
R, point and rotated 90 ° along the S _L to the great circle l _SS H _R, by obtaining the H _L, center C and the point H _R, straight h _R that connects the H _L, h _L columnar body B
It becomes a tangent to the circle that is the cross section of.

Therefore, if the bisector h _b of h _R and h _L is generated, it can be seen that the center Q of the circle which is the cross section of the cylindrical body B is on this bisector h _b .

Then, by repeating the process of generating the above bisector h _b on the same plane Ω while sequentially translating the camera, they are integrated at the position of the point Q. Therefore, as shown in FIG. 4, the center Q of the circle, which is the cross section of the cylindrical body B, can be found by finding the intersection point. Point Q is the foot of the perpendicular line drawn from the camera to the center axis of the cylinder, and if the position is known, the position of the center axis of the cylinder to be measured is uniquely combined with the previous center axis azimuth SS. Determined.

〔Example〕

Embodiments of the robot three-dimensional measurement system according to the present invention will be described below.

(1) Description of system hardware configuration: In FIG. 5, 30 is a microprocessor (hereinafter, MP).
U is abbreviated as U), 31 is a camera (for example, a spherical camera) having a moving mechanism as a three-dimensional positioning device, 32 is an original image memory, 34 is a moving direction measurement data creating section, 35 is a moving direction determining section, and 36- 38 is a mapping processor, 39 to 41 are mapping memories, and 42 is a parallel interface, which form the computing means 2.

In the above configuration, the MPU 30 includes the spherical camera 31, the original image memory 32, the moving direction measurement data creation unit 34, and the moving direction determination unit.
One that controls 35 etc., spherical camera 31 shoots the object,
The obtained image data is output in polar coordinates (r, θ) format, the original image memory 32 is output from the spherical camera 31, and the image data in polar coordinates format after the contour extraction processing is performed is associated with the grid shape. Is stored as N in the longitudinal direction,
The moving direction measurement data creation unit 34 has a storage area divided into M pieces in the latitude direction, and sets a plurality of arbitrary points in the measurement target space as points of interest. In the image, the vertices that intersect a plurality of line segments are set as the points of interest, or the difference between the original image when multiple light spots are irradiated on the measurement target space and the difference between the original image when not irradiated and obtained The movement direction determination unit 35 determines the movement direction of the spherical camera 31 based on which coordinate the intersection developed in the mapping memory 39 is stored. What is determined, the mapping processor 36 generates great circle information for the image data of each pixel stored in the original image memory 32, and the mapping result is sequentially stored in the mapping memory provided inside, the mapping memory
39 is a peak point (intersection point) which is the mapping result in the mapping processor 36 is transferred and stored, and the mapping processor 37 generates great circle information for the pixel having the peak point stored in the mapping memory 39. And a mapping memory 40 that sequentially stores the mapping results in a mapping memory provided internally. The mapping memory 40 stores the peak points that are the mapping results of the mapping processor 37 transferred from the internal memory in the mapping processor 37 and stored therein. Reference numeral 38 is a means for generating great circle information for the pixels having the peak points stored in the mapping memory 40 and sequentially storing the mapping result in a mapping memory provided inside, and the mapping memory 41 is a memory in the mapping processor 38. The peak points stored in the mapping memory are transferred and stored. In the parallel interface 42, the MPU 30 and the like are mapped memories 39 to 41.
To access.

(2) Description of operation: (a) Measurement of direction of line segment: First, the spherical camera 31 captures an image of the measurement target space under the control of the MPU 30 and displays the image data of the contour extracted in polar coordinates (r, θ) format. Output. Then, the MPU 30 writes the image data (coordinate position and density value) from the spherical camera 31 in the original image memory 32 in pixel units.

Next, the mapping processor 36 generates a spherical mapping function (great circle) for the image data of each pixel in the original image memory 32, and stores the peak extraction result in the mapping memory 39.

Therefore, the mapping memory 39 stores the intersections of the great circles generated corresponding to the respective points constituting each line segment obtained by the contour extraction, that is, the pole S point of the line segment.

After the spherical mapping function generation process in the mapping processor 36 is completed, the MPU 30 operates a moving mechanism (not shown) mounted on the spherical camera 31 to move the spherical camera 31 by a predetermined pitch.

Then, the above-mentioned processing is executed, and thereafter, the spherical camera 31 is moved straight (translationally) to photograph a predetermined number of frames.

Thereafter, the mapping processor 37 generates a spherical mapping function for the pixel of each peak point stored in the mapping memory 39, and stores the peak extraction result in the mapping memory 40. Therefore, in the mapping memory 40, information indicating the direction of the line segment existing in the measurement target space is stored as the intersection (SS point).

(B) Camera moving direction measurement: FIG. 6 is a diagram showing the basic principle of the camera moving direction measuring method. P ₀ ^t , P ₁ ^t , P ₂ ^t ... (t is time series information, t =
0,1,2 ...) is an arbitrary point of interest in the measurement target space,
A spherical surface A indicates a spherical memory for recording information of a camera for photographing the points of interest P ₀ ^t , P ₁ ^t , P ₂ ^t ....

Now, when the plurality of points of interest P ₀ ^t , P ₁ ^t , and P ₂ ^t that are stationary while the camera is linearly moved are viewed, it seems that the plurality of points are linearly moved in parallel.

Therefore, for example, as shown in FIG. 6, a plurality of respective points are spherically mapped (great circle generation) at each time point, and the results are stored in the spherical memory A (internal memory of the moving direction measurement data creation unit). When going, the mapping processor 36 obtains a point sequence Σ ₀ , Σ ₁ , Σ ₂ , ... Of the intersection points Σ of great circle information, and the mapping memory 39
Write in. When the mapping processor 37 performs spherical mapping on this Σ point sequence and writes great circle information on the spherical memory A, their intersection v is obtained and written on the mapping memory 40.

Here, the vector V connecting the intersection point v and the center of the sphere is parallel to a plurality of straight lines connecting the point sequences projected on the sphere due to the nature of the sphere.

On the other hand, since the plurality of straight lines projected on the spherical memory A and the moving direction of the camera coincide with each other, the moving direction of the camera coincides with this vector V.

Therefore, the three-dimensional direction of the camera can be known from the address of the intersection v on the spherical memory A.

Then, the moving direction determination unit 35 is controlled by the MPU 30 to read the intersection v in the mapping memory 40, and determine the moving direction of the spherical camera 31 corresponding to the read position.

The above-described line segment direction measurement (including line segment integration point measurement) and camera movement direction measurement processing operation are both items that have already been described in the above-described conventional example, and the present invention is based on these known techniques. The technique is used to extract three-dimensional information (center position and radius) of the cylindrical body as described below.

First, the parameters of the cylinder to be measured are re-arranged. The parameters of the cylindrical body to be measured here are five, as shown in FIG. That is, the azimuth SS of the central axis
(Α, β), central axis position Q on a cross section perpendicular to the axis
(Ρ, θ) and radius r. Of these, the method of measuring the position Q (ρ, θ) of the central axis and the radius r is relevant to the present invention. The azimuth SS (α, β) of the central axis can be obtained by the translational movement of the camera shown in FIG. 8 as in the above-mentioned conventional technique. Also, the moving direction v of the camera can be obtained as described above.

Hereinafter, an embodiment of a method of measuring the position Q of the central axis and the radius r according to the present invention will be described. Incidentally, FIG. 9 shows a schematic flow of the three-dimensional measurement processing of the cylindrical body to which the present invention is applied.

In the embodiment of the present invention, in order to obtain the position of the central axis of the cylindrical body, as shown in FIG. 10, in actual processing, the generation processing on the plane Ω is performed by another spherical surface (hereinafter, distance O) having the origin O. Spherical surface O
Call it). The distance sphere O is, as shown,
It may be considered that a sphere centering on the initial position C ^{0 of the} camera (a sphere onto which a contour image is projected: hereinafter referred to as an imaging sphere) is translated by an appropriate distance along the central axis of the cylinder. it can. Then, the correspondence between the plane Ω and the spherical surface O is O
If it is defined by a projection centered at, the algorithm on the plane Ω can be equivalently executed on the sphere O. This has the advantage that the measurable depth range can be expanded to infinity.

First, an imaging sphere whose center is the initial position C ⁰ of the camera and whose optical axis direction is the north pole Z is defined. Figure 11 shows the northern hemisphere of this photographic sphere in polar coordinates. By the spherical mapping on this photographing spherical surface, the following points are obtained as described above.

S (S _α ^t , S _β ^t ): Pole of great circle including line segment (time is t) SS (SS _α , SS _β ): Point representing axial direction of cylinder v (v α, v β): Camera The moving azimuth of the point is obtained on the spherical surface, where the coordinates are represented by (α coordinate, β coordinate).

These points S are stored in the mapping memory 39 shown in FIG.
And the moving direction v are stored in the mapping memory 40.

Next, the range sphere for measuring the position of the cylinder is defined in the mapping processor 38 as shown in FIG. As described above, this spherical surface is a spherical surface (distance spherical surface) whose axial direction SS is the north pole.

The mapping processor 38 first calculates a point C ^t (C _α ^t , C _β ^t ) obtained by projecting the position of the camera on the distance sphere in order to measure the center position of the cylindrical body B on the distance sphere. Among these, in order to obtain C _α ^{t of} α coordinate, on the imaging spherical surface,
Considering the point L _X where the great circle passing through SS and v intersects with the great circle l _SS having the SS pole, the distance between the great circle l _SS and the intersection X of the equator ▲
▼ ≡ δv should be calculated. That is, C _α ^t = δv = mod _2π (π / 2−ψ) where Further, C _β ^{t of} β coordinate is obtained by the following formula for each time t.

C _β ^t = tan ⁻¹ (η · sin γ · t) where η = ΔX / R ₀ γ = cos ⁻¹ {cosvβ cosSSβ + sinvβ sinSSβ cos (v
α-SSα)} ΔX: Moving pitch of camera R ₀ : Distance between imaging spherical surface and distance spherical surface H ^t (H _α ^t , H) that represents the direction of the tangent line drawn to the cylinder of interest on the plane Ω. _β ^t ) is also the length along the great circle l _SS on the imaging sphere ▲ ▼ ≡δ _S , ▲ ▼ ≡δ _h
Is calculated by the following formula.

_{^{δ S t = cos -1 {sinSβ}} t sin (Sα t -SSα)} = Mod _π (δ _S ^t + π / 2) Hβ ^t = π / 2 In the present invention, from the two tangent lines h _R ^t and h _L ^{t at} different times,
As shown in Fig. 13, find the bisector h _b ^t . For that purpose, first, a point H _b ^t (H _b α ^t , π / 2) indicating the direction of the bisector is obtained from two points H _R ^t and H _L ^t indicating the directions of both tangent lines.

The great circle passing through C ^t and H _b ^t representing the camera position is the desired bisector h _b ^t , and its pole B ^t (Bα ^t , Bα ^t ) can be calculated by the following formula.

^{Bβ t = | cot -1 {tanCβ} t + sin | H b α t -Cα t |}
| Here, the cumulative intersection Q (Qα, Qβ) is obtained by spherically mapping B ^t while moving the camera, and the position (ρ, θ) of the cylinder center on the plane Ω is calculated by the following equation. Can be obtained with.

ρ = R ₀ tanQβ, θ = Qα The mapping processor 38 that calculated these values is a mapping memory.
Store in 41.

If the position Q of the central axis of the cylindrical body B is obtained in this way,
It is back-projected on the original plane Ω, the distance between the position Q and each tangent line is calculated on the plane Ω, and the histogram thereof is created.
Since the distance at which the frequency is maximum is the radius, the radius of the cylinder can be obtained by detecting it.

〔The invention's effect〕

As described above, according to the robot three-dimensional measurement system of the present invention, the central axis azimuth is obtained from a plurality of images obtained by moving the camera by using the mapping method, and the center of the camera is measured perpendicularly to this axis. The tangent to the cylinder is obtained by cutting the cylinder, which is the target object, in the plane that passes through, and a plurality of bisectors of the tangent are generated to obtain the intersection point, so the position / orientation of the cylinder is 3 It becomes possible to measure dimensionally.

[Brief description of drawings]

FIG. 1 is a block diagram conceptually showing a three-dimensional measuring system for a robot according to the present invention, and FIG. 2 is a view for explaining a plan projection of a cylindrical body in order to generate a tangent line of the cylindrical body according to the present invention. FIG. 3 is a perspective view for explaining a spherical projection of the cylindrical body in order to generate the tangent line of the cylindrical body according to the present invention, and FIG. 4 is a view for explaining the basic concept of the present invention. 5, FIG. 5 is a hardware configuration diagram of a robot three-dimensional measurement system of the present invention, FIG. 6 is a diagram for explaining the principle of a camera movement direction measurement method, and FIG. 7 is a diagram for explaining parameters of a cylindrical body. Fig. 8 is an explanatory view for measuring the central axis direction of a cylindrical body, Fig. 9 is a flow chart schematically showing the algorithm of the present invention, and Fig. 10 is a distance spherical surface when the camera is moved. Fig. 11 is a perspective view showing the projected state. Fig. 12 is a diagram showing the northern hemisphere of the photographic sphere of La in polar coordinates, Fig. 12 is a diagram showing in polar coordinates a distance sphere where the axial direction of the cylinder is the north pole, and Fig. 13 is a tangent line of the cylinder produced by the present invention. Diagram showing bisectors in polar coordinates, Fig. 14 is a diagram showing a state when an object is subjected to plane projection and spherical projection, Fig. 15 is a diagram for explaining a spherical mapping to a point, and Fig. 16 is FIG. 17 is a diagram for explaining the generation of the accumulation points of spherically mapped line segments, FIG. 17 is a diagram for explaining the azimuth of the line segments obtained when the camera is translated, and FIG. FIG. 19 is an explanatory diagram for mapping and obtaining the distance to the camera, and FIG. 19 is a diagram showing that the target point is extracted by moving the camera. 1 ...... camera, 2 ...... calculating means, B ...... cylinder, S ...... accumulation point, SS ...... central axis orientation, v moving direction l _R of ...... camera, l _L ...... outline, l _SS ... … Great circle, Q …… Center axis position, h _R , h _L …… Tangential line, H _R , H _L …… Point indicating the azimuth of the tangent line. In the drawings, the same reference numerals indicate the same or corresponding parts.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yusuke Yasukawa 1015 Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa Fujitsu Limited Examiner Kohei Hirooka

Claims (3)

[Claims] 1. A system for recognizing an object three-dimensionally by mapping a plurality of image data taken by a camera (1) having a moving mechanism while translating the image data by a calculation means (2). When it is a cylindrical body (B), the calculation means (2)
Is obtained from the contour line (l _R , l _L ) of the cylindrical body (B), and a plane (perpendicular to the center axis azimuth (SS) and passing through the center (C) of the camera ( Ω) is used to find the left and right tangents (h _R , h _L ) tangent to the cross-section circle that cuts the cylindrical body (B), and a plurality of bisectors (h _b ) of the tangents are generated by translation of the camera. A three-dimensional measurement system for a robot, characterized in that a central axis position of the cylindrical body (B) is obtained by extracting an accumulated intersection (Q) caused by the above. 2. An accumulation point (S _R , S) obtained from said contour line (l _R , l _L ) by said computing means (2) along a great circle (l _SS ) having the center axis position as a pole. _A line connecting the point (H _R , H _L ) obtained by rotating _L ) by 90 ° and the center (C) of the camera is obtained as a tangent to the cylindrical body (B). 3 of the robot described in the first section
Dimensional measurement system. 3. The calculation means (2) obtains the radius of the cylindrical body (B) by taking a histogram of the distance between the central axis position and each tangent line. The robot three-dimensional measurement system according to item 1.