JPH0610602B2 - Robot 3D measurement system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] The present invention relates to a system for three-dimensionally measuring an object by mapping a plurality of image data taken by a camera having a moving mechanism while translating the image data, and a measuring object of a robot. For the purpose of enabling three-dimensional measurement when an object is a cylindrical body, when the measurement target is a cylindrical body, the calculation means obtains the central axis direction from the contour line of the cylindrical body, A tangential line that is tangent to the cross-sectional circle that cuts the cylinder in a plane that is vertical and passes through the center of the camera is obtained, and the accumulated intersection points that are generated by translating the tangential lines in parallel by the camera's translation The central axis position of the cylindrical body is obtained after extraction.

[Industrial application field]

The present invention relates to a system in which a robot measures an object three-dimensionally, and particularly, the object is three-dimensionally measured by mapping a plurality of image data taken by a camera having a moving mechanism while translating it. It is about the system.

In recent years, the use range of robots has been expanded not only in factories but also in environments where humans cannot enter, such as in nuclear power generation facilities. In order for a robot to work in such a complicated environment, it is necessary to accurately measure the state of the external world and adapt to it as needed. Especially in the nuclear power generation facility, many pipes are installed, and in order for the robot to move and work in it,
The function of measuring the position and orientation of an object in three dimensions is essential.

[Conventional technology]

As a powerful means to realize this environment recognition function,
Various techniques for measuring a three-dimensional shape of an object by using an image input by a camera (for example, a spherical camera) have been proposed so far.

That is, Japanese Patent Application Laid-Open No. Sho 59-184973 discloses the extraction of a straight line segment of an object and measurement of its orientation, and Japanese Patent Application No. 61-258141 discloses the distance between the straight line segment and the extracted straight line segment. It has already been proposed by the applicant. Each principle will be described below.

Extraction of a straight line segment of an object (Japanese Patent Laid-Open No. 59-184973): As shown in FIG.
Is not a plane projection method that projects
A spherical projection method is used to project the to the spherical surface 23. in this case,
If a monocular spherical camera is used, this camera will be located at the center of the spherical surface 23.

In Fig. 15, the points are straight lines (a great circle on the sphere) by spherical projection.
The point P in the three-dimensional space is projected onto the point P ′ on the spherical surface 23 by spherical projection as shown in FIG. Then, as shown in FIG. 3B, the point P ′ is mapped to the straight line l which is a great circle by the spherical mapping. This corresponds to the relation that the point P'is the North Pole and the great circle l is the equator.

The straight line segment is mapped using this, and in FIG.
A straight line L in the dimensional space is spherically projected onto a straight line L ′ on the spherical surface 23, and points P _1, P _2, P _3, ...
_{1 ', P 2', P} 3 ', are projected respectively ... to. Each of these points P ₁ ′, P ₂ ′, P ₃ ′, ... Is mapped by the spherical map to a group of straight lines l _1, l _2, l _{3 ,} . It intersects with the origin O at two symmetrical points (anti-center points). These two points are mathematically one point, and one of them is represented as a mapping accumulation point S. As a result, the information on the straight line L is condensed into one point S by the spherical mapping.

Measurement of orientation of extracted line segment (Japanese Patent Laid-Open No. 59-184973): In order to obtain the orientation of the extracted line segment, as shown in FIG. ), The extracted line segments should be parallel lines P, Q, and R like Zushi. In this case, the parallel lines P, Q, and R are spherically mapped to sequentially obtain the above-mentioned accumulation points S _P, S _{R, and} S _Q, and when they are spherically mapped, the drawn great circle groups intersect at one point SS. The vector direction connecting the intersection SS and the origin O coincides with the directions of the parallel lines P, Q, and R.

Therefore, this direction indicates the direction of the extracted line segment.

Measurement of distance from line segment from which direction is extracted (Japanese Patent Application No. 61-25)
No. 8141): Now, in order to obtain the distance between the line segment L and the camera position C, the first
This will be described with reference to FIG. 8. First, the line segment L is spherically projected to obtain the line segment L ′. If this line segment L'is obtained, its accumulation point S
Can be obtained in the same manner as above.

On the other hand, the azimuth of the line segment L is also required to be parallel to the vector connecting the point C and the pole SS as described above. Therefore, the 90 ° rotation along the great circle l _SS integrated point S located on the great circle l _SS of the pole SS, the point H is required on the line L ', the origin O and the point in this case A line segment L is vertically positioned on an extension of a straight line h connecting H and H.

Therefore, if the intersection of the straight line h and the line segment L is D, the distance between the points D and C corresponds to the distance between the line segment L and the line segment L.

Incidentally, in order to determine the point D, as shown in FIG. 19, the center C of the camera C ^0, C ^1, ... by relatively moving so that C ^i, sight Sh ⁰ respectively obtained, Sh ¹ , ... Sh ⁱ
The intersection point P is obtained, and this intersection point P corresponds to the point where the line segment L is viewed from above. Therefore, the distance between the caramel and the line segment L can be obtained. In obtaining the intersection point P, a generally known peak extraction method (method of multiplying a filter coefficient to raise the peak portion) is used. The applicant of the present invention is concerned with this peak extraction method separately in Japanese Patent Application No. 61-258136 and Japanese Patent Application No. 61-258136.
No. 258138 and Japanese Patent Application No. 61-258139 propose a peak extraction process on a spherical surface.

[Problems to be solved by the invention]

However, even if any of the above-mentioned conventional methods is used, measurement is difficult when the symmetrical object is a cylindrical body.

The reason is that the contour (for example, a vertical line segment) of the cylindrical body viewed from the camera changes due to the movement (horizontal movement) of the camera, so that the distance cannot be measured.

Therefore, according to the present invention, in a system in which a plurality of image data taken while a camera having a moving mechanism is translated is imaged by a calculation means and a symmetrical object is three-dimensionally measured, the measurement object of the robot is a cylindrical body. The purpose is to enable three-dimensional measurement of time.

[Means for solving problems]

FIG. 1 is a block diagram conceptually showing a three-dimensional measuring system for a robot according to the present invention for achieving the above object. 1 is a camera having a moving mechanism, 2 is a computing means, and this computing means 2 is a cylindrical body B with reference to FIG.
The center azimuth SS is obtained from the contour line l _h that is mapped, and the tangent h that is tangent to the cross section circle of the cylindrical body B cut by the plane Ω that is perpendicular to the center axis azimuth SS and passes through the center C of the camera 1 is further generated. A central axis position of the cylindrical body B is obtained by extracting and freezing the accumulated intersection points Q generated by generating a plurality of straight lines h _b obtained by translating each tangent line by the assumed radius length by the translation of the camera 1.

[Work]

In the present invention, if the cylindrical body B is imaged by a camera and the image is spatially differentiated, a contour straight line on the side surface of the cylindrical body can be obtained. L _h shown in FIG. 2 is an example of a contour straight line obtained by imaging the cylindrical body B on the imaging surface Λ of the camera 1.

Here, the central axis azimuth SS of the cylindrical body B is first obtained, and then the plane Ω perpendicular to SS and passing through the camera center C is considered. Let H be the intersection of the plane Ω and the line l _h, and the camera center C and the point H on the plane Ω.
If the line of sight passing through is h, then h is in contact with the cylindrical body B. Since the cross section of the cylindrical body B cut by the plane Ω is a circle, the line of sight h is a tangent line drawn from the camera 1 to the circle.

This also applies to the cylindrical body B obtained by projecting the spatial differentiation processing on the input image shown in FIG. 3, and if the azimuth SS of the central axis of the cylindrical body B is known, then the great circle l
_SS is determined, and the accumulation point S for the contour line l _{h is set} to the great circle l _SS
If the point H rotated by 90 ° along the line is obtained, the line of sight h connecting the center C and the point H becomes the tangent to the circle that is the cross section of the cylindrical body B.

Here, the radius r of the cylindrical body B is set assuming a certain value. Then, an offset line is generated by translating the obtained tangent line to the left and right by the assumed radius length. If the assumed radius is correct, either the left or right offset line should pass through the center Q of the cross section circle.

Therefore, if a plurality of offset lines are generated in the same plane Ω while the camera 1 is translated, they are integrated at the position of the point Q.

Point Q is the foot of the perpendicular line drawn from the camera to the center axis of the cylinder, and if the position is known, the position of the center axis of the cylinder to be measured is uniquely combined with the previous center axis azimuth SS. Determined. The assumed radius length when the point Q is obtained corresponds to the actual radius of the cylinder.

Therefore, when the radius information is known from the beginning, the position / orientation of the cylinder can be measured immediately, and even if the radius is unknown, if several assumed radii are set and the processing is repeated sequentially, an accurate cylinder It is possible to obtain three-dimensional information.

〔Example〕

Embodiments of the robot three-dimensional measurement system according to the present invention will be described below.

(1) Description of system hardware configuration: In FIG. 5, 30 is a microprocessor (hereinafter,
MPU is abbreviated), 31 is a camera (for example, a spherical camera) having a moving mechanism as a three-dimensional positioning device, 3
2 original image memory, 34 is a moving direction measurement data creation unit, 3
5 is a moving direction determination unit, 36 to 3 are mapping processors, 3
The mapping memories 9 to 41 and the parallel interface 42 form the computing means 2.

In the above configuration, the MPU 30 controls the spherical camera 31, the original image memory 32, the moving direction measurement data creation unit 34, the moving direction determination unit 35, and the like. The spherical camera 31 photographs an object and obtains image data. In polar coordinates (r, θ)
Image memory 32 is output in the form of a spherical camera 31
The image data of the post-polar coordinate format that has been output by the above and has been subjected to the contour extraction processing is stored in association with a grid, and has a storage area divided into N pieces in the longitude direction and M pieces in the latitude direction. The moving direction measurement data creation unit 34 sets arbitrary points in the measurement target space as points of interest. For example, in the original image from which contours have been extracted, vertices intersecting a plurality of line segments are set. Set as a point of interest, or find the difference between the original image when multiple light points are irradiated on the measurement target space and the original image when not irradiated, and set the coordinates of the obtained multiple light points as the point of interest. The moving direction determination unit 35 determines the moving direction of the spherical camera 31 based on which coordinate the rotated intersection is stored in the mapping memory 39. The mapping processor 36 stores the original image memory 32. Was done Large circle information is generated for image data of each pixel, and mapping results are sequentially stored in a mapping memory provided inside. A mapping memory 39 is a peak point (intersection point) which is a mapping result in the mapping processor 36. Is transferred and stored, the mapping processor 37 generates great circle information for the pixel having the peak point stored in the mapping memory 39, and sequentially stores the mapping result in the mapping memory provided therein. In the mapping memory 40, the peak point which is the mapping result of the mapping processor 37 is the mapping processor 3
7, which is transferred and stored from the internal memory in the mapping memory 38, the mapping processor 38 generates great circle information for the pixel having the peak point stored in the mapping memory 40 and sequentially transfers it to the mapping memory provided inside. The mapping memory 41 stores the mapping result, the mapping memory 41 stores and transfers the peak points stored in the mapping memory in the mapping processor 38, and the parallel interface 42 stores the mapping memory 3 in the MPU 30 or the like.
9 to 41 are accessed.

(2) Description of operation: (a) Measurement of direction of line segment: First, the spherical camera 31 captures an image of the measurement target space under the control of the MPU 30, and extracts the image data of the contour extracted in polar coordinates (r, θ) format. Output. Then, the MPU 30 writes the image data (coordinate position and density value) from the spherical camera 31 to the original image memory 32 in pixel units.

Next, the mapping processor 36 generates a spherical mapping function (great circle) for the image data of each pixel in the original image memory 32, and stores the peak extraction result in the mapping memory 39.

Therefore, the mapping memory 39 stores the intersections of the great circles generated for the respective points constituting each line segment obtained by the contour extraction, that is, the pole S point of the line segment.

After the spherical mapping function generation process in the mapping processor 36 is completed, the MPU 30 operates a moving mechanism (not shown) mounted on the spherical camera 31 to move the spherical camera 31 by a predetermined pitch.

Then, the above process is executed, and thereafter, the spherical camera 31 is moved linearly (translationally) to capture a predetermined number of frames.

After that, the mapping memory 37 is mapped by the mapping processor 37.
A spherical mapping function is generated for each pixel of each peak point stored in, and the peak extraction result is stored in the mapping memory 40. Therefore, the information indicating the direction of the line segment existing in the measurement target space is stored in the mapping memory 40 as the intersection (SS point).

(B) Measurement of moving direction of camera: FIG. 6 is a diagram showing the basic principle of the moving direction measuring method of Karea, P ₀ ^t , P ₁ ^t , P ₂ ^t ... (t is time series information, t = 0, 1, 2, ...) is an arbitrary point of interest on the open space of the measurement object, and the spherical surface A has points of interest P ₀ ^t , P ₁ ^t , P ₂ ^t
3 shows a spherical memory for recording information of a camera for photographing.

Now, looking at the plurality of points of interest P ₀ ^t , P ₁ ^t , and P ₂ ^t that are stationary while moving the camera in a straight line, it seems that the plurality of points are moving in a straight line in parallel.

Therefore, for example, as shown in FIG. 6, a plurality of respective points are spherically mapped (great circle generation) at each time point, and the results are stored in the spherical memory A (internal memory of the moving direction measurement data creation unit). When going, the map processor 36 obtains the point sequence E _0, Σ _1, Σ _2, ... Of the point sequence Σ of the great circle information intersection Σ and writes it in the mapping memory 39. The mapping processor 37 performs spherical mapping on this Σ point sequence, and when great circle information is written on the spherical memory A, their intersection v is obtained and written on the mapping memory 40.

Here, the vector V connecting the intersection point v and the center of the sphere is parallel to a plurality of straight lines connecting the point sequences projected on the sphere due to the nature of the sphere.

On the other hand, since the plurality of straight lines projected on the spherical memory A and the moving direction of the camera coincide with each other, the moving direction of the camera coincides with this vector V.

Therefore, the three-dimensional direction of the camera can be known from the address of the intersection v on the spherical memory A.

Then, the moving direction determination unit 35 is controlled by the MPU 30 to read the intersection v in the mapping memory 40 and determine the moving direction of the spherical camera 31 corresponding to the read position.

The above-described line segment direction measurement (including line segment integration point measurement) and camera movement direction measurement processing operation are both items that have already been described in the above-described conventional example, and the present invention is based on these known techniques. The technique is used to extract three-dimensional information (center position and radius) of the cylindrical body as described below.

First, the parameters of the cylinder to be measured are re-arranged. The parameters of the cylindrical body to be measured here are five, as shown in FIG. That is, the azimuth SS of the central axis
(Α, β), central axis position Q on a cross section perpendicular to the axis
(Ρ, θ) and radius r. Of these, the present invention relates to a method of measuring the position Q (ρ, θ) of the central axis and the radius r. The azimuth SS (α, β) of the central axis can be obtained by the translational movement of the camera shown in FIG. 8 as in the above-mentioned conventional technique. Also, the moving direction v of the camera can be obtained as described above.

Hereinafter, an embodiment of a method of measuring the position Q of the central axis and the radius r according to the present invention will be described. Incidentally, FIG. 9 shows a schematic flow of the three-dimensional measurement processing of the cylindrical body to which the present invention is applied.

In the embodiment of the present invention, in order to obtain the position of the central axis of the cylindrical body, as shown in FIG. 10, in actual processing, the generation processing on the plane Ω is performed by another spherical surface (hereinafter, distance O) having the origin O. It is projected on a spherical surface O). As shown in the figure, the distance sphere O is a sphere centered on the initial position C ^{0 of the} camera (a sphere onto which a contour image is projected: hereinafter referred to as an imaging sphere) at an appropriate distance along the central axis of the cylindrical body. It can be considered to be in the translated position. Then, if the correspondence between the plane Ω and the sphere O is defined by a projection centered on O, the algorithm on the plane Ω can be equivalently executed on the sphere O. This has the advantage that the measurable depth range can be expanded to infinity.

First, an imaging sphere whose center is the initial position C ⁰ of the camera and whose optical axis direction is the north pole Z is defined. FIG. 11 shows the northern hemisphere of this imaging sphere in a polar coordinate format. The following points are obtained from the spherical mapping on the imaging spherical surface as described above.

_{S (Sα t, Sβ t)} : great circle of electrode including a segment (time was a t) SS (SSα, SSβ) : point represents the axial orientation of the cylindrical v (vα, vβ): a movement direction of the camera Points obtained on the sphere where the coordinates are expressed as (α coordinate, β coordinate).

These points S are stored in the mapping memory 39 shown in FIG.
The SS and the moving direction v are stored in the mapping memory 40.

Next, the distance spherical surface for measuring the position of the cylindrical body is defined in the mapping processor 38 as shown in FIG. As described above, this spherical surface is a spherical surface (distance spherical surface) whose axial direction SS is the north pole.

In the mapping processor 38, the cylindrical body B is further arranged on the distance sphere.
In order to measure the center position of, the point C ^t (Cα ^t , Cβ ^t ) obtained by projecting one of the cameras onto the distance sphere is first calculated.
Among these, in order to obtain Cα ^t of α coordinate, consider a point Lx where a great circle passing through SS and v on the imaging sphere intersects a great circle l _SS having SS as a pole, and considers the great circle l _SS and the equator. The distance ▲ ▼ ≡δv from the intersection point X of _{^{That, Cα t = δv = mod 2π}} (π / 2-ψ) However, Further, Cβ ^t of β coordinate is obtained by the following formula for each time t.

Cβ ^t = tan ⁻¹ (η · sin γ · t) where η = ΔX / R ₀ γ = cos ⁻¹ {cosvβ cosSSβ + sinvβ sinSSβ cos (vα-SSα)} ΔX: camera movement pitch R ₀ : imaging sphere and Distance The distance of the spherical surface The point H ^t (Hα ^t , Hβ ^t ) representing the direction of the tangent line drawn to the cylinder of interest on the plane Ω is also the length along the great circle l _SS on the imaging spherical surface. ▲ ▼ ≡δ _S , ▲ ▼ ≡δ
_It is calculated by the following equation using _h .

The _{^{δ S t = cos -1 {sinSβ}} t sin (Sα t -SSα)} ∴Hα t = δ h t = mod π (δ s t + π / 2) Hβ t = π / 2 the invention, in FIG. 13 Set the radius r as shown,
Draw a great circle that translates each line of sight h ^t by r. For that purpose, first, the tangent pole G ^t (Gα ^t , Gβ ^t ) is obtained from the point H ^t indicating the azimuth of the line of sight and the position C of the camera.

Gaze right and left in parallel moved great circle of pole ^{B ± t (Bα ± t,} Bβ ± t) When each of the operation result is as follows.

Bα ± ^t = Gα ^t However, r: radius of the set cylindrical body R ₀ : distance between the shooting spherical surface and the distance spherical surface Here, by accumulating the spherical surface of B ± ^t while moving the camera, the accumulated intersection point Q (Qα, Qβ)) is obtained. The
The position (ρ, θ) of the center of the cylinder on the plane Ω is obtained by the following formula.

ρ = R ₀ tanQβ, θ = Qα The image pickup processor 38 that has calculated these values stores them in the mapping memory 41.

In this way, the position Q of the central axis of the cylindrical body B and the radius r can be obtained at the same time.

〔The invention's effect〕

As described above, according to the robot three-dimensional measurement system of the present invention, the central axis azimuth is obtained from a plurality of images obtained by moving the camera by using the mapping method, and the center of the camera is measured perpendicularly to this axis. The tangents to the cylinder are obtained by cutting the cylinder that is the object in the plane that passes through, and a plurality of straight lines with each tangent offset by the assumed radius length are generated by translation of the camera to obtain the accumulation intersection. Body position / posture 3
It becomes possible to measure dimensionally.

[Brief description of drawings]

FIG. 1 is a block diagram conceptually showing a three-dimensional measuring system for a robot according to the present invention, and FIG. 2 is a view for explaining a plan projection of a cylindrical body in order to generate a tangent line of the cylindrical body according to the present invention. FIG. 3 is a perspective view for explaining a spherical projection of the cylindrical body in order to generate the tangent line of the cylindrical body according to the present invention, and FIG. 4 is a view for explaining the basic concept of the present invention. 5, FIG. 5 is a hardware configuration diagram of a robot three-dimensional measurement system of the present invention, FIG. 6 is a diagram for explaining the principle of a camera movement direction measurement method, and FIG. 7 is a diagram for explaining parameters of a cylindrical body. Fig. 8 is an explanatory diagram for measuring the central axis azimuth of a cylindrical body, Fig. 9 is a flowchart schematically showing the algorithm of the present invention, and Fig. 10 is a distance spherical surface when the camera is moved. Fig. 11 is a perspective view showing the projected state Is a diagram showing the northern hemisphere of the photographing sphere of the camera in polar coordinates. FIG. 12 is a diagram showing in polar coordinates a distance sphere in which the axial direction of the cylinder is the north pole. FIG. 13 is a tangent line of the cylinder produced by the present invention. Fig. 14 is a diagram showing both left and right parallel lines in polar coordinates, Fig. 14 is a diagram showing a state when an object is projected on a plane and a spherical surface, Fig. 15 is a diagram for explaining a spherical mapping to a point, Fig. 16 Is a diagram for explaining the generation of the accumulation points of the spherically mapped line segments, FIG. 17 is a diagram for explaining the direction of the line segments obtained when the camera is moved in parallel, and FIG. FIG. 19 is an explanatory diagram for obtaining a distance to a camera by spherical mapping, and FIG. 19 is a diagram showing that a target point is extracted by moving the camera. 1 ...... camera, 2 ...... calculating means, B ...... cylinder, S ...... accumulation point, SS ...... central axis orientation, v moving direction of the ...... camera, _{l h} ...... outline, l _SS ...... Univ Circle, Q: central axis position, h: tangent line. In the drawings, the same reference numerals indicate the same or corresponding parts.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yusuke Yasukawa No. 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Fujitsu Limited Examiner, Genji Tabe

Claims (2)

[Claims] 1. A system for three-dimensionally measuring an object by mapping a plurality of image data taken by translating a camera (1) having a moving mechanism by a computing means (2). In the case of a cylindrical body (B), the computing means (2) obtains the central axis azimuth (SS) from the contour line (l _h ) of the cylindrical body (B), and determines the central axis azimuth (SS) perpendicularly to the central axis azimuth (SS). A tangent line (h) tangent to a cross-section circle cut through the cylindrical body (B) at a plane (Ω) passing through the center (C) of the camera is obtained, and a straight line obtained by translating each tangent line by an assumed radius length is taken by the camera. 3 of the robot characterized in that the central axis position and the radius of the cylindrical body (B) are obtained by extracting the accumulated intersection points (Q) generated by generating a plurality by the translation of
Dimensional measurement system. 2. The computing means (2) rotates an integration point (S) obtained from the contour line (l _h ) by 90 ° along a great circle (l _SS ) having a pole at the central axis position. The line connecting the point (H) obtained by the above and the center (C) of the camera is obtained as the tangent line (h) of the cylindrical body (B). Robot 3D measurement system.