JPH0497100A - Detecting method for position and attitude of segment at time of automatic assembly of segment - Google Patents

Abstract

PURPOSE:To improve the control of positioning by arranging a projector projecting slit light and a camera near the segment gripping section of an erector, inputting a picture data to an arithmetic unit and detecting the quality of the deviation of positional attitude. CONSTITUTION:Projectors 18a-18c projecting slit light and TV cameras 19a-19c receiving the slit reflected light are disposed near the segment gripping section of an erector. The upper section of an assembling segment 16 is irradiated with slit light by the projectors 18a-18c, and picture data from the RV cameras 19a-19c including slit light images A-A', B-B', C-C' are input to an arithmetic unit 25. The upper section of the boundary of the assembling segment 16 and an existing segment 17b is irradiated with slit light from the projectors 18a-18c, thus arithmetically operating and outputting the quantity of the deviation of positional attitude. Accordingly, the positioning of the segments can be controlled in a short time.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is designed to position the assembled segment in accordance with the existing segment when automatically assembling tunnel segments using an erector. The present invention relates to a method of detecting the amount of deviation in position and orientation.

[Conventional technology]

Conventional technology in this field includes the Japan Construction Mechanization Association, 1986 Construction Machinery and Construction Methods Symposium Proceedings, p. 2.
As described in ``Development of automatic assembly system for 0 to 23''segment'' and Japanese Patent Application Laid-Open No. 1999-263509, slit light is irradiated onto the boundary between the assembled segment and the existing segment,
The image data from the television camera that receives the slit reflected light is scanned horizontally to detect the address of the brightest point on the scanning line, and by determining its continuity, the slit light image (light cut image) on the screen is created. ), and from the coordinates of the end points of the slit light images on each of the assembled segment side and the existing segment side, a method is known in which the gap and step difference between both segments are determined.

[Problem to be solved by the invention]

In the above conventional technology, when determining the end point coordinates of a slit optical image,
Luminance variation and characteristics of slit light.

Furthermore, there are many unresolved issues regarding the surface condition of the segment, the effects of noise and disconnection caused by ambient light, etc., and there are problems in terms of detection accuracy. For example, when a semiconductor laser, which is less susceptible to disturbance light, is used as a light source, the cross-sectional shape of the laser beam generally has side lobes as shown in Figure 2 (a), which are unidirectionally oriented using a cylindrical lens, etc. The slit light expanded to has a similar site lobe. Further, when a light source such as a normal visible range laser beam or white light is used, noise due to disturbance light (for example, illumination light in a tunnel, welding light, etc.) increases. Furthermore, as an influence of the surface condition of the segment,
If there are scratches or dirt on the surface, it may cause disconnection of the slit light image. As a result, the image actually obtained from the television camera is not the original slit light image shown in No. 31ffi(a).

Generally, it contains a lot of noise information and disconnections as shown in FIGS.

Here, (b) is an example in which a disconnection of the slit light image occurs, and (C
) shows an example in which a noise image was generated due to the side lobe of the laser beam, and (d) shows an example in which both a disconnection of the slit light image and a noise image due to the side lobe were generated.

When detecting the end points of a slit light image using normal image processing technology from an image containing such noise information and disconnections, for example, if the end points of the slit light image to be detected are aQ and bo shown in Fig. 3, then , there is a high possibility that points other than the desired end points, such as al+t)l, ai, and bi shown in FIGS.

An object of the present invention is to detect the amount of deviation in the relative position and orientation of an assembled segment and an existing segment by an optical cutting method.
The object of the present invention is to provide a method that can accurately determine the coordinates of the end points of a slit light image, which serve as the basis for detecting the amount of deviation, without being affected by noise information or disconnections in the video described above.

[Means to solve the problem]

In order to achieve the above object, the present invention disposes a projector that emits slit light near the segment gripping part of the erector and a television camera that receives the reflected light from the slit when automatically assembling tunnel segments using the erector. When the erector grasps the assembled segment, the slit light is irradiated from the projector onto the assembled segment, and the image data from the television camera including the slit light image is input to the arithmetic unit, and the labeling of the binary image and each After detecting the end points of each label image, images that are inherently continuous are determined on the condition that the distance between the label images is within a predetermined range.
From among the images determined to be inherently continuous, the image with the smallest minimum point coordinate in the vertical direction of the screen and the largest maximum point coordinate is extracted, and the minimum point coordinate of that image is used as the assembly segment at this point. First, when the assembled segment moves to the vicinity of the existing segment, the slit light is irradiated from the projector onto the boundary between the assembled segment and the existing segment, and a television camera that includes the slit light image is determined as the end point coordinates of the side slit light image. The image data from is input to the arithmetic unit, and the binary image is labeled again, end points are detected for each labeled image, and images that are originally continuous are determined. Using the end point coordinates of the assembled segment side slit light image obtained when gripping the segment as reference coordinates, extract the image with the vertical minimum point coordinate closest to it, and use the vertical minimum point coordinate of that image as the reference coordinate. Find the end point coordinates of the segment side slit light image, then extract the image with the vertical maximum point coordinates closest to the end point coordinates of this assembled segment side slit light image, and calculate the vertical maximum point coordinates of that image on the existing segment side. The method is characterized in that the end point coordinates of the slit light image are obtained, and the position and orientation deviation amount of the assembled segment with respect to the existing segment is calculated and output based on the end point coordinates of the slit light images on the assembled segment side and the existing segment side obtained at this point. do.

After the assembled segment has moved to the vicinity of the existing segment, if the position/orientation deviation amount of the assembled segment with respect to the existing segment is to be detected twice or more, Instead, the end point coordinates of the assembled segment side slit light image obtained for the second and subsequent times may be determined using the end point coordinates of the assembled segment side slit light image obtained last time as the reference coordinates.

[For production]

In the present invention, when the erector grasps the assembled segment in advance, the end point coordinates (x std + y 5 td) of the slit light image on the assembled segment side shown in FIG. 4(a) are determined, and then the assembled segment is placed near the existing segment. At the point of movement, the end point coordinates (xstd + ys
td), the end point coordinates (x6+yδ
Find L<xp+yp). Originally, the relative position between the gripped segment and the visual device (projector + TV camera) should remain unchanged, but in reality, due to mechanical play in the segment grip, the relative position of the segment and the visual device changes during movement. Since the position may change slightly, (x i5 + y 6) + (x p + y
p), and at that time, the previously obtained (xst
d+ystd) as the reference coordinates to prevent false detection. When detecting the position/orientation deviation amount of the assembled segment with respect to the existing segment is repeated two or more times, the end point coordinates of the assembled segment side slit optical image obtained last time are used as the reference coordinates, and the end points of the assembled segment side slit optical image from the second time onwards are used as reference coordinates. The coordinate detection is also performed for the same reason.

When determining the end point coordinates of the slit light image, the actual image is
Since it contains various noises and disconnections as shown in Figures (b) to (d), after the image data is binarized,
Numbers are assigned to connected regions that are collections of high-brightness areas. This is a process usually called labeling, and each numbered connected region is called a label image. Next, the coordinates of the minimum point and maximum point in the vertical direction (y direction) of the screen are determined for each level image, and based on the minimum point coordinates and maximum point coordinates, images that are inherently continuous are determined. 1. Among the images determined to be originally continuous, when grasping a segment, the image with the smallest minimum point coordinate in the vertical direction and the largest maximum point coordinate is extracted, and after segment movement, the image with the smallest point coordinate in the vertical direction is extracted. An image having the closest vertical minimum point coordinates and an image having the vertical maximum point coordinates closest to the vertical minimum point coordinates are extracted, and the end point coordinates to be finally determined are detected. By doing this, the effects of noise and disconnection are removed,
No more false detections.

[Example 3 Embodiments of the present invention will be described below with reference to the drawings.

The configuration of the apparatus for carrying out the present invention is shown in FIGS. 6 and 7.
As shown in the figure, when the assembled segment 16 is grasped by the erector 26 (FIG. 7) and moved to the vicinity of the existing segments 17a and 1.7b as shown in FIG. .

Projectors 18a, 18b, 18c that irradiate slit light onto the boundary of 1.7b (three points in the illustrated example) and a television camera that captures the respective slit light images AA', BB', and CC'. 19a, ], 9b, and 19c, an image input/output device 22 that inputs and outputs video signals from these television cameras 19a, 19b, and 1.9c, an image memory 24 that stores image data, and an image on the image memory 24. A calculation device 25 that executes image processing on the data and calculates and outputs the position and orientation deviation amount of the assembled segment 16 with respect to the existing segments 17a and 17b based on the processing result, and a calculation device 25 that corresponds to the deviation amount output of the calculation device 25. Erecta 2G to correct the position and orientation of assembly segment 16.
It is equipped with a control device (not shown) that operates the upper jack and the like. Selection of video signals to be taken in from the television cameras 19a, 19b, and 19c is performed by operating the camera switch 21 in response to a signal from the arithmetic unit 25. Further, the status of image processing and the like can be monitored by the image monitor 23.

As shown in FIG. 7, floodlights 18a, 18b, 18
Three sets of visual devices 20a, 20b, and 20c are integrated with television cameras 19a, 19b, and 19c.
It is arranged near the segment gripping part 27 of the erector 26,
Together with the segment gripping portion 27, it is possible to slide in the x, y, and z axes directions in the figure and rotate around the x, y, and z axes.

These sliding movements and rotations are performed by various jacks on the erector 26, thereby making it possible to correct the position and orientation of the gripped segment.

The segment gripping part 27 shown in FIG. 7 consists of a screw shaft that is rotated by a rotation mechanism (not shown), and by screwing this screw shaft into the grout hole 28 of the assembly segment 16 placed below the erector, the assembly segment 16 grasp. Lift up the gripped segment,
Swinging and moving to a predetermined position is the same as a normal erector, and after positioning the segment, automatic assembly of the segment is performed by assembling it to an existing segment using a separately provided bolt fastening device.

In the above configuration, the arithmetic unit 25 performs processing according to the procedure roughly shown in FIG. First, when a segment grasping completion signal is input from the control device, the process proceeds from step 101 to step 102, and a process for determining the end point coordinates (reference coordinates) of the assembled segment side slit light image is executed (details are shown in FIG. 8). ). That is, the television camera 19a~
After selecting 19c, inputting the video, and performing preprocessing such as differential processing, the image data is binarized, and the high brightness portion including the slit light image is displayed with a signal at the it 1 ++ level.

Then, at this stage, only the slit light image on the assembly segment side exists within the field of view of the camera, so an image like the one on the So side shown in FIG. 5 is obtained. When labeling is performed on this image, label images 9 to 15 are obtained, and the first order is
If we calculate the coordinates of the minimum point and maximum point in the
n), is detected as the maximum point (X9X! y9x).

Next, the distances in the X direction and the X direction between each label image are determined, and the continuity of the label images is determined. For example, 1st and i+
The distance in the X direction of the first label image is 3'1x-y(iyal)n, and the distance in the X direction is Xix X(i+l). It is required to close. At that time, yix y(t+++nl
<ΔY1, and l X1xx (,++)n l
If <ΔX+ (ΔXl+ΔY1 is set as, for example, 1 to 2 pixels), the 1st and 1+1st label images are determined to be one continuous image. Next, if we extract the image with the smallest minimum point coordinate and the largest maximum point coordinate in the X direction from among these images that are determined to be inherently continuous, only images 9-10-14 will be obtained. .

Therefore, the coordinates of the end point of the slit light image when gripping the segment are the coordinates of the minimum point in the y direction of 9. That is, Fig. 4 (a
) The end point coordinates (x std + y 5td) are X5
td”X9n+ystd=y9n.

In this way, each television camera 19a, 19b,
Regarding the image of 19c, the slit light image end point coordinates (xstd
+ ystd), the erector replaces the assembled segment 16 with the existing segment 17a,
It is in a standby state until the setting in the vicinity of 17b is completed (step 103).

When the segment predetermined position setting completion signal is input from the control device, the process proceeds to step 104, where the end point coordinates (X'6ryE) r ( XP?y
p) is detected (details are shown in FIG. 9). That is,
From camera selection to determination of the continuity of label images, the same processing as the slit light image end point detection (step 102) when gripping a segment is performed, and a segment is selected from among those determined to be a continuous image. Reference seat I found during gripping
Extract the image with the minimum point coordinate in the y direction closest to I (xstd + ysta), and convert the minimum point coordinate of that image into the end point coordinate (x'
ii+yδ).

In Figure 9, x B 2X N (m) t yδ=Y
It is found as N(m). Next, find the image with the maximum point origin in the y direction closest to this (Xδ+yo), and set the maximum point coordinates of that image to the end point coordinates (X
p+yp). In Figure 9, XX(k), YX
(k) corresponds to this. If there are multiple images with the maximum point azalea in the y direction closest to (Xδ, y5), select the y
An image having the smallest minimum point coordinate in the direction and the largest maximum point coordinate is found, and the maximum point coordinate of that image is set as the end point coordinate (xp ry p ) to be found.

In this way, each television camera 19a, 19b, 1
For the image 9c, the slit light image end point coordinates (X 71
+ )' O) After determining +(xp+yp), the position/orientation deviation amount of the assembled segment with respect to the existing segment is calculated based on each of the coordinate values, and the deviation amount is output to the control device (steps 105 and 106). As shown in Fig. 1, by detecting the slit light images A-A', B-B', and C-C' at three points, the gap between the segments at each point can be determined from the coordinates of the end points on each screen. Since information regarding the step can be obtained, it is possible to determine the amount of deviation in position and orientation of the assembled segment in all directions based on that information, and it is possible to correct the positioning of the assembled segment accordingly.

In step 107, it is determined whether or not a retry is necessary based on the signal from the control device. For example, if the deviation amount output is within the allowable value and a retry is not necessary, the processing of the arithmetic device ends 7, but if a retry is necessary, The processing from step 104 onwards is executed again. In this way, when detecting the position and orientation deviation amount of the assembled segment twice or more, instead of the reference coordinate (xstd + ystd) obtained when gripping the segment, the end point coordinate (x
The detection accuracy is further improved by determining the end point coordinates of the second and subsequent assembly segment side slit light images using the reference coordinates (r, yε).

Next, the flowcharts shown in FIGS. 8 and 9 will be explained.

FIGS. 8(A) and 8(B) are flowcharts of image processing corresponding to step 102 in FIG.
Consists of 209.

In step 201, a television camera to be subjected to image processing is selected.

At 202, an image from a television camera is input.

In step 203, preprocessing such as differential processing is performed on the image data.

In 204, the image data is binarized.

In 205, labeling of the binary image is performed. In this example, the number of label images is n.

In 206, end point coordinates for each label image are determined.

In this case, (X in, 3' in) is the minimum point coordinate in the y direction, and (X iX+'/ix) is the maximum point coordinate in the y direction (where ]=1 to n).

In 207-0 to 207-8, short label images are considered to be noise and removed. That is, 207-1
Find the length ylx-y,n of each label image, and the value is Δ
If it is less than Qy (for example, 5 pixels, etc.), it is considered as noise,
The i-th data is deleted by operations 207-2 to 207-6. Therefore, the number of label images also decreases by one, and 2
As in 07-6, n=n-1.

In steps 208-0 to 208-13, continuity of the remaining label images is determined.

First, let the coordinates of each end point of the i-th label image be XN(i)=
Xin+ Y N(1)=ytn”・Minimum point X
) = XiX + Y X (i) = ytx - maximum point ((20g-2).

Next, the X direction and the
Find the distance in the direction and determine whether it is within the specified range (2
08-4.208-5). If it is within the specified range,
Assuming that the first and j-th images are originally continuous, the coordinate values of the maximum point of the label images are exchanged.

This operation is performed by XX(i)''Xjx+Y This operation is 20
8-7 to 208-10. Here again, the label image data is reduced by one, and n=n-1 due to 208-10. The above operations are executed for all label images.

In steps 209-0 to 209-6, it is determined which image indicates the end point of the assembled segment side slit light image from among the images determined to be originally continuous. This determination is based on the premise that if there is no influence of noise or the like, the end point of the uppermost image (minimum point in the X direction) among the remaining images indicates the end point of the slit light image. For this reason, first
The minimum point in the X direction (XN(1), YN(1)) of the th image is set as the reference point (x sta +ystd) (209-
0). If the minimum point in the X direction is smaller than the first image,
And if there is an image with a large maximum point, then (209-2
, 209-3), and replace the reference point (xstd+ystd> with the latter minimum point in the X direction (209
-4). Perform the above operations on all remaining statues,
Slit light image end point coordinates (xstd+
ystd).

FIG. 9 is a flowchart of image processing corresponding to step 104 in FIG. 1, and consists of steps 301 to 316.

In step 301, only label images that have a length of ΔQy or more and are determined to be inherently continuous are left by the same processing as in FIGS. 8(A) or 8(B). This process leaves only the remaining label image. Through this process, the coordinates of each end point of the remaining label image are X N (i), Y N (i
)...minimum point XX(i), YX(↓)...maximum point (however, 1=1 to n).

In 302, a value larger than the maximum y-coordinate value on the image memory (for example, if the maximum y-coordinate value is 512, at least 513 or more) is substituted as YDEL. In this example, YDEL=10000.

303 to 308, the end point coordinates (xStar
The label image having the X-direction minimum point coordinate closest to yst+) is extracted. As a method, the difference between the X-direction minimum point coordinate value of the label image and yStd is taken, and if the difference is smaller than YDEL, this value is set as YDEL again, and the label number at that time is stored as m (304 to 306 ). Finally, the X-direction minimum point coordinates of the label image closest to -ysLa are determined to be the end point coordinates of the assembled segment side slit light image at this point.

In steps 309 to 315, end point coordinates of the existing segment side slit light image are determined. This process is performed when the image on the existing segment side (Sp side in Figure 5) exists above the screen (lower label number) than the image on the assembled segment side (So side in Figure 5), and the end point coordinates to be found are This is based on the premise that it is closest to the end point coordinates of the slit light image on the assembly segment side.

Therefore, in 309, as in 302, Y D E L =
10000, and the coordinate value of the maximum point in the X direction and YN(m) for the image with label number m-1, which is one before the image with label number m, which indicates the end point of the slit light image on the assembly segment side.
(=y5), and if the difference is smaller than YDEL, set this value again to YDEL, store the label number at that time as k (310 to 313), and finally, YN( The coordinates of the maximum point in the X direction of the label image closest to m) are determined to be the coordinates of the end point of the slit light image on the existing segment side.

The coordinates are determined as Xδ=XN(m), yci=YN(m), and the end point coordinates of the slit light image on the existing segment side are determined as Xp=
Calculate as X X (k), yp = Y X (k).

As mentioned before, when performing the processes 301 to 316 twice or more, from the second time onwards, instead of the end point coordinates (xstd + ystd) obtained in FIG. 8 (A) or C),
The end point coordinates of the assembly segment side slit light image obtained last time (
xiy yo) can be used as the reference coordinates.

〔Effect of the invention〕

According to the present invention, the coordinates of the end points of the slit light image can be determined without error from images containing various noises and disconnections, and the position and orientation deviation amount of the assembled segment with respect to the existing segment can be calculated based on this information. Compared to the prior art, segment positioning control during automatic segment assembly can be performed more accurately and in a shorter time.

[Brief explanation of the drawing]

Figure 1 is a processing flow diagram of the arithmetic unit in the present invention, Figures 2 (a) and (b) are cross-sectional views of the laser beam used as slit light and its horizontal brightness distribution diagram, and Figures 3 to 5. is an input image diagram, FIGS. 6 and 7 are schematic configuration diagrams of an apparatus for carrying out the present invention, and FIGS. 8 and 9 are image processing flowcharts for determining slit light image end point coordinates. 1 to 15... Label image, 16... Assembly segment,
17a. 17b-=existing segment, 18a, 18b, 18
cm floodlight, 19a, 19b, 19cm television camera, 20a, 20b, 20cm visual device, 21...
・Camera switch, 22... Image input/output device, 24...
- Image memory, 25... Arithmetic device, 26... Erector, 27... Segment gripping section, A-A', B-B'
c-c'...Slit light image 6 Patent applicant: Hitachi Construction Machinery Co., Ltd. Representative Patent Attorney Tadashi Akimoto Figure 1 Figure 2 (a) (b) ~OG [D゛\koO) ( a) (c) 1 to 15 Label image (b) Figure (a) (b) Figure 2 Snake 20a to 20c - Visual Kansou 3 - Erecta r - Segment eI Terabe Togeraudshi - Ru

Claims (1)

[Claims] 1. When automatically assembling tunnel segments using the erector, a projector that emits slit light and a television camera that receives the reflected light from the slit are placed near the segment gripping part of the erector, and the erector When the assembly segment is gripped, a slit light is irradiated onto the assembly segment from the projector, and image data from the television camera including the slit light image is inputted to a calculation device to label the binary image and label each label image. After performing endpoint detection for each
Under the condition that the distance between label images is within a predetermined range, images that are originally continuous are determined, and among the images that are determined to be originally continuous, the minimum point coordinate in the vertical direction of the screen is the smallest, The image with the largest maximum point coordinate is extracted, the minimum point coordinate of that image is determined as the end point coordinate of the slit light image on the assembly segment side at this point, and then when the assembly segment moves to the vicinity of the existing segment, the above-mentioned A slit light is irradiated from a floodlight onto the boundary between the assembled segment and the existing segment, and the image data from the TV camera, including the slit light image, is input to the calculation device, and the binary image is labeled again, and the end points for each label image are input. Detection and discrimination of originally continuous images are performed, and from among the images judged to be originally continuous, the coordinates of the end point of the assembled segment side slit light image obtained when gripping the segment are used as reference coordinates, and the vertical direction closest to the reference coordinates is determined. Extract the image with the minimum point coordinates, find the vertical minimum point coordinates of that image as the end point coordinates of the slit light image on the assembled segment side at this point, and then find the coordinates of the end point closest to the end point coordinates of the slit light image on the assembled segment side at this point. Extract the image with the vertical maximum point coordinates, find the vertical maximum point coordinates of that image as the end point coordinates of the slit light image on the existing segment side, and calculate the slits on each of the assembled segment side and the existing segment side found at this point. A method for detecting a segment position and orientation during automatic segment assembly, characterized by calculating and outputting a position and orientation deviation amount of an assembled segment with respect to an existing segment based on end point coordinates of a light image. 2. After the assembled segment moves to the vicinity of the existing segment, if the position/orientation deviation amount of the assembled segment with respect to the existing segment is detected twice or more, the end point of the assembled segment side slit optical image obtained when the segment is gripped. 2. When automatically assembling segments according to claim 1, the end point coordinates of the second and subsequent assembled segment side slit light images are determined using the end point coordinates of the previously determined assembled segment side slit light images as reference coordinates instead of the coordinates. Segment position and orientation detection method.