JP3055649B2 - Image system for remote construction support - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote construction support image system, and more particularly to a remote construction work using a binocular stereoscopic superimposed image obtained by superimposing a pair of design images representing a design shape and a pair of operation images representing the current state. The present invention relates to an image system that supports

[0002]

2. Description of the Related Art With the advance of control technology and the like, when working in an area where entry of persons is prohibited or an area where entry of persons is difficult, a work machine (hereinafter referred to as a heavy machine) is required from a sufficiently distant operation room. May be performed in the construction area by remote control. For example, in the construction area 11 as shown in FIG. 2, the heavy equipment 2 is remotely operated while monitoring the video or image (hereinafter, referred to as an image) of the construction object 1, and
It is actually practiced to remotely carry out earth works and the like for the construction.

[0003]

In the conventional general earth work, a rope indicating the design shape of the construction object 1 is installed in the construction area 11 as a so-called tenter, and the work is carried out using the tenter as a guide. . However, it is extremely difficult to install a tenter in the construction area 11 where remote construction is performed, and in many cases, construction cannot be performed while comparing with a design shape only with the image of the construction object 1 alone. Therefore, the work by remote control has a problem that the work efficiency is reduced and the work quality tends to be inferior to the work using the conventional tenting.

Accordingly, it is an object of the present invention to provide an image system for remote construction in which a three-dimensional superimposed image of an image of a construction area by a camera and a perspective view of a design shape is used instead of tenting in remote construction.

[0005]

Referring to the embodiment of FIG. 1, the remote construction support image system of the present invention
Draw the blueprint I _g of 3D computer graphics can output perspective view I _p of any optical axis from an arbitrary viewpoint with respect to construction area 11 that contains the three-dimensional coordinates in the construction zone 11 and construction target 1 is known Construction with position / posture measuring device with multiple targets 8
, Taken from different predetermined photographing direction by the pair of imaging device 5 mounted on a predetermined position to create a pair of three-dimensional operation image I _wL and I _wR on use heavy machinery, the three-dimensional computer graphic on blueprints I _g draw each optotype 8, a pair of right and left perspective view of the optical axis of the photographing direction of the view point the shooting position of the imaging device 5 from the blueprint I _g that plot the target 8 and the imaging device 5
Outputs I _pL and I _pR, corresponding optotype 8 on the left perspective I _pL and right perspective image and a left three-dimensional operation image I of each optotype 8 on I _{pR wL} and right three-dimensional operation image I _wR Each perspective view I so that the image of
_pL and I _pR are scaled or rotated to create a pair of left and right perspective design images I _dL and I _dR , a left perspective design image I _dL and a right perspective design image I _dR , a left three-dimensional operation image I _wL and a right tertiary it is made to create a pair of left and right binocular vision stereo superimposed image I _sL and I _sR by superimposing the original operation image I _wR respectively.

[0006]

[Action] The operation of the present invention is applied to the case where the slope of section No. 12 in the construction area 11 such as the residential land development area shown in FIG. explain. Construction object 1 shown in FIG. 1 represents the slope of section No. 12 before completion. The following equipment is mounted on the heavy equipment 2 shown in FIGS.

[0007] (1) A pair of imaging devices 5: The construction object 1 is photographed. The operator in the operation room 3 performs a remote operation while monitoring the output image of the imaging device 5. Both imaging devices 5 are installed in a stereo system so that a pair of output images becomes a binocular stereoscopic image. That is, when the heavy equipment 2 is horizontal, the respective imaging devices 5 are fixed at a pair of predetermined horizontal positions on the heavy equipment 2 in a predetermined direction, and the optical conditions of the two imaging devices 5 are the same. (2) Satellite surveying device 22: For example, GPS (Global Positionin)
g System) is used to measure the coordinates of the heavy equipment 2 on the earth's surface. (3) Attitude measurement device 23: Measures the azimuth and inclination of heavy equipment 2. (4) Transmitting devices 6 and 24: image transmitting device 6 of the output image of image capturing device 5 and transmitting device 24 of the coordinate position and shooting direction of each image capturing device 5
including.

In the remote control room 3 shown in FIG.
, 24, a storage means 18 for storing a three-dimensional computer graphic design drawing (hereinafter sometimes referred to as a graphic design drawing) _Ig of the construction area 11, and an operation display 14. Remote monitoring means. For the graphic design diagram _Ig , for example, the coordinates of necessary points in the construction area 11 are input as relative coordinates with respect to the reference point 4 (point in section No. 16 in FIG. 2) by three-dimensional CAD belonging to the prior art, and It can be created by determining the three-dimensional coordinates of the reference point 4 with respect to the ground. Since the three-dimensional coordinates after completion of each point in the construction zone 11 by the graphic design diagram I _g is determined, it is determined and an arbitrary viewpoint and the optical axis in the coordinate system of the graphics design diagram I _g, construction area after completion A perspective view _Ip for 11 can be output. Reference numeral 19 of FIG. 1 shows a perspective view output means for outputting a perspective I _p from the graphic design diagram I _g based on viewpoint and optical axis are specified, one example of which performs coordinate calculations and graphic processing of the three-dimensional computer graphic It is a graphic computer.

The construction area 11 is controlled by the pair of imaging machines 5 of the heavy equipment 2.
A pair of left and right operation images I _wL and I _wR including a construction target 1 in the inside and a plurality of targets 8 whose three-dimensional coordinates in the construction area 11 are known. An example of the operation image I _wR is shown in FIG. The target 8 can be attached to the heavy equipment 2 as shown in the example of FIG. 5, for example. In this case, the three-dimensional coordinates of the heavy equipment 2 with respect to the ground measured by the satellite surveying device 22 and the mounting position of the target 8 on the heavy equipment 2 , The three-dimensional coordinates of each target 8 are determined. However, the target 8 used in the present invention is not limited to the embodiment shown in FIG.

[0010] The operation images I _wL and I _wR taken by each image pickup device 5 are transmitted by the image transmitter 6 to the image receiver 10 in the operation room 3. Further, the photographing position of each image pickup device 5 is calculated from the measured value of the satellite surveying device 22 and the mounting position on the heavy equipment 2, and the posture measuring device is used.
The imaging direction of each imaging device 5 is calculated from the measured value of 23 and the mounting posture on the heavy equipment 2, and the calculated imaging position and orientation are transmitted from the transmission device 24 to the reception device 26 of the operation room 3. However, only the measured values of the satellite surveying device 22 and the attitude measuring device 23 may be transmitted from the transmitting device 24, and the photographing position and the orientation of each imaging device 5 may be calculated based on the measured values received in the operation room 3. When the optotypes 8 are attached to the heavy equipment 2 as shown in FIG. 5, the three-dimensional coordinates of each optotype 8 are also transmitted from the transmitting device 24 to the receiving device 26.
Transmit to

The three-dimensional coordinates of the target 8 and the photographing position and orientation of each imaging device 5 are input to the perspective view output means 19 of the operation room 3.
The perspective view output means 19 is a graphic design drawing I _g of the storage means 18.
Reading, plotting plotting the target 8 by the three-dimensional computer graphics on a graphical design diagram I _g based on the three-dimensional coordinates inputted. Then from the graphic design diagram I _g that plot the target 8, and the viewpoint of the photographing position of the image pickup device 5 and the pair of left and right perspective I _pL of the photographing direction to the optical axis
And create an I _pR . An example of the perspective view _IpR is shown in FIG. As can be seen from FIG. 3 (B), the perspective views I _pL and I _pR correspond to the target 8
Including the image.

A pair of perspective views created by the perspective view output means 19
I _pL and I _pR , a pair of operation images received by the image receiver 10
It is input to the image synthesizing means 20 together with I _wL and I _wR . The image synthesizing means 20 _{converts the} left perspective view I _pL and the left operation image I _wL into a left superimposed image.
Create an I _sL, creating a right superimposed image I _sR from a right perspective I _pR and right operation image I _wR. Referring to FIG. 3, a method of creating the right superimposed image _IsR will be described. First, a right perspective view _IpR (FIG. 3B)
The right perspective view I _pR is scaled or rotated so that the image of each target 8 above and the image of the corresponding target 8 on the right operation image I _wR (FIG. 3 (A)) overlap, and FIG. A right perspective design image _IdR shown in FIG. By then the superposition of the image of the corresponding optotype 8 on the image of the optotype 8 right perspective design image I _dR and right operation image I _wR, create a right superimposed image I _sR shown in FIG. 3 (D) I do. The same applies to the method of creating the left superimposed image _IsL .

Preferably, the number of the targets 8 is three or more, and three or more images of the targets 8 on the operation image I _wR (or I _wL ) and three or more images on the perspective design image I _dR (or I _dL ). A superimposed image I _sR (or I _sL ) is created by superimposing the image of the corresponding target 8. In addition, in order to facilitate identification of the image of each optotype 8, each optotype 8 may have a different color. If a pair of image combining means 20 is provided as shown in FIG. 1, a pair of left and right superimposed images _{IsL
And IsR} can be created simultaneously.

_By photographing a pair of operation images I _wL and I _wR as a binocular stereoscopic image, a pair of superimposed images _IsL and I
_sR can also be a binocular stereoscopic image. For example displayed alternately in the operation display 14 while switching the pair of superimposed images I _sL and I _sR at high speed as shown in FIG. 4, the left eye and the right eye are alternately closed in synchronism with the switching of the image by viewing the liquid crystal display 14 through the liquid crystal shutter glasses 16 can be viewed stereoscopically pair of superimposed image I _sL and I _sR. However stereoscopic method superimposed image I _sL and I _sR is not limited to a liquid crystal display system of FIG. 4, it is also possible to stereoscopically by a two-colored glasses method or polarization method or the like belonging to the prior art. By performing the remote operation using the superimposed image I _sL and I _sR, it is possible to check the working state by superimposing operation image showing the working status and the design image work object 1, the felling of trees at a remote construction An alternative work index will be shown, and the same work efficiency and construction quality as the construction work using the conventional tenting can be expected.

In this manner, the object of the present invention is to achieve "providing an image system for remote construction in which a three-dimensional superimposed image of an image of a construction area by a camera and a perspective view of a design shape is used instead of tenting in remote construction". it can.

[0016]

In the embodiment of EXAMPLES 4, enter the left and right binocular vision stereo superimposed image I _sL and I _sR created by the image combining unit 20 to the image controller 15, for example, high speed of approximately 1/60 second Time-divisionally displayed on the operation display 14 while switching alternately with
By viewing the operation display 14 through the liquid crystal shutter glasses 16 in which the left and right eyes are alternately closed in synchronization with the image switching of the image controller 15, the superimposed images _IsL and I
I am stereoscopically viewing _sR . Such a method of switching images at such a high speed that the human eye does not feel the switching is known as, for example, an interlacing method of television images, in which one of the television images is a left superimposed image I _sL and the other is a right superimposed image I _sR. By doing so, a stereoscopic image display by a conventional television receiver is also possible. It is also possible to use liquid crystal shutter glasses 16 that open and close at a very high speed of, for example, about 120 Hz, and to switch images at a higher speed.

FIG. 5A shows a heavy equipment 2 which is a power shovel.
An embodiment in which the remote construction support image system of the present invention is applied is shown below. However, the application object of the present invention is not limited to the power shovel. The heavy equipment 2 of FIG. 5 (A) is provided with a target rod 8a and a target position control device 9 capable of moving the relative position of the target rod 8a with respect to the heavy equipment 2, and the target rod 8a has an interval S1,
Four optotypes 8 are attached across S2 and S3 (see FIG. 5D). However, the number of the optotypes 8 is not limited to the illustrated example, and a plurality is sufficient. The target position control device 9 includes a support table 9t fixed to the heavy equipment 2 and a front-rear movable plate 9a slidable on the support table 9t, as shown in FIGS. 5B and 5C, for example. The positioning unit includes a left and right moving plate 9b and a vertically moving element 9c, and a support arm 9d having one end fixed to the vertically moving element 9c. By monitoring the operation of the positioning unit, the relative position of the target rod 8a fixed to the other end of the support arm 9d with respect to the heavy equipment 2, that is, the relative position of the target 8 can be determined. The three-dimensional coordinates of the target 8 with respect to the ground are obtained from the relative position of the target 8 and the measurement value of the satellite surveying device 22.

[0018]

As described above, the image system for remote construction support of the present invention is created from a pair of operation images of a construction object in a construction area where a three-dimensional computer graphic design drawing is drawn, and the design drawing. Since the paired binocular stereoscopic superimposed images are created by superimposing the operation image and the pair of perspective design images having the same viewpoint, the following remarkable effects are obtained.

(A) Since the design image can be superimposed and displayed as a work index on the operation image of the remote construction, the construction efficiency in the remote construction can be expected to be increased to about the same level as when using the conventional tenting. . (B) Since the state of the construction target can always be compared with the state after completion, improvement in construction quality in remote construction can be expected.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of one embodiment of the present invention.

FIG. 2 is an explanatory view showing a construction subject of remote operation.

FIG. 3 is an explanatory diagram of a method of creating a superimposed image.

FIG. 4 is an explanatory diagram of stereoscopic viewing of a binocular stereoscopically superimposed image.

FIG. 5 is an explanatory view of another embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Construction target 2 heavy equipment 3 operation room 4 reference point 5 image pickup device 6 image transmitter 8 optotype 8a optotype rod 9 optotype position control device 9t support table 9a front and back moving plate 9b left and right moving plate 9c vertical moving element 9d support arm 10 image Receiver 11 Construction area 14 Operation display 15 Image controller 16 Liquid crystal shutter glasses 18 Storage means 19 Perspective view output means 20 Image synthesis means 22 Satellite surveying equipment 23 Attitude measuring equipment 24 Transmitting equipment 26 Receiving equipment _Ig 3D computer graphic design I _w operation image I _p perspective view I _d perspective design image Is _s superimposed image.

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-6-272282 (JP, A) JP-A-6-113340 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) E02F 9/20 H04N 7/18 G06T 15/00-17/50

Claims (6)

(57) [Claims] 1. A design drawing of a three-dimensional computer graphic capable of outputting a perspective view of an arbitrary optical axis from an arbitrary viewpoint with respect to a construction area including a construction object, wherein said construction object and three-dimensional coordinates in said construction area are known. Position / orientation with multiple targets
A pair of left and right three-dimensional operation images are created by photographing from different predetermined photographing directions by a pair of image pickup machines mounted at predetermined positions on the heavy equipment for construction with a measuring device, and the respective views are displayed on the design drawing by three-dimensional computer graphics. A target is drawn, and a pair of left and right perspective views are output from the design drawing in which the target is drawn, with the shooting position of each of the imaging devices as the viewpoint and the shooting direction of the imaging device as the optical axis. Each of the perspective views is scaled or rotated so that the image of each target on the perspective view and the corresponding target image on the left and right three-dimensional operation images are overlapped to create a pair of left and right perspective design images. And a left and right perspective design image and the left and right three-dimensional operation images are superimposed on each other to create a pair of left and right binocular stereoscopic superimposed images. 2. The image system according to claim 1, wherein the number of the targets is three or more. 3. The image system according to claim 1, wherein
The left and right binocular stereoscopic superimposed images are alternately switched at a predetermined high speed and displayed on an operation display, and the binocular stereoscopic superimposed image is stereoscopically viewed by looking at the operation display through liquid crystal shutter glasses. An image system for remote construction support that you can see. 4. A plurality of movable targets, and a pair of imagers for photographing each of the targets and a construction object from different directions at predetermined positions to create a pair of left and right three-dimensional operation images. A satellite surveying device that measures the three-dimensional coordinates of the optotypes and the shooting position of each imaging device, a posture measurement device that measures the shooting direction of each imaging device, and the three-dimensional operation image and three-dimensional coordinates. A heavy equipment for remote construction having a transmitting device for transmitting a shooting position and an orientation, and moving in a construction area including the construction object; a receiving device for receiving from the transmitting device; and an arbitrary optical axis for the construction area from an arbitrary viewpoint. Storage means for storing a design drawing of a three-dimensional computer graphic capable of outputting a perspective view of the object, and remote monitoring means provided with an operation display; three-dimensional coordinates of each of the optotypes, and a photographing position and a photographing direction of each imaging device. Input from the storage means. Read the schematic drawing, draw each of the optotypes on the design drawing by three-dimensional computer graphics, and take the shooting position of each of the imaging devices as a viewpoint from the design drawing that draws the optotype, and set the shooting direction of the imaging device. Perspective view output means for outputting a pair of left and right perspective views having an optical axis; inputting the pair of three-dimensional operation images and the pair of perspective views, and forming images of respective targets on the left and right perspective views, and The perspective views are scaled or rotated to create a pair of left and right perspective design images so that the corresponding optotype images on the right operation image overlap, and the left and right perspective design images and the left and right tertiary Image synthesizing means for creating a pair of left and right binocular stereoscopic superimposed images by superimposing with the original operation image; and the operation display while alternately switching the left and right binocular stereoscopic superimposed images at a predetermined high speed. Image control Comprising a chromatography La, remote installation support image system comprising stereoscopically the binocular vision stereo superimposed image by viewing the operation display through the liquid crystal shutter glasses. 5. An image system according to claim 4, wherein the number of said movable targets is three or more. 6. The image system according to claim 4, wherein
An image system for remote construction support, wherein the plurality of movable targets are fixed to one movable target rod attached to the heavy equipment.