JPH09234685A - Vision support device of manipulator operation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the working efficiency by matching the axis of tool movement with the optical axis of a camera. SOLUTION: A wrench tool 22 is arranged on a tip of a manipulator 20, the direction of the optical axis of a first monitoring camera 30 is matched with the axis of a tightening part 23, and the coordinate system of the camera is matched with the working coordinate system 37. A second monitoring camera 40 is installed in a place from which the distance between the wrench tool 22 and a nut 38 can be monitored. The image taken by the monitoring cameras is projected on a screen in an operation room, and an operator operates the manipulator 20 watching the screen. In this device, the information on five variables out of six variables to perform the position and orientation matching of the wrench tool 22 with the nut 38 by the image by the first monitoring camera 30 can be recognized. After the operation, the wrench tool is fitted to the nut 38 watching the image from the second monitoring camera 40, and the nut 38 is tightened. The working efficiency can be improved because almost all operations can be performed watching the image from the first monitoring camera 30.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for remotely controlling a manipulator for use in power distribution work, a nuclear power plant, etc. by using a surveillance camera, and a work tool and a work object attached to the manipulator. The present invention relates to a visual support device for accurately positioning a robot.

[0002]

2. Description of the Related Art There are many electric power distribution works, nuclear power plants, etc. that must be carried out at a high place or in an environment in which radioactive materials exist.
Therefore, when working in the above-mentioned bad environment, a manipulator for remote control is used to ensure the safety of workers.

FIG. 11 shows the construction of the essential parts when a nut is tightened using a manipulator. As shown in the drawing, a wrench tool 3 for tightening a nut 2 as a tool is provided at the tip of the manipulator 1. Around the manipulator 1, a wrench tool 3 and a nut 2 at the tip of the manipulator 1 can be monitored so that the first
A surveillance camera 4 and a second surveillance camera 5 are arranged. The locations of the two surveillance cameras are set so that the wrench tool 3 and the nut 2 cannot be caught by an obstacle or the like at the same time until the wrench tool 3 is attached to the nut 2. The first surveillance camera 4 and the second
The image of the surveillance camera 5 is displayed on the screen 6 of the cockpit 8 where the safety of the worker 7 is ensured, being isolated from the bad environment place where the manipulator 1 and the like are installed.

The cockpit 8 is equipped with a joystick type control device 9 for operating the manipulator 1, and an operator 7 operates the manipulator 1 while looking at the image on the screen 6 to operate the manipulator 1 and a wrench tool. Three
Is used to tighten the nut 2.

In the device thus constructed, the coordinate system for steering is appropriately determined. Therefore, before moving the manipulator 1, the operator must reconsider the steering direction of the steering device 9 from the image information on the screen 6. It was not possible to improve work efficiency. In order to improve the work efficiency, a coordinate system for operating the control device 9 includes a camera coordinate system in which the tip of the manipulator 1 is displayed on the screen 6 in the left-right direction, the front-back direction, and the depth direction. Further, a working coordinate system in which a coordinate origin is set at the tip of the manipulator 1 and a coordinate system set with the coordinate origin as a reference and a steering direction are made to coincide is devised.

In the camera coordinate system, the tip of the manipulator 1 is made to match the left, right, front and back, and the depth direction reflected on the screen 6, so that the operator 7 intuitively determines the steering direction from the image on the screen 6. You can decide. Further, in the case of using the working coordinate system, the coordinate system of the tip of the manipulator 1 and the steering direction of the control device coincide with each other, so that it is efficient in moving the tip of the manipulator 1 in any direction. It can be operated.

[0007]

Generally, in order to position a tool on a work object in a three-dimensional coordinate space, as shown in the position and orientation alignment explanatory diagram of FIG. 12, positioning in a three-dimensional space is performed. 6 variables (X,
It is necessary to be able to monitor Y, Z, θ _x , θ _y , and θ _z ) and steer in each direction.

However, in the case of operating with a joystick while watching the image from the surveillance camera,
Depending on where multiple surveillance cameras are installed, there are differences in the field of view and work feeling, and as a result, work efficiency greatly differs.

For example, as shown in FIG. 11, when tightening the nut 2 with the wrench tool 3, while watching the images of the first surveillance camera 4 and the second surveillance camera 5 on the screen 6,
Although the manipulator 1 is operated, the image on the screen 6 is a two-dimensional image, so it was difficult to grasp the information in the depth direction. Therefore, it is difficult to know whether or not the axis of the wrench hole of the wrench tool 3 exactly matches the axis of the nut 2, which is a factor of reducing the work efficiency.

Further, even when maneuvering, since the direction of the manipulator 1 on the screen 6 and the maneuvering direction of the manipulator 9 do not always match, the operator 7 cannot intuitively maneuver, and maneuverability is reduced. However, there was a problem in that

That is, in the maneuvering method by the camera coordinate system in which the tip of the manipulator 1 can be steered in the left, right, front and back, and the depth direction in which the screen 6 is reflected, a working coordinate system and a camera coordinate system suitable for the corresponding work are provided. There is a problem that it is difficult to work because the two do not match.

Further, if the manipulator 1 is to be operated in a working coordinate system which is the same as the coordinate system of the tip of the manipulator 1, there is a problem that the operation locus becomes difficult to see on the screen 6 and the work is difficult. .

Further, since the image of the surveillance camera is a plane image, the manipulator 1 must be operated while simultaneously observing the images of a plurality of surveillance cameras, and the maneuverability is not efficient.

As described above, in the conventional apparatus, the work point is generally observed while observing the work point from a plurality of directions by using the surveillance camera. However, the image of the surveillance camera is a plane image,
Since it is difficult to understand the depth, there is a demand for a method that enables more accurate and efficient alignment of the position and orientation of the tool than in the past.

The present invention has been made in view of the above-mentioned problems, and provides a visual support device for manipulator control provided with means for aligning the moving axis of a tool with the optical axis of a camera to improve work efficiency. The purpose is to do.

[0016]

In order to achieve the above-mentioned problems, the visual support device for a manipulator according to the present invention is a manipulator having a tool attached thereto, when the work is performed on a work object, the tool and the work are used. A visual support device for manipulator control, which monitors an object and a monitoring camera and provides an image captured by the monitoring camera, wherein the optical axis of the first monitoring camera is coaxial with the tool movement axis that finally approaches the work object. Alternatively, they are installed in parallel, and the second monitoring camera is arranged on the side of the tool moving shaft capable of monitoring the distance between the tool and the work object.

Further, in a manipulator-controlled visual support device for manipulating a manipulator to which a wrench tool is attached, the wrench tool and the nut are monitored by a surveillance camera when a nut is attached and detached, and an image of the surveillance camera is provided. , First
The optical axis of the monitoring camera is installed coaxially with the wrench tool moving shaft that finally approaches the nut, and a second monitoring camera is provided on the side of the wrench tool moving shaft capable of monitoring the distance between the wrench tool and the nut. I placed it in.

Further, in a manipulator-manipulated visual support device for manipulating a work object with a manipulator equipped with a tool, the tool and the work object are monitored by a surveillance camera and an image of the surveillance camera is provided. Therefore, through the field-of-view switch disposed in front of the first surveillance camera, the first
The optical axis of the surveillance camera is installed coaxially or in parallel with the tool moving axis that finally approaches the work object, and
A monitoring camera is arranged on the side of the tool moving shaft capable of monitoring the distance between the tool and the work object.

Further, when a manipulator equipped with a gripper tool is used to perform work on a work object, the gripper tool and the work object are monitored by a surveillance camera, and visual support for manipulator operation is provided by which an image of the surveillance camera is provided. A device, wherein an optical axis of the first surveillance camera is installed coaxially or in parallel with the movement axis of the gripper tool that finally approaches the work object via a visual field switching device arranged in front of the first surveillance camera. At the same time, the second monitoring camera is arranged on the side of the gripper tool moving shaft capable of monitoring the distance between the gripper tool and the work object.

Here, the field-of-view switch is formed by a mirror arranged obliquely with respect to the optical axis of the first surveillance camera, and by moving the mirror, the direction of the optical axis of the first surveillance camera is changed. It is configured to reflect in the moving axis direction.

The optical axis of the first monitoring camera is installed coaxially or in parallel with the moving axis of the tool which finally approaches the work object in the above-mentioned manipulator visual support device, and the coordinate system of the first monitoring camera is set. The remaining two axes other than the optical axis are arranged coaxially or in parallel with coordinate axes other than the movement axis of the tool. Further, the relative position between the coordinate system of the first surveillance camera and the coordinate axis of the tool is fixed.

[0022]

According to the above construction, the optical axis of the first surveillance camera is arranged so as to be coaxial with or parallel to the tool movement axis which the tool finally approaches the work object. By arranging the optical axis of the first surveillance camera in this way, the six variables (X, X, which determine the position and orientation of the tool in the three-dimensional coordinate space in the first surveillance camera)
Y, Z, θ _x , θ _y , θ _z ) of 5 variables (X, Y, θ _x ,
Since the situation of (θ _y , θ _z ) can be grasped, most of the control can be performed with the image from the first surveillance camera. Also the second
By disposing the surveillance camera on the side of the tool movement axis capable of monitoring the distance between the tool and the work target, the situation of one variable (Z) representing the distance between the tool and the work target can be grasped. Therefore, the operator first adjusts the position and orientation of the tool excluding one variable (Z) from the image of the first surveillance camera.
After that, from the image of the second surveillance camera, the distance between the tool and the work object, which is the remaining one variable (Z), is narrowed,
The predetermined work is performed after the tool is attached to the work target. In this way, most of the operations can be performed with the image from the first surveillance camera, so that the work efficiency can be improved.

Further, the optical axis of the first surveillance camera is arranged so that the wrench tool is coaxial with the wrench tool movement axis that finally approaches the nut. By arranging the optical axis of the first monitoring camera in this manner, the six variables (X, Y, and 6) that determine the position and orientation of the wrench tool in the three-dimensional coordinate space in the first monitoring camera.
Z, θ _x , θ _y , θ _z ) of 5 variables (X, Y, θ _x ,
Since the situation of (θ _y , θ _z ) can be grasped, most of the control can be performed with the image from the first surveillance camera. Also the second
By arranging the monitoring camera on the side of the wrench tool moving shaft capable of monitoring the distance between the wrench tool and the nut, the situation of one variable (Z) representing the distance between the wrench tool and the nut can be grasped. Therefore, the operator first adjusts the position and orientation of the wrench tool excluding the one variable (Z) from the image of the first surveillance camera. After that, from the image of the second surveillance camera, the distance between the wrench tool and the nut, which is the remaining one variable (Z), is reduced, and the predetermined work is performed after the wrench tool is attached to the nut. In this way, most of the operations can be performed with the image from the first surveillance camera, so that the work efficiency can be improved.

A field-of-view switch is installed in front of the first surveillance camera. Here, the optical axis of the first surveillance camera is arranged via the field-of-view switch so that the tool is coaxial or parallel to the tool movement axis that finally approaches the work target. Thus the first
If you arrange the optical axis of the surveillance camera,
Determining tool position and orientation in 3D coordinate space 6
Of the variables (X, Y, Z, θ _x , θ _y , θ _z ), the status of 5 variables (X, Y, θ _x , θ _y , θ _z ) can be grasped,
Most of the control can be performed with the image from the first surveillance camera. Further, if the second monitoring camera is arranged on the side of the tool moving axis capable of monitoring the distance between the tool and the work object, the situation of one variable (Z) representing the distance between the tool and the work object can be grasped. . Therefore, the operator first adjusts the position and orientation of the tool excluding one variable (Z) from the image of the first surveillance camera. After that, from the video of the second surveillance camera,
The remaining one variable (Z), that is, the distance between the tool and the work target is narrowed, and the predetermined work is performed after the tool is mounted on the work target. In this way, most of the operations can be performed with the image from the first surveillance camera, so that the work efficiency can be improved.

A field-of-view switch is installed in front of the first surveillance camera. Here, the optical axis of the first surveillance camera is arranged via the field-of-view switch so that the gripper tool is coaxial or parallel to the movement axis of the gripper tool that finally approaches the work object. By arranging the optical axis of the first surveillance camera in this way, the six variables (X, Y, Z, θ _x , which determine the position and orientation of the gripper tool in the three-dimensional coordinate space in the first surveillance camera,
Since theta _y, 5 variables (X of _{θ z), Y, θ x} , θ y, the status of theta _z) can be grasped, it is possible to perform most of the maneuvers in the image from the first surveillance camera. Further, if the second monitoring camera is arranged on the side of the gripper tool moving axis capable of monitoring the distance between the gripper tool and the work object, the situation of one variable (Z) representing the distance between the gripper tool and the work object. Can be grasped. Therefore, the operator first adjusts the position and orientation of the gripper tool excluding the one variable (Z) from the image of the first surveillance camera. After that, from the image of the second surveillance camera, the distance between the remaining one variable (Z), the gripper tool, and the work target is narrowed, and the predetermined work is performed after the gripper tool is mounted on the work target. is there. In this way, most of the control can be performed with the image from the first surveillance camera,
It is possible to improve work efficiency.

Here, the field-of-view switch is formed by a mirror arranged in a slanting plane with respect to the optical axis direction of the first surveillance camera, and the mirror can be moved in front of the first surveillance camera. Therefore, if a plurality of mirrors are switched according to a plurality of tool moving axes of a gripper tool or the like, various methods of using the tool are possible.

In the manipulator visual support device described above, if the two axes other than the optical axis in the coordinate system of the first surveillance camera are arranged coaxially or in parallel with the two axes other than the tool movement axis in the tool coordinate system, The steering direction of the tool can be intuitively understood from the image of the first surveillance camera. As a result, tool maneuverability is improved and work efficiency can be improved. Further, if the relative position between the coordinate system of the first monitoring camera and the coordinate axis of the tool is fixed, the image from the first monitoring camera is always the image from the position determined for the tool. Since the relative position between the tool and the work object can be easily grasped by the image from the first monitoring camera, workability can be improved.

[0028]

BEST MODE FOR CARRYING OUT THE INVENTION Specific embodiments of a visual support device for manipulator control according to the present invention will be described in detail below with reference to the drawings.

FIG. 1 shows the first embodiment and is a structural explanatory view when the visual support device according to the present invention is applied to the nut attaching and detaching work. Here, only the tip end portion of the manipulator 20 is depicted and the root side is not shown in the figure, but the main portion of the manipulator 20 is composed of a plurality of arms and a joint portion that connects the end portions of the arms. Has been done. And a driving device such as a motor is incorporated in this joint portion,
By operating this drive device, the arm can perform its own motion without being influenced by the motion of the adjacent arm. Therefore, if a working coordinate system that represents the position and orientation of the tip of the manipulator 20 is set and the movements of the joints are sequentially combined from the base side of the manipulator 20 based on the values of the working coordinate system, the cumulative movement of each arm is performed. Depending on the quantity
It is possible to move the position of the tip of the manipulator 20 to a desired place and to take an arbitrary posture.

A wrench tool 22, which serves as a tool, is installed from the tip of the manipulator 20 to the front, and the mounting direction of the wrench hole shaft core 24 in the tightening portion 23 is
The direction is orthogonal to the axial direction of the manipulator tip. Further, a tool power motor 26 is built in the tip of the manipulator 20, and the tool power motor 26 and the tightening portion 23 are connected via a speed reducer 28 inside the wrench tool 22. For example, here, the tool power motor 2
When 6 is rotated in the normal direction, the tightening portion 23 tightens the nut via the speed reducer 28, and conversely, when the tool power motor 26 is rotated in the reverse direction, the tightening portion 23 loosens the nut. . Therefore, the tightening portion 23 can be operated as a wrench tool by switching the rotation direction of the tool power motor 26.

On the back side of the wrench tool 22, a first surveillance camera 30 for controlling the manipulator 20 is provided. And this first surveillance camera 30
The optical axis direction is aligned with the wrench hole axis 24, and the optical axis direction is also aligned with the working coordinate system of the wrench tool 22 described later.

FIG. 2 shows the coordinate system of the tool and the surveillance camera.
It is a coordinate system relationship diagram which shows a relationship. Take the tool 31 here
Work coordinate system that represents the position and orientation of the attached manipulator 20
Is defined. First, in the tool 31, the tool 31
Align the axes to align with the elephant, and finally
The tool movement axis for approaching movement is the access axis 32 (Z axis
Or X axis, Y axis). And this access axis 32
Set the working coordinate system 34 of the tool 31 based on
When operating with a control device such as tick type,
There is no parallel movement or rotation movement based on this work coordinate system 34.
What action do you take? Further placed near the tool 31
The Z axis of the camera coordinate system in the first surveillance camera 30
The optical axis direction of the first surveillance camera 30 is set, and the Z axis is the work seat.
Arranged to be coaxial or parallel to the access axis 32 of the standard system
I do. Here, the remaining two axes (X axis, Y axis) of the camera coordinate system
Is also set to be coaxial or parallel to the working coordinate system,
The working coordinate system and the camera coordinate system are the same or parallel
Thus, it becomes possible to operate efficiently. Ie
In FIG. 1, align the wrench hole shaft core 24 with the
The access axis 36 is defined and the work coordinate system 37 is set. So
The first monitoring camera installed on the back side of the wrench tool 22
The optical axis direction of the laser 30 is the Z axis, and the optical axis direction is overlapped with the access axis 36.
At the same time, the remaining two axes (X axis, Y axis) of the camera coordinate system
Set the first surveillance camera 30 so that it matches the working coordinate system 37.
Install it. The installation of the first surveillance camera
In the image, six variables (X, Y,
Z, θ x, θ_y, Θ_zOf 5 variables (X, Y, θ_x,
θ_y, Θ_z) Position, attitude misalignment can be grasped
Then, most of the work can be done only with the image of the first surveillance camera 30.
Because

The manipulator 20 set as described above.
Is moved to the vicinity of the nut 38 that is the work target. At this time, the second monitoring camera 4 is provided on the side of the access shaft 36 capable of monitoring the distance between the nut 38 and the wrench tool 22.
0 is provided. The installation direction of the second surveillance camera 40 is such that the optical axis of the second surveillance camera 40 is the first
The orientation is perpendicular to the optical axis of the surveillance camera 30. According to the image from the second surveillance camera 40 arranged in this way, the wrench tool 22 and the nut 38
It is possible to recognize the distance from. In the present embodiment, the optical axis of the first surveillance camera 30 and the second surveillance camera 4
The second monitoring camera 40 is installed so that the optical axis of 0 is orthogonal to each other so that the distance between the wrench tool 22 and the nut can be easily determined. The optical axis of the second monitoring camera 40 may not be orthogonal to the optical axis of the first monitoring camera 30.

Although not shown, the manipulator 20 is separated from the place where the manipulator 20 works, and a cockpit of the manipulator 20 is provided in a place where the safety of the worker can be secured. This cockpit is equipped with a screen on which the images of the first surveillance camera 30 and the second surveillance camera 40 are displayed, and a joystick type control device for manipulating the manipulator 20. Then, the operator operates the manipulator 20 by moving the control device while watching the image on the screen.

The procedure for tightening the nut 38, which is the work object, using the manipulator-controlled visual support device thus constructed will be described. The tightening portion 2 of the wrench tool 22 will be described.
3 is attached to the nut 38, the work of aligning the shaft center of the tightening portion 23 and the shaft center of the nut 38, and then the access shaft 36 for aligning the peaks and valleys so that the hexagonal surfaces of both are aligned.
It is necessary to perform a work of adjusting the posture around the work, and a work of moving the wrench tool 22 along the access shaft 36 and mounting it on the nut 38.

FIG. 3 shows a screen image of the first surveillance camera 30. In order to perform the above-mentioned work, the operator operates the manipulator 20 while looking at the screen image, but the work of aligning the shaft center of the tightening portion 23 and the shaft center of the nut 38 and the mountains and valleys so that the hexagonal surfaces coincide with each other. The posture matching work can be performed only by looking at the screen image of the first surveillance camera 30. That is, the tightening portion 2
In the work of aligning the shaft core of 3 and the shaft core of the nut 38, X
The left and right and up and down positions are adjusted using the axes and Y-axis movements, and thereafter the axes are aligned by using the two degrees of freedom θ _x and θ _y movements. Further, in the work of aligning the peaks and valleys so that the hexagonal surfaces coincide with each other, θ _z which is one degree of freedom around the access axis 36 is used. After the shaft center of the tightening portion 23 and the shaft center of the nut 38 are aligned as described above, and the hexagonal surfaces of the both are further aligned, the worker uses the second monitoring camera 40 to set the tightening portion 2
The tightening portion 23 is attached to the nut 38 while monitoring the distance between the nut 3 and the nut 38. And after mounting the tightening portion 23,
The tool power motor 26 is rotated to tighten the nut 38.

By arranging the access axis 32 of the tool 31 and the optical axis direction of the first surveillance camera 30 in parallel in this way, and setting the remaining two axes of camera coordinates to be parallel to the working coordinate system, Since most of the work can be performed with the image from the first surveillance camera 30, work efficiency can be improved.

Further, as a control device, Japanese Patent Application No. 02-24
A control device capable of generating a command in the working coordinate system 34 in a Cartesian coordinate system, as shown in Japanese Patent Application No. 9792 and Japanese Patent Application No. 03-334314, is used. FIG. 4 shows an operation explanatory diagram of this operation device. As shown in the figure, the left and right arms of the control device,
Up and down, forward and backward, and commands by rotation around the respective axes are respectively given to six variables (X, Y, Z,
If it is made to coincide with θ _x , θ _y , and θ _z ), the direction of the operation arm and the movement direction of the tool 31 with respect to the positioning target match on the screen, so that an intuitive operation is possible.

FIG. 5 shows the second embodiment and is a structural explanatory view when the visual support device according to the present invention is applied to a gripper tool. As shown in the figure, a gripper tool 42 is provided at the tip of a manipulator (not shown), and a parallel gripper 44 is provided at the tip of the gripper tool 42. Then, the parallel gripper 44 can be moved by supplying hydraulic pressure or air pressure to the supply port 45 from the tip of the manipulator to perform an operation of gripping an object to be worked, or an operation of separating it. From such an operation, the gripper tool 42 can be used for various work purposes such as grasping a work target object and assembling it with another object.
It is more versatile than dedicated tools.

FIG. 6 is a display diagram showing the working coordinate system of the gripper tool 42. It is difficult to specify the work coordinate system of the gripper tool 42 due to its general-purpose nature, but it is considered that the gripper tool 42 is often operated along the three access axes depending on the intended use. That is, firstly, when the Z-axis positive direction in the figure is used as the access axis and the work object is grasped or inserted. Secondly, when the work object is grasped or inserted with the X-axis positive direction in the figure as the access axis. And thirdly, there is a case in which the work object is grasped or inserted with the negative direction of the X axis in the figure as the access axis.

In the gripper tool 42 for performing such work, the first monitoring camera 4 is provided at the base of the parallel gripper 44.
Install 6. The optical axis direction of the first surveillance camera 46 is arranged so as to be coaxial with the Z-direction access axis of the gripper tool 42. Further, the other two axes (X axis, Y axis) in the camera coordinate system are also arranged so as to be coaxial with the two axes (X axis, Y axis) of the work coordinate system of the gripper tool 42.

A field-of-view switch 48 is provided in front of the first surveillance camera 46. FIG. 7 is a structural explanatory view of the visual field switch 48. As shown in the figure, the field-of-view switch 48 includes two mirrors 50 and 52 arranged obliquely with respect to the optical axis direction of the first surveillance camera 46 and the mirrors 50 and 52 in front of the first surveillance camera 46. It is constituted by a slide mechanism (not shown) that is moved by. And the mirrors 50, 52
The inclination angle of is relative to the optical axis direction of the first surveillance camera 46,
It is set to ± 45 degrees. Here, from the state where the optical axis direction of the first surveillance camera 46 coincides with the Z-axis direction of the work coordinate system as shown in (2), the visual field switching device 48 is moved to perform the first surveillance as shown in (3). The mirror 50 is moved in front of the camera 46. By the movement of the mirror 50, the optical axis of the first surveillance camera 46 is reflected by the mirror 50 and becomes parallel to the positive direction of the X axis. Further, the means for operating the slide mechanism may be performed manually, or if it is not possible to perform the operation manually, a drive device such as a solenoid or a motor may be used for remote operation.

When performing the work using the above-mentioned structure, the second monitor camera 53 is arranged on the side of the access shaft capable of monitoring the distance between the work object and the gripper tool 22, and the gripper tool 42 is used. And the distance to the work object can be grasped. Further, the cockpit is provided in a place separated from the work place, and the operator operates the manipulator while looking at the screen, as in the first embodiment.

The procedure of gripping the pipe 54 which is the work target by applying the manipulator-controlled visual support device thus constructed to a gripper tool will be described. FIG. 8 is a work explanatory diagram when the access axis of the gripper tool 42 is set in the Z direction. Here, (1) shows the main configuration, and (2) shows the image from the first surveillance camera 46. In this case, the mirrors 50 and 52 are not arranged in front of the first surveillance camera 46, and the optical axis direction of the first surveillance camera 46 is aligned with the Z axis which is the access axis of the gripper tool 42. Then, the field-of-view switch 48 is set as described above, the five variables (X, Y, θ _x , θ _y , θ _z ) are adjusted based on the image of the first surveillance camera 46, and then the second surveillance camera 46 is set. The gripper tool 42 is approached to the pipe 54 while monitoring the image of 53. Pipe 5
After the gripper tool 42 is brought sufficiently close to 4, the parallel gripper 44 is operated and the pipe 54 is gripped. When the Z axis is used as the access axis, more intuitive operation can be performed because the work coordinate system and the camera coordinate system match as in the first embodiment described above.

FIG. 9 is a work explanatory view when inserting the sleeve 58 into the cable 56 with the access axis of the gripper tool 42 in the positive direction of the X axis. Here, (1) shows the configuration of the main part, and (2) shows the state of the visual field switching device 48 during work.
In this case, the mirror 50 of the visual field switch 48 is moved to the front of the first surveillance camera 46. Then, the first surveillance camera 46
The optical axis direction of is parallel to the X-axis positive direction by the mirror 50.
The camera coordinate system is also rotated by the field-of-view switch 48, and the remaining two axes are also parallel to the remaining two axes of the work coordinate system via the field-of-view switch 48. FIG. 10 is an image of the first surveillance camera 46 through the view switching device. First surveillance camera 4
Since 6 is provided at the base of the parallel gripper 44, the image does not coincide with the access axis and is as shown in FIG. Also, the working coordinate system and the camera coordinate system do not match. However, even in this case, from the image of the first surveillance camera 46, four of the five variables (X, Y, θ
_x , θ _z ) can be recognized, and the remaining one variable (θ _y ) can be roughly recognized. Therefore, after the cable 56 and the sleeve 58 are aligned by the first monitoring camera 46, the distance from the cable 56 to the gripper tool 42 can be monitored from the second monitoring camera 53 arranged on the side of the access shaft. The sleeve 58 is inserted into the cable 56 while monitoring the image.

Further, in the above-described embodiment, the case where the optical axis direction of the first monitoring camera 46 is reflected by the visual field switching device 48 in the positive X-axis direction has been described, but the optical axis direction of the first monitoring camera 46 is described. The same effect can be obtained by reflecting the image in the negative direction of the X-axis by the visual field switching device 48.

In this way, the optical axis direction of the first surveillance camera 46 is made parallel to the access axis of the gripper tool 42 via the visual field switching device 48, and the camera coordinate system is set to the visual field switching device 4.
If you rotate it by 8, although the steering direction is different and you can not grasp the situation of perfect position and attitude adjustment,
5 variables (X, Y, θ _x , θ _y , θ from the first monitoring camera 46)
_Since the situation of _z ) can be grasped, it can sufficiently contribute to the improvement of work efficiency.

In the above-mentioned first and second embodiments, the case where two surveillance cameras are used has been described.
When the field of view of the second surveillance camera becomes unclear due to an obstacle or the like, a plurality of surveillance cameras may be arranged laterally of the first surveillance camera in the optical axis direction. Then, by monitoring the distance between the tool and the work target from a plurality of places, it becomes possible to grasp a clear value and improve the work efficiency.

[0049]

As described above, according to the present invention,
Use the manipulator with the tool attached to work on the work target.
Monitor the tool and the work object with a monitoring camera
And manipulator control providing imaging of said surveillance camera
Of the first surveillance camera, wherein the optical axis of the first surveillance camera is
Coaxial with the tool movement axis where the tool finally approaches the work object
Or install them in parallel and set the second surveillance camera to
The tool transfer capable of monitoring the distance between the tool and the work object.
If it is placed on the side of the axis of motion, the image from the first surveillance camera
6 variables (X, Y,
Z, θ_x, Θ_y, Θ of z), 5 variables (X, Y, θ_x, Θ
_y, Θ_z) Status can be understood. And the first monitoring turtle
After adjusting the position and orientation with the
The remaining 1 variable (Z) is monitored by the camera, and
Move the tool closer. With such a configuration,
Most of the control can be done with the image from the first surveillance camera,
It is possible to improve work efficiency.

If the tool is a wrench tool or a gripper tool, tightening of nuts, gripping of an object to be worked,
Alternatively, it is possible to perform an inserting operation such as inserting a member or the like into the work target. Further, regarding the gripper tool, if the field-of-view switch is arranged in front of the first surveillance camera,
The optical axis direction of the surveillance camera is arranged coaxially or in parallel with the gripper tool movement axis direction. From this, it becomes possible to correspond to a plurality of gripper tool moving axes of the gripper tool, and it is possible to correspond to the versatility of the gripper tool.

In the manipulator visual support device described above, if the two axes other than the optical axis in the coordinate system of the first surveillance camera are arranged coaxially or parallel to the two axes other than the tool movement axis in the tool coordinate system, The steering direction of the tool can be intuitively understood from the image of the first surveillance camera. As a result, tool maneuverability is improved and work efficiency can be improved. Further, if the relative position between the coordinate system of the first monitoring camera and the coordinate axis of the tool is fixed, the image from the first monitoring camera is always the image from the position determined for the tool. Since the relative position between the tool and the work object can be easily grasped by the image from the first monitoring camera, workability can be improved.

[Brief description of drawings]

FIG. 1 is a structural explanatory view when a visual support device according to the present invention is applied to a nut attaching / detaching action.

FIG. 2 is a coordinate system relationship diagram showing a relationship between coordinate systems of a tool and a surveillance camera.

FIG. 3 shows a screen image of the first surveillance camera.

FIG. 4 is a control explanatory diagram of a control device that can generate a command in a work coordinate system in a rectangular coordinate system.

FIG. 5 is a structural explanatory view when the visual support device according to the present invention is applied to a gripper tool.

FIG. 6 is a display diagram showing a work coordinate system of the gripper tool.

FIG. 7 is a structural explanatory view of a visual field switching device.

FIG. 8 is an operation explanatory diagram when the access axis of the gripper tool is set in the Z direction.

FIG. 9: The access axis of the gripper tool is the X axis positive direction,
It is a work explanatory view when inserting a sleeve in a cable.

FIG. 10 is an image of the first surveillance camera through the visual field switching device.

FIG. 11 is a configuration diagram of a main part when a nut is tightened using a manipulator in a conventional technique.

FIG. 12 is an explanatory diagram of position and posture alignment for positioning.

[Explanation of symbols]

 1 Manipulator 2 Nut 3 Wrench Tool 4 1st Surveillance Camera 5 2nd Surveillance Camera 6 Screen 7 Operator 8 Cockpit 9 Control Device 20 Manipulator 22 Wrench Tool 23 Tightening Section 24 Wrench Hole Shaft Core 26 Tool Power Motor 28 Reducer 30 First Monitoring Camera 31 Tool 32 Access Axis 34 Working Coordinate System 36 Access Axis 37 Working Coordinate System 38 Nut 40 Second Monitoring Camera 42 Second Gripper Tool 44 Parallel Gripper 46 First Monitoring Camera 48 Field Switch 50 Mirror 52 Mirror 53 Second Surveillance camera 54 Pipe 56 Cable 58 Sleeve

Claims (5)

[Claims] 1. A manipulator visual support device for manipulating a work object with a manipulator having a tool, wherein the tool and the work object are monitored by a surveillance camera, and an image of the surveillance camera is provided. The optical axis of the first surveillance camera is installed coaxially or in parallel with the tool movement axis that finally approaches the work object, and the second surveillance camera monitors the distance between the tool and the work object. A visual support device for manipulator control, wherein the visual support device is arranged laterally of the possible tool movement axis. 2. A manipulator-controlled visual support device for monitoring the wrench tool and the nut with a surveillance camera when the nut is attached / detached by a manipulator having a wrench tool attached thereto, and provides an image of the surveillance camera. First
The optical axis of the monitoring camera is installed coaxially with the wrench tool moving shaft that finally approaches the nut, and a second monitoring camera is provided on the side of the wrench tool moving shaft capable of monitoring the distance between the wrench tool and the nut. A visual support device for manipulator control, which is characterized in that 3. A manipulator-controlled visual support device for monitoring the tool and the work object with a surveillance camera when performing work on the work object with a manipulator to which a tool is attached, and for providing an image of the surveillance camera. In addition, the optical axis of the first monitoring camera is installed coaxially or in parallel with the tool moving axis that finally approaches the work object, via a visual field switch disposed in front of the first monitoring camera, A visual support device for manipulator control, wherein a second monitoring camera is arranged on a side of the tool moving shaft capable of monitoring a distance between the tool and the work object. 4. A manipulator control visual system for monitoring the gripper tool and the work object by a surveillance camera when performing work on the work object with a manipulator equipped with a gripper tool, and providing an image of the surveillance camera. A support device, wherein an optical axis of the first surveillance camera is installed coaxially or in parallel with the movement axis of the gripper tool that finally approaches the work object via a visual field switching device arranged in front of the first surveillance camera. In addition, the visual support device for manipulator control, wherein the second monitoring camera is arranged laterally of the gripper tool moving shaft capable of monitoring the distance between the gripper tool and the work object. 5. The field-of-view switch is formed by a mirror that is arranged obliquely with respect to the optical axis of the first surveillance camera,
The manipulator control visual support device according to claim 3 or 4, wherein the optical axis direction of the first surveillance camera is reflected in the movement axis direction by moving the mirror.