JPH08132372A - Control method for robot - Google Patents

Abstract

PURPOSE: To surely feedback to an operator force and torque information that a slave arm receives without adopting a bilateral control method. CONSTITUTION: Image display is carried out on a display 12 by putting the moving image information of the position of an arm tip portion obtained from a camera 5 fitted to a robot together upon force and torque information to act on an arm tip portion obtained from a force and torque sensor 7. An operator who conducts work while looking at the display 12, can simultaneously confirm on the display 12 in which direction of an arm 3 load is acting at present, and can execute work so as not to impose overload on the arm 3.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot control method, and more particularly to a robot control method having a function of feeding back force / torque information to an operator.

[0002]

2. Description of the Related Art When remotely controlling a robot, it is known to employ a so-called master-slave system in which a slave arm at a remote place is driven so as to follow a master arm operated by an operator.

In such a master-slave system, bilateral control is used in which the reaction force received by the slave arm from the work object or the surrounding environment is fed back to the master arm, so that the operator senses the work situation at a remote place. The feature is that the work can be grasped and the work can be performed without overloading the work target or the slave arm.

[0004]

However, in such a bilateral control system, an actuator (a motor, a speed reducer, etc.) for driving the master arm in accordance with the reaction force fed back is required. The structure of the arm becomes large and complicated. for that reason,
Since a simple control device such as a joystick cannot obtain force / torque information, its use as a master arm is limited.

Further, when the force / torque information is fed back, the control system tends to become unstable depending on the rigidity of the object, and the entire control system vibrates, making it difficult to carry out the work.

Therefore, an object of the present invention is to provide a robot control method capable of reliably feeding back force / torque information received by a slave arm to an operator without employing a bilateral control method.

[0007]

In order to achieve the above-mentioned object, the present invention displays the position information of the arm tip of a robot as an image, and the force acting on the arm tip with respect to this image information. The robot control method is to display the image by superimposing the torque information.

Further, the control method of the robot is such that the force / torque information acting on the arm tip portion is superposed on the moving image information of the position of the arm tip portion of the robot to display the image.

Further, the force / torque information acting on the arm tip portion obtained from the force / torque sensor attached to the robot is superimposed on the image information of the arm tip portion obtained from the camera attached to the robot. A robot control method that also displays images is used.

The image information at the tip of the arm is C
It may be G (computer graphic) processed image information. Further, the force / torque information may be displayed so as to overlap with the image information of the arm tip portion, or the force / torque information may be displayed as a vector image. Further, only a part of the force / torque information may be selectively displayed.

[0011]

By using the control method described above, the operator can capture the force / torque information as image information, so that the force / torque information received by the slave arm can be assured without using the bilateral control method. It is possible to feed back to the operator.

Particularly, since the operator pays attention to the tip of the arm to perform the work, if the force / torque information is displayed at the same time as the position information of the arm tip, the work efficiency is greatly improved and the risk of the work is sufficiently increased. Can be reduced to

[0013]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a control method of a robot of the present invention, FIG. 2 is a conceptual diagram showing an example of a working state by the robot, and FIG. 3 is a diagram showing an example of a display method of an arm tip portion.

Here, the pin insertion work by the robot will be described as an example. Robot 1 as shown in Fig. 2 (a)
Includes a robot body 2, an arm 3, and a hand 4 attached to the tip of the arm, and a camera 5 is arranged on the robot body 2. The arm 3 has a plurality of joints 6 (6a, 6b, 6c) and connects the robot body 2 and the hand 4. On the other hand, a force / torque sensor 7 is fixed to the base portion of the hand 4 and detects an external force acting on the pin P gripped by the hand 4. Here, for example, a 6-axis force sensor is used as the force / torque sensor 7.

The work method by the robot 1 having such a configuration is performed based on the block shown in FIG. First,
Position / angle information and force / torque information are detected from the arm 3. The position / angle information is detected by the position / angle sensor 8 provided at each joint 6a, 6b, 6c, and the force / torque information is detected by the force / torque sensor 7 described above.

Further, from the camera 5 arranged on the robot main body 2, camera setting information such as its position / orientation (pan / tilt etc.) and angle of view / zoom ratio, and an image captured near the tip of the arm. Information is output independently.

The camera setting information is detected by the camera information detector 9 and input to the arithmetic unit 10. The signal from the position / angle sensor 8 described above is also output to the arithmetic unit 10. Therefore, the calculation unit 10 calculates at which position within the angle of view of the camera the arm tip is projected.

For example, the reference coordinate system is set as the origin position of the camera 5, and one end of the arm 3 (joint 6 is seen from the reference coordinate system).
coordinate conversion to a) Tcam, one end of arm 3 (joint 6a)
Tar transformation from the other side of arm 3 to the other end (joint 6c)
If m is the coordinate transformation from the other end of the arm 3 (joint 6c) to the hand 4 as E, the transformation X from the coordinate system of the camera 5 to the coordinate system of the hand 4 is

[0019]

## EQU1 ## X = Tcam.Tarm.E ... (1) can be expressed. From this equation (1), the position / orientation of the hand 4 (arm end) viewed from the coordinate system of the camera 5 can be obtained. In addition, the conversion X shown in the equation (1)
The conversion process is performed as shown in the vector drawn on the robot body 2 in FIG. 2 (b).

On the other hand, by changing the camera setting information such as the position / orientation (pan / tilt etc.) and the angle of view / zoom ratio of the camera 5, the position and size of the arm tip part on the angle of view of the camera can be changed. Changes each time. The position of the arm tip portion on the camera angle of view is accurately calculated by performing the calculation of Expression (1) in the calculation unit 11 and performing the correction in consideration of the camera setting information.

In this way, it is calculated at which position within the angle of view of the camera the arm tip is projected, and the information is sent to the screen control unit 11. Screen control unit 11
Shows the state of the arm tip obtained by the calculation unit 10 on the CRT or LC.
Performs a mapping conversion operation for display on the display 12 including D.

Information from the force / torque sensor 7 is also input to the screen control section 11. At this time, force
The force / torque information obtained by the torque sensor 7 is temporarily output to the coordinate conversion unit 13, and the force / torque information in an arbitrary coordinate system is output.
Converted to torque information. For example, when the calculation of the position / angle information by the calculation unit 10 converts the coordinate system of the camera 5 into the coordinate system of the arm tip (hand 4) viewed from
The calculation in the coordinate conversion unit 13 also converts the arm tip (hand 4) into the coordinate system.

Then, the force taken in by the screen control unit 11
The torque information is processed so as to be superimposed on the moving image of the arm tip portion output to the display 12 side. That is, the screen control unit 11 superimposes the image information relating to the position of the arm tip and the force / torque information acting on the arm tip and simultaneously displays the image.

As a display method on the display 12, a method of superposing force / torque information on the actual image captured by the camera 5 may be used, but the actual image is a CG (computer graphics) image. A method of superposing force / torque information on information may be adopted. That is, the environment model prepared in advance in the computer,
By displaying a virtual robot arm (represented by a computer) that operates in conjunction with the actual movement of the arm 3 based on the position information from the position / angle sensor 8, the tip of the arm is displayed. It is possible to feed back the situation of to the operator more clearly.

FIG. 3 shows a moving image of the tip of the arm displayed on the display 12. As shown in the figure, here, force / torque information is displayed as a vector so as to overlap the position of the pin P gripped by the hand 4. For the force / torque information here, the coordinate system of the pin P is adopted and only the three translational axes orthogonal to each other are displayed as a vector.

The operation of the present invention having the above configuration will be described. While monitoring the image information from the camera 5 on the display 12, the operator uses the joystick at hand to operate the robot 1 having a structure as shown in FIG. A pin P is attached to the hand 4 attached to the tip of the arm 3 of the robot 1.
Is being held, and the insertion work into the hole H provided in the object A is being performed.

In such work, since the operator operates the joystick which has only the function of instructing the position, it is usually impossible to reliably grasp the contact state between the pin P and the object A. . That is,
Since it is not possible to grasp any information about the contact force generated between the object A and the object A, whether the pin P is properly inserted in the hole H, or the pin P is inserted in an obliquely twisted state with respect to the hole H. There is a risk that the pin P may be bent or the object A may be damaged without knowing whether or not it is attached.

Therefore, according to the present invention, with respect to the display 12 which displays the image information obtained from the camera 5, the pin P is provided with a hole H by using the coordinate system of the hand 4 holding the pin P.
The contact forces F1, F2, and F3 in the translational three-axis directions that are generated when inserting / pulling in and out are displayed graphically on the image of the hand 4.

Here, since the coordinate system of the hand 4 is the center of two fingers, the origin of the vector is just the pin P.
It is displayed on the central axis of. Also, here, the magnitude of the force is displayed in proportion to the length of the vector.

Therefore, an operator working while looking at the display 12 can simultaneously confirm on the display 12 in which direction the load is currently applied to the pin P, and the pin P is overloaded. Work can be done in a non-intrusive manner. Since the operator can capture the force / torque information as image information, the arm 3 can be used without adopting the bilateral control method.
The force / torque information received by the operator is reliably fed back to the operator.

In particular, the operator often looks at the arm 4 such as the hand 4 or the pin P to perform the work. According to the present invention, by displaying the force / torque information on the arm tip in this way, the operator can work without looking at images or numerical data on another display or looking at other than the gazing point. The work efficiency can be significantly improved and the work risk can be sufficiently reduced.

Since the tip of the arm is constantly monitored during work, if the display position of the force / torque information is set to the tip of the arm, the force / torque information can be obtained even when the camera 5 is zoomed or moved closer. Always display 12
Will be displayed in, and the workability will be extremely improved.

Various display methods on the display 12 will be described below. In FIG. 3, the force / torque sensor 7
The force / torque information detected in was converted to the hand coordinate system and displayed. That is, the force displayed as a vector
The torque information represents the direction in which the force actually acts.

On the other hand, in the display method shown in FIG. 4, the vector is displayed in the direction opposite to that of the hand coordinate system shown in FIG. That is, in this display method, a vector in the direction opposite to the load is shown. Therefore, the operator drives the arm 3 in the direction of this vector, so that the load acting on the arm 3 is reduced.

By adopting such a display method, the operator can immediately and intuitively determine in which direction the arm 3 should be retracted even if the pin P is twisted or otherwise twisted. Can be useful in an emergency.

In the display of FIG. 4, it is necessary to consider that the color of the vector is changed from that of the case of FIG. 3, and the display method of FIG. It is also possible to switch and use the display method of FIG. For example, the display method of FIG. 3 can be set as the normal mode, and in the case of complicated work, the work can be performed by switching to the special mode of FIG. In FIG. 4, the vector display is outlined.

The coordinate system for displaying force / torque information is a sensor coordinate system based on the position of the force / torque sensor 7, a coordinate system based on the base of the arm 3 (joint 6a), and a hand 4 It is preferable to prepare a plurality of coordinate systems based on, such as a coordinate system, depending on the work content and the degree of recognition of the operator, and set them so that arbitrary ones can be displayed.

Such various coordinate systems are subjected to arithmetic processing by the above-mentioned screen control section 11 and coordinate conversion section 14 in accordance with the input designation from the personal computer or the like by the operator, and the display 12
Displayed appropriately above.

For example, FIG. 5 shows the translational force in the sensor coordinate system with the position of the force / torque sensor 7 as a reference. Such a display is realized by adopting the conversion E'in the sensor coordinate system of the force / torque sensor 7 viewed from the other end of the arm 3 in place of E in the above-mentioned formula (1).

Further, here, the hand 4 has a cylindrical pin P.
However, if there are other parts handled by the hand 4, various coordinate systems corresponding to the shapes of those parts are prepared so that they can be selected as needed. deep. Also, when using another tool (end effector) instead of the hand 4, the coordinate system of the tool should be selectable.

Further, the force / torque information may be changed in length or thickness in proportion to the size of each component along the coordinate axis, or may be displayed as a combined vector. In FIG. 6, the translational force in the hand coordinate system is displayed as the thickness corresponding to the magnitude of the force.

If a part of the vector coincides with the depth direction of the screen, there arises a problem that the force / torque information is not accurately transmitted to the operator. Therefore, it is preferable that the display 12 is displayed within a range in which a predetermined angle is maintained so that the depth direction of the display 12 does not match any component of the coordinate system of the force / torque information. At this time, the vector in the direction close to the depth direction of the display 12 may be displayed in a conspicuous color.

When a large force or torque of a certain value or more acts on the arm 3, the object A or the arm 3
Is more likely to be damaged. Therefore, regardless of how the coordinate system is selected, it is preferable to transmit the force / torque information by a special method such as changing the color or generating an alarm when a force / torque of a certain value or more is applied.

FIG. 7 is an example in which the translational forces for three axes and the moment force around each axis are all displayed on the coordinate axis. As described above, it is possible to display all the six degrees of freedom detected by the force / torque sensor 7 on the display 12, but on the contrary, there is a risk that the operator cannot immediately and intuitively understand. Therefore, it is preferable to display only necessary force information according to the work content, the tool (end effector) used, and the like.

For example, in FIG. 8, only the translational force in the Z-axis direction and the moment My about the Y-axis are displayed conspicuously, and the other force information is displayed inconspicuously (for example, by a broken line). . Such a display method is, for example, Y
Z tools that rotate around the axis (such as drills and grinders)
It is particularly useful when used by moving in the axial direction.
Although this is an example in which only force information with only two degrees of freedom is displayed, it may be easier for the operator to understand the situation by selectively displaying force / torque information in this way.

Not only the work content and the tool to be used, but also the translational force or moment information is displayed only when the translational force or moment exceeds a certain value, or only the maximum translational force or moment information is displayed. It may be displayed.

The actual image from the camera may be used as it is on the display screen, and only the force / torque information may be superimposed and displayed. Further, as described above, the arm tip portion may be displayed in CG, and the force / torque information actually acting on the arm may be displayed thereon.

In practicing the present invention, the operating device handled by the operator is not limited to the joystick, but a simple input device such as a mouse may be used, and in some cases, a switch or the like may be substituted. Of course, it is also possible to use a master arm as used in the conventional bilateral control system.

Further, a pressure sensor or a displacement sensor may be used instead of the force / torque sensor, and the arm 3 may be distributedly arranged in various places so that the direction of the force / torque can be understood.
Further, the present invention is not limited to a robot intended for remote control, and may be used as a display for monitoring work in an autonomous work robot or an automatic work robot that does not have a pilot operation. As a result, the robot work can be executed more safely and reliably.

[0050]

As described above, according to the present invention, the operator can capture the force / torque information as image information, so that the force / torque information received by the slave arm can be obtained without adopting the bilateral control system. Can be reliably fed back to the operator.

[Brief description of drawings]

FIG. 1 is a block diagram showing a robot control method of the present invention.

FIG. 2 is a conceptual diagram showing an example of a working state by a robot.

FIG. 3 is a diagram showing an example of a display method of an arm tip portion.

FIG. 4 is a diagram showing another example of the display method of the arm tip portion.

FIG. 5 is a diagram showing another example of the display method of the arm tip portion.

FIG. 6 is a diagram showing another example of the display method of the arm tip portion.

FIG. 7 is a diagram showing another example of the display method of the arm tip portion.

FIG. 8 is a diagram showing another example of the display method of the arm tip portion.

[Explanation of symbols]

 1 ... Robot 2 ... Robot body 3 ... Arm 4 ... Hand 5 ... Camera 6 ... Joint 7 ... Force / torque sensor 8 ... Position / angle sensor 9 ... Camera information detector 10 ... Arithmetic unit 11 ... Screen control unit 12 ... Display 13 … Coordinate conversion unit A… Object H… Hole P… Pin

Claims (7)

[Claims] 1. A robot characterized by displaying position information of an arm tip portion of a robot as an image and superimposing force / torque information acting on the arm tip portion on the image information. Control method. 2. A method of controlling a robot, characterized in that the force / torque information acting on the arm tip portion is superimposed on the moving image information of the position of the arm tip portion of the robot and displayed as an image. 3. The force / torque information acting on the arm tip portion obtained from a force / torque sensor attached to the robot is superimposed on the image information of the arm tip portion obtained from a camera attached to the robot. A robot control method characterized by displaying images together. 4. The robot control method according to claim 1, wherein the image information of the arm tip portion is image information subjected to CG (computer graphic) processing. 5. The robot control method according to claim 1, wherein the force / torque information is displayed so as to overlap the image information of the arm tip portion. 6. The robot control method according to claim 1, wherein the force / torque information is displayed as a vector image. 7. The robot control method according to claim 1, wherein only a part of the force / torque information is selectively displayed.