JPH06114766A - Remote control device for manipulator - Google Patents

Abstract

PURPOSE:To improve the operability series by independently computing a relative state by an operation series control system and a working series control system operated by means of a work message. CONSTITUTION:A ground series control system as an operation steries control system comprises a central processing layer CPU 101, an image measuring means 102 connected thereto, a graphic display 103, and an input output device 104. An on-track control system as a working series control system comprises a central processing unit CPU 101, a image measuring device 102, and a slave arm input output device 105. An on-track series system is caused to execute a work according to a command from the ground series control system. The respective CPUs 101 for the two systems individually control a work environment model, and perform control based on respective image measurement results such that working environment where the working is actually executed is kept equal to a work environment model, and work image level communicated is carried out therebetween.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a manipulator operated by an operator, and more particularly to a manipulator remote control device suitable for reliably performing work in an environment and human space that are intolerable to humans.

[0002]

2. Description of the Related Art Conventional manipulators have tended to rely on the skill of the operator to perform complex manipulator operations. For example, in a master-slave manipulator, the master arm and the slave arm have the same or similar shapes, so that the master arm has a structure unsuitable for human operation, or the operator himself interferes with the operation of the master arm. It was so big that it was impossible to perform detailed operations.

In the remote operation, the manipulator itself obstructs the visual field of the operator, and the operator sometimes has to operate while moving the face left and right. For this reason, ITV and the like have been used, but one image cannot provide a sufficient field of view, and when a large number of TV cameras are arranged, the relationship between the image and the manipulator changes, and the operator may make an operation error. .

In recent years, due to rapid progress in computer technology, computers have been incorporated into the control of manipulators and robots, and the control has become more sophisticated. As an example, the 4th of I-Econ '84
As shown on page 0 to page 45, there is one in which the motions of the hand of a master arm and a slave arm having different shapes are made to correspond one-to-one by a high-speed coordinate conversion calculation using a computer.

[0005]

The above-mentioned conventional technique is
It can be said that it has not been completed as a system technology that includes human beings by examining manipulators from a comprehensive viewpoint, while remaining in each narrow technical field. For example, even in the case of the above-described high-speed coordinate conversion calculation technique, since the two points, that is, the reference coordinates of the master arm and the position of the hand of each arm, are matched, the motion ratio of each arm is constant. Has become. For this reason, when the slave arm is to perform a precise work, the master arm must perform a fine and minute operation as required by the slave arm, and on the contrary, the slave arm requires a large operation. If this is the case, the master arm must also be moved largely.

As described above, the work of the slave arm puts a heavy burden on the operator, and the feasibility of the work is determined by the technique.

In addition to the operator's burden, I
When operating while watching TV, the master arm is operated by looking at the image and considering the direction of the TV camera, which increases the mental burden.

Further, in the case of operating an artificial satellite device moving on the satellite orbit of space from the ground via a manipulator, the operation signal reaches the satellite after the master arm on the ground is operated, and the manipulator is operated. It takes a few seconds for the image to reach the ground. Therefore, the image monitored on the ground shows a state before the actual state in time, and the illusion is that the manipulator arm has actually reached the required position but has not yet arrived. Then the arm is overrun.

An object of the present invention is to provide a remote manipulator operation device capable of improving the operability of a master arm operated by an operator.

[0010]

The above-mentioned object is to provide an operation system management system operated by an operator and a work system management system for controlling a manipulator for performing work based on a work instruction issued from the operation system management system. Each has its own central processing unit, manages the environment model that individually expresses the state of the work place and the state of the work target, and the work management system carries out work according to the work instructions issued from the operation management system. In addition, the operation system management system performs the same calculation as the work system management system independently of the work system management system and displays the calculation result.

[0011]

[Operation] An operation system management system operated by an operator,
The work system management system that controls the slave arm has its own central processing unit, individually manages the environment model, and the operation system management system and the work system management system independently calculate the mutual situation. Match the environment models managed by each. By matching the environment models managed by each, it becomes possible to perform work image level communication between the operation system management system and the work system management system. Further, with such a system, in the operation system management system, it is possible to simulate the real-time operation of the slave arm and display it on the screen together with the environment model. By simulating the operation in real time and displaying it on the screen, deterioration of operability due to communication delay can be eliminated and communication reliability can be improved.

[0012]

【Example】

(First Embodiment) A first embodiment of the present invention will be described below with reference to the drawings.

First, prior to describing an embodiment of the apparatus of the present invention, the operating principle of the present invention will be described with reference to FIG. The master-slave manipulator shown in FIG.
It comprises a master arm 201 and a slave arm 202 having a structure different from that of the master arm 201. Now
The reference coordinate system of the master arm 201 is M, the coordinate transformation matrix from this reference coordinate system M to the tip of the master arm 201 is T ₆ m, the reference coordinate system of the slave arm 202 is S, and the reference coordinate system S is referred to as the slave arm 202. Assuming that the coordinate conversion matrix up to the hand of the above is T ₆ s and the scale conversion matrix between the coordinate conversion matrices T ₆ m and T ₆ s described above is K, the coordinate conversion calculation procedure is as follows.

That is, the coordinate conversion matrix T ₆ m from the above-mentioned master arm end to the above-mentioned coordinate conversion matrix T ₆ m can be obtained from the parameter of each link of the master arm 201 and the position of each link connecting axis. Then the coordinate transformation operation contains a scale transformation and origin shift between the coordinate transformation matrix T ₆ s to coordinate transformation matrix T ₆ m and the slave arm the hand up to the master arm hand is expressed by the following equation (1) Be done.

T ₆ s = K · T ₆ m (1) where K is

[0016]

[Equation 1]

Then, when d = e = F = 0, T ₆ s is T ₆ m a times in the x axis direction of the reference coordinate system M of the master arm 201, b times in the y axis direction, and z times in the z axis direction. It will be c times. Then, in the case of uniformly expanding in the three axis directions, a = b
= C will do. Then, the link parameter of the slave arm 202 is given to the conversion matrix T ₆ s obtained by the above-mentioned equation (1), and the target value of each axis of the slave arm is obtained by the inverse coordinate conversion calculation. If the target value of each axis of the slave arm 202 obtained in this way is servo-controlled for each axis, the master arm 2 will move within the movable area of each arm regardless of the shape of the arm.
The slave arm 202 can be operated by arbitrarily expanding or contracting the motion of 01.

Next, when a = b = c = 1, T ₆ s is T ₆
m in the x-axis direction of the reference coordinate system M of the master arm 1, d, y
It is a parallel translation of c in the direction and f in the z direction. Then, the conversion matrix T ₆ obtained by the above equation (1)
The link parameter of the slave arm 202 is given to s, and the target value of each axis of the slave arm 202 is obtained by the inverse transformation calculation. If the target value of each axis of the slave arm 202 thus obtained is servo-controlled for each axis, the reference point of the master arm and the reference point of the slave arm can be arbitrarily shifted to operate.

Next, based on the principle of the present invention described above,
An embodiment of the device of the present invention will be described with reference to FIG. The master-slave manipulator shown in FIG. 4 includes a master arm 201 and a slave arm 202 having a structure different from that of the master arm 201 and working on a work target 301. The central processing unit 101 performs coordinate conversion calculation and control of both arms. The coordinate conversion adding device 218, which is a feature of the present invention, creates K of the equation (2) based on the input from the keyboard of the input / output device 217, and the central processing unit 1
An instruction to add the coordinate transformation shown in the equation (1) is given to 01.
The input / output device 204 of the master arm 201 inputs each joint angle and the like of the master arm 201 to the central processing unit 101, and also servo-amplifies a command value from the central processing unit 101 to the master arm 201, so that Acts to drive joint actuators. The input / output device 206 of the slave arm 202 is
Each joint angle and the like are input to the central processing unit 101, and the command value from the central processing unit 101 to the slave arm 202 is servo-amplified to drive the actuator of each joint of the slave arm 202.

By switching inside the central processing unit 101, the joystick 207 outputs a command value for the slave arm 202 instead of the master arm 201 and an animation image described later. The input / output device 208 of the joystick 207 inputs a signal of the tilt angle of the joystick 207 to the central processing unit 101, servo-amplifies a force feedback signal from the central processing unit 101 to the joystick 207, and drives the actuator of the joystick 207. It has the function of driving. The image input device 209 is
The work of the slave arm 202 is monitored from various directions. The image processing device 210 performs an operation such as FFT on the image input by the image input device 209 to perform pattern identification and the like. The simulation means 211 is an animation computer that displays the state of the slave arm in real time by computer graphics. In addition to displaying the animation image by the simulation means 211, the operator presentation screen 103 can also display the animation image and the real image input by the image input device 209 in a superimposed manner. Further, menu display such as mode switching is started and the center is displayed. Processing device 101 and simulation means 2
It is also used for output of the man-machine interface with 11. The TV monitor 213 has an image input device 209.
The image of is displayed. The input / output device 217 is used for inputting a scale conversion constant, a base point shift instruction menu, and an interactive message input of a man-machine interface between the central processing unit 101 and the simulation means 211.

Next, the main flow of signals in the present embodiment will be described.
The operation system management system will be described with reference to FIG. 2 and the work system management system part shown in FIG. It should be noted that although the actual one has 6 degrees of freedom for each of the master arm, the slave arm, and the joystick, the following description will be made with 3 degrees of freedom for simplicity.

Position detecting sensors 214A to 214C are provided on each joint axis of the master arm 201. The signals of the position detecting sensors 214A to 214C are processed by the master arm input / output device 204, and the simulation means 21.
Input to 1. Position detection sensors 215A to 215C are provided on each joint axis of the slave arm 202,
The signals of the position detection sensors 215A to 215C are processed by the slave arm input / output device 206 and input to the manipulation means 230. Manipulation means 2
At 30, the coordinate conversion calculation to the generalized coordinates is performed, and further the coordinate conversion to the second generalized coordinate is performed based on the instruction of the coordinate conversion adding device 218, so that the slave arm 202
Determine the specified value for. This signal drives the actuators 216A to 216C provided on each joint axis of the slave arm 202 via the slave arm input / output device 206.
At the same time, a signal is sent from the manipulation means 230 to the simulation means 211, and an animation image of the slave arm is displayed on the operator presentation screen 103.

The joystick is provided with position detection sensors 217A to 217C for detecting its angle.
The signals from the position detection sensors 217A to 217C are processed by the joystick input / output device 208 and input to the simulation means 211. Simulation means 211
Then, when a command to the slave arm is determined, it is determined based on a switching signal from the outside, which signal from the master arm or the joystick is referred to. Further, the image input by the image input device 209 is the image processing device 2
After FFT and pattern identification are performed in 10, the real image is displayed on the television monitor 213 and at the same time sent to the simulation means 211, and displayed on the operator presentation screen 103 so as to be superimposed on the animation image as necessary.

Next, the position detecting sensors 214A to 214C
FIG. 5 shows a specific example of processing a signal from the master arm input / output device 204. This is the same for joysticks and slave arms. In FIG. 5, rotary pulse generators are used as the position detection sensors 214A to 214C. This position detection sensor 214A
From -214C, a pair of pulse signals 90 degrees out of phase, that is, A-phase and B-phase are generated according to the rotation angle, and are input to the direction determination circuit 401 to determine the direction of the rotation angle. On the other hand, the A-phase or B-phase signal is input to the counter 402 and the number of pulses is counted. The direction discrimination circuit 4
The direction signal 403 output from 01 is input to the counter 402, and the increase / decrease of the pulse number is switched. Therefore, the value of the counter 402 increases / decreases in accordance with the increase / decrease of the rotation angle, so that the rotation angle can be detected by reading the output 404 of the counter 402 from the outside.

FIG. 6 shows a concrete configuration example of the central processing unit 101. In this, a processor 501 for controlling input / output of data and addition / subtraction, a memory 502 for storing data such as a trigonometric table and link parameters of a manipulator,
The multiplier 503 and the divider 504 are connected to each other via a bus circuit 505. Further, the bus circuit 505 includes a serial or parallel interface circuit 506.
A to 506E are connected. Interface circuit 5
06A to 506E include arm input / output devices 206, 20
4, a joystick input / output device 208, a simulation means 211, and a coordinate conversion adding device 218 are connected to each other. The processor 501 is a bus circuit 505.
All devices connected to the bus circuit 505 can be accessed through the to process data.

Next, the operation of the above-described embodiment of the apparatus of the present invention will be described. When the master arm 201 is operated, each joint angle of the master arm 201 is detected by the position detection sensor 214A.
Detected by ~ 214C. This detection signal is sent to the simulation means 2 via the master arm input / output device 204.
11 is input. The simulation means 211 calculates the relative positional relationship of the hand coordinate system MC of the master arm 201 with respect to the master arm reference coordinate system M by a coordinate conversion matrix T _6.
It is stored as m, and a coordinate conversion operation to generalized coordinates is performed. Further, the coordinate conversion adding device 218 is the
4, the dimensional ratio of the hand movement of the slave arm 202 to the hand movement of the master arm 201, that is, the scale conversion constant, and the master arm 201.
A matrix K indicating the shift amount of the reference point between the position of the tip of the slave arm 202 and the position of the tip of the slave arm 202 is stored, and the simulation unit 211 is instructed to add coordinate conversion. Then, the simulation means 211 performs an operation to apply K to the master arm coordinate conversion matrix T ₆ m to obtain the slave arm coordinate conversion matrix T ₆ s. Next, the simulation means 211 makes the slave arm 2 when the relative position of the hand coordinate system SC of the slave arm 202 to the slave arm reference coordinate system S matches the slave arm coordinate conversion matrix T ₆ s.
The joint axis target value of 02 is obtained by inverse coordinate transformation calculation, and this is output to the slave arm input / output device 206.

The slave arm input / output device 206 drives the actuators 216A to 216C. This allows
The movement of the hand of the master arm 201 can be transmitted to the movement of the hand of the slave arm 202 by performing scale conversion, shift of the base point, or both. As a result, in the movable region of each arm, the movement of the hand of the master arm 201 can be arbitrarily enlarged or reduced and transmitted to the hand of the slave arm 202 regardless of the shape of the arm, and the operation of the master arm 201 can be performed. On the other hand, the slave arm 202 can be finely moved, and a rough but large movement can be given. If necessary, the simulation means 211 temporarily disconnects the slave arm 202 from the master arm 201 and makes it stand still by a command from the coordinate transformation adding device 218 by an input from the input / output device 104, and the operator operates the master. Only the arm 201 can be moved to any position. With the master arm 201 moved to the required position, the coordinate conversion adding device 218
The shift amount between the positions of the master arm 201 and the slave arm 202 is stored in the d, e, and f portions of the equation (2) again. In this state, the operation reference points of the master arm 201 and the slave arm 202 can be freely set again by interlocking the master arm 201 and the slave arm 202 with the input of the input / output device 104 again. It can be moved at a position where the operator can easily operate it.

On the other hand, the signals from the joystick position detection sensors 217A to 217C are input to the simulation means 211 via the joystick input / output device 208. The simulation means 211 temporally integrates this signal and stores the relative positional relationship of the virtual hand coordinate system JC of the joystick 207 with respect to the joystick reference coordinate system J as a coordinate conversion matrix T ₆ j. Then, when calculating the above-mentioned T ₆ s, by using T ₆ j instead of T ₆ m by a switching signal from the outside,
The slave arm 202 can be moved by manipulating the joystick 207. Master arm 20
1 outputs a command to the slave arm 202 as a position command, and the joystick outputs a command to the slave arm 202 as a speed command, so that the operator can select the one that is easy to use according to the situation. Joystick 2
Even when 07 is used, there is an advantage that the moving speed of the slave arm when the joystick is tilted can be freely set by scale conversion.

Further, instead of outputting the second generalized coordinates, that is, KT ₆ m of the equation (2) to the slave arm input / output device 206, or outputting to the slave arm input / output device 206, it is also output to the simulation means 211. As a result, on the operator presentation screen 103,
It can be seen as an animated image. By displaying the animation image without moving the slave arm 202, the movement due to the effect of scale conversion or the like can be confirmed, so that a dangerous situation does not occur.

Furthermore, when operating the slave arm in space from the ground, the delay time of communication can be used to see the simulated image faster than the actual movement of the slave arm, and to check the erroneous operation of the master arm. It can also be used to modify. If you want to know the actual (real-time) movement of the slave arm, you can display the animation image with a delay of communication delay.

When an animation is displayed at the same time as the slave arm 202 is moved, the animation can be displayed as seen from any direction, so that the operability is improved.

Further, the image input by the image input device 209 can be displayed on the television monitor 213 and can be displayed over the animation image on the operator presentation screen 103. As a result, more realistic image information can be obtained, and the part of the real image that is a shadow of the object in the foreground and becomes a blind spot can also be displayed in the animation image, so that the real image and the animation can complement each other's defects. it can.

At this time, the coordinate conversion adding device 218 needs to determine the conversion matrix K to the second generalized coordinates so that the animation image overlaps the actual image without shifting.

Further, as shown in FIG. 4, since there are a plurality of image input devices 209 and the slave arm is viewed from various directions, when the operator operates while watching the television monitor 213, the master arm is operated. The correspondence with the direction of 201 must always be kept in mind, and it becomes difficult to perform the base point shift and the like. Therefore, when the slave arm coordinate conversion matrix T ₆ s is obtained, if the coordinate conversion matrix for correcting the direction of the i-th image input device 209 and the slave arm 202 is Ri, then T ₆ s = Ri · K · T ₆ m As a result, by obtaining T ₆ s, the operator can operate the master arm 201 without being aware of the difference in direction. Ri · K is provided by the coordinate transformation adding device 218.

Here, the roll angle, the pitch angle and the yaw angle in the direction of the line of sight of the i-th image input device 209 are αi, βi and γ.
If i is given, Ri is given by the following equation.

[0036]

[Equation 2]

By introducing such a coordinate transformation matrix R, a signal corresponding to thermal deformation and inertial deformation of the slave arm in space can be given to the slave arm,
Operability is improved. For the above purpose, the roll angle, pitch angle, yaw angle of the posture of the tip due to arm deformation are δ, ε,
Let ζ be the displacement positions in the x, y, z directions due to strain, p, q, r

[0038]

[Equation 3]

It can be expressed as

Next, another embodiment of the present invention will be described with reference to FIG. In this embodiment, scale conversion calculation is performed on the minute displacement of the hand of the master arm 201, and the result is transmitted to the slave arm 202 as the minute displacement of the hand of the slave arm 202.
Then, similarly to the embodiment shown in FIGS.
The reference coordinate system of 01 is M, the coordinate conversion matrix T ₆ m from the reference coordinate system M to the hand of the master arm 201, and the conversion matrix for scale conversion calculation and the like. Further, the reference coordinate conversion matrix of the slave arm 202 is set to T ₆
If s, the procedure of the coordinate conversion calculation is as follows. That is, the coordinate conversion matrix T ₆ m is obtained from each link parameter of the master arm 201 and the position of each joint axis thereof. Further, if each link parameter of the slave arm 202 and the coordinate conversion matrix T ₆ s indicating the position of its hand are given, the target value of each axis of the slave arm 202 can be obtained.

Assuming that the movements of the master arm 201 and the slave arm 202 are synchronized at a certain point in time, the following relationship exists between the minute displacement dT ₆ of the hand position and the minute displacement dQ of each axis of the manipulator.

DT ₆ = JdQ (3) (J: Jacobian matrix) Now, when the master arm 201 is subjected to minute movement MD, the change in displacement of each joint axis is set to dQ, and the master arm 20
When the Jacobian matrix of 1 is J, the minute movement dT ₆ m of the hand of the master arm 201 is obtained from the following equation.

DT ₆ m = Jm · dQm (4) Here, the scale conversion of dT ₆ m is performed and the slave arm 20
The second minute movement dT ₆ s is obtained from the following equation.

DT ₆ s = KdT ₆ m (5) Next, a minute displacement dQs of each joint axis of the slave arm 202.
Is the inverse matrix 1 / of the Jacobian matrix Js of the slave arm 202.
Obtained by solving (Js). That is, dQs = 1 / (Js) · dT ₆ s (6) The small displacement dQs of each joint axis of the slave arm 202 obtained by the above equation is added to the position of each joint axis of the slave arm 202, and this is added to the slave. The target value of the servo control circuit of each joint axis of the arm 202 is set.

Next, another embodiment of the control device of the present invention will be described with reference to FIG. 8 based on the above principle. Since the handling of the joystick 207, the image input device 209, the simulation means 211, etc. is based on the first example, only the relationship between the master arm 201 and the slave arm 202 will be described here.

In FIG. 8, the same reference numerals as those in FIGS. 2 and 3 denote the same or corresponding portions. 701 is a difference circuit,
Reference numeral 702 is an incremental circuit. The difference circuit 701 detects a change amount of the sensor signals of the sensors 214A to 214C at the sampling time. The central processing unit 101 has been described above.
Perform the calculations shown in equations (3) to (6) and set slave arm 2
The change amount of each joint axis 02 is obtained, and this change amount is output to the increment circuit 702. Incremental circuit 702 is slave arm 2
The change amount obtained by the central processing unit is input to the current target value for each joint axis 02. The slave arm input / output device 206 drives actuators 216A to 216C provided on each joint axis of the slave arm 202. The slave arm 202 is driven by these actuators 216A to 216C, and the amount of movement is detected by the detectors 215A to 215C.
Is detected by and is fed back to the slave arm input / output device 206. As a result, the master arm 201
It is possible to scale the movement of the hand of the robot and transmit it to the hand of the slave arm.

The arithmetic processing operation of the arithmetic circuit in another embodiment of the above-described control device of the present invention will be described with reference to FIG. First, when starting at the initial position, the initial value of each joint of the master arm 1 is read. Next, the joint angles of the master arm 1 and the slave arm are input, and the change amount dQm of the joint angle is obtained from the difference from the previous data. Next, the triangular relationship is obtained by referring to the table, and the Jacobian matrix Jm of the master arm 1 is calculated. From the joint angle variation dQm and the Jacobian matrix Jm, the hand end displacement dT ₆ m of the master arm 1 is obtained.
The scale conversion constant K is obtained using the input data.
The hand end displacement dT ₆ m of the master arm is multiplied by K to obtain the hand end displacement dT ₆ s of the slave arm. Next, the Jacobian inverse matrix 1 / Js of the slave arm is obtained. 1 to this dT ₆ s
The joint angle displacement dQs of the slave arm is obtained by multiplying / Js, and the sum of the joint angles Qs and dQs of the slave arm is calculated and the result is output to each servo system of the slave arm. The above procedure is repeated until the end of operation.

According to this embodiment, the same effect as that of the embodiment shown in FIGS.
The slave arm 202 and the slave arm 202 can be synchronously started wherever the hand position is, and any scale can be displaced.

FIG. 11 shows still another embodiment of the apparatus of the present invention, in which the same reference numerals as those in FIGS. 2 and 3 designate the same parts. In this embodiment, the slave arm 2
The scale conversion constant of the motion of the slave arm 202 with respect to the motion of the master arm 201 is changed according to the change of the zoom ratio of the image input device 209 that projects the hand of 02. A sensor 1001 that detects the movement of the zoom lens is provided, and this sensor information is input to the coordinate conversion adding device 218. The coordinate conversion adding device 218 determines the scale conversion matrix K by performing a correction calculation using this sensor information and the data obtained in advance, and adds the coordinate conversion to the central processing unit 101 as in the above-described embodiment. The central processing unit 101 calculates a target value for causing the movement of the slave arm 202 to perform scale conversion operation with respect to that of the master arm 201. In FIG. 11, 213 is a television monitor.

Next, the operation of another embodiment of the above-mentioned device of the present invention will be described with reference to the flow chart shown in FIG.
First, when starting at the initial position, each joint angle of the master arm 201 is read. Next, the value of the trigonometric function is obtained by referring to the table, and the hand coordinate T ₆ m is obtained using the obtained value of the trigonometric function. As described above, when synchronizing with the zoom lens of the image input device 209, the zoom ratio is detected by the sensor 1001 attached to the zoom lens, and the scale conversion matrix K is determined. When not synchronized with the zoom lens, the scale conversion matrix K input in advance is used. Next, the master arm 20
Multiply K by the hand position T ₆ m of 1 and slave arm 202
Find the hand position T ₆ s. The target value of each joint of the slave arm 202 is obtained from this T ₆ s by the inverse coordinate transformation calculation, and this is output to each servo system of the slave arm. The above operation is repeated until the operation ends.

With this configuration, even if the zoom ratio of the image input device 209 is arbitrarily changed, the movement of the hand of the master arm 201 and the movement of the hand image of the slave arm 202 on the television monitor 213 can be adjusted. The ratio can always be kept constant. As a result, a proper operation feeling is always obtained and operability is improved.

(Second Embodiment) 1. Description of Second Embodiment A second embodiment of the present invention will be described with reference to FIG.

This embodiment comprises an operation system management system and a work system management system. The operation system management system includes a central processing unit 101, an image measuring means 102 connected to the central processing unit 101, and a central processing unit. Graphic display 103 connected to 101 and image measuring means 102
And an input / output device 104 connected to the central processing unit 101
The work management system consists of the central processing unit 1
01, an image measuring means 102 and a slave arm 105 connected to the central processing unit 101. In this system, for example, the operating environment management system and the central processing unit 101 of the working management system separately manage the working environment model in orbit. The work environment in which the work is actually performed (for example, in a spacecraft in orbit) and the work environment model simulating this work environment (the environment built in the computer) are centered so that they are always equal based on the image measurement results. It is managed by the processor. In addition, when the slave arm on the orbit performs work by a command from the ground, the central processing unit of the operation system management system simulates the work in accordance with the progress of the work on the orbit, thereby creating a work environment model. It matches the work environment. The graphic display 103 shows the status of the system to the operator and serves as an interface for interactive operation between the operator and the system using the input / output device 104. The slave arm input / output device 105 inputs / outputs information for controlling the slave arm.

2. Outline of Second Embodiment A system configuration will be described with reference to FIGS. 2 and 3.

This system comprises an operation system management system 10
6 and a work management system 107.
The operation system management system 106 includes a simulation unit 211, an image measurement unit 102 connected to the simulation unit 211, an operator presentation screen 103,
Input / output device 104, joystick input / output device 20
8, 228, 2i8, joystick 207, 2i
7, master arm input / output devices 204, 224, 2j4,
It is configured to include master arms 201 and j1. The work system management system includes a manipulation unit 230 and slave arm input / output devices 206, 226, 2 m connected to the manipulation unit 230.
6, slave arm 202, 2m2, image measuring means 10
2 and a camera 209 connected to the image measuring unit 102,
229 and 2n9. The joystick input / output device, the joystick,
There may be a plurality of the master arm input / output device, the master arm, the slave arm input / output device, the slave arm, the image measuring unit, and the camera.

The simulation means 211 and the manipulation means 230 are communicably connected to each other by a communication means.

In the present embodiment, a ground operation system management system (hereinafter referred to as a ground system management system or simply a ground system) is provided for the purpose of rationalizing work performed in an orbit system including space work equipment and a work object. The database required for the work is autonomously generated through an interactive operation with the operator presentation screen 103 installed in.

(1) Creation of Environmental Model In order to equalize the work environment on the orbit and the environment model 1107 of the ground system (refer to the description of 5.3 Environment model), first, the environment model 1107 is created. This is a work of newly registering an unregistered object.

The operator uses the input / output device 104 such as a mouse or a keyboard to set the environment model 1107.
If you give the name of the object you want to register to and the approximate position,
The image measuring means 102 described in “Construction apparatus (Japanese Patent Laid-Open No. 2-209562)” measures an accurate position and posture of an object placed on the track and registers it in the environment model 1107. After that, the image measuring unit continues to measure the position and orientation of the registered object in a cycle of 1 second or less, and constantly updates the environment model 1107. The image measuring means is a ground management system 10
6 and an operation system management system (hereinafter, referred to as an on-orbit system management system or simply an on-orbit system) 107,
The position and orientation of the work target object is measured from the images captured by the cameras 209, 229, and 2n9.

(2) Planning of on-orbit work The plan of on-orbit work is interactive with the operator presentation screen 103 by expressing the work concept level on the environment model 1107 created by the operator (hereinafter referred to as work message). It is done by operation. The terrestrial management system 106 creates a work network from the work message input by the operator using the input / output device 104. The work network is such that minute changes in the work environment or mechanically occur,
Since it uses a description method that is abstracted to the extent that it is not affected by errors, it can be stored for a longer time than robot languages.

(3) Information communication between ground system and on-orbit system From ground system management system 106 to on-orbit system management system 1
To 07, the work network and the on-orbit system environment data (work target object name, object position and orientation) observed in the ground system are sent. From the on-orbit system to the terrestrial system, image data, orbital system environment data (work target object name,
Object position / posture, slave arm position / posture) is sent.

(4) Execution of planned work by on-orbit system and work monitoring on ground system Referring to FIGS. 12 and 13 showing the simulation means 211 of the ground system and the manipulation means 230 of the on-orbit system, respectively, the on-orbit system The execution of planned work and the monitoring of work on the ground will be explained. Based on the environment model 1107, work networks are independently developed in the on-orbit system and the ground system, and a robot language is generated.
The robot language is data that is dynamically generated based on the image measurement value. The robot language is immediately converted into time series data of the joint angles of the robot, and the ground system performs work simulation, and the orbital system operates the robot. The simulation of work performed on the ground system is used for monitoring the work actually performed by the on-orbit system. When an abnormality occurs in the system, the ground system monitoring means 1104 and the on-orbit system monitoring means 1204 independently or in cooperation (the communication abnormality can be detected in cooperation) detect the abnormal portion and respond accordingly. To do.

3. Operation system management system (ground system management system) 3.1 Description of configuration Simulation means 21 of the ground system management system 106
As shown in FIG. 12, an offline simulator 1100, an online simulator 1101 connected to the offline simulator 1100 and a ground system monitoring means 1104, and an offline simulator 1100.
And an environment model management unit 1103 connected to the online simulator 1101, and an offline simulator 1
100 and an environment model management means 1103 connected to an animator 1102, and an offline simulator 1100.
Data receiving device 1105 connected to the Internet and data transmitting device 110 connected to the online simulator 1101
6 and an environment model 1107 connected to the environment model management means 1103. The off-line simulator 1100 includes a work network generating means 11
08 and interface control means 1109 connected to the work network generation means 1108.
The online simulator 1101 includes a work network management unit 1110 and a robot language generation unit 1111 connected to the work network management unit 1110.
And a robot language interpretation means 1112 connected to the robot language generation means 1111. The environment model management means 1103 is the class frame search means 11
13, an instance frame generating means 1114 connected to the class frame searching means 1113, and an instance frame updating means 1115 connected to the instance frame generating means 1114. The environment model 1107 (refer to the description of 5.3 environment model) includes a class frame and an instance frame.

3.2 Description of Functions The offline simulator 1100 has the environment model 110.
7 and the on-orbit work plan are created by using an interactive operation with the operator presentation screen 103. The work network generation means 1108 generates a work network (see 5.2 Description of work network) based on the work message input by the operator using the input / output device 104. The interface control means 1109 uses the operator and the operator presentation screen 10
Controls the interactive operation with 3 (For details of the control, refer to the section of 7. Screen for presenting operator).

The online simulator 1101 generates data for executing the on-orbit work represented by the work network. Work network management means 1110
Manages the work network and transfers it to the robot language generation means 1111 in response to the operator's request. The robot language generation means 1111 generates a robot language based on the work network. Robot language interpreting means 111
2 converts the robot language into time series data of joint angles.

The environment model management means 1103 performs the work by the operator's instruction input via the interface control means 1109, the measurement data input by the image measurement means 102, the work environment update accompanying the execution of the on-orbit work, and the like. The environment model 1107 is managed so that no contradiction occurs between the environment and the environment model 1107. The class frame search unit 1113 uses the input / output device 104 to extract the work target object name, that is, the class name input by the operator via the interface control unit 1109.
Search for class frames (see 5.3 Description of environment model). The instance frame generating means 1114 generates an instance belonging to the searched class. The instance frame update means 1115 updates the instance frame based on the image measurement value, the data passed from the interface control means 1109, and the robot language interpretation means 1112.

The animator 1102 is the environment model 110.
A graphic animation is created based on 7, and displayed on the operator presentation screen 103. The data receiving device 1105 receives the data sent from the orbital manipulation means 230. Data transmission device 110
6 transmits data to the manipulation means 230. The terrestrial system monitoring means 1104 constantly monitors the operating state of the system and notifies the offline simulator 1100 of any abnormality.

3.3 Description of Operation Next, the operation of the system when the environment model 1107 is created will be described. When the environment model 1107 is created, the target instruction process by the operator interrupt and the target object automatic measurement process by the system are simultaneously functioning. Here, one process is hierarchically configured by a plurality of processes, but each process is autonomously distributed. That is, each process starts and does some work only when a certain condition is satisfied. In such a system, there is no execution order (flow chart) between processes.

Therefore, first, the target designation process by the operator will be described. Interface control means 1109
Outputs an interactive message and a command menu to the operator presenting screen 103 through the animator 1102. Then, the operator displays the operator presentation screen 10
Input / output device 1 according to the dialogue message displayed in 3.
Use 04 to input the work target object class name. This class name is passed to the environment model management means 1103 via the interface control means 1109. The environment model management unit 1103 searches the environment model 1107 for a class frame (see 5.3 Description of environment model) corresponding to the class name, and generates an instance frame belonging to the class (see 5.3 description of environment model). Environment model 1107. Meanwhile, animator 1102
Always converts the environment model into drawing data and displays it on the operator presentation screen 103, so the update result of the environment model (in this case, the registration status of the work target object) is
Immediately, it is reflected on the operator presentation screen 103.

When the position of the work target object is corrected by the mouse, the interface control means 1109 transforms the mouse coordinates into three-dimensional coordinates (work target object position coordinates). Then, the instance frame updating means 111
5 receives this data and updates the position slot of the created instance (see 5.3 Description of environment model).

Next, the target object automatic measurement process by the system will be described. When the above process is not working, the image measuring unit 102 automatically measures the position and orientation of the work target object based on the image measuring request issued from the interface control unit 1109. Then, the instance frame updating means 1115 updates the position and orientation slot of the generated instance frame based on this measured value. Then, the animator 1102
These update statuses are output to the operator presentation screen 103.

Next, the on-orbit work plan and the operation of the system when the planned work is executed will be described.

The interface control means 1109 makes the operator presenting screen 103 via the animator 1102.
Display an interactive message and a command menu on. Accordingly, the operator uses the input / output device 104 to
Enter a work message. Interface control means 1
109 receives the input work message and passes it to the work network generation means 1108. Then, the work network generation means 1108 interprets the work message and generates a work network (see 5.2 Description of work network). The generated work network is stored in the work network management means 1110. Upon receiving the work execution request from the operator, the work network management means 1110 immediately passes the designated work network to the robot language generation means 1111. The robot language generation means 1111 uses a work environment model 1107.
, The working network is expanded to the level of a motion schema (the minimum unit of motion, details will be described in 5.2), and then converted into a robot language. The obtained robot language is passed to the robot language interpreting means 1112, which interprets the robot language to generate time series data of joint angles for moving the slave arm simulator, and updates the instance frame. Pass to means 1115. The instance frame update status of the environment model 1107 by the instance frame update means 1115 is immediately displayed on the operator presentation screen 103 via the animator 1102. At this time, the operator can monitor the work status of the robot installed on the track while looking at the operator presentation screen 103. When an image measurement request is issued in the process of generating a robot language, the environment model 1107 is modified based on the measurement result, and the environment model 1107 is converted into a robot language.

The ground system monitoring means 1104 periodically notifies the offline simulator 1100 of the operating state of the system (whether there is an abnormality). In addition, the ground system monitoring means 1104
Stops the system immediately upon receiving an emergency stop command from the offline simulator.

4. Work system management system (on-orbit system management system) 4.1 Description of configuration Manipulation means 230 of the on-orbit system management system 107 includes an online simulator 1101, an environment model management means 1103 connected to the online simulator 1101, and slave arm control. Device 1216 and data receiving device 1205, and slave arm control device 12
16 and a data transmission device 1206 connected to the 16; an on-orbit system monitoring means 1204; and an environment model 1107 connected to the environment model management means 1103. The online simulator 1101 includes a work network management unit 1110 and a robot language generation unit 1111 connected to the work network management unit 1110.
And a robot language interpretation means 1112 connected to the robot language generation means 1111. The environment model managing means 1103 is configured to include an instance frame updating means 1115 and is connected to the image measuring means 102. The environment model 1107 (see 5.2) is composed of a class frame and an instance frame. Data receiving device 1205 and data transmitting device 1206
Can communicate with the data transmitting device 1106 and the data receiving device 1105 of the terrestrial simulation means 211, respectively. Slave arm controller 1216
The satellite operation means 260 and the slave arm input / output device 105 are connected to the.

4.2 Description of Function The online simulator 1101 uses the work network to generate data for executing on-orbit work. The work network management means 1110 manages the work network and transfers it to the robot language generation means 1111 in response to a request from the operator or the on-orbit system monitoring means 1204 (see 6. System safety management).
The robot language generation means 1111 generates a robot language from the work network, and the robot language interpretation means 111.
Pass to 2. The robot language interpretation unit 1112 converts the robot language into joint angle time series data. The environment model management unit 1103 manages the environment model 1107 so that the work environment and the work environment model 1107 are always the same with respect to the measurement data by the image measurement unit 102 and the update of the work environment accompanying the execution of the on-orbit work. .

The data receiving device 1205 receives the data sent from the simulation means 211. The data transmission device 1206 transmits data to the simulation means. The on-orbit system monitoring means 1204 constantly monitors the operating state of the system and notifies the offline simulator 1100 of any abnormality. The slave arm control device 1216 creates slave arm control data,
Control data is passed to the slave arm input / output device 105.
When performing the attitude control, the slave arm control device 1216 creates attitude control data and sends it to the satellite operation means 260. As for the control data generation method, the method described in "Control device for artificial satellite equipped with movable object" (Japanese Patent Application No. Hei 10) is applicable.

4.3 Description of Operation When the work network management means 1110 receives a work network execution request from the operator, it passes the work network to the robot language generation means 1111. The robot language generation unit 1111 refers to the environment model 1107, expands the work network to the level of the operation schema (the minimum unit of operation, details will be described in 5.2), and then converts it into the robot language. When an image measurement request is issued in the process of generating a robot language, the environment model 1107 is modified based on the measurement result, and based on this, the environment model 1107 is converted into a robot language. The obtained robot language is passed to the robot language interpretation means 1112 and the slave arm control device 1216. Robot language interpretation means 1
112 interprets the robot language, generates time series data of joint angles for moving the slave arm, and passes it to the instance frame updating means 1115. When the slave arm control device 1216 performs attitude control, attitude control data is created and passed to the satellite operation means 260. The slave arm control device 1216 generates data for controlling the slave arm and transfers the data to the slave arm input / output device 206. Finally, the on-orbit system monitoring means 1204
Operating status of the on-orbit management system 107 (whether there is any abnormality)
Is sent to the offline simulator on a regular basis. Also,
Upon receiving the emergency stop command from the offline simulator, the on-orbit system management system 107 is stopped immediately.

5. Communication Method 5.1 Communication Hierarchy at Work Concept Level Next, a communication method exchanged between the ground system and the on-orbit system of the present embodiment will be described with reference to FIG. From the ground system to the on-orbit system, a work network and ground system monitoring means 1
A message from 104 is sent. From the on-orbit system to the ground system, image data, image measurement values (position and orientation of the work target object) by the image measurement means of the on-orbit system, and a message from the on-orbit system monitoring means 1204 are sent.

The communication system shown in FIG. 14 is a database transmission 1300, an operation code transmission 1301, a return code reception 1302, a ground system orbit system condition calculation 13
03, situation match? 1304, operation progress processing 1
305, abnormality avoidance processing 1306, database reception 13
07, operation code reception 1308, return code transmission 1309, ground system, orbit system status calculation 131
0, match situation? 1311, operation progress processing 13
12, and the abnormality avoidance processing 1313 is included. Procedures 1300 to 1306 are procedures on the transmitting side, and procedures 1307 to 1313 are procedures on the receiving side.

The operation of this communication system will be described below. First,
When the sender transmits the database in procedure 1300, the receiver receives the database in procedure 1307. next,
When the transmitting side transmits the operation code in procedure 1301, the receiving side receives the operation code in procedure 1308. Next, when the receiving side transmits the return code in 1309, the transmitting side receives the return code in step 1302. Next, in steps 1303 and 1310, the transmitting side and the receiving side both calculate the status of the terrestrial system and the on-orbit system, and in steps 1304 and 1311, the status of the terrestrial system and the orbital system (status calculation result) is calculated. Check if you are doing. Here, if the situations match, operation progress processing is performed. If the situations do not match, it is determined that the system is abnormal, and the abnormality avoidance processing is performed according to the rule indicated by the safety management network.

5.2 Description of Work Network The work network is the work network generating means 110.
8 is a work machine work expression system at the work concept level, which is generated from an operator's input message. If the work network is used, the work planned for the work machine (manipulator, etc.) can be described in an abstract manner to the extent that it is not affected by fluctuations (small fluctuations) in the work environment. Here, the description method and meaning of one embodiment carry of the work network will be shown.

Carry (A, [P1, P2], B) (7) Meaning: The object A is carried to the point B through the points P1 and P2.

Here, both A and B are the names of the objects to be worked, A is the object to be carried, and B is the object on which A is installed. P1 and P
2 is the position coordinate in the world coordinate system. The robot language generation means 1111 first expands the work network to the level of the operation schema according to the following expansion rules. Next, referring to the environment model 1107,
Robot language is generated from motion schema. The expansion rules are shown below.

Carry (A, [P1, P2], B) = grasp (A) → path [P1, P2] → attach_to (B) (8) In this expression, grasp (A) and attach_to (B) operate. It is a schema.

Grasp (A) = approach_point → grsp_point (9) attach_to (B) = approach_point → attach_point (10) Expressions (3) and (4) are expansion expressions of the operation schema. In these expressions, approach_point (reference position and posture for the next motion), grasp_point (position and posture that the slave manipulator hand must take for the object gripping motion) and attach_point (to set the gripped object) The position and orientation that the slave manipulator hand must take are determined depending on the shape, position and orientation of the work target object (A or B). Therefore, the frame (5.3 of A or B of the environment model 1107)
See). Furthermore, grasp_point and attach_poin
Since t is an operation that requires accurate and reliable positioning accuracy, it is remeasured when approach_point is encountered. Through these steps, a robot language is generated.

5.3 Description of Environment Model The environment model 1107 is described by a frame structure. Also, the relationships between the frames are all hierarchical. Also, the frame itself has a hierarchical structure,
Below the frame, several slots are defined. At the same time, fasit is defined under the slot, and value is defined under the fasit. Hereinafter, first, the structure of the environment model 1107 will be described.

The environment model is composed of a class frame and an instance frame. Instances are subordinate to classes. The class frame stores the shape of an object and items that are characteristic for operation (grasping, mounting, sensing method, etc.).
Since detailed data is required to describe the operational characteristics, a new class is defined in the lower hierarchy. As you can see, there are several kinds of classes, and the kind names and their definitions are clarified. In the instance frame, the name unique to the object, the position and orientation of the object, the connection relationship between other objects, and the connection state (differences such as being simply placed or being attached) are described.

6. System Safety Management 6.1 Overview When an abnormality occurs in the system, the on-orbit system management system 107 moves to a safe state by autonomous operation,
Wait or end. First, the work network management means 1110 causes the system abnormal state (communication interruption, failure of the on-orbit system management system 107, ground system management system 1).
(06 failure) is analyzed and the work network for safety management is passed to the robot language generation means 1111. The subsequent operation is the same as the operation of the on-orbit system management system described in 5.3.

6.2 In case of communication interruption In the case of temporary communication failure due to noise, the working machine is temporarily stopped and waits until a communication recovery notice comes.

In the case of a permanent communication failure due to a communication device failure or the like, the following steps are taken to return to the initial posture.

(1) Return the working machine to the initial posture.

(2) When the work machine is working (for example, when a gripping object is present), a minimum amount of work is performed at that time to set the work target at a safe position (for example, when gripping the object. Attach it to the nearest attachment device) and then return to the initial posture.

(3) It waits until a communication return notification is received from the ground system monitoring means 1104.

6.3 Failure of the on-orbit system management system (1) The on-orbit system monitoring means 1204 indicates that an abnormal situation has occurred.
Notify the ground system monitoring means 1104.

(2) After stopping the working machine, the on-orbit system management system 107 is stopped.

6.4 Failure of Ground System Management System (1) The ground system monitoring means 1104 notifies the on-orbit system monitoring means 1204 of the occurrence of an abnormal situation and stops the ground system management system 106.

(2) The on-orbit system management system 107 notified of the occurrence of the abnormal situation returns the working machine to the initial posture.

(3) When the working machine is working (for example, when a gripping object is present), a minimum amount of work is performed at that time, and the work target is set at a safe position (for example, when the gripping object is placed). Attach it to the nearest attachment device) and then return to the initial posture.

(4) It waits until a ground system management system return notification is received from the ground system monitoring means 1104.

6.5 Safety Management Network The safety management rule of the system is expressed by the state transition diagram as shown in FIG. In the state transition diagram, one "section" to proceed to is determined from the state variables of the network. The "branch" of the network corresponds to the working network. Therefore, for disturbances from any anomaly in the system,
One state variable is determined regardless of the work procedure up to the present time, and one "branch" and "section" to be selected next is determined. Due to this property, this method (action plan rule expression method using state transition diagram) is effective for an autonomous system that requires dynamic action planning ability.

By the way, the state transition diagram of FIG. 15 is an embodiment of the action plan rule expression method using the state transition diagram. In the on-orbit system, two attachment devices A and B, a working machine, and a work are installed. It is assumed that the target object exists and that the work machine moves the object from A to B or from B to A. Information that becomes the state variable in this case, and the state variable (r **
Is shown below).

(1) Abnormality notification from on-orbit system monitoring means-Communication error / recovery of communication ... (r00 / r01) -Orbit system management system error / recovery・ ・ ・ ・ (R10 / r11) ・ Abnormality / recovery of ground management system ・ ・ ・ ・ ・ ・ (r20 / r21) (2) Working status of work equipment ・ Gripping object / Not ... (r30 / r31) ・ Close to the mounting device A / Close to B ... (r40 / r41) (3) Time change of state variable ・ x No change / change for more than a second ... (r50 / r51) Next, the rule list created from the state transition diagram is shown.

If (r01 & r11 & r21 & r31 & r4 * & r5 *) then n0 if (r00 & r11 & r21 & r31 & r4 * & r5 *) then n1 if (r00 & r11 & r21 & r30 & r40 & R5 *) then n3 if (r00 & r11 & r21 & r30 & r41 & r5 *) then n4 if (r01 & r10 & r2 * & r3 * & r4 * & r51) then n2 if (r01 & r10 & r2 * & R3 * & r4 * & r50) then n5 if (r01 & r11 & r20 & r31 & r4 * & r5 *) then n2 if (r01 & r11 & r20 & r30 & r40 & r5 *) then n3 if (r01 & r11 & r20 & r30 & r41 & r5 *) then n4 7. Operator Presenting Screen The operator presenting screen 103 is displayed by the operator from the ground system management system 106 to the on-orbit system management system 107.
In order to remotely control the robot, it is equipped with a dedicated environment for creating a new work environment model, planning work, and executing work while operating the graphic animation of the work environment in orbit with the mouse. FIG. 16 shows an external view of the operator presentation screen 103. As shown in FIG. 16, on the operator presentation screen 103, in addition to the main window that displays the work environment model 1107, two key command input windows and one real image window for presenting the image measurement status to the operator are displayed. To be done.

The system of this embodiment is roughly divided into (1) creation of an environment model, (2) planning of on-orbit work, and
(3) It has three types of operation modes: execution of planned work.

7.1 Environment Model Creation Mode The environment model creation mode is a mode in which a work target object is newly registered in the environment model 1107. In particular,
From the list of candidate models displayed on the operator presentation screen with icons etc., the operator selects with the mouse an object model that exists in the work environment but is not registered in the environment model, and overlays it on the actual image. .

Here, the position ambiguity that causes operator error (the possibility that the actual image and the object model are not aligned correctly) and the shape ambiguity (similar but different models are selected). (Possibility) can be processed by the image measuring means 102. A specific example of the position ambiguity processing is described in “Construction device” (Japanese Patent Laid-Open No. 2-209562).
It is described in. The ambiguity of shapes exists in the work environment from the list of candidate models (class frame group of environment model) using the device described in "Real-time generalized Hough transform device" (Japanese Patent Application No. 02-313350). However, it can be processed by automatically searching for an object model that is not registered in the environment model and registering it in the environment model (generating an instance frame of the environment model and storing position and orientation data). it can. In this method, an object candidate pattern is extracted from the image captured by the camera, and a correct candidate is finally determined by repeating verification and verification with an object model group in the environment model. However, in order for the system to have a high level of environment recognition (pattern recognition) ability, it is necessary to combine collation algorithms with different collation layers, but on the other hand, there are as many collation model description methods as there are collation layers. However, it also becomes a problem in realizing the system. On the other hand, the environment model can be given by CAD data. Therefore, the CAD data is once converted into image data through the animator 1102, and then Hough transformed to generate a matching model described by Hough parameters.

The Hough parameter extraction algorithm is 25
An image space of 6 × 256 (pixels) is divided into 16 × 16 (pixels) small windows, and each window is individually used by the device described in “Real-time generalized Hough transform device” (Japanese Patent Application No. 02-313350). Hough parameters are extracted by performing Hough transform.

The matching verification rule is shown below.

First, the notation of the matching pattern is defined as follows.

Object pattern extracted from the image: P Object model extracted from the database: Q Here, the reliability of the matching pattern Q is a small window that overlaps with each other in the small windows Wp and Wq from which P and Q are extracted. It is defined as the ratio R of the total sum Sp∩q of windows to the total sum Sq of windows from which Q is extracted.

Next, windows Wp that overlap each other
The sum of the distances in the ρ-θ space of the line segment parameters (ρpi, θpi) and (ρqi, θqi) extracted from Hq transform from ∩q is defined as D. The matching pattern obtained by normalizing this D with Sp∩q is defined as the matching degree C.

That is, R = Sp∩q / Sq (11) C = {(│ρpi-ρqi│ + k│θpi-θqi│)} / Sp∩q = D / Sp∩q (12) k: constant (11 ) And (12), the following judgment rule is obtained.

R> θ ₁ (13) C> θ ₂ (14) θ ₁ , θ ₂ : Thresholds Here, a combination of matching patterns (Pi, Qj) that simultaneously satisfy the expressions (13) and (14). Of the maximum C
Let j be the search pattern.

7.2 On-Orbit Work Planning Mode In the on-orbit work planning mode, a work message is input by a direct method of operating the animation of the work machine with a mouse. Details of the method of inputting the work message are described in “Three-dimensional selection method” (Japanese Patent Laid-Open No. 1-112374).

7.3 Planned Work Execution Mode In the planned work execution mode, a work execution command is issued and a message for confirming the status of image measurement that is timely performed during work execution is issued.

8. Supplement This system is a remote control device, and it is effective for remote control of the management system placed under any environment such as the ground or the seabed. In addition, a plurality of management systems may remotely control each other. What is important for realizing such a system is (1) that the modules (in the present embodiment, the work network generation means 1108, the interface control means 1109, etc.) that constitute the management system operate in an autonomous distributed manner. (2) The management system individually or shares a common storage unit (in this embodiment, the environment model 1107) for reference when each module operates. Also,
The "action plan rule expression method using a state transition diagram" described in Section 5.5 above is effective for describing the rules for operating in an autonomous distributed manner. Further, in this embodiment, it is not necessary to create the environment model exclusively for the environment recognition process.

(Third Embodiment) The present invention can be applied to not only the actual operation but also the training of the operator or the pre-practice test of each equipment. First, an on-orbit system management system is installed on the ground experimental site, and the relative motion calculation means described in "Relative motion simulator for robot and work target" (Japanese Patent Application No. 3-343947) is set as a slave of the manipulation means 230. By connecting to the arm controller 1216, operator training equipment for a spacecraft recovery mission can be configured.

[0119]

According to the present invention, the operability of the master arm of the manipulator is improved, and as a result, the operability of the manipulator itself is also improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a main part of a first embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration example of a portion of the embodiment shown in FIG.

FIG. 3 is a block diagram showing a detailed configuration example of a portion of the embodiment shown in FIG.

FIG. 4 is a perspective view showing a specific overall configuration example of an embodiment of the present invention.

5 is a block diagram showing a detailed configuration example of a portion of the embodiment shown in FIG.

6 is a block diagram showing a specific configuration example of a central processing unit 101 shown in FIG.

FIG. 7 is an explanatory diagram showing the operating principle of the present invention.

FIG. 8 is a block diagram of a manipulator control system according to another embodiment of the present invention.

FIG. 9 is a flowchart showing an arithmetic processing operation in the embodiment of the present invention.

FIG. 10 is a flowchart showing an arithmetic processing operation according to the embodiment of the present invention.

FIG. 11 is a block diagram of a manipulator control system according to another embodiment of the present invention.

FIG. 12 is a block diagram showing details of a portion of the embodiment shown in FIG.

13 is a block diagram showing details of a portion of the embodiment shown in FIG.

FIG. 14 is a block diagram showing a communication protocol in the embodiment of the present invention.

FIG. 15 is a state transition diagram of the safety management network used in the embodiment of the present invention.

[Explanation of symbols]

101 Central Processing Unit 102 Image Measuring Means 103 Operator Presentation Screen (Graphic Display) 104 Input / Output Device 105 Slave Arm Input / Output Device 106 Operation System Management System (Ground System Management System) 107 Work System Management System (Orbit System Management System) 201 Master arm 202 Slave arm 204,224,2j4 Master arm input / output device 205 Central processing unit 206,226,2m6 Slave arm input / output device 207 Joystick 208,228,2i8 Joystick input / output device 209,229,2n9 Image input device 210 images Processing device 211 Simulation means 213 Television monitor 214A, 214B, 214C Position detection sensor 216A, 216B, 216C Actuator 217 Input / output device 217A, 217B, 217C Position detection sensor 218 Coordinate conversion addition device 230 Manipulation means 260 Satellite operation means 301 Work target 401 Direction determination circuit 402 Counter 501 processor 502 Memory 503 Multiplier 504 Divider 505 Bus circuit 506A, 506B, 506C, 506D, 506E
Interface circuit 701 Difference circuit 702 Incremental circuit

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 2, 1992

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0016

[Correction method] Change

[Correction content]

[0016]

[Equation 1]

Front Page Continuation (72) Inventor Tomoyuki Hamada 502 Jinritsu-cho Machinery Research Center, Tsuchiura-shi, Ibaraki Prefecture Machinery Research Institute, Ltd. (72) Inventor Hironari Kikuchi 502 Jinritsucho, Tsuchiura-shi, Ibaraki Machinery Research Institute, Hiritsu Manufacturing Co., Ltd.

Claims (11)

[Claims] 1. An operation system management system including a central processing unit, an image measuring unit, a screen for presenting an operator, and an input / output unit, and a work system including an image measuring unit, a slave arm input / output unit, and a central processing unit. In the remote control device of the manipulator consisting of the management system, the work system management system performs work based on the work message transmitted from the operation system management system, and the work system management system and the work system management system work independently. A remote control device for a manipulator, characterized in that the operating system management system and the work system management system independently calculate mutual conditions. 2. An operation system management system including a central processing unit, an image measuring unit, a screen for presenting an operator and an input / output unit, and a work system including an image measuring unit, a slave arm input / output unit and a central processing unit. In a remote control device of a manipulator including a management system, a work system management system performs work by a slave arm based on a work message transmitted from the operation system management system, and the operation system management system based on the work message. A remote control device for a manipulator, wherein the operation of a slave arm is simulated independently of the work system management system and is displayed on the operator presentation screen. 3. An operation system management system and a work system management system are respectively stored in a storage unit that stores an environment model that can be accessed by each component of the own system and a storage unit that the operation system management system has. 3. An image measuring unit that works so as to always make the environment model stored in the storage unit of the work system management system equal to each other.
A remote control device for a manipulator according to. 4. The environment model includes a shape of an object, a portion describing a characteristic feature for operating the object, a name unique to the object, a position and orientation of the object, and a connection with another object. The manipulator remote control device according to claim 3, wherein the remote control device comprises a relationship and a portion describing a connection state. 5. An operation system management system for instructing work,
In a remote control device of a manipulator including a work system management system for controlling the work of a slave arm based on the instruction, a work instruction from the operation system management system to the work system management system is described at a work concept level. Remote control device for manipulators, characterized in that it is performed using a working network. 6. The manipulator according to claim 5, wherein the work network is configured to include at least a work name and data necessary for executing the work, and is described in a hierarchical structure. Remote control device. 7. An operation system management system and a work system management system, wherein each component of the operation system management system is stored in a storage means provided in the work system management system, and each component of the work system management system is included in the operation system management system. 5. The remote control device for a manipulator according to claim 3, further comprising means capable of accessing each of the storage means provided in, and capable of reading and writing the environment model stored in each storage means. 8. An operation system management system for instructing a work system management system for controlling the work of the slave arm via a communication means, and a work system management system for controlling the work of the slave arm based on the command. In the remote control device of the manipulator including, the work management system detects a communication abnormality occurrence, a means for storing the safety management rule expressed by the state transition diagram, and a communication abnormality occurrence is detected. A remote control device for a manipulator, further comprising means for instructing an operation of a slave arm with reference to the state transition diagram. 9. The operation system management system is provided with means for managing information necessary for an operation controlled by the work system management system by using an interactive operation with an operator presentation screen. The remote control device for a manipulator according to any one of claims 1 to 4 and 7. 10. The operation system management system includes means for inputting a work target object to an environment model, and position ambiguity and shape ambiguity when a work target object is input to the environment model via the input means. The remote control device for a manipulator according to any one of claims 3, 4 and 7, further comprising: an image measuring means for processing. 11. The operating system management system and each component of the work system management system are configured to operate in an autonomous distributed manner while accessing the environment model. , 4, 7 and 10, a remote control device for a manipulator.