JP2820639B2 - Robot control method and robot system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot control method and a robot system.

[0002]

2. Description of the Related Art When a variety of articulated robots such as a robot that simulates a human motion perform a predetermined operation, the amount of drive of each actuator that drives the robot is conventionally determined by programming the operation. A method of individually setting the time series is adopted. In this case, in the operation of a certain robot, when the movements of a plurality of robot parts occur simultaneously or in cooperation, the drive amount of the actuator must be specified for each of the robot parts. For example, in the case of the operation of "walking", for example, in addition to the movement of both legs, this operation is accompanied by the operation of swinging the arms and raising and lowering the shoulders, and the programs for the operations of these parts are individually assembled. And if you want to program a more complex higher-level action, such as "go one step forward and thank and go down and go back to where you were,"
The programming operation is performed by disassembling the higher-order operation into lower-order operations of “going forward one step”, “grate”, and “go down one step”.

[0003]

In the above-mentioned conventional method, since the driving amounts of a large number of actuators and the operation order of the actuators must be set for each robot part, programming work takes time and complicated upper-level operations are required. It is hard to say that it is suitable for creating programs efficiently. Further, when it is desired to change the contents of the higher-order operation, it is necessary to rewrite the program or data at the drive amount level of each actuator, and the operation takes a long time. That is, since it is not so easy to change the operation content of the robot, there is a disadvantage that the robot lacks flexibility in various operations.

[0004] It is an object of the present invention to provide a robot capable of easily performing programming and rewriting of a higher-order robot operation including a series of lower-order operations, and thereby allowing the robot to perform various operations relatively easily. It is to provide a control method and a robot system.

[0005]

In order to solve the above-mentioned problems, a robot control method according to the present invention forms lower-order operation data in units of drive data relating to driving actuators provided in a plurality of robot parts. A middle-level operation routine that defines the basic operation of the robot is constructed by integrating a plurality of the lower-level operation data, and a middle-level operation command is assigned to each of the middle-level operation routines. And controlling the driving of the actuator based on a combination of the middle-level operation routines corresponding to the read-out middle-level operation commands while reading out the higher-level operation program in units of the middle-level operation commands. Features.

A robot system according to the present invention based on the above control method is characterized by including the following requirements. Program storage unit: stores a higher-level operation program defined by a combination of middle-level operation commands. The middle-level operation command corresponds to a middle-level operation routine that defines the basic operation of the robot. The middle-level operation routine includes lower-order units of drive data related to driving actuators provided in a plurality of robot parts. A plurality of operation data are integrated and constructed. Program reading means: reads a higher-level operation program in units of medium-level operation commands stored in a program storage unit. Actuator operation control unit: controls the driving of the actuator based on a combination of the middle operation routines corresponding to the read middle operation commands.

[0007] In the control method and the robot system having the above-described configuration, for example, a program of a higher-level operation of "go forward one step and give thanks and go back one step and return to the original place" includes "one step forward", It is defined as a combination of middle motion commands corresponding to the middle motions of "thank you" and "go down one step", respectively. And, for example, "go one step"
In the case of the middle-level motion, lower-level motions such as "leg stepping out" and "arm swing" are described in lower-level motion data in units of drive data of actuators of each robot part, and the lower-level motion data Are integrated to form a middle-level operation routine as a kind of module.

If a required number of such middle-level operation routines (modules) are prepared in advance, a program for a higher-level operation can be created only by appropriately combining the corresponding middle-level operation commands. But it's relatively easy to program. The contents of the program
Rewriting can be easily performed only by adding, deleting, or replacing the middle-level operation command, and the contents of the higher-level operation can easily be changed or modified. Furthermore, since the higher-order operation program is mainly composed of middle-order operation commands, the capacity is small,
Even when a large number of higher-level operation programs need to be prepared, the occupation amount of the disk or the memory is small, and the installation work is also simple.

Here, at least one of rearrangement, addition, and deletion of the middle-level operation command is instructed to the higher-level operation program based on the combination of the middle-level operation commands stored in the program storage section, and A higher-order operation program rewriting means for rewriting can be provided.

Next, the drive data forming the lower-order operation data can be used as drive amount calculation data for calculating the drive amount of the actuator. Based on the calculation data, each drive amount of the actuator included in the robot part is calculated, and the control unit is configured to include calculation / drive control means for controlling the drive of the actuator based on the calculated drive amount. Can be. With this configuration, the drive amount of the actuator is sequentially calculated with the progress of the robot operation. Therefore, it is not necessary to previously store all drive amount data of each actuator in the program storage unit. The amount can be reduced. Here, a calculation / drive unit is provided for each robot part, and data transmission for transmitting drive amount calculation data to the calculation / drive control unit in accordance with execution of lower-order operation data based on the middle operation routine. Means can also be provided. With this configuration, the calculation of the drive amount of the actuator is performed in parallel for each robot part, so that
Responsiveness of actuator drive can be improved.

Next, at least a part of the plurality of robot parts may be provided with a skeleton having a plurality of skeleton units and an actuator disposed between the skeleton units. In this case, the driving amount calculation data corresponding to the robot part may include displacement data of each skeleton unit at each time of the robot operation. Then, the calculation / drive control means calculates the
The drive amount of the actuator corresponding to the skeleton unit can be calculated based on the displacement data that is temporally successive. More specifically, any one skeleton unit of the skeleton (hereinafter, referred to as a first skeleton unit);
The calculation and drive control means may calculate the drive amount as described below for the first skeleton unit and the second skeleton unit connected via the actuator. That is, at least one of the rotational movement and the translational movement is performed on the position of the first skeleton unit and the position of the second skeleton unit at successive times, and the first skeleton units at those times are overlapped with each other. . Then, in that state, the displacement of the second skeleton unit between these times is calculated, and the driving amount of the actuator is calculated based on the calculation result.

When at least a part of a plurality of robot parts is provided with the above-mentioned skeleton, a drive data generating device including the following requirements can be provided. Image display means: A series of model images in which the operation of the model corresponding to the robot operation is recorded with time and an image of the robot skeleton (skeleton image) are displayed. Skeleton positioning means: Positions the skeletal image on the image display means at a position corresponding to the model image. Specifically, the display unit can be configured to position the skeleton image on the model image by superimposing the skeleton image on the model image. Driving data calculation means: Skeleton displacement data and inter-skeleton angle data (these two are regarded as driving amount calculation data) based on the displacement of the aligned skeletal images with respect to the model images preceding and succeeding each other on the time axis. ) Or actuator drive amount data,
Calculate as drive data.

For example, a model that changes over time can be obtained by causing a human model to actually perform a certain intermediate movement, recording this as a series of images, and aligning each of these images with a robot skeleton image. At each specific time of the operation, positioning of the skeleton image with respect to the model image is performed, and drive data is calculated based on the displacement of the skeleton between adjacent images. In this way, lower-order operation data for each part included in the intermediate operation can be collectively obtained, and if an intermediate operation command is added and stored in the storage unit, the intermediate operation It can be used.
The operation for creating the drive data is mainly performed by the operation of aligning the skeleton image with the model image on the screen of the image display means, so that the content is easy to understand, and
Since it is not necessary to set the drive amount of the actuator for each part of the robot, it is possible to obtain a large amount of drive data necessary for creating a higher-level operation program in a rational and simple manner.

The above-described drive data generating device is provided with drive data storage means for storing drive data calculated and generated based on drive data calculation means, and drive data editing means for editing the stored drive data. be able to. By using the editing means, the generated drive data can be easily corrected, added, or deleted. In addition, the editing means can edit the drive data for each type of middle motion or for each part of the robot, thereby replacing lower motion data or exchanging lower motion data between different middle motion routines. Therefore, the versatility of the data can be enhanced and the robot can perform various operations. Further, the relatively similar lower-order motions of the same robot part can be unified into one lower-order motion between different middle-order motions, which has the effect of reducing the entire data amount and increasing the responsiveness of the robot motion.

In the above-mentioned drive data creating apparatus, for example, when a robot is to copy a human motion, the motion performed by the human model is recorded as a moving image by a photographing means such as a video camera, and the moving image is sent to the image display means by frame. During playback, a skeleton image positioning operation may be performed for each frame of the moving image. Here, the dimensions of the recorded model image may vary depending on the dimensions (or body shape) of the selected model. In this case, the length of each skeleton unit of the skeleton image is displayed on the display means. It can be made expandable and contractible according to the dimensions of the model image.

The time interval between the model images can be changed according to the operation of the robot. For example,
When playing back video images frame by frame, the frames of those images are taken at fixed time intervals, but when converting relatively simple model operations into data, by thinning out a fixed number of frames, The time interval between adjacent model images can be increased. Thereby, the number of images required for teaching is reduced, and teaching work can be made more efficient. Further, since the amount of data required for the calculation is reduced, the responsiveness of the robot operation can be improved.

Next, for a part of the skeleton of the robot, the driving data is calculated by the driving data calculating means based on the displacement of the input skeleton image with respect to the model images preceding and succeeding each other on the time axis. On the other hand, auxiliary input means are provided in correspondence with the remaining robot skeleton and predetermined robot parts other than the skeleton,
The auxiliary input means may be configured to individually input drive data for the remaining skeleton and robot parts. For example, for a robot part (for example, movement of a wrist, a finger, an eye, a mouth, etc., lighting of a lamp, or the like) from which it is difficult or impossible to calculate drive data from a skeleton image aligned with a model image, an auxiliary input unit may be used. Driving data can be directly input, whereby a detailed operation for the robot details can be realized.

Specifically, the skeleton of the robot includes a plurality of skeleton units connected by joint mechanisms, and these skeleton units can be configured to be driven by actuators provided in the joint mechanisms. in this case,
A reference posture is set in advance for the model, and reference data including the reference position of each actuator for the skeleton image aligned with the model image of the reference posture is stored in the drive data calculation means. Reference data storage means may be provided. The drive data calculation means calculates the displacement of each skeleton unit at each time based on the reference data, or calculates the drive amount of each actuator based on the displacement.

Next, the input of the alignment position of the skeleton image with respect to the model image is performed by designating the positions on the display screen of the display means of the predetermined mark points at both ends of each skeleton unit. Can be. As a more specific device configuration for that purpose, for example, the following can be exemplified. That is, a pointing device such as a mouse or a trackball is provided in the skeleton positioning means, and a pointer that moves on the screen along with the operation of the pointing device is displayed on the display screen together with the model image and the skeleton image on the display screen. Let it. Then, when aligning the skeleton image with the model image, select the landmark point to be aligned, position the pointer at the alignment position set on the model image, and attach the pointer to the pointing device. By operating the input unit and designating the alignment position of the landmark point, the skeleton unit is positioned at a position corresponding to the landmark point of the positioning. In this way, the operation of aligning the skeleton image with respect to the model image can be made more direct and easy to understand.

Further, a plurality of skeleton units form a group, and a predetermined constraint condition can be set for the relative positional relationship between the skeleton units included in the group. And
By designating the alignment position with respect to the model image for some of the landmark points of the skeleton unit belonging to the set, the alignment position of the remaining landmark points is determined based on the constraint condition. can do.
By doing so, the position adjustment operation can be simplified and the working time can be shortened, and also the effect of preventing the inconvenience of specifying a structurally impossible position for the robot part can be obtained.

Here, the order of positioning with respect to the model image may be determined in advance for the skeleton unit, and the positioning operation may be performed according to the order.
In this case, when the positioning operation of a certain skeleton unit is completed, the display state of the skeleton unit to be operated next is changed from the other landmark points by changing the color of the landmark point or blinking the landmark point. Can be configured.

Next, a mechanism for calculating the drive amount of the actuator for driving the skeleton unit from the displacement of the skeleton unit is described below. Specifically, the following mechanism can be employed. That is, for any one skeleton unit of the robot skeleton (hereinafter, referred to as a first skeleton unit) and a second skeleton unit coupled to the first skeleton unit via an actuator, For at least one of the skeleton images that precede and follow each other on the time axis, the first skeleton unit and the second skeleton unit are subjected to at least one of rotational movement or translational movement, and the first skeleton unit of both skeleton images Overlap each other. Then, in the superimposed state, the displacement of the second skeleton unit between the two skeleton images is calculated, and the driving amount of each actuator is calculated based on the calculation result.

The driving data calculation means is provided in the image display means.
It can be configured to calculate the three-dimensional position of the skeleton unit based on the image of the skeleton unit displayed in a three-dimensional manner and the image of the standard body having a known dimension.

[0024]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an overall configuration of a robot system 200 according to an embodiment of the present invention, which includes a robot 201, a robot control unit 202, a drive data creation device 1, and a graphic simulator 151, each of which has a communication interface 203. , 152 and 153 via a communication line 150.

The robot control unit 202 includes a CPU 204, a RAM 205 and a ROM 206 connected thereto, a storage device 207 including a hard disk device, an input device 208 such as a keyboard, a display control unit 210, and the like.
The display control unit 210 includes a display device 21 such as a CRT.
1 is connected, and a communication interface 203 is connected to the CPU 204.

On the other hand, the robot 201 is configured as, for example, a robot that mimics the movement of a human, and includes a skeleton 19 in which skeleton units 20 to 30 are connected to each other by joint mechanisms 31 to 37 as shown in FIG. Then, this skeleton includes, for example, the following robot parts (hereinafter, parts A, B, and C)
.. Etc.). -Head (20): Equipped with a drivable eye 20a and a mouth 20c, a blinkable lamp 20b is arranged in the eye 20a, and the mouth 20c can be opened and closed. -Torso: includes shoulder 21, spine 22, and hip 23. -Left and right arms: provided with a humerus 24, a lower humerus 25, and a wrist 29, respectively. Left and right legs: provided with an upper limb bone 26, a lower limb bone 27, and an ankle 30, respectively.

Then, as shown in FIG.
Are provided with calculation / drive control units (calculation / drive control means) 212 corresponding to the respective parts A, B, C, etc., and are connected to the CPU 204 of the robot control unit 202, respectively. Then, these calculation / drive control units 212
, An actuator 213 included in each part is connected. The types of actuators (pulse motors, servomotors, piston cylinders, etc.) and the number of shafts, the degree of freedom of shaft rotation, and the range of shaft rotation or piston sliding provided in each of the joint mechanisms 31 to 37 in FIG. Are set individually for each skeleton unit.

Next, as shown in FIG. 3, the storage device 207 of the robot control unit 202 stores a higher-order operation program storage unit 207a, a middle-order operation command analysis program storage unit 2
07b, a medium operation routine storage unit 207c, a medium operation correction program storage unit 207d, and a rewriting program storage unit 207f, each of which stores a corresponding program or routine (or data).

The operation of each program is as follows. (1) Upper-level operation program: A desired higher-level operation is described by selecting and arranging appropriate middle-level operation commands prepared in advance. For example, when the robot 201 wants to perform a series of higher-order operations such as "walk rightward, walk four steps, turn right, step one and a half steps, and pose for karate chop", the higher-order operation program is as shown in FIG. "Turn right", "right leg half step", "left leg one step"
The commands of the middle operation (middle operation commands) constituting the higher operation are arranged in the order of execution in principle.

(2) Middle operation routine: formed for each of the above middle operation commands,
For each of B, C,..., Drive data DA1, DA2,.
2, ..., etc., and drive data of each part
Data transmission programs PA, PB,... For transmission to the drive control unit 212 are included. These data transmission programs PA, PB,..., Etc. perform the operations defined by the parts A, B, C,. For example, for a middle motion of “one step of the right leg”, the right leg takes a stepping action,
And the left and right arms perform the forward and backward swing motions in accordance with the stepping), and sequentially transmit the drive data DA1 and DA2 of each actuator to the drive data calculation unit 212 of each part. Here, each data transmission program PA, P
B, etc. are the corresponding drive data sets (DA1, DA2,
..), (DB1, DB2,...), Etc., constitute lower operation data. Here, drive data DA1, D
A2,..., And the like include displacement data of both ends of the skeleton unit driven by each actuator at each time, but may also be angle data between skeleton units. On the other hand, the drive data DA1, DA2,..., Etc. may be the drive amount of each actuator for each time.

(3) Medium-level operation command analysis program: A high-level operation program is read in units of medium-level operation commands, and the corresponding medium-level operation programs are sequentially activated. (4) Intermediate motion correction program: When the robot's operation start position based on the intermediate operation routine does not match the robot operation end position according to the immediately preceding intermediate operation routine, a correction operation necessary for connecting the operations. Generate (5) Rewriting program: Rewrites or newly creates a higher-level operation program. Specifically, a program creation screen is opened on the display device 211 (FIG. 1), a higher-level operation command is input using the input device 208 in a format as shown in FIG. 4, and a higher-level operation program name is given to the input content. do it,
It is stored in the higher-level operation program storage unit 207a of the storage device 207. Also, the stored higher-level operation program is displayed on a program creation screen, and upper-level operation commands are added, deleted, inserted, and the like based on an input from the input device 208, and the program is corrected and edited.

On the other hand, as shown in FIG. 5, the RAM 205 has a higher-level operation program storage section 205a, a medium-level operation command analysis program storage section 205b, a medium-level operation routine storage section (I) 205c, and a medium-level operation routine storage section. (II) 20
5d and the like are formed to store corresponding programs, data, and the like read from the storage device 207, respectively. Here, the intermediate operation routine storage unit is (I)
(II) is provided in such a manner that the intermediate operation routine to be executed next is read and stored in advance, so that the robot 201 reads data from the storage device 207 when the robot 201 moves to the next operation. This is to reduce the response delay based on

As shown in FIG. 6, the calculation / drive control unit 212 includes a CPU 214, a RAM 215, and an R
A drive amount calculation program 216a is stored in the ROM 216. The calculation / drive control unit 212 drives the drive data DA1, DA2,.
.., Etc., the drive amounts of the actuators 213 (FIG. 1) connected to the CPU 214 are calculated in chronological order,
Each actuator 213 is driven based on the calculation result. The calculation method will be described later.

FIG. 7 is a block diagram showing an example of the configuration of the drive data creation device 1. The drive data creation device 1 includes a CPU 2, a ROM 3, and a RAM that constitute drive data calculation means.
An image capture control unit 6, a display control unit 7, an input device interface 8, a program storage device 9, a drive data storage device 10, and a moving image data storage device 11 are connected to the device control unit 5 including . The input device interface 8 includes a mouse 12 and a keyboard 1 as skeleton positioning means having a click button (input unit).
3 are connected.

The display control section 7 has C as an image display means.
RT14 is connected. On the other hand, the image capture control unit 6
Is connected to a video playback device (VTR) 15 and includes a VRAM 16 and a data transfer processor 17 so that image display data stored in the VRAM 16 can be directly transferred to the display controller 7 via a bus 18. Have been.

Next, the programs shown in FIG. 8 are stored in the program storage device 9, read out by the CPU 2 as necessary, written into a predetermined area of the RAM 4, and used. (1) Moving image capturing program 90: The image of the model operation reproduced by the VTR 15 is captured as moving image data via the image capturing control unit 6, and is stored in the moving image data storage device 11 including a magneto-optical disk device. . (2) Model image display program 91: The model image of the designated frame is displayed on the CRT 14 based on the moving image data stored in the moving image data storage device 11. (3) Skeleton image display program 92: Robot skeleton 19
The image data of the skeleton image corresponding to (FIG. 2) is created, and the skeleton image 38 based on the data is displayed on the CRT 14 together with the model image 39 as shown in FIG.

(4) Skeleton image alignment program 94:
Using the mouse 12, the skeletal image 38 displayed on the CRT 14 is superimposed on the model image 39 and aligned. Here, the process of adjusting the size of the skeleton image 38 to the size of the model image 39 on the screen of the CRT 14 is also performed by this program. (5) Displacement calculation program 96: Calculates the three-dimensional position of each skeleton unit in the actual space from the image of each skeleton unit displayed two-dimensionally on the CRT 14, and based on it calculates the three-dimensional positions The displacement of each skeleton unit between the two images is calculated. The calculation process compares the dimensions of the standard body image obtained by photographing a standard body of known dimensions prepared for each skeleton unit from a predetermined direction with the dimensions on the screen of the corresponding skeleton unit. It is performed by The standard body size data is stored in the standard body size data storage unit 112 of the program storage device. (6) Actuator drive amount calculation program 98: The drive amount of each actuator corresponding to the displacement of the skeleton unit is calculated, and the drive data storage device 10 is provided together with the displacement data.
To memorize.

(7) Operating time setting program 100: For a moving image frame to be used, a time on the real time axis of the robot operation is set, and a driving speed of each actuator is calculated from a time interval between adjacent frames. This is stored in the drive data storage device 10 as a part of the drive data. (8) Auxiliary input program 102: Auxiliary input processing of drive data to a skeleton unit or a robot part specified in advance is performed by using the mouse 12 or the keyboard 13. Such auxiliary input processing is performed for the skeleton image 3
8 is applied to a skeletal unit or a site where it is relatively difficult to generate drive data by the alignment process between the model 8 and the model image.
The movement of b, the blinking of the lamp 20b disposed on the eye 20a, and the like correspond to this. On the other hand, for some skeleton units, it may be difficult to generate drive data by alignment processing only for a specific operation mode, but in such a case, the drive data can be input by the above method. It is. This method includes, for example, tilting the head 20 forward and backward, right and left, rotating, or the like, or rotating the humerus 24 and the lower humerus 25 about the longitudinal axis with respect to the joint mechanism 33 (equivalent to the elbow).
The present invention is applicable to rotation of the wrist 29, bending of a finger, and the like. In the present embodiment, such an input process is a screen (or another window) different from a screen (FIG. 28) for performing a positioning process of the skeleton image 38 with respect to the model image 39.
Open and do it.

(9) Frame correction program 104: Changes and corrects (eg, changes the position of skeleton units, etc.) the drive data input by the skeleton image alignment process in the specified frame. When one of the positions of the skeleton units connected under the above-described constraint conditions is changed, the positions of other skeleton units are automatically corrected so that the restriction conditions are not broken. (10) Frame deletion / addition program 105: Deletes data of a specified frame, and adds / inserts a frame for inputting new drive data next to the specified frame position. (11) Sequential display program 106: The skeletal images of each frame are displayed on the screen sequentially at a fixed time interval in chronological order. As a result, a rough change in the movement of the skeleton image can be confirmed, and the skeleton image can be used as a simple simulation means for actual robot operation. (12) Unnecessary part setting program 107: A part that does not require calculation of drive data is set at the time of the alignment processing of the skeleton image 38. For example, if it is known that a specific part hardly moves in the target operation, the unnecessary part is set for that part, so that the part is fixed at a predetermined position and Then, the positioning process of the skeleton image 38 with respect to the part is skipped.

(13) Drive data file editing program 1
08 (drive data editing means): The drive data is divided into files according to the type of the middle action and the robot part, and a middle action command name is assigned to the drive data for storage in the drive data storage device 10.
save. (14) Simulator start program 109: The driving data of the specified robot operation is transferred to the graphic simulator 1
51, and prompts the start of a trial (simulation) process of the robot operation on the simulator 151 side.

The CPU 2 of the drive data generating apparatus 1 has communication interfaces 152 and 153 and a communication line 1
A graphic simulator (hereinafter, simply referred to as a simulator) for performing a demonstration of a robot operation via the 50
151 are connected. The simulator 151 is shown in FIG.
As shown in the figure, the CPU 154 and the RO connected thereto
M155, RAM 156, VRAM 157, display control unit 158, and input device interface 160,
The display control unit 158 is connected to a display device such as a CRT 159 for displaying a test moving image of the robot operation, and the input device interface 160 is connected to an input unit such as a keyboard 161. Further, a communication interface 153 for communicating with the drive data creation device 1 is connected to the CPU 154.

Next, the skeleton 1 of the robot 201 shown in FIG.
Some of the skeletal units forming 9 form a set, and a predetermined constraint condition is set for the relative positional relationship between the skeletal units included in the set. For example, taking the skeletal unit constituting both legs as an example, as shown in FIG. 10, the lower limb bone 27 moves only in a plane P formed by a straight line along the upper limb bone 26 and a straight line perpendicular to the hip bone 23. The constraint conditions are set (note that the joint mechanisms 35 and 36 (FIG. 2) are not shown in FIG. 10).
Therefore, if the upper end position of the upper limb bone 26 and the lower end position of the lower limb bone 27 are designated, the position of the connection point (portion corresponding to the knee) between them is automatically determined according to the above-described constraint condition. Such constraint conditions are stored in the constraint condition storage unit 110 (FIG. 8) of the program storage device 9 and used for calculating drive data.

Hereinafter, the operation of the robot system 1 of the present embodiment will be described. First, the process flow for operating the robot 201 is as follows. When a higher-level operation program name is designated and activated by the input device 208 or the like in FIG. 1, a middle-level operation command analysis program is activated, and FIG.
As shown in the flowchart of FIG. 1, the first middle-level operation command of the higher-level operation program is read in M1, and M2
Are stored in the middle-level operation routine storage (I) 205c of the RAM 205. Next, a signal is issued to the stored middle-level operation routine to activate it (M3). Also, during execution of the middle operation routine, the next middle operation command is read by M4, and the corresponding middle operation routine is stored in the empty middle operation routine storage section of the RAM 205 (in this case, the storage section (II) ).
Further, if a middle-level processing completion signal has been transmitted from the middle-level operation routine (storage unit (I) side) that is being started and executed in M6, the middle-level operation routine stored in the storage unit (II) 205d (M7). Then, if there is a next middle operation command in M8, the process returns to M4, reads the next command, and stores the corresponding middle operation routine in an empty storage unit (in this case, storage unit (I)), Thereafter, by repeating the same processing, the higher-level operation program is executed in units of middle-level operation commands. If the next intermediate operation command does not exist or is an end command, the process ends.

Next, regarding the processing on the middle-level operation routine side, as shown in the flow chart of FIG. 12, upon receiving a start signal from the higher-level operation program side, drive data of the actuator of each part is read (B1). , B2), by activating the drive data transmission program, the drive data described as displacement data for each time of the actuator is transmitted to the calculation / drive control unit 212 of each part in chronological order. Then, the calculation / drive control unit 212 calculates the drive amount of each actuator based on the displacement data based on the drive amount calculation program 216a, and drives the actuators based on the calculation result.

As a method of calculating the drive amount, for example,
A method based on the following principle can be adopted. First, in the skeleton 19 shown in FIG. 2, an index is given to both end positions of each skeleton unit, for example, as shown in FIG. 27 (W1). -Head 20: H, f1. -Shoulder 21: A1, H and H, A2. -Spine 22: G and H. Here, G is used as a reference position when calculating the drive amount of the actuator from the displacement of each skeleton unit. -Hipbone 23: L1 and L2. G is positioned at the midpoint of the line connecting L1 and L2. -Humerus 24, Humerus 25: A1, Ae1, Ah1, and A
2, Ae2, Ah2. -Upper limb bone 26, lower limb bone 27: L1, Lk1, Lf1, and L
2, Lk2, Lf2. The displacement data is calculated as the displacement at both ends of each skeleton unit.

For example, for the left leg in FIG.
As schematically shown in FIG. 13A, when the position of the point G is unchanged and the positions of L1 and Lk1 at a certain time T1 move to L1 ′ and Lk1 ′ at the next time T2, (b)
As shown in (2), the line segment GL1 and the line segment L1Lk1 (the first skeleton unit and the second skeleton unit in the claims, respectively) are rotationally moved to overlap the line segment GL1 with the line segment GL1 '. Then, from the spatial coordinates of each point after the rotational movement, the time T
1 line segment L1Lk1 and a time segment T1 line segment L1′Lk1 ′.
Is calculated, the rotation angle of both skeleton units (that is, each axis of the actuator disposed between the upper arm 26 and the lower leg 27 ) can be obtained. By sequentially performing such processing for determining the rotational movement of the skeleton unit and the rotation angle of the second skeleton unit in the direction toward the end of the skeleton, the drive amount of the actuator included in each joint mechanism is sequentially calculated. Becomes In the above description, the actuator is configured by a motor, and the first and second skeleton units are rotated by rotation of the motor shaft.
Although only a relative rotational motion is performed, when a piston rod or the like is used as an actuator, a relative translational motion may be performed between skeleton units due to expansion and contraction of the piston. In such a case, the translation distance is calculated in addition to the rotation angle of the second skeleton unit.

The drive amount of the actuator is calculated in this way, and when a series of drive of the actuator based on the drive data of the middle operation routine is completed, the drive amount calculation program 216a sends a lower processing completion signal to the middle operation routine. Sent to the routine side. In the middle operation routine,
In response to this in B4 of FIG. 12, a middle-level processing completion signal is transmitted to the higher-level operation program. When the drive amount of the actuator itself is transmitted as drive data, the above-described calculation processing on the calculation / drive control unit side is unnecessary.

FIG. 14 shows a robot 2 based on the above flow.
13 shows a specific execution example of the higher-order operation of No. 01. That is, in order to execute a higher-order operation of “walking four steps”, the higher-order operation program includes “right leg half step”, “left leg 1”.
Step, right leg one step, and left leg half step. For example, the contents of execution of the middle action routine of the first “half step of the right foot” are “half step” and “step down” for each part of the right leg and the left arm, respectively. The drive data of the actuator is transmitted to the calculation / drive control unit 212 in chronological order.
Then, the calculation / drive control unit calculates the drive amount based on the transmitted drive data, drives the corresponding actuator, and shifts to the execution of the next middle-level motion command “left leg one step”.

The drive data as described above can be created by the drive data creation device 1. Hereinafter, the operation flow of the drive data creation device 1 (FIG. 7) will be described using a flowchart. FIG. 15 shows a general flow of the entire process, and one of the following processes is selected. At the time of selection, a predetermined selection screen is displayed on the CRT 14 as a window, and by selecting the selection item displayed there using the mouse 12, individual processing windows are opened. (1) Input (processing from S1000 to S1100): superimposition processing of a skeleton image on a model image is performed, and drive data is created (in other words, operation of the robot 201 is taught).
I do. (2) Correction (processing from S2000 to S2100): The generated drive data is corrected. (3) Demonstration (processing from S3000 to S3100): A demonstration of the robot operation is performed using the graphic simulator 151. (4) Termination instruction (processing from S4000 to termination): An instruction to terminate the processing is issued.

If an input is selected, an input process is performed in S1100 in units of middle-level operation. Here, prior to the input, the intermediate action is performed by a human model. For example, as shown in FIG. 25, the model image 39 is photographed by a video camera at predetermined frame time intervals. Then, the moving image reproduced by the VTR 15 in FIG.
The moving image data is captured via the image capturing control unit 6 and stored in the moving image data storage device 11. Here, as shown in FIG. 26, the moving image data stored in the moving image data storage device 11 may be arranged such that when the model is performing a relatively complicated movement, the intervals between the frames 80 are close. Conversely, when the movement is relatively monotonous, the frames of the moving image are appropriately thinned out at the time of capturing the moving image data or at the time of editing the data after the capturing so that the interval between the frames 80 becomes coarse.

FIG. 16 is a flow chart showing the details of the input processing in S1100. First, after setting a portion that does not require calculation of drive data (S1101), windows W1 and W2 as shown in FIG. Display each (S1102, S
1103). The model image 39 is displayed in one window W1, and the skeleton image 38 is displayed in the other window W2.
Is displayed. The skeleton image 38 is displayed as a simplified diagram of the robot skeleton 19 shown in FIG.

Returning to FIG. 16, in step S1104, a model image 3 photographed with the model facing the video camera.
9 (FIG. 27) is performed to adjust the size of the skeleton image 38 . Details of the processing are shown in FIGS. That is, in FIG. 17, the model image 39 of the facing pose is shown.
Is displayed in the window W1 and the skeleton image 38 is displayed in the window W2 (C101 to 103), and the length of each skeleton unit is adjusted to the corresponding portion of the model image (C104). Here bone
Standard dimensions are set in advance for each skeleton unit of the case image 38 . FIG. 18 shows an example of the adjustment process.
As shown in the figure, first, the position of the reference point G is adjusted (C2
01) Then, end points of each skeleton unit are sequentially adjusted so as to gradually move away from the reference point G.

Here, the skeleton image is compared with the model image 39.
FIG. 2 shows an input method for adjusting the end points of 38 .
As shown in FIG. 7, each end point is displayed as a mark on the window W2, and the mouse 1 is displayed on the window W1.
A pointer 40 that moves on the screen with the movement of 2 is displayed. The alignment of the mark points is performed in a preset order in a direction starting from the point G and gradually toward the end of the skeleton. On the skeleton image 38 displayed in the window W2, the next alignment is performed. The end point at which the embedding is to be performed flashes to indicate this. And
By moving the pointer 40 to a position to be designated on the model image 39 on the window W1 and operating the click button of the mouse 12, the overlapping position of the end point blinking on the window W2 on the model image 39 is changed. The length of each skeleton unit is sequentially calculated based on the determined and the already determined end point positions (C202
To C210). Further, the skeleton unit whose length is determined is sequentially drawn on the model image 39 of the window W1.

Here, the model image 3 in the facing pose
9 is substantially symmetrical, the end points of the humerus 24 and the lower humerus 25, and the upper limb bone 26 and the lower limb bone 27 (FIG. 2) can be determined if one of the right and left ends is positioned. Is also performed so that the position is automatically determined. Also, the dimensions of the head 20 (the distance between H and F1, FIG.
8) compares the standard size of the skeleton unit other than the head 20 with the calculated size, and is determined based on the size ratio and the standard size of the head 20 (C210). Note that a skeleton image having standard dimensions set in advance in window W1 is used as a model.
The display may be performed together with the image 39, and a process may be performed on the window W1 to match the end point of each skeleton unit of the skeleton image to the model image 39 . In this case, the window W2 can be omitted. Next, returning to FIG. 17, the skeleton image 3 of the facing pose is adjusted according to the calculated size of the skeleton unit.
8 is redrawn in the window W1 (C105).
The process proceeds to S1105 of No. 6. Here, for the skeleton image 38 corresponding to the facing pose, the drive position of each actuator is set in advance as reference data, and the set contents are stored in the reference data storage unit 113 of the program storage device 9. I have.

Subsequently, in S1105 to S1109, by designating the frame number of the moving image for performing the positioning of the skeleton image 38, the data of the model image of the frame is acquired.
Move to the skeleton image alignment processing (skeleton image input). FIGS. 19 to 21 are flowcharts showing the details of the contents of the positioning process (S1109). That is, FIG.
As shown in C301 to C305, the model image 39 of the captured frame is gradually moved away from the point G, starting from the point G.
The position of each end point of the skeleton image 38 is adjusted to the model image 39. FIG. 28 shows an example of the input.
The procedure is almost the same as for the dimension matching process described above,
End point window W1 blinking on window W2
While the mouse 12 is used to specify the position to be aligned, the position input for the model image 39 is performed. Also in this case, the skeletal image 38 is sequentially drawn on the model image 39 on the window W1 from the position where the alignment is completed. Here, input processing is performed individually for the left and right legs and arms, unlike the above-described dimension matching processing. It should be noted that the position input is skipped for the teaching unnecessary part set in S1101 (FIG. 16).

FIG. 20 shows, as an example, the flow of the positioning process of the parts corresponding to the two legs. For each of the right leg and the left leg, the upper end point L1 (L1) of the upper limb bone 26 is shown.
After the alignment of 2), the lower end point Lf of the lower limb bone 27 is obtained.
1 (Lf2) is performed ( C401 to C403). Here, if L1 (L2) and Lf1 (Lf2) are designated by the constraint conditions shown in FIG. 10, the input is not performed because the position of Lk1 (Lk2) corresponding to the knee is automatically determined. In this manner, the coordinates of each end point on the screen are calculated, and the three-dimensional coordinates of the end point in the actual space are calculated based on the screen coordinates and the standard body size data 112 (FIG. 8) of each skeleton unit.

Next, returning to FIG. 16, the process proceeds to step S1110. For the skeleton units or parts for which it is difficult or impossible to generate drive data in the above-described alignment processing, auxiliary input processing of their spatial positions is performed by using the mouse 12 or This is performed by using the keyboard 13. In this input processing, a window different from the above-described alignment processing is opened, an image of the corresponding skeleton unit or site is displayed on the window, and various processing commands (for example, skeleton unit) prepared in advance for position setting are set. Of the skeletal unit and each part are selected and executed by inputting from the mouse 12 or the keyboard 13.

The input processing for one frame is completed as described above. However, when the processing moves to the next frame, the frame number is updated, and the skeleton image 38 and the model image 39 of the immediately preceding frame are redrawn (S1111). To S1113), and returns to S1107 to repeat the processing. On the other hand, when the input operation to the frame is completed (S1114 to S1116), the windows W1 and W2 are closed and the process proceeds to S1200 of FIG.

Here, if the displacement data of each end point obtained as described above is used as it is as drive data, the process proceeds to S1300, but if the drive data of each actuator is calculated from the displacement data. Contains S12
The process of 00, that is, the process of calculating the drive amount of the actuator that drives the skeleton unit based on the spatial position coordinates of the end points of the skeleton unit for each frame obtained by the above input is inserted. That is, as shown in FIG. 21, the numbers of two adjacent frames are designated, the spatial position coordinates of the end points of each skeleton unit are read, and based on the spatial position coordinates, provided between each skeleton unit. The drive amount of the actuator is calculated (C501 to C503). The content of the calculation process is substantially the same as the process of calculating the drive amount of each actuator by the above-described drive amount calculation program 216a , so that the detailed description is omitted. If the skeleton image of the first frame is not of the facing pose, the reference data storage unit 1
The drive position of each actuator with respect to the first frame may be calculated based on the reference position of the actuator stored in 13 (FIG. 8).

Next, the flow advances to S1300 in FIG. 15, and the time on the real time axis of the robot operation is set for each frame for which input has been completed. Details of the processing are shown in the flowchart of FIG. That is, a time setting window is opened, skeleton image data is read out in the order of frame numbers, and time is input while sequentially displaying skeleton images in the window (C601 to C601).
C605). Then, in C6051, the driving speed of each actuator is calculated from the time interval with the immediately preceding frame, and in C606, it is determined whether or not all the actuators can be completed driving at that time interval. If it can be completed, the input time and the actuator drive speed are stored in C607, the frame number is incremented in C608, and C6 is stored.
Return to 03 and repeat the process. If the driving cannot be completed, the process returns to C605 and the time is input again. And
When the time input process is completed, the window is closed at C612 and the process proceeds to S1400 in FIG.

In S1400, the finally obtained drive data is stored and stored in the drive data storage device 10. The flow of the process is as shown in FIG. 23. First, a middle operation command name is input in C701. Then, the drive data is divided for each part (C702), and a data transmission program (PA, PB,..., Etc., FIG. 3) is inserted to form a sub-file of lower-order operation data (C702). Then, these sub-files are integrated under the input middle operation command name, and are stored in the storage device 1 as a middle operation routine.
0 (C703). The sub-file of each lower-level operation data can be exchanged or replaced between different middle-level operation routines.

Next, when it is desired to correct the drive data created in this way, the processing from S2000 onward in FIG. 15 is executed. First, a drive data file is designated by inputting an operation name in S2100, and the data is read in S2200. Then, the correction processing is performed in S2300, and details of the processing are shown in FIG. First, a correction input window (same type as the positioning input window shown in FIG. 28 or the like) is opened (C801, C802), and skeleton image data is read in order from the frame in the first order, and the skeleton image is displayed in the window. It is drawn (C803, C804). And
A desired item is selected from a processing menu prepared for correction, such as frame correction, frame deletion, frame addition, frame sequential display, and the like, and processing corresponding to that item is performed.

That is, when the frame correction is selected, the skeleton image of the displayed frame is corrected (for example, the overlapping position of the skeleton unit is changed) (C808, C80).
9). When the frame deletion is selected, the displayed frame is deleted (C810, C811). When the addition of the frame is selected, a new frame is inserted next to the displayed frame, and the input frame is deleted. By the same operation and processing as in (1), the skeleton image is overlaid on the model image, etc. (C812, C8)
13). Further, when the sequential display of the frames is selected, a process of sequentially displaying the skeleton images of the respective frames on the window in a time series at a constant time interval is performed (C814,
C815). When the processing is completed, the frame number is updated in C817 , and the process returns to C804 to repeat the processing.
Upon completion of the process, the window is closed at C818, and the process proceeds to S2400 in FIG. In step S2400, the drive data is updated before and after the corrected, added, or deleted frame. In step S2500, the drive data is updated, and the process ends.

Next, based on the created driving data, the actual motion of the robot can be tested by the graphic simulator 151. In this case, FIG.
The process is performed after S3000. First, when an operation name desired to be performed in S3100 is input, a test process of the operation is started in S3200. The outline of the process is as follows. First, the drive data stored under the operation name is read out and transferred to the graphic simulator 151 together with the trial start command signal. The graphics simulator 151 receives this, synthesizes a robot image by the robot image data generation program 155a stored in the ROM 155 (FIG. 9), and performs a robot operation trial on the CRT 159 by the simulation program 155b based on the received drive data. Display a movie. It is to be noted that an image can be controlled by inputting from the keyboard 161, such as fast-forwarding or slow-playing the trial moving image or enlarging and displaying the movement of a specific portion. Note that the graphic simulator 151 can be omitted.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the overall configuration of a robot system according to the present invention.

FIG. 2 is a schematic diagram of a robot skeleton.

FIG. 3 is an explanatory diagram showing contents of a storage device of a robot control unit.

FIG. 4 is an explanatory diagram showing an example of a higher-level operation program.

FIG. 5 is an explanatory diagram showing the contents of a RAM of a robot control unit.

FIG. 6 is a block diagram showing a configuration of a calculation / drive control unit.

FIG. 7 is a block diagram illustrating a configuration example of a driving data creation device.

FIG. 8 is an explanatory diagram showing the contents of the program storage device.

FIG. 9 is a block diagram showing a configuration of a graphic simulator.

FIG. 10 is an explanatory diagram of a constraint condition for a leg skeleton.

FIG. 11 is a flowchart showing the flow of processing by a higher-level operation program.

FIG. 12 is a flowchart showing the flow of processing of a middle-level operation routine.

FIG. 13 is an explanatory diagram of a method of calculating a drive amount of an actuator.

FIG. 14 is a view for explaining the flow of the operation of the robot in association with the program hierarchy.

FIG. 15 is a flowchart showing the overall processing flow of the drive data creation device.

FIG. 16 is a flowchart of the input process.

FIG. 17 is a flowchart showing a flow of processing for determining a skeleton unit length in the input processing;

FIG. 18 is a flowchart showing the flow of a process for aligning end points of each skeleton unit.

FIG. 19 is a flowchart showing the flow of processing for calculating coordinates of end points of a skeleton.

FIG. 20 is a flowchart showing the flow of processing for calculating coordinates of end points of a skeleton unit forming a leg.

FIG. 21 is a flowchart showing the flow of drive data calculation.

FIG. 22 is a flowchart showing the flow of a time input process.

FIG. 23 is a flowchart showing the flow of a drive data storage process.

FIG. 24 is a flowchart showing the flow of a correction process.

FIG. 25 is a schematic diagram of a frame of a moving image.

FIG. 26 is a schematic diagram showing an example of an array of frames on a time axis.

FIG. 27 is an explanatory diagram of a skeleton image size matching process for a directly-facing model.

FIG. 28 is an explanatory diagram of a registration process of a skeleton image with respect to a model image .

[Explanation of symbols]

Reference Signs List 1 drive data creation device 5 system control unit (drive data calculation unit) 200 robot system 201 robot 202 robot control unit 204 CPU (read control unit) 205 RAM 205a higher-order operation program storage unit 205b middle-order operation command analysis program storage unit 205c Medium operation routine storage unit (I) 205 d Medium operation routine storage unit (II) 205 e Medium operation correction program storage unit 207 Storage device 207 a Upper operation program storage unit 207 b Medium operation command analysis program storage unit 207 c Medium operation routine Storage unit 207d Medium operation correction program storage unit 207f Rewriting program storage unit 208 Input device (program rewriting unit) 212 Calculation / drive control unit 213 Actuator

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Teruo Kato 2-28-24 Izumi, Higashi-ku, Nagoya City, Aichi Prefecture Inside Ricoh Elemex Co., Ltd. (56) References JP-A-60-151716 (JP, A) 63-257005 (JP, A) JP-A-5-265531 (JP, A) JP-A-2-81102 (JP, A) (58) Fields studied (Int. Cl. ⁶ , DB name) G05B 19 / 4155 B25J 13/00

Claims (8)

(57) [Claims] 1. A medium-level operation routine that forms lower-level operation data in units of drive data relating to driving of actuators provided in a plurality of robot parts and integrates the plurality of lower-level operation data to define a basic operation of the robot. A middle operation command corresponding to each of the middle operation routines, a higher operation program is defined by a combination of the middle operation commands, and the higher operation program is read out in units of the middle operation command. Based on the combination of the middle operation routine corresponding to each of the read middle operation commands,
A method for controlling a robot, comprising controlling the driving of the actuator. 2. A middle-level operation routine for defining a basic operation of a robot, which is constructed by integrating a plurality of lower-level operation data in units of drive data relating to driving actuators provided in a plurality of robot parts, and a corresponding operation. A program storage unit that stores a higher-level operation program defined by a combination of these middle-level operation commands, using the attached middle-level operation command; and stores the higher-level operation program stored in the program storage unit in the middle-level operation command. Program reading means for reading in units of commands, and an actuator operation control unit for controlling the driving of the actuator based on a combination of medium operation routines corresponding to the read medium operation commands. Characteristic robot system. 3. The high-order operation program based on the combination of the middle-level operation commands stored in the program storage unit is commanded to perform at least one of rearrangement, addition, and deletion of the middle-level operation command, and 3. The robot system according to claim 2, further comprising a host operation program rewriting means for rewriting the operation program. 4. The drive data forming the lower-order operation data is drive amount calculation data for calculating a drive amount of the actuator, and the actuator operation control unit is provided for each of the robot parts. A calculation / drive control unit that calculates a drive amount of an actuator included in the robot part based on the drive amount calculation data, and controls the drive of the actuator based on the calculated drive amount; 4. The data transmission means for transmitting the drive amount calculation data to the calculation / drive control means, respectively, in accordance with the execution of the lower-order operation data based on the middle-level operation routine. Robot system. 5. At least a part of the plurality of robot parts includes a skeleton having a plurality of skeleton units and the actuator disposed between the skeleton units. The corresponding drive amount calculation data includes displacement data of each of the skeleton units for each time of the robot operation, and the calculation / drive control unit sets the skeleton units in time sequence. The robot system according to claim 4, wherein a drive amount of an actuator corresponding to the skeleton unit is calculated based on the displacement data. 6. An arbitrary one of the skeleton units of the skeleton (hereinafter, referred to as a first skeleton unit) and a second skeleton unit bonded to the first skeleton unit via an actuator.
The calculation / drive control unit performs at least one of a rotational movement and a translational movement on each of the positions of the first skeleton unit and the second skeleton unit at successive times, and the first skeleton at each of the times. 6. The apparatus according to claim 5, wherein a process of superimposing the units is performed, a displacement of the second skeleton unit between the times is calculated in that state, and a driving amount of the actuator is calculated based on the calculation result. Robot system. 7. At least a part of the plurality of robot parts includes a skeleton having a plurality of skeleton units and the actuator disposed between the skeleton units. A drive data creation device for creating data, comprising: a series of model images in which the operation of a model corresponding to a robot operation is recorded with time; image display means for displaying the image of the skeleton; A skeleton positioning unit that positions the skeleton image with respect to the model image, and a displacement of the skeleton image that is positioned with respect to each of the model images that precede and follow each other on the time axis.
The robot system according to any one of claims 2 to 6, further comprising a driving data generating device, comprising: a data calculating unit that calculates the driving data. 8. The skeleton locating means of the drive data generating device, wherein the skeleton image is superimposed on the model image on the display means, thereby aligning the skeleton image with the model image. Item 7. The robot system according to Item 7.