JPH08314524A - Control method for robot and robot system - Google Patents

Abstract

PURPOSE: To provide a robot system which can easily execute the programming of a high-order robot operation including a series of low-order operations and the rewriting of it. CONSTITUTION: The robot system 1 is provided with a program storage part 205, a program read means 204 and an actuator operation control part 212. A high-order operation program regulated by the combination of middle-order operation commands is stored in a program storage part 207. A middle-order operation routine constituted integrally of plural pieces of low-order operation data in units of driving data of actuators 213 provided for plural robot parts is stored in the storage part 207 in accordance with the middle-order operation commands. The program read means 204 reads the high-order operation program in the unit of the middle-order operation command. An actuator operation control part 212 controls the driving of the actuators 213 based on the combination of the middle-order operation routines corresponding to the respective middle-order operation commands.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot control method and robot system.

[0002]

2. Description of the Related Art When various articulated robots, such as a robot that imitates human movements, perform predetermined movements, conventionally, in order to program the movements, the drive amount of each actuator for driving the robots is changed. The method of setting individually according to the time series is adopted. In this case, when the movements of a plurality of robot parts occur simultaneously or in association with each other in the operation of a certain robot, it is necessary to specify the drive amount of the actuator for each robot part. For example, in the case of an action of "walking", this involves movements of both legs, as well as movements of an arm and shoulders up and down, and the programs for the movements of the respective parts are individually assembled. And if you want to program a more complicated higher-level action, such as "Go ahead one step, thank you, and then go back to the original place",
The programming operation is performed by decomposing the higher-order motions into lower-order motions such as "go ahead one step", "give thanks", and "fall one step".

[0003]

In the above-mentioned conventional method, since the driving amount of a large number of actuators and the operation order of the actuators must be set for each robot part, it takes a long time to perform programming work, and complicated high-order operation is required. It is hard to say that it is suitable for writing programs efficiently. Further, when it is desired to change the contents of the higher-order operation, the program or data must be rewritten at the drive amount level of each actuator, which takes time. That is, since it is not so easy to change the operation content of the robot, there is a drawback in that it lacks flexibility in various operations of the robot.

An object of the present invention is to provide a robot capable of easily programming and rewriting an upper robot motion including a series of lower motions, and thus allowing the robot to perform various motions relatively easily. It is to provide a control method and a robot system.

[0005]

In order to solve the above-mentioned problems, a robot control method of the present invention forms subordinate operation data in units of drive data relating to the drive of actuators provided in a plurality of robot parts, A plurality of lower-order movement data are integrated to build a middle-order movement routine that defines the basic movements of the robot, a middle-order movement command is associated with each middle-order movement routine, and a higher-order movement program is created by combining the middle-order movement commands. Is specified, and the drive of the actuator is controlled based on the combination of the middle-order operation routines corresponding to the read middle-order operation commands while reading the higher-order operation program in units of the middle-order operation commands. Characterize.

A robot system of the present invention based on the above control method is characterized by including the following requirements. Program storage unit: Stores an upper operation program defined by a combination of middle operation commands. The middle-level motion command corresponds to a middle-level motion routine that defines the basic motion of the robot, and the middle-level motion routine is a low-order unit in which drive data relating to the driving of actuators provided in a plurality of robot parts is used as a unit. It is constructed by integrating multiple motion data. Program reading means: Reads out a higher-order operation program in units of middle-order operation commands stored in the program storage section. Actuator operation control unit: Controls the drive of the actuator based on a combination of the medium-level operation routines corresponding to the read medium-level operation commands.

In the control method and the robot system configured as described above, for example, the program of the higher operation of "go ahead one step and thank you, and return to the original place even if you step back one step", "take one step", It is defined as a combination of middle-level motion commands corresponding to the middle-level motions of "give thanks" and "one step down", respectively. And, for example, "take one step"
In the case of the middle-level motion, the lower-level motion data such as "stepping on the leg" and "arm swing" is described in the lower-level motion data in units of the drive data of the actuator of each robot part, and the lower-level motion data is described. By integrating a plurality of, the middle level operation routine is constructed as a kind of module.

If a required number of such middle-level operation routines (modules) are prepared in advance, a program for higher-level operation can be created by simply combining the corresponding middle-level operation commands, and a complicated series of operations can be performed. But programming is relatively easy. Also, the contents of the program are
It can be easily rewritten simply by adding, deleting, or replacing the middle-level operation command, and the contents of the higher-level operation can be easily changed or modified. Furthermore, since the program for higher-level operation is composed mainly of medium-level operation commands, its capacity is small,
Even if it is necessary to prepare a large number of higher-level operation programs, the amount of disk and memory occupied is small, and the installation work is simple.

Here, at least one of rearrangement, addition, and deletion of the middle-order operation command is instructed to the higher-order operation program based on the combination of the middle-order operation commands stored in the program storage unit. A high-order operation program rewriting means for rewriting can be provided.

Next, the drive data forming the lower order operation data can be drive amount calculation data for calculating the drive amount of the actuator. In response to this, the actuator operation control section causes the drive amount to be calculated. The driving amount of the actuator included in the robot part is calculated based on the calculation data, and the driving amount of the actuator is controlled based on the calculated driving amount. You can By doing so, the drive amount of the actuator is sequentially calculated as the robot operation progresses, so that it is not necessary to previously store all the drive amount data of each actuator in the program storage unit, and the entire data The amount can be reduced. Here, the calculation / driving means is provided for each robot part, and further, the data transmission for transmitting the driving amount calculation data to the calculation / driving control means with the execution of the lower-order movement data based on the middle-order movement routine. Means can also be provided. In this way, the drive amount of the actuator is calculated in parallel for each robot part,
The response of the actuator drive can be improved.

Next, at least a part of the plurality of robot parts can be provided with a skeleton having a plurality of skeleton units and actuators arranged between the skeleton units. In this case, the drive amount calculation data corresponding to the robot part may include displacement data of each skeleton unit for each time of robot operation. Then, the calculation / drive control means is
The drive amount of the actuator corresponding to the skeletal unit can be calculated based on the displacement data that are temporally behind each other. Further, more specifically, an arbitrary one skeleton unit of the skeleton (hereinafter referred to as a first skeleton unit),
The calculation / drive control means may calculate the drive amount for the second skeleton unit coupled to the first skeleton unit via the actuator as follows. That is, at least one of rotational movement and translational movement is performed on the positions of the first skeleton unit and the second skeleton unit at successive times, and the processing for superimposing the first skeleton units at those times is performed. . Then, in that state, the displacement of the second skeleton unit between those times is calculated, and the drive amount of the actuator is calculated based on the calculation result.

Further, when at least a part of the plurality of robot parts is provided with the above-mentioned skeleton, it is possible to provide a drive data creating device including the following requirements. Image display means: Displays a series of model images in which the motion of the model corresponding to the robot motion is recorded over time and an image of the skeleton of the robot (skeleton image). Skeleton positioning means: Aligns the skeleton image on the image display means with the position corresponding to the model image. Specifically, the display means can be configured to be aligned with the model image by superimposing the skeleton image on the model image. Drive data calculation means: Skeleton displacement data and skeleton angle data based on the displacements of the skeleton images that are aligned with respect to the model images moving forward and backward on the time axis (these two are regarded as drive amount calculation data). Or the actuator drive amount data,
Calculate as drive data.

For example, a model that changes over time is obtained by causing a human model to actually perform a certain intermediate motion, recording this as a series of images, and aligning each of these images with the skeleton image of the robot. The skeleton image is positioned with respect to the model image at each specific time of operation, and drive data is calculated based on the displacement of the skeleton between adjacent images. By doing this, it is possible to collectively obtain the lower-order motion data for each part included in the middle-level motion, and if a middle-level motion command is given and saved in the storage unit, it will be used as a middle-level motion routine. It can be used.
Then, the work for creating the drive data is centered on the operation of aligning the skeleton image with the model image on the screen of the image display means, so the contents are easy to understand, and
Since it is not necessary to individually set the drive amount of the actuator for each part of the robot, it is possible to reasonably and easily obtain a large amount of drive data necessary for creating the higher-order operation program.

The above-described drive data creation device is provided with drive data storage means for storing drive data calculated and generated based on the drive data calculation means, and drive data editing means for editing the stored drive data. be able to. By using the editing means, it is possible to easily correct, add or delete the generated drive data. In addition, the editing means can edit the drive data for each type of middle-level motion and the part of the robot, whereby replacement of lower-level motion data, exchange of lower-level motion data between different middle-level motion routines, etc. Therefore, the versatility of the data can be enhanced and various operations can be performed by the robot. Further, it is also possible to unify relatively similar lower-level motions of the same robot part between different middle-level motions into one lower-level motion, which has the effect of reducing the overall data amount and increasing the responsiveness of the robot motion.

In the above drive data creating apparatus, for example, when a robot's motion is copied, the motion of the human model is recorded as a moving image by a photographing means such as a video camera, and the moving image is frame-fed to the image display means. The position of the skeleton image may be adjusted for each frame of the moving image during playback. Here, the dimension of the recorded model image may change depending on the dimension (or body shape) of the selected model. In this case, the length of each skeleton unit of the skeleton image is displayed on the display means. It can be made expandable and contractable according to the dimensions of the model image.

Further, the time intervals of the model images can be changed according to the operation contents of the robot. For example,
When playing back video images frame by frame, the frames of those images are taken at fixed time intervals, but when converting a relatively simple model motion into data, by thinning out a fixed number of frames, The time interval between adjacent model images can be increased. As a result, the number of images required for teaching is reduced, and teaching work can be made more efficient. Moreover, since the amount of data required for the calculation is reduced, the responsiveness of the robot operation can be improved.

Next, with respect to a part of the skeleton of the robot, the driving data calculating means generates driving data based on the displacements of the skeleton images respectively inputted to the model images which are in front of and behind each other on the time axis. While calculating, auxiliary input means are provided corresponding to the remaining parts of the robot skeleton and predetermined robot parts other than the skeleton,
The auxiliary input means may be configured to individually input the drive data for the remaining skeleton and the robot part. For example, for a robot part (for example, movement of wrist, finger, eye, mouth, etc., lighting of a lamp, etc.) for which it is difficult or impossible to calculate drive data from the skeleton image aligned with the model image, the auxiliary input means is used. It is possible to directly input the drive data, and thus it is possible to realize a fine movement for the details of the robot.

Specifically, the skeleton of the robot includes a plurality of skeleton units connected by an articulation mechanism, and the skeleton units can be configured to be driven by an actuator provided in the articulation mechanism. in this case,
A reference attitude is set in advance for the model, and the drive data calculation means stores reference data including the reference position of each actuator for the skeleton image aligned with the model image of the reference attitude. Reference data storage means can be provided. Then, the drive data calculation means calculates the displacement of each skeleton unit at each time based on the reference data, or calculates the drive amount of each actuator based on the displacement.

Next, the input of the alignment position of the skeleton image with respect to the model image is performed by designating the positions on the display screen of the display means of the predetermined mark points at both ends of each skeleton unit. You can As a more specific device configuration therefor, the following can be exemplified. That is, a pointing device such as a mouse or a trackball is provided in the skeleton positioning means, and the display means displays a model image and a skeleton image on the display screen and a pointer that moves on the screen in accordance with the operation of the pointing device. Let Then, when aligning the skeleton image with the model image, one of the mark points to be aligned is selected, the pointer is aligned with the alignment position set on the model image, and the mark is attached to the pointing device. The skeleton unit is aligned with the position corresponding to the alignment mark point by operating the input unit to specify the alignment position of the alignment mark. By doing so, the positioning operation of the skeleton image with respect to the model image can be made more direct and easy to understand.

Further, a plurality of skeleton units form a set, and a predetermined constraint condition can be set for the relative positional relationship between the skeleton units included in the set. And
A part of the skeleton mark points belonging to the set is specified as a matching position with respect to the model image, so that the matching position of the remaining mark points is determined based on the constraint condition. can do.
By doing so, the positioning operation can be simplified and the working time can be shortened, and the inconvenience of designating a position that is structurally impossible for the robot part can be obtained.

Here, it is possible to predetermine the order of alignment of the skeleton unit with respect to the model image, and perform the alignment operation according to the order.
In this case, when the alignment operation for a certain skeleton unit is completed, the color of the mark point of the skeleton unit to be operated next should be changed or blinked to make the display state different from other mark points. Can be configured to.

Next, the mechanism for calculating the drive amount of the actuator for driving the skeleton unit from the displacement of the skeleton unit, specifically, the following can be adopted. That is, the drive data calculation means for an arbitrary one skeleton unit (hereinafter, referred to as a first skeleton unit) of the skeleton of the robot and a second skeleton unit connected to the first skeleton unit via an actuator, The first skeleton unit of both skeleton images is obtained by subjecting at least one of the skeleton images moving forward and backward on the time axis to at least one of rotational movement and translational movement of the first skeleton unit and the second skeleton unit. Overlap each other. Then, in the superposed state, the displacement of the second skeleton unit between both skeleton images is calculated, and the drive amount of each actuator is calculated based on the calculation result.

The drive data calculation means is provided in the image display means.
The three-dimensional position of the skeleton unit can be calculated based on the image of the skeleton unit displayed dimensionally and the image of the standard body having a known dimension.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows the overall configuration of a robot system 200 as an embodiment of the present invention, which includes a robot 201, a robot controller 202, a drive data creation device 1, and a graphic simulator 151, which are each a communication interface 203. , 152, 153, and are connected to each other by a communication line 150.

The robot controller 202 includes a CPU 204, a RAM 205, a ROM 206 connected to the CPU 204, a storage device 207 including a hard disk device, an input device 208 such as a keyboard, and a display controller 210.
The display control unit 210 includes a display device 21 such as a CRT.
1 is connected, and the communication interface 203 is connected to the CPU 204.

On the other hand, the robot 201 is constructed, for example, as a robot that imitates the movement of a human, and as shown in FIG. 2, the skeleton units 20 to 30 are provided with skeletons 19 connected to each other by joint mechanisms 31 to 37. Then, this skeleton has, for example, the following robot parts (hereinafter, parts A, B, and C).
..., etc.). Head (20): A drivable eye portion 20a and a mouth portion 20c are provided. A blinkable lamp 20b is arranged in the eye portion 20a, and the mouth portion 20c can be opened and closed. -Torso: Includes shoulder 21, spine 22, and hip bone 23. -Left and right arm parts: Each has a humerus 24, a lower humerus 25, and a wrist 29. Left and right legs: Upper limb bone 26, lower limb bone 27, and ankle 30, respectively.

Then, as shown in FIG.
A calculation / drive control section (calculation / drive control means) 212 is provided corresponding to each of the parts A, B, C, etc., and is connected to the CPU 204 of the robot control section 202. Then, the calculation / drive control unit 212
On the other hand, the actuator 213 included in each part is connected. The types of actuators (pulse motor, servo motor, piston cylinder, etc.) provided in each joint mechanism 31 to 37 in FIG. 2, the number of shafts, the degree of freedom of shaft rotation, and the range of shaft rotation or piston sliding. Etc. are individually set for each skeleton unit.

Next, as shown in FIG. 3, the storage device 207 of the robot control unit 202 includes a high-order operation program storage unit 207a and a middle-order operation command analysis program storage unit 2.
07b, a middle operation routine storage unit 207c, a middle operation correction program storage unit 207d, and a rewriting program storage unit 207f are formed, and each stores a corresponding program or routine (or data).

The operation of each program is as follows. (1) Upper operation program: A desired upper operation is described by selecting an appropriate middle operation command prepared in advance and arranging it. For example, when the robot 201 wants to perform a series of higher-level actions such as "walk to the right for four steps, turn to the right for one and a half steps, and take a karate chop pose", the higher-level action program is as shown in FIG. As shown in, "turn right", "half step of right leg", "one step of left leg"
As a general rule, middle-order operation commands (middle-order operation commands) composing the higher-order operations are arranged in the order of execution.

(2) Middle operation routine: formed for each of the above middle operation commands, the part A of the robot 201,
For each of B, C ..., Drive data DA1, DA2, ..., DB1, DB of actuators included in that part
2, ..., etc. and those driving data are calculated for each part.
Data transmission programs PA, PB, etc. for transmission to the drive control unit 212 are included. These data transmission programs PA, PB, ... Etc. perform the operations in which the parts A, B, C, ... Are respectively defined for the middle operation specified by the middle operation command ( For example, for the middle movement of "one step of the right leg", the movement of the right leg to step
Further, the driving data DA1 and DA2 of each actuator are sequentially transmitted to the driving data calculation unit 212 of each part so that the left and right arms perform forward and backward swinging motions in accordance with the stepping-out). Here, each data transmission program PA, P
B, ..., etc. are the corresponding drive data sets (DA1, DA2, ...
・ ・), (DB1, DB2, ...) and so on, which constitutes the lower-order operation data. Here, drive data DA1, D
A2, ... Are supposed to include displacement data at both ends of the skeleton unit driven by each actuator, but may be angle data between the skeleton units. On the other hand, the drive data DA1, DA2, ... Can be used as the drive amount for each time of each actuator.

(3) Medium-level operation command analysis program: A high-level operation program is read in units of medium-level operation commands, and corresponding intermediate-level operation programs are sequentially activated. (4) Medium motion compensation program: Corrective motion required for joining motions when the robot motion start position based on the medium motion routine does not match the end position of the robot motion by the immediately previous medium motion routine. To generate. (5) Rewriting program: Rewrites or newly creates a host operation program. Specifically, the program creation screen is opened on the display device 211 (FIG. 1), the upper operation command is input using the input device 208 in the format as shown in FIG. 4, and the upper operation program name is given to the input content. do it,
It is stored in the upper operation program storage unit 207a of the storage device 207. Further, the stored higher-order operation program is displayed on the program creation screen, and the higher-order operation command is added, deleted, inserted, etc. based on the input from the input device 208, and the program is corrected / edited.

On the other hand, as shown in FIG. 5, the RAM 205 includes a high-order operation program storage section 205a, a middle-level operation command analysis program storage section 205b, a middle-level operation routine storage section (I) 205c, and a middle-level operation routine storage section. (II) 20
5d and the like are formed to store the corresponding programs and data read from the storage device 207, respectively. Here, the intermediate operation routine storage section is (I)
Two of (II) are provided. By reading and accumulating a middle-level operation routine to be executed next, the data is read from the storage device 207 when the robot 201 moves to the next operation. This is to reduce the delay of the response based on.

Further, as shown in FIG. 6, the calculation / drive control section 212 has a CPU 214, a RAM 215, and an R, respectively.
The OM 216 and the like are provided, and the drive amount calculation program 216a is stored in the ROM 216. Then, the calculation / drive control unit 212 executes the middle-level operation routine, and the drive data DA1, DA2, ...
.., etc., and calculates the drive amount of each actuator 213 (FIG. 1) connected to the CPU 214 in chronological order,
Each actuator 213 is driven based on the calculation result. The calculation method will be described later.

FIG. 7 is a block diagram showing an example of the structure of the drive data creating apparatus 1. The drive data creation device 1 includes a CPU 2, a ROM 3, and a RAM that constitute drive data calculation means.
4 is provided with a device control unit 5 including an image capture control unit 6, a display control unit 7, an input device interface 8, a program storage device 9, a drive data storage device 10, and a moving image data storage device 11. . The input device interface 8 has a click button 12a (input section).
A mouse 12 and a keyboard 13 are connected as a skeleton positioning means having the above.

The display control unit 7 has C as an image display means.
RT14 is connected. On the other hand, the image capture controller 6
Is equipped with a VRAM 16 and a data transfer processor 17 while being connected to a video playback device (VTR) 15, and image display data stored in the VRAM 16 can be directly transferred to the display control unit 7 via a bus 18. Has been done.

Next, each program shown in FIG. 8 is stored in the program storage device 9 and is read by the CPU 2 as necessary and written in a predetermined area of the RAM 4 for use. (1) Video capture program 90: The image of the model motion reproduced by the VTR 15 is captured as video data via the image capture control unit 6 and stored in the video data storage device 11 composed of a magneto-optical disk device or the like. . (2) Model image display program 91: The model image of the designated frame is displayed on the CRT 14 based on the moving image data stored in the moving image data storage device 11. (3) Skeleton image display program 92: robot skeleton 19
Image data of a skeleton image corresponding to (FIG. 2) is created, and as shown in FIG. 28, the skeleton image 38 based on the data is displayed on the CRT 14 together with the model image 39.

(4) Skeleton image registration program 94:
Using the mouse 12, the skeleton image 38 displayed on the CRT 14 is superimposed on the model image 39 and aligned. Here, the processing for matching the size of the skeleton image 38 with the size of the model image 39 on the screen of the CRT 14 is also performed by this program. (5) Displacement calculation program 96: Calculates the three-dimensional position of each skeleton unit in the actual space from the images of each skeleton unit displayed two-dimensionally on the CRT 14, and moves in chronological order based on it. The displacement of each skeleton unit between the two images is calculated. The calculation process compares the size of a standard body image obtained by photographing a standard body of known size prepared for each skeleton unit from a predetermined direction with the size of the corresponding skeleton unit on the screen. It is executed by The standard body size data is stored in the standard body size data storage unit 112 of the program storage device. (6) Actuator drive amount calculation program 98: The drive amount of each actuator corresponding to the displacement in the skeleton unit is calculated, and the drive data storage device 10 together with the displacement data.
To memorize.

(7) Motion time setting program 100: The time on the real time axis of the robot motion is set for the moving image frame to be used, and the driving speed of each actuator is calculated from the time interval between adjacent frames. This is stored in the drive data storage device 10 as a part of the drive data. (8) Auxiliary input program 102: Auxiliary input processing of drive data for a skeletal unit or robot part designated in advance is performed by using the mouse 12 or the keyboard 13. Such auxiliary input processing is performed by the skeleton image 38.
It is applied to the skeletal unit or site where it is relatively difficult to generate drive data by the registration process between the
For example, the movement of the eye portion 20a and the mouth portion 20b of the head portion 20 shown in FIG. 2 and the blinking of the lamp 20b arranged in the eye portion 20a correspond to this. On the other hand, for some skeletal units, it may be difficult to generate drive data by alignment processing only for a specific operation mode, but even in such a case, drive data can be input by the above method. Is. This method is performed by, for example, tilting the head 20a forward and backward, tilting left and right, rotating, or rotating the humerus 24 and the lower humer 25 around the longitudinal axis with respect to the joint mechanism 33 (corresponding to an elbow),
It is applicable to the rotation of the wrist 29, the bending state of the finger, and the like. It should be noted that, in the present embodiment, such an input process is a screen (or another window) different from the screen (FIG. 28) in which the skeleton image 38 is aligned with the model image 39.
It is designed to be opened.

(9) Frame correction program 104: The drive data input by the skeleton image position adjustment processing is changed and corrected (for example, the position of each skeleton is changed) in the designated frame. When any position of the skeleton unit connected by the above-mentioned constraint condition is changed, the positions of other skeleton units are automatically corrected so that the constraint condition is not broken. (10) Frame deletion / addition program 105: Deletes data of a specified frame, and adds / inserts a frame for inputting new drive data next to the specified frame position. (11) Sequential display program 106: The skeletal images of each frame are sequentially displayed on the screen in chronological order at regular time intervals. Accordingly, it is possible to confirm a rough change in the movement of the skeleton image, and it can be used as a simple simulation means of an actual robot operation. (12) Unnecessary part setting program 107: When the position of the skeleton image 38 is aligned, a part for which driving data is unnecessary is set. For example, when it is known that a specific part hardly moves in the target motion, the unnecessary part is set to the part so that the part is fixed at the determined position. The process of aligning the skeleton image 38 with respect to the part is skipped.

(13) Driving data file editing program 1
08 (driving data editing means): The driving data is divided into files according to the type of medium-level movement and the part of the robot, and the medium-level movement command name is given and stored in the driving data storage device 10.
save. (14) Simulator start-up program 109: The drive data of the specified robot movement is transferred to the graphic simulator 1.
Then, the process is transferred to 51, and the simulator 151 is prompted to start the robot motion demonstration process.

Further, the CPU 2 of the drive data creating apparatus 1 has the communication interfaces 152 and 153 and the communication line 1.
A graphic simulator (hereinafter simply referred to as a simulator) for performing a trial demonstration of robot operation via 50
151 is connected. The simulator 151 is shown in FIG.
As shown in, the CPU 154 and the RO connected to it
An M155, a RAM 156, a VRAM 157, a display controller 158, and an input device interface 160 are provided,
The display control unit 158 is connected to a display device such as a CRT 159 for displaying a demonstration movie of robot operation, and the input device interface 160 is connected to an input unit such as a keyboard 161. Also, a communication interface 153 for communicating with the drive data creation device 1 is connected to the CPU 154.

Next, the skeleton 1 of the robot 201 shown in FIG.
Some of the skeletal units forming 9 form a set, and a predetermined constraint condition is set for the relative positional relationship between the skeletal units included in the set. For example, in the case of a skeletal unit that constitutes both legs, as shown in FIG. 10, the lower limb bone 27 moves only within a plane P formed by a straight line along the upper limb bone 26 and a straight line perpendicular to the hip bone 23. The constraint condition is set (the joint mechanisms 35 and 36 (FIG. 2) are not drawn in FIG. 10).
Therefore, if the upper end position of the upper limb bone 26 and the lower end position of the lower limb bone 27 are designated, the position of the connecting point between them (the part corresponding to the knee) is automatically determined according to the above-mentioned constraint condition. Such constraint conditions are stored in the constraint condition storage unit 110 (FIG. 8) of the program storage device 9 and used for calculation of drive data.

The operation of the robot system 1 of this embodiment will be described below. First, regarding the flow of processing when operating the robot 201, when a higher-order operation program name is designated by the input device 208 or the like in FIG. 1 and activated, the middle-level operation command analysis program is activated, and FIG.
As shown in the flow chart of No. 1, the first middle operation command of the upper operation program is read in M1 and M2
The corresponding middle-level operation routine is stored in the middle-level operation routine storage section (I) 205c of the RAM 205. Next, a signal is issued to the stored intermediate operation routine to activate it (M3). Also, while the middle operation routine is being executed, the next middle operation command is read by M4, and the corresponding middle operation routine is stored in the empty middle operation routine storage section of the RAM 205 (in this case, the storage section (II)). ).
Further, if a middle-level processing completion signal is transmitted from the middle-level operation routine (storage unit (I) side) that is being started and executed in M6, the middle-level operation routine stored in the storage unit (II) 205d. Send a start signal to (M7). Then, if there is a next middle-level operation command in M8, it returns to M4, reads the next command, stores the corresponding middle-level operation routine in the empty storage section (in this case, storage section (I)), By repeating the same processing thereafter, the upper operation program is executed in units of middle operation commands. If the next middle operation command does not exist or if it is an end command, the process ends.

Next, regarding the processing on the side of the intermediate operation routine, as shown in the flowchart of FIG. 12, by receiving a start signal from the upper operation program side, the drive data of the actuator of each part is read (B1 , B2) and the drive data transmission program is activated to transmit the drive data described as displacement data for each time of the actuator to the calculation / drive control unit 212 of each part in chronological order. Then, the calculation / drive control unit 212 side calculates the drive amount of each actuator based on the displacement data based on the drive amount calculation program 216a, and drives the actuators based on the calculation result.

As a method of calculating the driving amount, for example,
A method based on the following principle can be adopted. First, in the skeleton 19 shown in FIG. 2, indexes are given to both end positions of each skeleton unit as shown in FIG. 27 (W1), for example.・ Head 20: H, f1. -Shoulder 21: A1, H and H, A2. -Spine 22: G and H. Here, G is used as a reference position when calculating the drive amount of the actuator from the displacement of each skeleton unit. -Hipbone 23: L1 and L2. G is located at the midpoint of the line segment connecting L1 and L2. Humerus 24, lower humerus 25: A1, Ae1, Ah1 and A
2, Ae2, Ah2. -Upper limb bone 26, lower limb bone 27: L1, Lk1, Lf1 and L
2, Lk2, Lf2. Then, the displacement data is calculated as the displacement of both ends of each skeleton unit.

For example, for the left leg of FIG. 27,
As shown schematically in FIG. 13A, when the position of the point G is unchanged and the positions of L1 and Lk1 at a certain time T1 move to L1 ′ and Lk1 ′ at the next time T2, As shown, the line segment GL1 and the line segment L1Lk1 (the first skeleton unit and the second skeleton unit in the claims, respectively) are rotationally moved to superimpose the line segment GL1 on the line segment GL1 ′. Then, the angle θ formed by the line segment L1Lk1 at time T1 and the line segment L1′Lk1 ′ at time T2 is calculated from the spatial coordinates of the respective points after the rotational movement, whereby both skeleton units (that is, the upper arm 26 And the rotation angle of each axis of the actuator disposed between the lower leg portion 27. The process of determining the rotational movement of the skeleton unit and the rotation angle of the second skeleton unit is performed in the direction toward the end of the skeleton. The driving amount of the actuators included in each joint mechanism is sequentially calculated by applying the actuators to the first and second skeleton units in the above description. Although only the relative rotation between them is performed by the rotation of their motor shafts, when piston rods etc. are used as actuators, expansion and contraction of the piston causes relative translational movement between skeleton units. In such a case, in addition to the rotation angle of the second skeleton unit, the translation distance is also calculated.

When the drive amount of the actuator is calculated in this way and a series of drive of the actuator based on the drive data of the intermediate operation routine is completed, the drive amount calculation program 216a sends the lower processing end signal to the intermediate operation. Sent to the routine side. On the medium operation routine side,
Receiving this at B4 in FIG. 12, the middle side processing completion signal is transmitted to the upper operation program. When the drive amount of the actuator itself is transmitted as drive data, the above calculation process on the calculation / drive control unit side becomes unnecessary.

FIG. 14 shows a robot 2 based on the above flow.
The specific execution example of the upper operation of 01 is shown. That is, in order to execute the higher-order motion of “walking four steps”, the higher-order motion programs are “right leg half step” and “left leg part 1”.
It includes the middle-level motion commands of "step", "one step on the right leg", and "half step on the left leg". Then, for example, the contents of the execution of the first middle-step motion routine of "half step of the right foot" are "half stepping" and "front stepping down" for the respective parts of the right leg and the left arm, and the relations between these parts. The drive data of the actuator is calculated in time series and transmitted to the drive / control unit 212.
Then, the calculation / drive control unit calculates the drive amount based on the transmitted drive data, drives the corresponding actuator, and shifts to the execution of the next middle-order operation command "left leg 1 step".

The drive data as described above can be created by the drive data creating apparatus 1. The flow of operation of the drive data creation device 1 (FIG. 7) will be described below with reference to a flowchart. FIG. 15 shows a rough flow of the entire processing, and one of the following processings is selected. In addition, at the time of selection, a predetermined selection screen is displayed on the CRT 14 as a window, and individual processing windows are opened by selecting the selection items displayed there with the mouse 12. (1) Input (process from S1000 to S1100): The skeleton image is superposed on the model image to create drive data (in other words, teaching of the operation to the robot 201).
To do. (2) Correction (processing from S2000 to S2100): The created drive data is corrected. (3) Demonstration (process from S3000 to S3100): Using the graphic simulator 151, a robot motion is demonstrated. (4) End instruction (process from S4000 to end): Gives an instruction to end the process.

When the input is selected, the input processing is performed in the unit of middle level operation in S1100. Here, prior to the input, the human model is caused to perform the middle operation, and as shown in FIG. 25, the model image 38 is taken as a moving image by the video camera at predetermined time intervals. Then, the moving image reproduced by the VTR 15 of FIG. 7 is captured as moving image data via the image capture control unit 6,
It is stored in the moving image data storage device 11. Here, the moving image data stored in the moving image data storage device 11 is
As shown in FIG. 26, when the model is moving in a relatively complicated manner, the intervals of the frames 80 are close. On the contrary, when the motion is relatively monotonous, the intervals of the frames 80 are small. To make it coarse, the frames of the moving image are appropriately thinned when the moving image data is loaded or when the data is edited after the loading.

FIG. 16 is a flow chart showing the details of the input processing of S1100. After first setting the portion where the drive data calculation is unnecessary (S1101), the windows W1 and W2 as shown in FIG. 27 are displayed. Display each (S1102, S
1103). Of these, the model image 39 is displayed in one window W1, and the skeleton image 38 is displayed in the other window W2.
Is displayed. The skeleton image 38 is displayed as a simplified diagram of the robot skeleton 19 shown in FIG.

Returning to FIG. 16, in S1104, the model image 38 photographed with the model facing the video camera.
A process for fitting the dimensions of the skeleton image 39 to (FIG. 27) is performed. Details of the processing are shown in FIGS. 17 and 18. That is, in FIG. 17, the model image 38 in the facing pose is displayed in the window W1 and the skeleton image 39 is displayed in the window W2 (C101 to 103), and the length of each skeleton unit is adjusted to the corresponding portion of the model image (C104). ). Here, standard dimensions are set in advance for each skeleton unit of the skeleton image 39.
The matching process is as shown in FIG.
First, the position of the reference point G is adjusted (C201), and then the end points of the respective skeleton units are sequentially adjusted so as to gradually move away from the reference point G.

Here, the input method for fitting the end points of the skeleton image 39 to the model image 38 is shown in FIG.
As shown in FIG. 7, each end point is displayed as a mark point on the window W2, and the mouse 12 is displayed on the window W1.
A pointer 40 that moves on the screen is displayed along with the movement of. The registration of the mark points is performed in a preset order in the direction starting from the point G and gradually moving toward the end of the skeleton, and the next matching is performed on the skeleton image 38 displayed in the window W2. The end points to be included in the flashing flashes to inform you. Then, by moving the pointer 40 to the position to be designated on the model image 38 on the window W1 and operating the click button of the mouse 12, the blinking end points on the window W2 are overlaid on the model image 38. The position is fixed, and the length of each skeleton unit is calculated sequentially based on the positions of the end points that have already been fixed (C202 to C2
Ten). Further, the skeleton unit of which the length is determined is sequentially drawn on the model image 38 of the window W1.

Here, the model image 3 in the facing pose 3
Since 8 is substantially symmetrical, each end point of the humerus bone 24 and the inferior arm bone 25, and the upper limb bone 26 and the lower limb bone 27 (FIG. 2) is the other one if the left or right one is positioned. Is also automatically positioned. Also, the size of the head 20 (distance between H and F1, FIG.
8) is determined based on the dimensional ratio and the standard size of the head 20 by comparing the standard size of the skeleton unit other than the head 20 with the calculated size (C210). The window W
A skeleton image having a standard size preset to 1 may be displayed together with the model image 38, and the end point of each skeleton unit of the skeleton image may be adjusted to the model image 38 on the window W1. In this case, the window W2 can be omitted. Next, returning to FIG. 17, after the skeleton image 38 in the facing pose is redrawn in the window W1 according to the calculated size of the skeleton unit (C105), S1105 in FIG.
Proceed to. Here, with respect to the skeleton image 38 corresponding to the facing pose, the driving position of each actuator is preset as reference data, and the setting content is stored in the reference data storage unit 113 of the program storage device 9. There is.

Subsequently, in S1105 to S1109, by specifying the frame number of the moving image for aligning the skeleton image 38, the data of the model image of the frame is fetched,
The process proceeds to the skeleton image registration processing (skeleton image input). 19 to 21 are flowcharts showing details of the contents of the alignment processing (S1109). That is, FIG.
As shown in C301 to C305, in the direction starting from point G and gradually moving away from the model image 39 of the captured frame,
The positions of the end points of the skeleton image 38 are adjusted to the model image 39. The input example is shown in FIG. 28.
The procedure is almost the same as the case of the dimension matching process described above,
Window W1 at the end point blinking on window W2
While designating the alignment position in (1) with the mouse 12, the alignment input for the model image 39 is performed. Also in this case, the skeleton image 38 is sequentially drawn on the model image 39 on the window W1 from the position where the alignment is completed. Here, the left and right legs and arms are individually subjected to input processing, unlike the above-described dimension matching processing. Note that the alignment input is skipped for the teaching unnecessary portion set in S1101 (FIG. 16).

FIG. 20 shows, as an example, the flow of the alignment processing of the parts corresponding to both legs. The upper end point L1 (L1) of the upper limb bone 26 of each of the right leg and the left leg is shown.
After performing the alignment of 2), the lower end point Lf1 of the lower limb bone 27
Positioning of (Lf2) is performed (C
410-C403). Here, according to the constraint condition shown in FIG.
If L1 (L2) and Lf1 (Lf2) are designated, the position of Lk1 (Lk2) corresponding to the knee is automatically determined, so no input is made. In this way, the coordinates of each end point on the screen are calculated, and the three-dimensional coordinates of the end point in the actual space are calculated based on the screen coordinates and the standard body size data 112 (FIG. 8) of each skeleton unit.

Next, returning to FIG. 16 and proceeding to S1110, auxiliary input processing of the spatial position of the skeleton unit or site for which it is difficult or impossible to generate drive data by the above-mentioned alignment processing is performed by the mouse 12 or. This is done by using the keyboard 13. This input processing opens a window different from the above-mentioned registration processing, displays the image of the corresponding skeleton unit or site on the window, and displays various processing commands (for example, skeleton unit) prepared in advance for position setting. Parallel movement, rotation, etc.) are selected and executed by inputting from the mouse 12 or the keyboard 13, while inputting individually for each skeleton unit or site.

When the input process for one frame is completed as described above, but when the process proceeds to the next frame, the frame number is updated, and the skeleton image 38 and the model image 39 of the immediately preceding frame are redrawn (S1111). ~ S1113), and returns to S1107 to repeat the process. On the other hand, when finishing the input work to the frame (S1114 to S1116), the windows W1 and W2 are closed and the process proceeds to S1200 in FIG.

Here, when the displacement data of each end point obtained as described above is used as it is as drive data, the process proceeds to S1300. However, when the displacement data is calculated from the displacement data In step S1200, a process for calculating the drive amount of the actuator that drives the skeleton unit is inserted based on the spatial position coordinates of the end points of the skeleton unit for each frame obtained by the above input. That is, as shown in FIG. 21, the numbers of two adjacent frames are designated, the spatial position coordinates of the end points of the respective skeleton units are read, and the space positions are provided between the skeleton units based on the spatial position coordinates. The drive amount of the actuator is calculated (C501 to C503). The content of the calculation process is almost the same as the process of calculating the drive amount of each actuator by the drive amount calculation program 216 described above, and thus detailed description thereof will be omitted. If the skeleton image of the first frame is not of the facing pose, the drive position of each actuator for the first frame is determined based on the reference position of the actuator stored in the reference data storage unit 113 (FIG. 8). Should be calculated.

Next, proceeding to S1300 in FIG. 15, the time on the real time axis of the robot operation is set for each frame for which input has been completed. The details of the processing are shown in the flowchart of FIG. That is, the window for setting the time is opened, the skeleton image data is read in the order of frame numbers, and the time is input while sequentially displaying the skeleton images in the window (C601-
C605). Then, in C6051, the driving speed of each actuator is calculated from the time interval from the immediately preceding frame, and in C606, it is determined whether or not all the actuators can be driven at that time interval. If it can be completed, the input time and actuator drive speed are stored in C607, the frame number is incremented in C608, and C6 is incremented.
Return to 03 and repeat the process. If the driving cannot be completed, the process returns to C605 and the time is input again. And
When the time input process ends, the window is closed at C612 and the process proceeds to S1400 in FIG.

In S1400, the drive data finally obtained is stored / stored in the drive data storage device 10. The flow of the process is as shown in FIG. 23. First, the middle level operation command name is input at C701. Then, the drive data is divided for each part (C702), and a data transmission program (PA, PB, ..., FIG. 3) is inserted into each to form a sub-file of the lower operation data (C702). Then, these subfiles are integrated by the input middle-level operation command name, and the storage device 1 is integrated as a middle-level operation routine.
It is saved in 0 (C703). In addition, the sub-files of the respective lower-order operation data can be exchanged or replaced between different middle-level operation routines.

Next, when it is desired to correct the drive data created in this way, the processing from S2000 onward in FIG. 15 is executed. First, in S2100, a drive data file is specified by inputting an operation name, and the data is read in S2200. Then, the correction process is performed in S2300, and the details of the process are shown in FIG. First, open the correction input window (the same model as the positioning input window shown in FIG. 28) (C801, C802), the skeletal image data is read in order from the previous frame, and the skeletal image is displayed in the window. It is drawn (C803, C804). And
A desired item is selected from a processing menu prepared for correction such as correction of a frame, deletion of a frame, addition of a frame, and sequential display of frames, and a process corresponding to the selected item is performed.

That is, when the frame correction is selected, the skeleton image of the displayed frame is corrected (for example, the overlapping position of the skeleton unit is changed) (C808, C809). Also,
If you select to delete frames, the displayed frame will be deleted (C810, C811). If you select to add a frame,
A new frame is inserted next to the displayed frame, and the skeleton image is overlaid on the model image by the same operation and processing as at the time of input (C812, C813). Further, when the sequential display of the frames is selected, a process of sequentially displaying the skeleton images of the respective frames in a time-sequential order in a window at regular time intervals is performed (C814, C815). And when the process ends,
The frame number is updated with C817, returned to C805, and the process is repeated. When the processing is completed, the window is closed at C818 and the process proceeds to S2400 in FIG. Also, in S2400, the drive data is updated before and after the frames that have been corrected, added, or deleted.
Save this with 0 to end the process.

Next, based on the created drive data, the graphic simulator 151 can demonstrate the actual movement of the robot. In this case,
The process is from S3000 onwards, but first enter the name of the action you want to perform in S3100, and the demonstration process for that action will start in S3200. The outline of the processing is as follows. First, the drive data stored under the operation name is read out, and this is transferred to the graphic simulator 151 side together with the trial start command signal. In response to this, the graphic simulator 151 synthesizes the robot image by the robot image data generation program 155a stored in the ROM 155 (FIG. 9), and based on the received drive data, the simulation program 155b causes the CRT 159 to perform a robot operation demonstration. Display the video. By inputting from the keyboard 161, it is possible to perform image control such as fast-forwarding or slow-moving the demonstration movie, or enlarging and displaying the movement of a specific part. The graphic simulator 151 may be omitted.

[Brief description of drawings]

FIG. 1 is a block diagram showing the overall configuration of a robot system of the present invention.

FIG. 2 is a schematic diagram of a robot skeleton.

FIG. 3 is an explanatory diagram showing the contents of a storage device of a robot controller.

FIG. 4 is an explanatory diagram showing an example of a higher-level operation program.

FIG. 5 is an explanatory diagram showing the contents of a RAM of a robot controller.

FIG. 6 is a block diagram showing a configuration of a calculation / drive control unit.

FIG. 7 is a block diagram showing a configuration example of a drive data creation device.

FIG. 8 is an explanatory diagram showing the contents of the program storage device.

FIG. 9 is a block diagram showing a configuration of a graphic simulator.

FIG. 10 is an explanatory diagram of constraint conditions for the leg skeleton.

FIG. 11 is a flowchart showing the flow of processing by a higher-level operation program.

FIG. 12 is a flowchart showing a processing flow of a middle-level operation routine.

FIG. 13 is an explanatory diagram of a method of calculating a drive amount of an actuator.

FIG. 14 is a diagram illustrating the flow of the operation of the robot in association with the program hierarchy.

FIG. 15 is a flowchart showing the overall processing flow of the drive data creation device.

FIG. 16 is a flowchart of the input process.

FIG. 17 is a flowchart showing a flow of processing for determining a skeleton unit length in the input processing.

FIG. 18 is a flow chart showing a flow of alignment processing of the end points of each skeleton unit.

FIG. 19 is a flow chart showing the flow of processing for calculating the coordinates of the end points of the skeleton.

FIG. 20 is a flow chart showing the flow of processing for calculating the coordinates of the end points of the skeleton units that make up the legs.

FIG. 21 is a flow chart showing a flow of driving data calculation.

FIG. 22 is a flow chart showing a flow of time input processing.

FIG. 23 is a flowchart similarly showing the flow of drive data storage processing.

FIG. 24 is a flowchart showing the flow of correction processing.

FIG. 25 is a schematic diagram of moving image frames.

FIG. 26 is a schematic diagram showing an example of an arrangement of frames on a time axis.

FIG. 27 is an explanatory diagram of a skeletal image dimension matching process for the facing model.

FIG. 28 is an explanatory diagram of alignment processing of a skeleton image with respect to a model image.

[Explanation of symbols]

 1 Drive Data Generation Device 5 System Control Unit (Drive Data Calculation Means) 200 Robot System 201 Robot 202 Robot Control Unit 204 CPU (Read Control Unit) 205 RAM 205a Upper Operation Program Storage 205b Medium Operation Command Analysis Program Storage 205c Medium Position operation routine storage unit (I) 205c middle level operation routine storage unit (II) 205e data transmission program storage unit 207 storage device 207a upper level operation program storage unit 207b middle level operation command analysis program storage unit 207c middle level operation routine storage unit 207d Medium operation correction program storage unit 207e Data transmission program storage unit 207f Rewriting program storage unit 208 Input device (program rewriting unit) 212 Calculation / drive control unit 213 Actuator

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Teruo Kato 2-28-24 Izumi 2-chome, Higashi-ku, Nagoya-shi, Aichi Ricoh Elemex Co., Ltd.

Claims (8)

[Claims] 1. A medium-level operation routine that forms lower-order operation data in units of drive data relating to the drive of actuators provided in a plurality of robot parts, and integrates the plurality of lower-order operation data to define a basic operation of the robot. The middle-level operation command is made to correspond to each of the middle-level operation routines, and the upper-level operation program is defined by the combination of the middle-level operation commands. Based on the combination of medium-level operation routines corresponding to each medium-level operation command read,
A method for controlling a robot, comprising controlling driving of the actuator. 2. A middle-level operation routine that defines a basic operation of the robot, which is constructed by integrating a plurality of lower-order operation data in units of drive data related to the drive of actuators provided in a plurality of robot parts, and corresponding thereto. Using the attached middle-level operation command, the program storage section that stores the upper-level operation program defined by the combination of these middle-level operation commands, and the higher-level operation program stored in the program storage section Program read means for reading in units of commands, and an actuator operation control unit for controlling the drive of the actuator based on a combination of the intermediate operation routines corresponding to the read intermediate operation commands. Characteristic robot system. 3. The high-order operation program is instructed to the upper-order operation program based on the combination of the middle-order operation commands stored in the program storage unit, and at least one of the middle-order operation commands is rearranged, added, or deleted. The robot system according to claim 2, further comprising a higher-order operation program rewriting means for rewriting the operation program. 4. The drive data forming the lower order motion data is drive amount calculation data for calculating the drive amount of the actuator, and the actuator operation control unit is provided for each robot part. Calculating / driving control means for respectively calculating the driving amount of the actuator included in the robot part based on the driving amount calculation data, and controlling the driving of the actuator based on the calculated driving amount; 4. A data transmission unit that transmits the drive amount calculation data to the calculation / drive control unit when the lower-order operation data is executed based on the middle-level operation routine. Robot system. 5. At least a part of the plurality of robot parts is provided with a skeleton having a plurality of skeletal units and the actuator arranged between the skeletal units, and the robot parts are provided in the robot parts. The corresponding drive amount calculation data is assumed to include displacement data of each skeleton unit for each time of robot operation, and the calculation / drive control means sequentially moves the skeleton units in time. The robot system according to claim 4, wherein the drive amount of the actuator corresponding to the skeleton unit is calculated based on the displacement data. 6. An arbitrary one skeleton unit (hereinafter, referred to as a first skeleton unit) of the skeleton and a second skeleton unit bonded to the first skeleton unit via an actuator,
The calculation / drive control means performs at least one of rotational movement and translational movement on each position of the first skeleton unit and the second skeleton unit at successive times, and the first skeleton at each time. The process of superposing units is performed, the displacement of the second skeleton unit between the times is calculated in that state, and the drive amount of the actuator is calculated based on the calculation result. Robot system. 7. At least a part of the plurality of robot parts includes a skeleton having a plurality of skeleton units and the actuator arranged between the skeleton units, and the drive is provided. A drive data creation device for creating data, comprising a series of model images in which model motions corresponding to robot motions are recorded over time, image display means for displaying the image of the skeleton, and the display means. Based on the skeleton positioning means for aligning the skeleton image with respect to the model image, and the displacement of the skeleton image respectively aligned with respect to the model images that are in front of and behind each other on the time axis,
The robot system according to any one of claims 2 to 6, further comprising: a drive data creation device including: a data calculation unit that calculates the drive data. 8. The skeleton positioning means of the drive data creating apparatus is adapted to position the skeleton image on the model image by superimposing the skeleton image on the model image on the display means. Item 7. The robot system according to Item 7.